Wednesday, June 27, 2012

15 CAPTCHA Testing Ideas







This is a continuation to my friend Santhosh Tuppad's blog entry (http://tuppad.com/blog/2012/06/26/captcha-testing-dedicated-to-andy-glover/). He decided to make a fantastic list what could be used when testing a CAPTCHA. I read it, liked it, and decided I want to add my few cents to the topic! Before I go to the test ideas, I want to note CAPTCHA will not increase security and it should not be used for such purpose.

Now I am sure some of these things will overlap with either each other and/or with Santhosh's list. This is fine by me because I wrote the list as things were coming to my mind:

1. Not to pass the information to the server in plain text.
2. Not to have the same CAPTCHA repeat (in a reasonable interval).
3. Try sending a decoded old CAPTCHA value with an old CAPTCHA/Session ID.
4. Catch the HTTP(S) request and check all parameters (for example possible ecnrypted CAPTCHA ID).
5. Server-side validation for inputs (for ex. injections like here http://osvdb.org/show/osvdb/82267).
6. Using dynamic noise in the CAPTCHA is harder to break automatically than static noise (however, obscurity is not security).
7. Avoid possibility to "random success" like selecting an answer from a list.
8. If you are using a visual CAPTCHA, check out from here if it's of any good http://www2.cs.sfu.ca/~mori/research/gimpy/#results.
9. Test the CAPTCHA with some CAPTCHA breaking tools to see if it's any good.
10. If you have access to the code (or someone can tell you technical details), verify from Open Source Vulnerability DataBase if the CAPTCHA has known issues http://osvdb.org/search/advsearch.
11. Test if the session is destroyed after a correct phrase is entered. Reusing session ID of a known image could make it possible to automate requests to the page.
12. Should try to avoid using I, l, 1, 0, o, O etc. because users have problems with them.
13. Can your granny register? Should she be able to?
14. Will the CAPTCHA make people disappear from the service?
15. Do you actually need a CAPTCHA or could you use another means for the purpose you had on your mind?





5 comments:

  1. I like the Granny scenario, I might like to think of more role-based tests. Could Homer Simpson register? A busy executive using a smartphone?

    ReplyDelete
  2. Hi Lisa,

    Thanks for the comment!

    You are absolutely right; the granny scenario is an example to remind the tester about how different people could be using the service/system. Usability/Accessibility tests are easy to forget if the tester doesn't have any major problems for example with vision or hearing.

    I didn't include any moderation-based test ideas here, but in reality it's good to see the system is not possibly offending for ex. different nationalities. Testing for such a thing can be a fun challenge and many sources are needed for oracles.

    Have wonderful last days of June!


    Best regards,
    Jari

    ReplyDelete
  3. Jari, I have heard this comment from some (Not more than 10) software testers who say that, audio version of reCaptcha is not usable (It cannot be understood). Now, one of the possible answer that I could give them is: May be people with disabilities (Partially blind / Color blind / Completely blind) would have some special skill of concentrating on the audio? I am not sure if I could find any survey where blind people were considered to test reCaptcha audio feature. Somehow, people with eyes (who could see) have found it difficult to make out the characters of CAPTCHA when there is back-ground noise. To them it's lot of back-ground noise, may be not for people with disabilities.

    What are your thoughts on this brother?

    ReplyDelete
    Replies
    1. Ho ho ho Santhosh,

      I'm glad you asked about that! I myself never tried the audio one. I think it's nice to have options for people who have problems with sight. But I know one thing already; my ability to understand words from speech/singing is quite bad, so I could imagine having lots of problems with it by myself. :-)

      I think we, and some other people, should put our heads together some day and think what alternatives there could be for the current implementations of CAPTCHAs. Now that would be a challenge!


      Best regards,
      Jari

      Delete
  4. This granny finds captcha very difficult. It feels like a hearing and vision test.

    Lillian Blessing
    Lakeland Florida

    ReplyDelete