1

I often analyse data that involves participants completing a selection of psychological tests online. A concern with online testing is that some participants may not complete the task conscientiously. For example, they may response options at random or just click the same response option repeatedly.

In general, I like to get response times per item, as I find that they are useful in diagnosing people who are deliberately skipping through the test. However, there are cases where that is not possible.

To make discussion concrete, think about where you have responses to 10 psychological scales. Each scale has 20 items. Some items may be reverse coded.

  • What is a good way to identify participants who were not concentrating or not completing the items conscientiously?
  • How do you ensure that you don't filter out participants with unusual patterns of responses?
Jeromy Anglim
  • 30,741
  • 11
  • 93
  • 221
  • Isn't that the same as http://cogsci.stackexchange.com/questions/3267/analyzing-control-questions-data-for-a-survey ? If not, how is it different? I'd not measure response times in an online questionnaire, because when I take an online survey, I'm usually getting up to get tea in the middle of a question, or answering email etc. –  May 07 '13 at 08:53
  • As for short response times, the upper limit needs to include a highly intelligent, totally focussed participant trying to break the speed record. I'm not sure if you can differentiate this from someone trying to emulate randomness (who will need time to make choices). Otherwise non-random patterns such as always the same value, "stairs", left-right-left-right etc., indicate a high probability (but no certainty) of "just clicking without reading". So, nothing to rely on. –  May 07 '13 at 09:05
  • 2
    You're right. I thought I remembered seeing a question like this before. It looks like I even answered it :-) – Jeromy Anglim May 07 '13 at 12:11

0 Answers0