9

Knowing that sleep quantity and quality affects cognitive performance across many domains, why aren't pre-test sleep measures or intra-test measures of arousal a standard part of all cognitive test paradigms?

Artem Kaznatcheev
  • 12,323
  • 5
  • 70
  • 164
stowler
  • 91
  • 2
  • Jeff did a suggested edit, removing 'mediators' from the title. "if sleep is a mediator, what relationship is it mediating? i think OP simply means 'moderator'" If this is the case, could you please edit your question accordingly? – Steven Jeuris Feb 29 '12 at 11:45

1 Answers1

7

Measures of arousal surely do play a role in subject performance on a wide variety of cognitive tasks. Generally, scientists can safely ignore this factor as it is assumed to introduce random noise between participants. From a hypothesis testing perspective, scientists are much more worried about factors that introduce systematic bias, or factors that skew performance in one condition but not another.

As an example, consider a psychologist that is testing whether some intervention has an effect on performance. If we assume that arousal levels throughout the sample are randomly distributed, it should not affect our ability to detect an effect of our independent variable. In fact, there are potentially infinitely other factors that may affect performance on a cognitive task. To name a few: prior experience, mood, fatigue, cognitive load, intelligence, or attention. Each one of these factors in turn is the result of numerous other incidental variables (for instance, my mood may be affected by the weather, or how long it's been since I've eaten).

It would be infeasible to test for all of these factors, but again, if we assume that each measure is randomly distributed throughout a sample, it won't make a big difference-- in fact, it's somewhat statistically convenient:

Many psychological measures are normally distributed, which allow us to run statistical tests that assume normality (e.g., t-tests!) The reason this is so is because what we are measuring is actually the sum of many independent and identically distributed (i.i.d.) variables-- such as arousal, experience, attention, etc. The Central Limit Theorem tells us that the sum of all of these factors lead to a normally distributed measure.

It is still important to know when and how sleep measures may affect performance. In knowing that information, scientists can watch out for situations in which arousal may cause systematic bias between conditions, a potential confound. But in general, it often just doesn't really matter for the tests we are conducting.

Jeff
  • 5,488
  • 29
  • 44
  • 1
    Isn't every variable "random noise" until it is controlled for? – Preece Feb 29 '12 at 20:17
  • 1
    @Preece no. as an example, let's suppose i put a test online to collect data from any participant who is willing. those who take the test voluntarily likely share certain characteristics-- perhaps they are more inquisitive than the population as a whole. in this case my sample is biased, because inquisitiveness is not normally distributed in my sample with respect to the population. – Jeff Feb 29 '12 at 21:10
  • The OP did not explicitly ask about controlled experiments, though. This logic does not fly if you are thinking about (neuro)psychological tests for diagnostic purposes. Does someone know about the practice in this domain? – Gala Mar 01 '12 at 08:27
  • Also it's not true that it does not make a difference or that it is better to have as many source of error as possible. Variables like mood or fatigue can in principle reduce measurement reliability and therefore statistical power. It would be very feasible to identify the most important ones and develop better measures, it's just that we can get away with not doing it (see below). Also, if you can actually measure mood, it doesn't matter much whether it depends on the weather or not. – Gala Mar 01 '12 at 08:32
  • The reason we can get away with incredibly noisy measures and a general disregard for measurement issues in many fields of experimental psychology is that running extra student participants is cheap and everybody mostly care about (statistical) significance (meaning your measure is so noisy that you have no idea about the magnitude of an effect, just that it is not nil but you can still happily publish it and make a career out of it). – Gala Mar 01 '12 at 08:33
  • One last point: As you note, the central limit theorem relies on the fact that all variables have an identical distribution and that does not only mean normal but also with the same variance. There is absolutely no guarantee that it is generally the case. Consider gender and height, gender has a lot more effect than all other variables, the resulting distribution is clearly bimodal, not normal. If you are comparing heights, measuring gender and including it in your randomization strategy and analysis would undoubtedly be valuable. – Gala Mar 01 '12 at 08:49
  • Yes, I agree with almost everything you say; I think I just did a poor job of wording my argument. I'll try to edit my post to be more clear, but you bring up some good points as well-- maybe you should delete your comments and turn them into an answer? – Jeff Mar 01 '12 at 21:20
  • @Gala I also think it would be a good idea to change your comments into an answer – Seanny123 Nov 24 '17 at 17:19