In the statistical gargon used in psychology, p refers to the probability that the difference you see between two groups (of introverts and extroverts, say, or males and females) could have occurred by chance. As a general rule, psychologists report a difference between two groups as ‘significant’ if the probability that it could have occurred by chance is 1 in 20, or less. The possibility of getting significant results by chance is a problem in any area of research, but it’s particularly acute for sex differences research. Supppose, for example, you’re a neuroscientist interested in what parts of the brain are involved in mind reading. You get fifteen participants into a scanner and ask them to guess the emotion of people in photographs. Since you have both males and females in your group, you rin a quick check to ensure that the two groups’ brains respond in the same way. They do. What do you do next? Most likely, you publish your results without mentioning gender at all in your report (except to note the number of male and female participants). What you don’t do is publish your findings with the title “No Sex Differences in Neural Circuitry Involved in Understanding Others’ Minds.” This is perfectly reasonable. After all, you weren’t looking for gender difference and there were only small numbers of each sex in your study. But remember that even if males and females, overall, respond the same way on a task, five percent of studies investigating this question will throw up a “significant” difference between the sexes by chance. As Hines has explained, sex is “easily assessed, routinely evaluated, and not always reported. Because it is more interesting to find a difference than to find no difference, the 19 failures to observe a difference between men and women go unreported, whereas the 1 in 20 finding of a difference is likely to be published.” This contributes to the so-called file-drawer phenomenon, whereby studies that do find sex differences get published, but those that don’t languish unpublished and unseen in a researcher’s file drawer.
In the statistical gargon used in psychology, p refers to the probability that the difference you see between two groups (of introverts and extroverts, say, or males and females) could have occurred by chance. As a general rule, psychologists report a difference between two groups as ‘significant’ if the probability that it could have occurred by chance is 1 in 20, or less. The possibility of getting significant results by chance is a problem in any area of research, but it’s particularly acute for sex differences research. Supppose, for example, you’re a neuroscientist interested in what parts of the brain are involved in mind reading. You get fifteen participants into a scanner and ask them to guess the emotion of people in photographs. Since you have both males and females in your group, you rin a quick check to ensure that the two groups’ brains respond in the same way. They do. What do you do next? Most likely, you publish your results without mentioning gender at all in your report (except to note the number of male and female participants). What you don’t do is publish your findings with the title “No Sex Differences in Neural Circuitry Involved in Understanding Others’ Minds.” This is perfectly reasonable. After all, you weren’t looking for gender difference and there were only small numbers of each sex in your study. But remember that even if males and females, overall, respond the same way on a task, five percent of studies investigating this question will throw up a “significant” difference between the sexes by chance. As Hines has explained, sex is “easily assessed, routinely evaluated, and not always reported. Because it is more interesting to find a difference than to find no difference, the 19 failures to observe a difference between men and women go unreported, whereas the 1 in 20 finding of a difference is likely to be published.” This contributes to the so-called file-drawer phenomenon, whereby studies that do find sex differences get published, but those that don’t languish unpublished and unseen in a researcher’s file drawer.
