Home > Tools, Websites > PsychFileDrawer.org

PsychFileDrawer.org

PsychFileDrawer LogoPsychFileDrawer.org is a web site that addresses ‘the file drawer problem’, the name for the bias introduced into scientific literature by a reluctance on the part of journals to publish negative or nonconfirmatory results (especially, perhaps, ones that contradict studies published earlier in the same journal.)

The maintainers (including Robert Rosenthal, the originator of the term) allow submitters to indicate the degree to which they modified the original study, and what their results were, along with a brief description of the perceived difference.

The results aren’t peer reviewed, but they’re based on peer-reviewed studies, and include of course real names, so they are easy enough to judge. In addition to one failure to replicate which has received publication (January & Kako’s 2007 exhaustive failure to replicate Boroditsky 2001), I was surprised to see a failure to replicate one of my favorite studies – Bargh, Chen, and Burrows 1996.

The Bargh et al. study buried words related to old age in a crossword puzzle, and had subjects unscramble them and use them in sentences. While subjects believed at this point that the study was over, in actuality it was just beginning in earnest; they were timed walking to the exit of the lab, and subjects who unscrambled words related to old age were slower than other subjects.

The replication, performed in 2008, made only small modifications – more than doubling the number of subjects, and replacing the stopwatch-equipped experimenter with automated infrared timing cones that measured walking rate.

The final result was that subjects in the old-age-related condition walked faster, but not significantly (p=.38, about a 0.2 sec ∂ over 27′). The submitter commentary focused on potential lack of blindness to outcome on the part of the experimenter (an angle subsequently developed by a later study published last month.)

I hardly need to rehash why results like this are significant. Like the Boroditsky result, this is an example I’ve used (and heard other instructor use) in class to talk about priming. It’s doubtless found its way into a few textbooks, as well. But as the site maintainers note, knowing about replication failures like this is frequently a matter of being plugged in to an informal social network, since the studies don’t find their way to publication very often. (Whether this is an entirely bad thing is available as a subject for debate.)

In this case, a failure to replicate with some additional work suggesting a more nuanced view between these two analyses was published six weeks ago on PLoS One by Doyen et al. – but the value of the FileDrawer project is clear.

PsychFileDrawer.org

Doyen et al. 2012: Behavioral Priming: It’s all in the mind, but whose Mind? 

Advertisements
Categories: Tools, Websites
  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: