[Jdm-society] Experimenter bias

Tue May 8 16:24:02 CDT 2007

Thanks so much everyone - I had a feeling you'd all know where this literature resided. Ken will be thrilled. Laura 

________________________________

From: jdm-society-bounces at mail.sjdm.org on behalf of Norm Constantine
Sent: Tue 5/8/2007 12:21 PM
To: jdm-society at mail.sjdm.org
Subject: Re: [Jdm-society] Experimenter bias

Laura, there's quite a bit of relevance to this question (of experimenter bias toward matching previous results) in the areas of confirmation bias, motivated reasoning, and scientific reasoning. The following might provide a good start:
Ditto, P.H. & Lopez, D.F. (1992). Motivated skepticism: Use of differential decision criteria for preferred and nonpreferred conclusions. Journal of Personality and Social Psychology, 63, 568-584. 

Evans, J.S.B.T. (2002). The influence of prior beliefs on scientific thinking. In Carruthers, P., Stich, S., and Siegal, M. (Eds.) The cognitive basis of science. Cambridge: Cambridge University Press.

Faust, D. (1984). The limits of scientific reasoning. Minneapolis: University of Minnesota Press.

Koslowski, B. (1996). Theory and evidence:  The development of scientific reasoning.  Cambridge, MA: MIT Press/Bradford Books.

Kunda, Z. (1990). The case for motivated reasoning. Psychological Bulletin, 108, 480-498.

Mahoney, M.J. (1977). Publication prejudices: An experimental study of confirmatory bias in the peer review system. Cognitive Therapy and Research, 1, 161-175.

--
Norman A. Constantine, Ph.D.
Clinical Professor, School of Public Health 
University of California, Berkeley
Director, Center for Research on Adolescent Health and Development 
Public Health Institute
Phone: 925.284.8118  Fax: 925.284.8107
http://sph.berkeley.edu/faculty/constantine.html
http://crahd.phi.org/

----- Original Message ----- 
From: McNamara, Laura A <mailto:lamcnam at sandia.gov>  
To: jdm-society at mail.sjdm.org 
Sent: Tuesday, May 08, 2007 7:57 AM
Subject: [Jdm-society] Experimenter bias

Hello everyone,

Usually I lurk on this list, but this question came across my e-desk and I wondered if any of you can comment. Feel free to contact Ken off list - he's a physicist at Los Alamos National Laboratory.

Laura 

Laura,

I hope all is going well with you.

I have been thinking about a topic that I believe is right up your
alley, and am wondering whether you can point me in a useful direction.

I have been looking at the historical record of measurements of
fundamental quantities in physics. I believe there is evidence to
support the notion that experimenters may be influenced, perhaps
subconsciously, by previously published results. Those of us who are
or have been involved in physics experiments recognize this
possibility. We realize that there is a definite tendency to question
what you have missed, if your results differs from earlier ones. On
the other hand, it is difficult the get scientists to talk candidly
about this effect.

Incidentally, I have signed up to talk in a few months about what we
can learn about likelihood functions from the history of
nuclear-physics measurements. My main purpose is to determine whether
the data support long-tailed likelihood functions, which are often
invoked to handle outliers.

I am wondering whether you have considered or studied the issue of
experimenter bias toward matching previous results. Is this a topic
that has been addressed by statisticians or elicitation experts, and
if so, what keywords would apply? 

I can imagine that one could even model this effect, to some extent.
The problem is that people handle differences of opinion (measured
results) in various ways. Some would question themselves, others the
previous experimenters. Anyway, it is an interesting topic.

Cheers,
Ken Hanson

MS D413/T-16
Los Alamos National Laboratory
Los Alamos, NM 87545, USA

PHN: 505 662 4370
E-MAIL: kmh at lanl.gov
Group office, PHN: 505 667 4835, FAX: 505 667 1931
At home: kennethmhanson at msn.com, 505 662 4370
WWW: http://www.lanl.gov/home/kmh

________________________________

_______________________________________________
Jdm-society mailing list
Jdm-society at mail.sjdm.org
http://www.sjdm.org/mailman/listinfo/jdm-society

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.sjdm.org/mail-archive/jdm-society/attachments/20070508/1dfc5e30/attachment-0001.htm 