[Jdm-society] sat scores and interviews

Wed May 28 00:49:56 EDT 2008

Regarding the SAT and related tests, I'd suggest Sackett et al.'s "High-Stakes 
Testing in Higher Education and Employment" appeared in the latest issue of 
the American Psychologist (April-June, 2008).

Quite easy to read, I think.

Edgar Kausel
University of Arizona

On Tue, 27 May 2008 19:14:13 -0500
  jdm-society-request at mail.sjdm.org wrote:
> Send Jdm-society mailing list submissions to
> 	jdm-society at mail.sjdm.org
> 
> To subscribe or unsubscribe via the World Wide Web, visit
> 	http://www.sjdm.org/mailman/listinfo/jdm-society
> or, via email, send a message with subject or body 'help' to
> 	jdm-society-request at mail.sjdm.org
> 
> You can reach the person managing the list at
> 	jdm-society-owner at mail.sjdm.org
> 
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Jdm-society digest..."
> 
> 
> Today's Topics:
> 
>   1. sat scores and interviews (Eric R Stone)
>   2. Re: sat scores and interviews (Jonathan Baron)
>   3. Re: sat scores and interviews (David Budescu)
>   4. Re: sat scores and interviews (Scott Highhouse)
> 
> 
> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Tue, 27 May 2008 16:48:15 -0400 (EDT)
>From: Eric R Stone <estone at wfu.edu>
> Subject: [Jdm-society] sat scores and interviews
> To: List <jdm-society at mail.sjdm.org>
> Message-ID: <Pine.A41.4.58.0805271636020.475634 at f1n11.sp2net.wfu.edu>
> Content-Type: TEXT/PLAIN; charset=US-ASCII
> 
> Hi everyone,
> 
> Our administration, in their ultimate wisdom, has recently decided to
> eliminate the SAT requirement and replace it, among other things, with
> personal interviews of each potential candidate.  From my read of the
> literature, this is replacing a (weakly) valid predictor with a
> nondiagnostic (at best) predictor of academic performance.
> 
> I have two questions:
> 
> 1. Although I would like to require each of the decision makers to read
> the collected works of Dawes and Meehl (among others), I suspect that at
> best I could convince a couple of people to read one very short piece,
> preferably something written in a mainstream source for laypeople.  Any
> suggestions for a very short, readable piece of information on the
> drawbacks of interviews?
> 
> 2. My understanding is that for this sort of decision making, interviews
> are pretty much worthless, and may actually degrade the process if they
> call attention to non-predictive factors (e.g., charisma, whether one's
> parents attended the university) and thus away from more predictive ones.
> But i'm not current on this line of research.  If anyone knows of any work
> that would lead me to feel better about this decision, I'd love to hear
> it!
> 
> thanks,
> eric
> 
> Eric Stone
> Department of Psychology
> Wake Forest University
> Box 7778 Reynolda Station
> Winston-Salem, NC  27109
> (336)-758-5729
> estone at wfu.edu
> 
> 
> ------------------------------
> 
> Message: 2
> Date: Tue, 27 May 2008 17:02:28 -0400
>From: Jonathan Baron <baron at psych.upenn.edu>
> Subject: Re: [Jdm-society] sat scores and interviews
> To: Eric R Stone <estone at wfu.edu>
> Cc: List <jdm-society at mail.sjdm.org>
> Message-ID: <20080527210228.GA5177 at psych.upenn.edu>
> Content-Type: text/plain; charset=us-ascii
> 
> Although I agree about interviews, I'm not sure that SAT aptitude
> tests are any better.  (The achievement tests are better.)
> 
> See http://www.psych.upenn.edu/~baron/sat.pdf and
> http://www.sas.upenn.edu/~baron/sat.htm
> 
> A group at the University of California found the same thing several
> years later.
> 
> These findings, like those concerning interviews, were roundly
> ignored.
> 
> Jon
> -- 
> Jonathan Baron, Professor of Psychology, University of Pennsylvania
> Home page: http://www.sas.upenn.edu/~baron
> 
> 
> ------------------------------
> 
> Message: 3
> Date: Tue, 27 May 2008 16:18:34 -0500 (CDT)
>From: David Budescu <dbudescu at cyrus.psych.uiuc.edu>
> Subject: Re: [Jdm-society] sat scores and interviews
> To: Jonathan Baron <baron at psych.upenn.edu>
> Cc: Eric R Stone <estone at wfu.edu>, List <jdm-society at mail.sjdm.org>
> Message-ID: <Pine.SOL.4.64.0805271612130.4375 at stat.psych.uiuc.edu>
> Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
> 
> 
> I agree with Eric and Jon about the interviews.
> 
> You can see a re-analysis of the CA data in table 13 (page 146) of
> 
> Azen, R. & Budescu, D.V. Dominance analysis: A method for 
> comparing predictors in multiple regression. Psychological Methods, 
> 2003, 8, 129-148.
> 
> which you can down-load from http://www.psych.uiuc.edu/~dbudescu
> 
> The aptitude test (SAT I) is the weakest predictor (dominated by HS 
> grades and the achievemt (SAT II) tests), but the 3 combined are better.
> 
> 
> 
> 
> On Tue, 27 May 2008, Jonathan Baron wrote:
> 
>> Although I agree about interviews, I'm not sure that SAT aptitude
>> tests are any better.  (The achievement tests are better.)
>>
>> See http://www.psych.upenn.edu/~baron/sat.pdf and
>> http://www.sas.upenn.edu/~baron/sat.htm
>>
>> A group at the University of California found the same thing several
>> years later.
>>
>> These findings, like those concerning interviews, were roundly
>> ignored.
>>
>> Jon
>> -- 
>> Jonathan Baron, Professor of Psychology, University of Pennsylvania
>> Home page: http://www.sas.upenn.edu/~baron
>> _______________________________________________
>> Jdm-society mailing list
>> Jdm-society at mail.sjdm.org
>> http://www.sjdm.org/mailman/listinfo/jdm-society
>>
> 
> 
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> |	David V. Budescu					|
> |	Department of Psychology, University of Illinois	|
> |	603 E. Daniel Street, Champaign, IL 61820		|
> |	Tel (217) 333 6758 or (217) 840 1586			|
> |	Fax (217) 244 5876	Email: dbudescu at uiuc.edu 	|
> |	http://www.psych.uiuc.edu/~dbudescu 			|
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 
> 
> ------------------------------
> 
> Message: 4
> Date: Tue, 27 May 2008 18:36:17 -0400
>From: "Scott Highhouse" <shighho at bgnet.bgsu.edu>
> Subject: Re: [Jdm-society] sat scores and interviews
> To: "Eric R Stone" <estone at wfu.edu>
> Cc: List <jdm-society at mail.sjdm.org>
> Message-ID: <1211927777-15332.00021.00112-smmsdV2.1.6 at smtp.bgsu.edu>
> Content-Type: text/plain; charset="us-ascii"
> 
> Eric: I have attached a forthcoming article that addresses these issues:
> 
> Highhouse, S. (feature article). Stubborn reliance on intuition and 
>subjectivity in employee 
> selection. Industrial and Organizational Psychology: Perspectives on Science 
>and Practice.
> 
> Also, Nathan Kuncel has done a lot of work on the validity of *g* measures 
>like the SAT:
> 
> Kuncel, N. R., Hezlett, S. A., & Ones, D. S. (2004). Academic performance, 
>career potential, 
> creativity, and job performance: Can one construct predict them all? Journal 
>of Personality 
> and Social Psychology [Special Section, Cognitive Abilities: 100 Years after 
>Spearman (1904)], 
> 86, 148-161.
> 
> Kuncel, N. R., Hezlett, S. A., & Ones, D. S. (2001). A comprehensive 
>meta-analysis of the 
> predictive validity of the Graduate Record Examinations: Implications for 
>graduate student 
> selection and performance. Psychological Bulletin, 127, 162-181.
> 
> ---------Included Message----------
>>Date: 27-May-2008 16:53:34 -0400
>>From: "Eric R Stone" <estone at wfu.edu>
>>To: "List" <jdm-society at mail.sjdm.org>
>>Subject: [Jdm-society] sat scores and interviews
>>
>>Hi everyone,
>>
>>Our administration, in their ultimate wisdom, has recently decided to
>>eliminate the SAT requirement and replace it, among other things, with
>>personal interviews of each potential candidate.  From my read of the
>>literature, this is replacing a (weakly) valid predictor with a
>>nondiagnostic (at best) predictor of academic performance.
>>
>>I have two questions:
>>
>>1. Although I would like to require each of the decision makers to read
>>the collected works of Dawes and Meehl (among others), I suspect that at
>>best I could convince a couple of people to read one very short piece,
>>preferably something written in a mainstream source for laypeople.  Any
>>suggestions for a very short, readable piece of information on the
>>drawbacks of interviews?
>>
>>2. My understanding is that for this sort of decision making, interviews
>>are pretty much worthless, and may actually degrade the process if they
>>call attention to non-predictive factors (e.g., charisma, whether one's
>>parents attended the university) and thus away from more predictive ones.
>>But i'm not current on this line of research.  If anyone knows of any work
>>that would lead me to feel better about this decision, I'd love to hear
>>it!
>>
>>thanks,
>>eric
>>
>>Eric Stone
>>Department of Psychology
>>Wake Forest University
>>Box 7778 Reynolda Station
>>Winston-Salem, NC  27109
>>(336)-758-5729
>>estone at wfu.edu
>>_______________________________________________
>>Jdm-society mailing list
>>Jdm-society at mail.sjdm.org
>>http://www.sjdm.org/mailman/listinfo/jdm-society
>>
>>
> ---------End of Included Message----------
> 
> Scott Highhouse, Ph.D.
> Professor of Industrial-Organizational Psychology
> Bowling Green State University
> http://shighho.bgsu.eportfolio.us
> 
> 
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: not available
> Type: application/msword
> Size: 318976 bytes
> Desc: not available
> URL: 
><http://www.sjdm.org/mail-archive/jdm-society/attachments/20080527/8c13129b/attachment.wiz>
> 
> ------------------------------
> 
> _______________________________________________
> Jdm-society mailing list
> Jdm-society at mail.sjdm.org
> http://www.sjdm.org/mailman/listinfo/jdm-society
> 
> 
> End of Jdm-society Digest, Vol 113, Issue 19
> ********************************************