Assessing Performance: Investigation of the Influence of Item-Context and Rater PersonalityDaniel C. KuangABSTRACT
Context effects in performance appraisals have been operationalized as a person level phenomenon. The present study extends this literature by investigating context effects at the instrument level by examining: the influence of rating contextual items on subsequent ratings of task items, and the influence of rating task items on subsequent ratings of contextual items. Item response theory (IRT) methods of differential item functioning (DIF) detection were applied to investigate the influence of context on item ratings of task and contextual performance. The order in which task and contextual items were presented was found to influence rating responses. Specifically, when task items were preceded by contextual items, the relatedness between task items and their underlying construct was reduced. These observed slope effects, as conceptualized within the IRT framework, demonstrate an advantage of IRT methods over traditional approaches for detecting the influence of context on responses to performance appraisal questions. Overall, an asymmetry in the influence of context between task and contextual scales was observed. Results are discussed in terms of the qualitative differences between task and contextual performance and how these differences influence memory retrieval. A major implication from this is that scale order may influence the validity of a performance appraisal. The rater personality study examined the relationship between levels of rater agreeableness and conscientiousness and ratings of task and contextual performance. Use of a more precise operationalization of performance detected complex relationships between personality and ratings on task and contextual performance. While rater agreeableness and conscientiousness were both related to task performance ratings, conscientiousness moderated the relationship between rater agreeableness and contextual performance ratings. Overall, high agreeable raters tend to give higher ratings than low agreeable raters. However, these differences were greatest among low conscientious raters but minimized among high conscientious raters.
Monday, June 7, 2004
|
