Page 55 - Special Topic Session (STS) - Volume 4

STS560 James Houran et al.
interviewer useful follow-up questions during behavioural interviews. We use
the example of the 20|20 Skills assessment, which contains over 125 questions
that address eleven HR related factors, including: Leadership, Ethical
Awareness, Service Orientation, Creativity, Problem Solving, Self-Efficacy,
Awareness of Diversity, Team-Building, Group Process, Company Loyalty, and
Humour Appreciation (Houran & Lange, 2007; Lange & Houran, 2009). Each
factor was assessed via ten questions using a 4-point rating scale. The
assessment also contains a “Lie Scale” to gauge potential impression
management. The design and development of the 20|20 Skills™ assessment is
based on collaborative effort between the School of Hotel Administration at
Cornell University and two private companies (Integrated Knowledge Systems,
Inc. and AETHOS Consulting Group, Inc.). Across the eleven factors, the
average reliability was 0.84 (Median = 0.82, SD = 0.04), indicating high
reliability. Inspection of items’ group-specific residuals indicated that item
misfit (if any) cannot be attributed to age-, gender-, job-level-, or ethnicity-
related biases.
Figure 3: 20|20 Skills Item-level Feedback map.

Figure 3 shows an example where a hypothetical test-taker obtained an IRT
scaled score of 87 - which is a linear transform of Tj, (in Logits) - on the
Leadership subscale. Condensed versions of the 10 Leadership items are
shown on the left sorted in order of their difficulty Di (“easy” items occur near

44 | I S I W S C 2 0 1 9