December 27, 2018
How to Create a Scoring Key to Rate Interviews
A walk through the nuances of setting a rating scale to assess competencies during the interview process, and sharing some templates that might come in handy
As we pointed out in a previous piece on how to design interview questions, competency-based interviews (CBI) – or structured interviews, in general – require interviewers have a set of scoring keys, or a rating scale, to be able to evaluate every candidate’s competency as objective as possible.
It is of vital importance that all interviewers share the same scale. The first step is to establish one proficiency-level scale for all competencies (i.e., a range from 0 to 4, or a range from 0 to 6; where 0 indicates no proficiency at all and 4 or 6 the highest proficiency level possible).
After deciding about the scale then it is time to define each of the proficiency-levels. For instance, here we present a 5-scale evaluation: excellent, good, average, poor, and no evidence. See Figure 1 for more detail:
Other potential scales could look like:
A) Far exceeds requirements (Score 4), Exceeds requirements (3), Meets requirements (2), Less than requirements (1), Misses requirements (0).
B) Expert (Score 5), Advanced (4), Intermediate (3), Basic (2), Awareness (1), Not Aware (0).
“More than the actual values of the scale itself, what is relevant is that all competencies are measured using the same scale and that all interviewers apply the same rating criteria”
Rather than the specific range and labels that we want to assign to the scales, which is more or less a question of organizational preferences, what is relevant is that all competencies are measured using the same scale and that all interviewers apply the same criteria.
Additionally, before the interview, we might want to determine which sort of answers, for each question, would score positive points and which would count as negative scores.
Let’s take the competency Proving solving skills and its formulation into the following question: “Describe a situation in which you had to address a problem whose cause was not clear to the organization.” The positive and negative points could be, for this example:
Positive: recognizes her limitations, uses effective strategies, demonstrates a constructive approach towards the issue, takes ownership.
Negative: tries unsuccessfully to fix the situation by herself, uses inappropriate strategies, does not reframe the problem as a challenge, does not take ownership.
The positive/negative points might help us better assess interviewees. Imagine for instance the case where two candidates are given the same score for a specific question: the observation of positive/negative points can help to decide which candidate fared best.
At this stage we should have for each competency: the specific question we will use in the interview together with any follow-up or probing question to clarify either context, action or results (see practical guide How to Design Interview Questions), the scoring key, and the positive and negative points. With all these pieces we are in the position to put together a question-assessment template (see Figure 2).
Finally, if we consider that one competency, or a set of competencies, is more relevant than other/s, then competencies might also be assigned a specific weight. For example, given the particular role we are selecting for, we might conclude that Judgment is more relevant than the others because the position requires a great deal of experience in a cross-cultural environment. We could, therefore, determine that this competency 50% more relevant than the others in the interview.
Figure 3 represents an example of an Interview Assessment template, with a summary of all the scores from the candidate, the different weights to each competency (optional), and the overall score for the interview.
Remember that, more than the rating scale per se, what is most important is that all interviewers have been properly briefed and share the same rulebook when measuring up competencies across interviewees.
Equipped with a scoring key and a definition of what each proficiency level entails for a given competency, we now have a toolkit that will help us reduce the heterogeneity and variance of our assessments. Thus making for a more systematic, scalable recruiting process.
This does not mean we are out of the wood yet. There are still some bumps ahead. Our ingrained biases and stereotypes might steer us to jump into conclusions too fast. If you want to learn more about biases and other rating mistakes, check our article The Biases That Fooled Us.
Recruitment Consultant at Direct HR
L: Chinese, English
T: +86 574 8848 7007
Editor at ChinaHRnews.com
L: English, Spanish
T: +86 21 6010 5000
Free Bonus Report:
Accelerate the Recruitment Cycle with Technology
Download here our full interview with Anne-Marie McCaughan, APAC Henkel Talent Acquisition Head, and get a first-hand account of how Henkel leverages on technology in China and APAC to accelerate talent acquisition. She also tells us what good management means for her and very openly shares a list ...
You May Also Like
Guide: How to Profile Candidates in China
The cultural and experience gaps between Chinese and foreign managers often results in conflicting assumptions with regards to job content and candidate qualification criteria during the hiring process. A well-executed profiling exercise will serve to bridge these gaps.