How to choose the right performance review rating scale

“Let me simply say that after a decade of discussion on the topic, the concepts of ratings themselves are not the issue. Organizations need to make decisions about people… and these decisions themselves are essentially evaluative by nature. The key today is to use lots of data and feedback to make these decisions; do them in a transparent and fair way; clearly communicate what is valued in the company; and give people visibility into others’ goals and projects.”

– Josh Bersin

In this blog post, we explore why rating scales are a necessary component of performance reviews and share recommendations on how to create one that works for your company.

Why rate employee performance?

You may be wondering why we have to rate performance in the first place. The simple answer is companies need an understanding of how employees are performing so the organization can continue to compete and grow. Further, employees want to understand what’s expected of them to get a raise, get promoted, or move forward in their careers, and that requires some form of measurement around established performance criteria.

Properly-designed performance ratings help differentiate high performers from low performers, identify areas for improvement, and offer transparency in decision making. Research shows that top performers can outperform the rest by over 400% – yet without rating performance, you wouldn’t be able to systematically identify and act on these performance differences.

What makes using rating scales difficult?

One reason why many people are against the idea of a rating scales is that traditional performance reviews are frequently the source of bad rating data – in the sense that they don’t measure the behaviors and business impact they should be measuring. This means you could unintentionally be overlooking or promoting the wrong people.

Unaddressed rater biases are also a common problem that can lead to inflation or deflation of employee ratings, which can have serious implications for performance reviews. Two common performance evaluation biases that can be impacted by rating scales are Leniency Bias, which is the tendency to give high ratings to everyone, and Centrality Bias, which is the tendency to rate people somewhere in the middle. Lack of awareness around these biases can lead to skewed ratings that don’t offer valuable or accurate data.

How to create a rating scale that works for your company

The good news is that you have the power to create a customized rating scale for your company that both produces useful data and helps reduce common biases. This also presents a great opportunity to bring focus to attributes and values you want to encourage within your organization. By identifying the behaviors you want to see in your employees, you can build a culture where those actions are valued.

Here are some recommendations to get you started on the right path with performance review rating scales:

Tip #1: Understand spread and validity

The most important concepts to understand when creating an employee performance rating scale are spread and validity. These are the two areas where most traditional performance ratings and reviews tend to be weak. We break down the definitions of both below:

• Spread: This is also known as variance, variation, or range. If you use a measuring stick to measure the performance of different people, does that measuring stick actually pick up on real differences in performance? Many old-school performance tools don’t effectively differentiate and create any meaningful spread. That’s why when these tools are in the real world, most employees get a 4 or 5 on a 5-point scale. Designing a scale with multiple, well-defined response options for “above average” performance in addition to training raters and running calibration sessions are a few ways you can address leniency bias, for example.
• Validity: Does the question or tool measure what we say it measures… and what the organization really cares about? In other words, do the ratings actually matter and help drive better decisions?

For example, if you have a measure of caloric intake, does it predict, affect, or drive anything else in the real world like your weight, height, or longevity? That’s why it’s always important to ask yourself whether a question is identifying some outcome that’s important to your company.

Tip #2: Customize your wording

It’s critical to make sure you articulate a rating scale that aligns with your unique brand and business goals. This means being flexible with regard to how you customize the wording and design of your question wording and response options, based on the criteria you’re trying to measure and the behaviors of your employees.

For instance, if you know that Centrality Bias has been an issue for managers in past performance reviews, you may want to eliminate any “neutral” options on your scales. Or, if you’re a company focused on the personal and career development of your employees, choose language like “getting there” over “incomplete” to encourage people to adopt a growth mindset rather than a fixed mindset.

Another option is to use strongly worded language that focuses on the rater’s personal intentions instead of the rating of the employee to reduce bias, which is a problem that has been noted in past research. Further, the more specific you are in defining each of the response option descriptions (i.e., “anchors”), the better and more consistent your raters will be in using the scales.

We encourage customization, and our platform allows you to reformat your questions and rating scales to address your organization’s goals and minimize the biases you may have seen in previous performance review cycles.

Tip #3: Be transparent

Finally, be prepared to share and train your employees on the scales you’re going to use. Transparency helps build trust and perceptions of fairness around how employees are being measured, and part of that is making sure everyone understands how the company defines success. One of the biggest mistakes you can make is to tell employees you’re eliminating the rating system only to use it behind closed doors with the executive and management teams.

Performance review rating scale examples

3-point rating scale

Most companies are familiar with the standard 3-point rating scale:

On a scale of 1 to 3, I would say this person handles team conflicts well.

As mentioned earlier, the validity of scales can be improved by adding more detail and specificity to the response options. Here’s an example for adding context to a more well-defined 3-point scale:

1. Below Expectations—Does not consistently meet expectations that are appropriate for the position.  Additional direction and coaching needed. Willing or able to improve but lacks results.
2. Meeting Expectations—Consistently meets expectations and sometimes exceeds expectations.  Requires occasional direction and/or coaching. Achieves a majority of core goals.
3. Exceeding Expectations—Always exceeds expectations.  Require little to no direction and/or coaching.  Always achieves or exceeds goals. Influences others to perform better.  Top 10% of peer group.

While this type of scale may work for measuring certain criteria, it often lacks the nuance needed to make appropriate assessments. Culture Amp’s Director of Product for Performance, Srinivas Krishnamurti, recommends developing a scale that has more gradations for top performers and fewer gradations for lower performers.

“Let’s say you have three ratings: ‘not meeting,’ ‘meeting,’ and ‘exceeding.’ It’s hard to justify giving bonuses to everyone who is in the ‘exceeding’ category. So you want to make the ratings more fine-grained and maybe introduce another rating where you’re going beyond ‘exceeding’ and should, therefore, be paid more. On the other hand, if you’re underperforming, you don’t want multiple gradations of that. If you’re not cutting it, that should be clear.”

Reducing leniency bias & improving “Above Average” spread

If your organization runs into challenges differentiating among top performers, consider using a scale that helps identify variation between “above average” contributors, such as:

How well does this person deliver on their objectives?

Of course, we recommend using the above response options as a starting point and customizing them with clarifying descriptors that align with your organization’s objectives and culture.

Evaluating “softer” skills

For areas of evaluation focused on “softer” topics like interpersonal skills, you can use an observation scale to indicates how frequently the desired behavior is being demonstrated by the employee. With this type of scale, it’s critical that the rater works closely enough with the employee to have a well-informed understanding of their behavior – though that’s important for all types of performance ratings. For example:

This person demonstrates leadership qualities in high-pressure situations.  

Rating scales have the potential to be a powerful piece of a holistic performance management system that your people trust and embrace. To learn more about how to rebrand and rebuild your performance review process to align with company goals, download our Performance eBook today.