Combatting Rater Bias in Performance Reviews
Ratings, scoring, and assessments of people don’t tend to invite warm, fuzzy feelings in most. On the contrary, negative connotations often emerge including disdain, judgment, and for many, a lack of fairness.
In some of our other work, we discuss reasons why traditional performance reviews don’t tend to produce the desired results; namely, they address only one of the three forms of feedback, evaluation. Ignoring the other two forms – coaching and appreciation – sets evaluation feedback up for failure right out of the gate.
But even if you get the feedback process “right” there may still be a distrust for the process.
Speaking with hundreds of HR Professionals and Organizational Leaders every year, we know this is true. Many employees often are skeptical of a performance management process whereby their manager assigns a score or rating of some sort, especially if that rating will affect their pay and career movement.
One necessary factor that must exist for employees to be merely “satisfied” is that they feel valued and rewarded fairly. If employees sense rater bias exists and do not feel rewarded fairly, their productivity may drop to 77%, versus 100% for satisfied employees.
Rater bias has been written about and studied for some time, but we still see many organizations struggle with how to address this.
Our goal for this article is to provide tips and best practices for your organization to reduce rater bias, or the Idiosyncratic Rater Effect, and increase trust for your performance management process.
Research on the Idiosyncratic Rater Bias
A 1998 study in Personnel Psychology found that over 60% of a manager’s rating was actually a reflection of the individual manager, not the employee being rated. Another study in 2000 and in 2010 confirmed these findings.
In these studies, they used what is called a multi-trait, multi-rater (MTMR) process to rate the managers that participated in the studies. In other words, MTMR is what most people would recognize as a 360-degree feedback process. The studies evaluated feedback not only from the manager to employee, but also from two of the manager’s peers, bosses, and subordinates as well as a self-assessment.
This research brings to the surface a serious issue. With performance ratings being such an important factor in individual raises, promotions, bonuses, and overall career growth, there is a lot at stake. If these decisions are being made with biased data, how can anyone trust they are being evaluated and/or compensated in a fair and equal manner?
Companies Move to Eliminate Performance Reviews
Trends to eliminate performance reviews became popular at companies like Adobe, Accenture, and GE several years back. They surmised that removing them altogether would solve the problems created by issues such as the idiosyncratic rater effect. But is eliminating performance ratings really the best option?
In a recent study by Gartner, companies that had removed ratings altogether were actually seeing a decline in overall worker performance by 10% and a decline in engagement by 6%. The study revealed managers struggled to give clear feedback without a defined performance management process.
Performance Reviews Still have a lot of Value
Just because there is potential for bias doesn’t mean you shouldn’t do performance reviews. In fact, performance reviews are even more important when the potential for bias exists. In the absence of some guidance, bias factors increase. Stereotypes increase. Inclusion can decrease.
When done correctly, the review process can minimize bias which will lead to more accurate data, increased confidence in how organizations reward their employees and will boost overall employee engagement.
However, if organizations are not going to commit to doing performance reviews beyond the evaluation portion, it may be better for them to not do reviews at all.
As briefly mentioned in our intro, for an organization to have an effective performance review process they must ensure they are evaluating fairly and giving the right type of feedback along the way.
Another big win outside of reducing bias is that a good performance management process will foster authentic conversations that bubble up employee feedback – feedback that’s critical for organizational success.
Bain & Company dug further into this and determined how engagement and productivity are related. The following criteria in the pyramid of employee needs represents how they classified different levels of engagement.
Satisfied employees met a baseline productivity level of 100% compared to unsatisfied employees at only 77%. Engaged employees saw productivity levels of 144% and inspired employees saw productivity levels of 225%.1
Organizations have a high incentive to ensure they are fostering a culture of inspired employees. Notice the pyramid structure – each level builds off of the level below it. Now notice the baseline level of satisfied employees, specifically the employee’s need of being valued and rewarded fairly. Nothing will undermine this vital employee need faster than a flawed (or missing) performance management and compensation process.
So employees need feedback and ratings, but ratings are often flawed and biased. How do we fix this?
Using Performance Management Tools
Technology enables modern-day organizations to utilize performance management tools to more easily ensure their teams are receiving the right type of feedback. Combining tools like Check-ins, the Performance-Values Matrix, and 360 Degree Feedback helps managers provide appropriate feedback and minimize the potential for idiosyncrasy.
Check-ins
Even if you have a great manager who checks in with you frequently, chances are a few days after the hallway conversation you won’t even remember anything you discussed! Remember the other two-thirds of feedback we mentioned? These check-ins are a crucial part of the coaching and appreciation portion of feedback.
Having a formal check-in process where these conversations are documented is extremely helpful, especially when it comes to the annual performance review. Check-ins help remove bias in the performance appraisal because the manager now has documented, frequent conversations they can reference when evaluating an employee.
At Performance Culture, our Check-ins are employee-driven, whereby employees are asked to answer relevant questions around their wins, needs, goal progress, and behaviors. Writing forces clarity, and when the employee drives the conversation, managers have a better chance of learning their employee’s perspective and simultaneously decreasing their own bias.
Think about entering into the performance review with twelve or more concise, but relevant documented conversations initiated by the employee being evaluated. Not only will your time spent on performance reviews dramatically decrease, but also your insight into the employee’s actual performance will increase.
This becomes even more helpful when managing more than one direct report. When you start managing multiple people, having conversations with each of them without a guided (and documented) approach, the likelihood of remembering details decreases dramatically. And thus one more way rater bias sneaks in. In the absence of information, we fill in the blanks with the stories in our heads. The way WE remember it.
Check-ins ensure employees are getting the appreciation and coaching they need along the way, plus they introduce a method for documenting results and conversations, one method for fighting rater bias.
The Performance-Values Matrix
At Performance Culture, we use a tool called the Performance-Values Matrix. This is a simple, yet powerful tool that helps team members visualize how their performance impacts a company and how their behaviors align with an organization’s core values.
In the references mentioned at the beginning of this article, researchers observed examples of Idiosyncrasy only AFTER the feedback had been finalized. Performance Culture utilizes the Performance-Values Matrix to help managers and supervisors compare evaluations BEFORE finalizing performance reviews. This allows you to calibrate ratings and reduce rater bias.
As a manager, I can perform a “self-check” by examining the scatter plot of my team based on my initial ratings (see image below). By hovering over the data points, I can see the individuals represented by that score, and quickly compare them to the rest of the team. Maybe I was in a good mood on Tuesday when I scored Alex but had an off day on Wednesday when I scored Jennifer. Rating employees in silos without an easy way to check fairness will inherently lead to rater bias, whether subconscious or unconscious.
Additionally, one role of a leader is to raise up other leaders. How often do we promote individuals to management without providing initial or ongoing people skills training? An enormous opportunity cost exists when managers of managers do not coach and prep their direct reports before evaluation time.
Through the use of data filters, Performance Culture allows for leaders to “inspect what they expect” of their managers. By filtering for how my direct report is evaluating his or her team, I can quickly spot anomalies or concerns and dig into those further versus sifting through every. single. review. I can also compare how various managers are rating their teams compared with one another, or department comparisons. Do we have a problem with applying consistent rating methodology? I can quickly ascertain this and provide coaching and training with the Performance-Values Matrix.
We hear countless Human Resource professionals speak about their concerns for consistent and fair ratings by managers. We don’t blame them. But lamenting after the fact doesn’t help much. Correcting a problem before it has an impact is the proactive and right approach.
360 Degree Feedback
The multi-trait multi-rater (MTMR) approach, otherwise known as 360 Degree Feedback, can also be a tool to reduce rater bias. This is a contested statement, but allow us to explain.
We see through the lens of our own biases, perspectives, and emotions. Gaining additional perspectives can help to reduce those factors that may be more attributable to us, the rater, than the ratee.
Reference the image below of Performance Culture’s 360 Degree Feedback Summary Report. Customizable, it allows managers to request feedback from anyone in the organization or outside the organization! Question and answer format is helpful with the aggregated responses by question.
Asking the right style and type of question is key with 360 Degree Feedback. In the Performance Culture System, we provide a starter group of questions to help guide this process:
- On a scale of 0 to 10, how likely are you to recommend this person for help when you need extraordinary results? What is the reason for your rating?
- What should this person do more of?
- Is there something this person should improve or stop doing?
If augmenting questions by requesting informational ratings are desired, the group’s averaged ratings present another quick way to assess if a manager’s ratings are on par or off base. Now, we know that a statistically relevant data set is hardly ever available in most organizations, but perception is reality. What do I mean by this?
That part of the idiosyncratic rater effect is OKAY. What?! Yes, I said that. How your manager, your peers, and your subordinates, if applicable, perceive your drive, your motivation, your acumen and quality of work, matters. It’s part of team chemistry and teams don’t function as well if there is a perception – whether real or biased – of someone not pulling their weight.
In some of our other work referencing DISCⓇ behavioral and driving forces, we learn it’s vital to understand how others perceive you, especially when you are encountering heightened moments of stress. Requesting 360 Degree Feedback on myself helped me to be aware of my blind spots and where I was hurting, not helping, team chemistry.
Emotional intelligence and being aware of others’ perceptions of my contributions to the team can be incredibly helpful in my ability to perform team-based work, and ironically, meet my performance objectives.
In Patrick Lencioni’s book The Ideal Team Player, he highlights these characteristics as Humble, Hungry, and Smart – EQ (emotional quotient). Possessing these traits marks an ideal team player, and people want to row the boat with a person they perceive to be a great team player.
In the Performance Culture System, individual 360-degree responses can be electronically attached to the review, helping the employee to see their areas of strength and improvement from a broader range, and hopefully, feel evaluated fairly.
An Overlooked Method for Fighting Rater Bias
An often overlooked method for fighting rater bias is to more concretely define the rating scale. Applying a simplistic “meets or does not meet expectations” approach leaves enormous room for interpretation.
At Performance Culture, we encourage our clients to carefully think through and define clearly what expectations are, with examples explaining this, in the rating scale. Even for quantifiable or measurable goals, examples or ranges for what is “above expectations” is generally lacking. Is 100% goal attainment meeting expectations? What does exceeding expectations look like? 125%? 150%?
This is particularly important for less measurable goals. Behavior or Core Value performance criteria need definitions on what success looks like as well. Read more about this in our post Creating a Framework for your Core Values.
Take “Communication” as an example. You might define the value of Communication as: “Conveys information and ideas through a variety of media to individuals or groups in a manner that engages the audience and helps them understand and retain the message.” This is a great starting point for managers to assess their employees – and for employees to understand what is expected of them – but going one step further to provide examples of unacceptable and acceptable forms of communication and what exceeding expectations looks like empowers your managers to provide a more accurate and less biased rating.
Performance Management is Changing
Performance management has changed significantly over time and new tools and methodologies are being introduced often. However, these tools will only be effective when used correctly. Training the craftsman (manager) remains one of the most effective methods for reaping the benefits of performance management. This is why Performance Culture puts so much focus on elevating the Manager to one of Coach and Mentor.
Some experts argue that the idiosyncratic rater effect leaves current performance management practices obsolete and new solutions need to be found to counteract this rater bias. We believe there is a solution; a performance management system that not only evaluates, but also facilitates genuine and authentic conversations that help leaders build better teams to achieve better results.
If your organization is ready to change the way that managers give feedback, let us know and we will show you how we can help your team develop a high-performance culture.
1Pyramid of Employee Needs; Bain & Company 2015 EIU Research (300 Companies)
