Summary
The article discusses the negative connotations associated with traditional performance reviews and how many organizations struggle with reducing rater bias. Research shows that over 60% of a manager’s rating is a reflection of the individual manager, not the employee being rated. When done correctly, the review process can minimize bias and lead to more accurate data, increased confidence in how organizations reward their employees, and boost overall employee engagement.
Ratings, scoring, and assessments of people don’t tend to invite warm, fuzzy feelings in most. On the contrary, negative connotations often emerge including disdain, judgment, and for many, a lack of fairness.
In some of our other work, we discuss reasons why traditional performance reviews don’t tend to produce the desired results; namely, they address only one of the three forms of feedback: evaluation. Ignoring the other two forms – coaching and appreciation – sets evaluation feedback up for failure right out of the gate. But even if you get the feedback process “right” there may still be a distrust for the process.
Speaking with hundreds of HR Professionals and organizational leaders every year, we know this is true. Many employees often are skeptical of a performance management process whereby their manager assigns a score or rating of some sort, especially if that rating will affect their pay and career movement.
One necessary factor that must exist for employees to be merely “satisfied” is that they feel valued and rewarded fairly. If employees sense rater bias exists and do not feel rewarded fairly, their productivity may drop to 77%, versus 100% for satisfied employees.
Rater bias has been written about and studied for some time, but we still see many organizations struggle with how to address this. Our goal for this article is to provide tips and best practices for your organization to reduce rater bias, or the Idiosyncratic Rater Effect, and increase trust in your performance management process.
Research on Performance Bias
A 1998 study in Personnel Psychology found that over 60% of a manager’s rating was a reflection of the individual manager, not the employee being rated. Another study in 2000 and in 2010 confirmed these findings.
In these studies, they used what is called a multi-trait, multi-rater (MTMR) process to rate the managers who participated in the studies. In other words, MTMR is what most people would recognize as a 360-degree feedback process. The studies evaluated feedback not only from the manager to the employee, but also from two of the manager’s peers, bosses, and subordinates, as well as a self-assessment.
This research brings to the surface a serious issue. With performance ratings being such an important factor in individual raises, promotions, bonuses, and overall career growth, there is a lot at stake. If these decisions are being made with biased data, how can anyone trust they are being evaluated and/or compensated in a fair and equal manner?
Companies Move to Eliminate Performance Reviews
Trends to eliminate performance reviews became popular at companies like Adobe, Accenture, and GE several years back. They surmised that removing them altogether would solve the problems created by issues such as the idiosyncratic rater effect. But is eliminating performance ratings the best option?
In a study by Gartner, companies that had removed ratings altogether were actually seeing a decline in overall worker performance by 10% and a decline in engagement by 6%. The study revealed managers struggled to give clear feedback without a defined performance management process.
Performance Reviews Are Still Valuable
Just because there is potential for bias does not mean you shouldn’t conduct reviews or utilize performance review tools. In fact, performance reviews are even more important when the potential for bias exists. In the absence of some guidance, bias factors increase. Stereotypes increase. Inclusion can decrease.
When done correctly, the review process can minimize bias which will lead to more accurate data, increased confidence in how organizations reward their employees, and will boost overall employee engagement.
However, if organizations are not going to commit to doing performance reviews beyond the evaluation portion, it may be better for them to not do reviews at all.
For an organization to have an effective performance review process they must ensure they are evaluating fairly and giving the right type of feedback along the way. Another big win outside of reducing bias is that a good performance management process will foster authentic conversations that bubble up employee feedback – feedback that’s critical for organizational success.
Bain & Company dug further into this and determined how engagement and productivity are related. The following criteria in the pyramid of employee needs represents how they classify different levels of engagement.
Satisfied employees met a baseline productivity level of 100% compared to unsatisfied employees at only 77%. Engaged employees saw productivity levels of 144% and inspired employees saw productivity levels of 225%.
Organizations have a high incentive to ensure they are fostering a culture of inspired employees. Notice the pyramid structure – each level builds off of the level below it. Now, notice the baseline level of satisfied employees, specifically the employee’s need to be valued and rewarded fairly. Nothing will undermine this vital employee need faster than a flawed (or missing) performance management and compensation process.
How To Prevent Performance Bias
Ratings are often flawed and biased. How do we fix this?
1. Use Performance Management Tools
Technology enables modern-day organizations to utilize performance management tools to more easily ensure their teams are receiving the right type of feedback. Combining Check-in software, the Performance-Values Matrix, and 360 Degree Feedback Software helps managers provide appropriate feedback and minimize the potential for idiosyncrasy.
2. Use One-To-One Check-Ins
Even if you have a great manager who checks in with you frequently, chances are a few days after the hallway conversation you won’t remember anything you discussed. Remember the other two-thirds of feedback we mentioned? Performance check-ins are a crucial part of the coaching and appreciation portion of feedback.
Having a formal check-in process where these conversations are documented is extremely helpful, especially when it comes to the annual performance review. Check-ins help remove bias in the performance appraisal because the manager now has documented, frequent conversations they can reference when evaluating an employee.
At WorkDove, our Check-ins are employee-driven, whereby employees are asked to answer relevant questions around their wins, needs, goal progress, and behaviors. Writing forces clarity, and when the employee drives the conversation, managers have a better chance of learning their employee’s perspective and simultaneously decreasing their own bias.
Think about entering into the performance review with twelve or more concise, but relevant documented conversations initiated by the employee being evaluated. Not only will your time spent on performance reviews dramatically decrease, but also your insight into the employee’s actual performance will increase. This becomes even more helpful when managing more than one direct report. When you start managing multiple people, having conversations with each of them without a guided (and documented) approach, the likelihood of remembering details decreases dramatically. Thus, one more way rater bias sneaks in. In the absence of information, we fill in the blanks with the stories in our heads the way WE remember it.
Check-ins ensure employees are getting the appreciation and coaching they need along the way. Plus, they introduce a method for documenting results and conversations, reducing rater bias.
What Is The Performance-Values Matrix?
At WorkDove, we utilize a tool called the Performance-Values Matrix. This is a simple, yet powerful tool that helps team members visualize how their performance impacts a company and how their behaviors align with an organization’s core values.
As mentioned before, researchers observed examples of Idiosyncrasy only AFTER the feedback had been finalized. WorkDove utilizes the Performance-Values Matrix to help managers and supervisors compare evaluations BEFORE finalizing performance reviews. This allows you to calibrate ratings and reduce rater bias.
As a manager, I can perform a “self-check” by examining the scatter plot of my team based on their initial ratings. By hovering over the data points, I can see the individuals represented by that score and quickly compare them to the rest of the team. Rating employees in silos without an easy way to check fairness will inherently lead to rater bias, whether subconscious or unconscious.
Additionally, one role of a leader is to raise up other leaders. How often do we promote individuals to management without providing initial or ongoing people skills training? An enormous opportunity cost exists when managers of managers do not coach and prep their direct reports before evaluation time.
Through the use of data filters, WorkDove allows leaders to “inspect what they expect” of their managers. By filtering for how my direct report is evaluating his or her team, I can quickly spot anomalies or concerns and dig into those further versus sifting through every. single. review. I can also compare how various managers are rating their teams compared with one another, or department comparisons. Do we have a problem with applying consistent rating methodology? I can quickly ascertain this and provide coaching and training with the Performance-Values Matrix.
360 Degree Feedback
The multi-trait multi-rater (MTMR) approach, otherwise known as 360 360-degree feedback, can also be a tool to reduce rater bias. This is a contested statement, but allow us to explain.
We see through the lens of our own biases, perspectives, and emotions. Gaining additional perspectives can help to reduce those factors that may be more attributable to us, the rater, than the ratee.
Asking the right style and type of question is key with 360 Degree Feedback. We provide a starter group of questions to help guide this process:
- On a scale of 0 to 10, how likely are you to recommend this person for help when you need extraordinary results? What is the reason for your rating?
- What should this person do more of?
- Is there something this person should improve or stop doing?
If augmenting questions by requesting informational ratings is desired, the group’s averaged ratings present another quick way to assess if a manager’s ratings are on par or off base. Now, we know that a statistically relevant data set is hardly ever available in most organizations, but perception is reality. What do we mean by this?
That part of the idiosyncratic rater effect is OKAY. How your manager, your peers, and your subordinates, if applicable, perceive your drive, your motivation, your acumen, and your quality of work, matters. It’s part of team chemistry and teams don’t function as well if there is a perception – whether real or biased – of someone not pulling their weight.
Performance Review Rating Scale
An often overlooked method for fighting rater bias is to more concretely define the rating scale. Applying a simplistic “meets or does not meet expectations” approach leaves enormous room for interpretation.
It is important to carefully think through and define clearly what expectations are, with examples explaining this, in the rating scale. Even for quantifiable or measurable goals, examples or ranges for what is “above expectations” are generally lacking. Is 100% goal attainment meeting expectations? What does exceeding expectations look like? 125%? 150%?
This is particularly important for less measurable goals. Behavior or core values performance criteria need definitions of what success looks like as well. Take “Communication” as an example. You might define the value of Communication as “Conveys information and ideas through a variety of media to individuals or groups in a manner that engages the audience and helps them understand and retain the message.” This is a great starting point for managers to assess their employees (and for employees to understand what is expected of them) but going one step further to provide examples of unacceptable and acceptable forms of communication and what exceeding expectations looks like empowers your managers to provide a more accurate and less biased rating.
Performance Management and Performance Reviews Are Changing
Performance management has changed significantly over time and new tools and methodologies are being introduced often. However, these tools will only be effective when used correctly. Training the craftsman (manager) remains one of the most effective methods for reaping the benefits of performance management.
Some experts argue that the idiosyncratic rater effect leaves current performance management practices obsolete and new solutions need to be found to counteract this rater bias. We believe there is a solution; performance management software that not only evaluates but also facilitates genuine and authentic conversations that build trust and foster growth and development.
If your organization is ready to change the way that managers give feedback, WorkDove can show you how to help your team develop a high-performance culture.
