Given the focus on data, metrics, and a/b testing that’s all the rage in tech startups these days, you’d think that more tech companies would be using empirical research on hiring practices and diversity initiatives to fix their ‘women, and men of color problem’.
But they aren’t, so here’s an easy diversity hack — based on real science.
The one simple change that tech companies can make, when they hire people and when they evaluate and promote employees, is to do consider people in ‘batches’ rather than serially, one after another.
Data on a huge range of decision making contexts show that we evaluate things differently when we look at them one after another versus as a batch. This is true for emergency spending, flavors of jam, and employee performance. 
Check out this research, conducted by Iris Bohnet, Max Baxerman, and Alexandra van Geen, all at Harvard:
When Performance Trumps Gender Bias: Joint Versus Separate Evaluation
Using actual experiments (a/b testing, people!), these researchers determined that employees were significantly more likely to be selected based on their performance when evaluated jointly rather than separately. In contrast, employees were significantly more likely to be selected based on their gender when evaluated separately.
“Put differently, employers were significantly more likely to select the higher-performing employee rather than the lower-performing employee in joint rather than in separate evaluation…. The size of the effect is considerable.”
You can download the paper and read the details of the research yourself. Or, you could check out these two blog posts summarizing the work and adding commentary from the authors.
Better by the Bunch: Evaluating Job Candidates in Groups, by Maggie Starvish, Working Knowledge, HBS
Use Group Evaluation to Mitigate Gender Bias, by Frank Kalman at Diversity Executive
What’s going in the batch evaluation process?
We think differently about both the people and the performance that we are evaluating when we consider candidates in groups (batches) rather than individually.
When we look at people in batches, we are “more likely to focus on individual performance” rather than on the candidate’s social category.
On the other hand, when we evaluate people one at a time, we are more likely to unconsciously include information drawn from our stereotypes about the candidate’s social category.
Obviously, when these stereotypes are either positive or negative, they bias our decisions— even when the information they supposedly include is irrelevant to the decision being made.
Batch processing triggers a critical cognitive shift.
Three things happen when we look at more than one person at a time.
1. We simply have more information available to us. “By definition, more information is available in joint than in separate evaluation.”
2. We use the other person/s as a standard of comparison, which lets us look at real quantities and real qualities. We’re less able to shift our criteria midstream or to adjust the abstract standard we’re holding as the ideal.
Think of the difference between saying “She does not have ‘enough’ experience” and saying “She has three years of experience compared to the other candidate’s two years”.
3. We shift from an “intuitive” evaluation of an individual to a data-driven evaluation of the group.
It’s not just ‘more’ information, but the idea that we treat this information as ‘data’, that leads us to think more about actual performance criteria that to rely on intuition, feel, or —dare I say — subjective cultural “fit”.
What does this science mean for your company?
There a simple action step you can take to improve the chances that you’ll hire based on qualities other than race, gender, or social category:
Batch process your hiring and promotion decisions.
When you evaluate candidates, consider two or more people at a time for the job. Seriously– sit there in the conference room looking at more than one person’s resume, test results, and interview reports. Compare Person A to Person B to Person C. Don’t compare Person A to “Abstract perfect candidate in your mind.”
These batches don’t have to be big— just comparing two candidates at a time versus one can be enough to trigger the shift from social category to past performance.
You may think it’s impossible to do this in a small company or a startup. You’re running lean, you have only one job open at a time. In that case, fake it role play.
- Compare the current candidate to two or three previous candidates. Or,
- Compare the candidate to other candidates you’d like to have but who aren’t looking for a new job. Or,
- Wait until you have three decent candidates and consider them all at once.
You should also use batch processing when you evaluate current employees’ performance. Again, this may seem ‘impossible’ if you have a small company, but if you’re innovative and committed, you can figure out a way.
For example, consider using rough clusters or groups of similar employees when you evaluate your human resource metrics. Rather than keeping categories so specific that they include only one or two employees (e.g., the API manager for Enterprise Software), use larger general categories (like ‘all software developers’ or ‘all employees promoted this year’).
Consider who’s getting promoted and who’s getting raises across the whole category. This helps you see bigger-picture trends in hiring and promotion decisions that get broken up and missed when you parse the data too finely.
It is true that the idea of ‘batch’ processing goes against the trend towards continuous deployment and many small cycles of iteration over chunk by chunk advances. There’s a way this philosophy confronts how “nimble” tech firms want to operate.
And obviously, there is much more to making an industry inclusive than just hiring people who look different from what’s been the norm. We can’t simply ‘add women and stir’. We can’t simply declare that our organization is a meritocracy and pretend to be “objective” about the ways we evaluate each other. It’s more complex, so why not use the data, the actual science, we already have?
It’s not rocket science. It’s really simple, useful science, that we can start using right. away.
If we are committed to achieving parity and real inclusion, we should use science to revise the processes that shape our actions, so that we actually get to parity.
Batch processing is a simple, easy hack that anyone at any firm can use to increase diversity. So, will you?
Download the research here: Bohnet, Iris and Bazerman, Max H. and van Geen, Alexandra, When Performance Trumps Gender Bias: Joint Versus Separate Evaluation (March 16, 2012). HKS Working Paper No. RWP12-009. Available at SSRN: http://ssrn.com/abstract=2087613 or http://dx.doi.org/10.2139/ssrn.2087613
I am an organizational consultant, change advocate, and organizational identity/reputation scholar with a PhD in leadership & organizations. I research, write about, and consult with organizations on the relationships between organizational identity, actions, and purpose. I teach Technology Management, part-time, at Stevens Institute of Technology.
My current research focuses on how social technologies in the workplace can drive organizational change, generate meaning, and catalyze purpose. See the 
Comments on this entry are closed.