Unbiased Data & Debiased Data

Kristen Flores
September 21, 2020

In this session, hear from Dr. Lewis Baker, Director of Data Science Strategy at pymetrics, on unbiased versus debiased data and how to leverage each to foster a diverse workforce.   

Watch the session here

What does it mean to be diverse? 

Diverse groups will, by definition, be different. A diverse workforce can take many different forms. Diversity can manifest in communication styles, as some people may be extroverted and others more introverted, while some are multilingual and others only speak one native language. The individuals who comprise your workforce may also have completely different experiences by virtue of their education and the cities or towns they have lived in. Experiences drive values, so some people may be financially driven, while others may value their time or relationships. With a workforce composed of different communication styles, experiences, and values, comes diverse ideas-- which is a great thing. However, diversity can also pave the way for bias when you treat diverse groups like they are homogenous. 


Debiased does not mean unbiased. While unbiased suggests that there are no differences within a diverse group, debiasing does not imply that such differences don’t exist. Debiased measures incorporate diversity by design. pymetrics uses data to maintain diversity by measuring qualities that matter, collecting representative data,  auditing, and remediating. 

Debiasing Data Examples.

Let’s say you're interested in measuring ability. An approach that will result in biased data is a generic IQ test, which was designed for only one specific definition of intelligence. An IQ test selects two white men for every one woman or minority. In contrast, a debiased approach is to measure job-relevant ability and seek cognitive diversity in skills that aren’t exclusively job-relevant. For other metrics such as success or communication skills, pymetrics offers assessments that provide a debiased approach for workforce decision making.  

Three Steps for Debiasing. 

1. Representation  

- Diverse data collection 

- Ensure data defines success the same for everyone 

- In historically biased sectors, look at comparable sectors 

2. Fair Measurement 

- Measure things that lead to success 

- Race and gender do not matter to the job, so measurements should not be biased 

- If an important measure suffers from systemic bias, reduce its weight in decisions

3. Auditing 

- Monitor your diversity outcomes 

- Make sure your data behaves correctly

- Do not ignore bias 

Debiasing at pymetrics. 

In order to maintain a debiased approach to data collection and usage, pymetrics promotes ethics at every stage. 

Never compromise.

Do you have to compromise between unbiased and debiased data for a diverse workforce? The answer is no, a plurality of ideas will only make your workforce stronger. Shift your focus from unbiasing your data to meet a quota, to debiasing data to make your workforce stronger. 

To hear from Dr. Lewis Baker directly, feel free to check out the session in full here.