MIT Technology Review is reporting that artificial intelligence models can develop their own biases and stereotype job applicants more severely than humans. New research from Princeton University and the University of Chicago found that large language models, or LLMs, pick up human biases from their training data and can also form novel biases from experience.
The researchers ran LLMs, including ChatGPT, Claude and Gemini, through a simulated hiring game adapted from a psychology study. The models were tasked with hiring for 20 jobs in a fictional city, selecting candidates from four fictional ethnic groups. All candidates were equally likely to succeed, but the models quickly began segregating groups into different job niches based on early, limited observations. For example, if an Aima candidate failed as a doctor, the model would then tend to hire Aimas as janitors.
The study found that LLMs stereotyped people by demographic group significantly more than human participants in the original study. On a segregation scale where 2 is maximum, humans scored 0.84, while OpenAI’s o3 model scored 1.83. Ryan Liu, a Princeton PhD student and coauthor, said LLMs are optimized to create generalizations from limited data, which makes them quick to stereotype in social settings. Newer models with higher reasoning capabilities showed even stronger biases.
The findings are particularly relevant as AI companies develop agentic models with improved memory and personalization features. While telling models to be fair did not significantly alter their behavior, offering an additional bonus for diverse hiring made them far less biased. Providing more personal information about individuals, such as age and education, also reduced segregation by ethnicity in a separate experiment.
MIT Technology Review said the extent to which AI systems will stereotype job applicants in the real world remains an open question, but the research highlights a serious implication for companies deploying LLMs to screen résumés and conduct interviews.
Full Article: AI is more likely than humans to form biases when hiring