Photo by Katie Mukhina on Pexels

A new investigation reveals that OpenAI’s AI models, including ChatGPT and Sora, exhibit significant caste bias, potentially reinforcing discriminatory views in India. The research, conducted by MIT Technology Review in collaboration with Harvard student Jay Chooi, tested the models for bias using fill-in-the-blank questions and text-to-video prompts. GPT-5, for instance, overwhelmingly associated positive traits with the Brahmin caste and negative traits with the Dalit caste. Sora even generated harmful and stereotypical imagery of oppressed castes, highlighting the severity of the issue.

The bias stems from the models’ training data, which reflects existing societal stereotypes prevalent in India. Researchers found that the AI models reproduce socioeconomic and occupational stereotypes, portraying Dalits in a negative light. As OpenAI expands its ChatGPT Go subscription plan in India, experts warn that such biases could amplify existing inequalities if left unaddressed.

The problem isn’t limited to OpenAI. Studies suggest that open-source LLMs may exhibit even greater caste bias. The AI industry currently lacks standardized testing for caste bias, with existing benchmarks failing to address this critical issue. Researchers are advocating for the development of new benchmarks and the implementation of thorough testing to ensure AI models are free from caste discrimination before deployment. The incident of ChatGPT automatically changing a user’s surname to one associated with a privileged caste further underscores the urgent need for caste consciousness in AI development.