A recent experiment examined bias in six major large language models (LLMs) using steelman prompting, a method that involves presenting a counterargument to a default perspective. The study focused on a contentious interpretive question from 1 Corinthians 6-7, a passage often cited in discussions of Christian sexual ethics.

Six AI platforms – Claude, ChatGPT, Grok, Llama, DeepSeek, and an uncensored DeepSeek clone (Venice.ai) – were tested using identical prompts to analyze the original Greek and historical context of the passage. The results showed that every platform’s default analysis favored one interpretive framework, while their steelman analyses produced more nuanced and historically grounded reasoning.

The study found that 63% of recommended scholarly sources across all platforms came from a single theological tradition, while none came from the peer-reviewed subdiscipline that supports an alternative reading. Interestingly, DeepSeek and its uncensored clone, which share the same base model, diverged significantly on the steelman prompt, suggesting that output-layer filtering can shape interpretive conclusions in unexpected ways.

The research highlights the importance of auditing bias in AI systems and demonstrates the effectiveness of steelman prompting as a systematic technique for doing so. The study’s findings have implications for the development of more balanced and nuanced AI models, and the technique has the potential to be applied to a wide range of domains beyond this specific study.

Photo by Céline | on Pexels
Photos provided by Pexels