A novel AI benchmark, dubbed Xbench, has emerged to rigorously evaluate AI models’ reasoning capabilities and proficiency in executing real-world tasks. Created by HongShan Capital Group, Xbench distinguishes itself by addressing a crucial question: Does an AI model genuinely reason, or does it merely reproduce information from its training data?
Xbench goes beyond conventional benchmarks by assessing models on both their performance in standardized tests and their aptitude in practical scenarios. To ensure continued relevance in the rapidly evolving landscape of AI, Xbench is designed to be regularly updated.
HongShan Capital has open-sourced a portion of the Xbench question set, offering free access to researchers and developers. A public leaderboard tracks the performance of leading AI models on Xbench, with ChatGPT o3 currently topping the charts across all categories. The benchmark incorporates two distinct assessment systems: academic tests (Xbench-ScienceQA and Xbench-DeepResearch) and a real-world readiness evaluation. ScienceQA gauges a model’s knowledge in STEM fields, while DeepResearch concentrates on its ability to navigate and process information from the Chinese-language web. The real-world readiness assessment presents models with practical workflows in recruitment and marketing.
Future iterations of Xbench are slated to incorporate additional dimensions, including creativity, collaboration, and reliability. While acknowledging the inherent challenges in quantifying subjective attributes, Zihan Zheng, lead researcher on LiveCodeBench Pro, praised Xbench as “a promising start” in the pursuit of more comprehensive AI evaluation.