Photo by Nathan Cowley on Pexels

Recent claims of exponential growth in AI task completion capabilities may be overstated, according to a new analysis. A deeper dive into the dataset behind a study asserting a doubling in AI task length reveals a less optimistic trend. Researchers found that while the average length of *successful* task completions has seen a modest increase, the length of *failed* tasks has increased significantly. This suggests that AI models are attempting, and often failing, to tackle more complex problems, rather than consistently completing longer and more demanding tasks. The previously reported exponential growth, attributed to logistic regression’s sensitivity to increasing overall task volume, may not accurately reflect true improvements in maximal task performance. Preliminary forecasts utilizing ARIMA modeling are being augmented by ongoing research focused on developing a more robust explanatory model incorporating factors such as computational costs and specific task characteristics. The independent analysis, initiated by Reddit user /u/Murky-Motor9856, has ignited a debate within the AI research community regarding the best metrics for evaluating progress.