Photo by Nikita Igonkin on Pexels

DeepSeek has offered a compelling new angle on understanding Large Language Models (LLMs) by framing them as complex dynamical systems. This perspective suggests that LLMs, with their vast parameter spaces (DeepSeek itself has 8 billion parameters, compared to ChatGPT 3’s 175 billion), exhibit emergent order arising from their dynamic interactions. This insight was sparked by a discussion of the research paper, “Transformer Dynamics: A neuroscientific approach to interpretability of large language models” by Jesseba Fernando and Grigori Guitchounts. The paper delves into phase space reconstruction and uncovers attractor-like dynamics within the residual stream of a 64-layer model. The initial conversation surrounding this intriguing concept took place on Reddit, with further details available at https://old.reddit.com/r/artificial/comments/1nmzted/conversing_with_an_llm_as_perturbing_a_dynamical/.