A groundbreaking study has uncovered the significant impact of system prompt framing on the generative capabilities of large language models (LLMs). The research, comprising 3,830 runs across 5 model architectures, revealed that the relational framing of a prompt profoundly influences the token-level Shannon entropy of the model’s output, independent of its instructions or topic.

The investigation centered on two crucial variables: relational presence and epistemic openness. The findings showed that the interplay between these variables yields a superadditive effect, wherein collaborative and epistemically open framing produces more pronounced effects than either factor alone. Moreover, the study confirmed that this effect is mediated through attention mechanisms, as evidenced by an ablation study.

The implications of this research are far-reaching for users of large transformers, such as ChatGPT, Claude, and Mistral. The framing of a system prompt can substantially alter the model’s generation dynamics, affecting not only the output topic but also the underlying distributional parameters. The full paper, code, and data are available online for further exploration, offering a valuable resource for those seeking to optimize their AI interactions.

Photo by Sound On on Pexels
Photos provided by Pexels