The butterfly effect in LLM. Persona format alone (prose vs bullets) flipped an LLM’s behavior by 76 points.

Reddit r/ArtificialInteligence 05/22/26, 05:33 PM Papers

llm persona formatting prisoners-dilemma sensitivity behavioral-economics ai-simulation

Summary

A study demonstrates that simply changing the formatting (prose vs bullet points) of a persona prompt dramatically flips an LLM's behavior in a Prisoner's Dilemma, from 96% cooperation to 20%, illustrating extreme sensitivity to format despite identical content (p < 0.001).

Two persona prompts, identical content, same model (gpt-5.2). Only difference is formatting: one prose, one bullet points. In a 10-round Prisoner’s Dilemma the prose version cooperated \~96% of the time, the bullet version \~20%. A 76pp gap, p < 0.001. Same meaning, opposite behavior. Authors call it the butterfly effect in LLM simulations. https://arxiv.org/pdf/2605.18890

Original Article

Similar Articles

Evaluation Drift in LLM Personality Induction: Are We Moving the Goalpost?

arXiv cs.CL

This paper investigates whether fine-tuning LLMs on long-form essays with associated Big Five personality profiles stabilizes questionnaire responses and can induce target profiles, finding that while variance reduces, accuracy on the full five-dimensional profile remains near chance.

When Does Personality Composition Matter for Multi-Agent LLM Teams?

arXiv cs.AI

This paper systematically examines how manipulating personality traits in multi-agent LLM teams affects performance across coding, research collaboration, and bargaining tasks, finding that effects depend critically on task structure.

Prompt Injection as Role Confusion

Simon Willison's Blog

Research paper shows that LLMs suffer from 'role confusion', where they prioritize the style of text over its actual role tags, enabling prompt injection attacks. Destyling text reduces attack success from 61% to 10%, indicating a fundamental challenge for LLM security.

The Story Shapes the Agent: Narrative Priors in LLM Behavior

arXiv cs.CL

This paper investigates how the narrative framing of a task (e.g., disease investigation vs. murder mystery) acts as a stronger driver of LLM agent behavior than assigned personas, introducing the concept of 'narrative priors' that explain 5–31x more behavioral variance and are negatively associated with task success in two of three domains.

Ψ-Bench: Evaluating Persona-Sensitive Influencing in Persuasive Dialogues

Hugging Face Daily Papers

Introduces Ψ-Bench, a benchmark for evaluating LLMs' ability to influence users through persuasive dialogues with personalized profiles. Tests 10 frontier LLMs and finds significant room for improvement, with profile access boosting performance by 18.24%.