CreativityNeuro: Steering Language Model Weights to Improve Divergent Thinking and Reduce Mode Collapse

# Steering Language Model Weights to Improve Divergent Thinking and Reduce Mode Collapse
Source: [https://arxiv.org/html/2607.01433](https://arxiv.org/html/2607.01433)
Samuel Schapiro Univeristy of Illinois, Urbana\-ChampaignCore Francisco Park Center for Brain Science, Harvard University CBS\-NTT Program in Physics of Intelligence, Harvard University Prior Computers &Felix Sosa Prior Computers &Lav R\. Varshney AI Innovation Institute, Stony Brook University

###### Abstract

Divergent thinking is a crucial aspect of creativity, yet large language models \(LLMs\) tend to consistently generate similar responses to open\-ended questions, in what has been termed the artificial hivemind effect\. Here, we introduce CreativityNeuro, a data\-free method for enhancing divergent thinking in LLMs via contrastive weight steering\. We evaluate our method across multiple creativity assessments and report several main findings\. On the Divergent Association Task \(DAT\), a vocabulary\-space creativity test, CreativityNeuro improves performance by up to 14 human percentile points\. Next, in a large\-scale human evaluation \(N=720\) on the Alternative Uses Test \(AUT\) and the Task Task, CreativityNeuro achieves significant improvements in originality, surprise, and creativity, transferring to longer\-form and more open\-ended tasks\. Importantly, we find that across all three tasks, CreativityNeuro demonstrably reduces measures of mode collapse\. Moreover, activation steering achieves comparable performance to CreativityNeuro on the DAT, but it does not transfer to the AUT and Task Task, demonstrating the effectiveness of weight\-space steering in generalizing to unseen tasks\. In conclusion, CreativityNeuro improves divergent thinking and reduces mode collapse without requiring behavioral data, re\-training, or gradient\-based fine\-tuning, providing a straightforward way to enhance LLM performance in creative domains\.

![Refer to caption](https://arxiv.org/html/2607.01433v1/figures/cover_figure_new.png)Figure 1:CreativityNeuro \(CN\) pipeline\.Given a pair of contrastive creative prompts, CN computes parameter importance scores, selects a sparse subset of creativity\-relevant parameters, and applies a scaled weight perturbation—without requiring behavioral datasets or gradient\-based finetuning\. CN improves divergent thinking across various tasks\. Subplot \(b\) visualizes CN*thinking outside of the “box”*\(i\.e\., the convex hull of baseline DAT responses\), despite baseline responses falling within CN’s convex hull in subplot \(a\)\.## 1Introduction

Recent advances in large language models \(LLMs\) have renewed interest in a longstanding question:*how can we understand and enhance creativity in intelligent systems?*\(Boden,[2004](https://arxiv.org/html/2607.01433#bib.bib7)\)\. While this question has deep roots in cognitive science\(Quetelet,[1842](https://arxiv.org/html/2607.01433#bib.bib43); Galton,[1870](https://arxiv.org/html/2607.01433#bib.bib44); Hadamard,[1954](https://arxiv.org/html/2607.01433#bib.bib4); Guilford,[1956](https://arxiv.org/html/2607.01433#bib.bib27); Mednick,[1962](https://arxiv.org/html/2607.01433#bib.bib2); Koestler,[1964](https://arxiv.org/html/2607.01433#bib.bib16); Simonton,[2004](https://arxiv.org/html/2607.01433#bib.bib17); Dietrich, Arne,[2004](https://arxiv.org/html/2607.01433#bib.bib3); Fauconnier and Turner,[2008](https://arxiv.org/html/2607.01433#bib.bib38); Rothenberg,[2014](https://arxiv.org/html/2607.01433#bib.bib45)\), it is now increasingly studied in the context of large\-scale generative models\(Maher,[2010](https://arxiv.org/html/2607.01433#bib.bib39); Varshney,[2019](https://arxiv.org/html/2607.01433#bib.bib10); Schapiroet al\.,[2025](https://arxiv.org/html/2607.01433#bib.bib40)\)\. Recent work has begun to assess the capacity for LLMs to engage in creative and open\-ended tasks\(Siet al\.,[2024](https://arxiv.org/html/2607.01433#bib.bib46);[2025](https://arxiv.org/html/2607.01433#bib.bib47); Sanyalet al\.,[2025](https://arxiv.org/html/2607.01433#bib.bib58); Bellemare\-Pepinet al\.,[2024](https://arxiv.org/html/2607.01433#bib.bib23); Wanget al\.,[2025](https://arxiv.org/html/2607.01433#bib.bib1);[2024](https://arxiv.org/html/2607.01433#bib.bib48)\), where a recurring issue has surfaced: models tend to consistently generate similar responses to open\-ended questions, in what has been termed the*artificial hivemind*effect\(Jianget al\.,[2025](https://arxiv.org/html/2607.01433#bib.bib37)\)\.

Within the creativity literature, a common distinction is made between*divergent thinking*, the capacity to generate multiple diverse solutions to a problem, and*convergent thinking*, the ability to find a single correct solution that unifies multiple diverse stimuli\(Dietrich,[2019](https://arxiv.org/html/2607.01433#bib.bib12); Guilford,[1956](https://arxiv.org/html/2607.01433#bib.bib27)\)\. Studying ways to enhance divergent thinking offers a promising pathway to encourage greater diversity and novelty in model responses, combating the homogenization issues that have emerged thus far\. Here, we introduce a weight\-space steering method that improves divergent thinking in LLMs\. Our method outperforms prior approaches—including decoding, prompting, and activation steering—and generalizes better to unseen tasks, without requiring behavioral data or gradient\-based fine\-tuning\. In detail, our main contributions are as follows:

1. 1\.In[Section3](https://arxiv.org/html/2607.01433#S3), we introduce CreativityNeuro, a data\-free method for steering creative behavior\.
2. 2\.In[Section4](https://arxiv.org/html/2607.01433#S4), we find that CreativityNeuro significantly improves divergent thinking on the Divergent Association Task \(DAT\), outperforming baselines such as prompting, activation steering, and decoding baselines\.
3. 3\.In[Section5](https://arxiv.org/html/2607.01433#S5), we conduct a large\-scale human evaluation on the Alternative Uses Test \(AUT\) and Task Task \(TT\) and find that CreativityNeuro improves originality, surprise, and creativity on the AUT and TT, while activation steering transfers poorly to the AUT and TT\.
4. 4\.In[Section6](https://arxiv.org/html/2607.01433#S6), we find that CreativityNeuro reduces mode collapse across all three tasks\.
5. 5\.In[Section7](https://arxiv.org/html/2607.01433#S7), we find evidence that divergent thinking and factual reasoning are non\-separable in weight space\.

## 2Related Work

We start by briefly reviewing related work before introducing our method in[Section3](https://arxiv.org/html/2607.01433#S3)\.

Evaluating the creativity of LLMs\.Previous work has evaluated LLMs on divergent creativity assessments–including the DAT\(Olsonet al\.,[2021](https://arxiv.org/html/2607.01433#bib.bib9)\), AUT\(Guilford,[1956](https://arxiv.org/html/2607.01433#bib.bib27)\), the Task Task\(Chuet al\.,[2024](https://arxiv.org/html/2607.01433#bib.bib36)\)–and in various real\-world settings like scientific ideation\(Siet al\.,[2024](https://arxiv.org/html/2607.01433#bib.bib46);[2025](https://arxiv.org/html/2607.01433#bib.bib47)\)and open\-ended user queries\(Jianget al\.,[2025](https://arxiv.org/html/2607.01433#bib.bib37)\)\. On the DAT, LLMs can achieve scores well into the 90th percentile of humans\(Bellemare\-Pepinet al\.,[2024](https://arxiv.org/html/2607.01433#bib.bib23); Wanget al\.,[2025](https://arxiv.org/html/2607.01433#bib.bib1)\), whereasStevensonet al\.\([2022](https://arxiv.org/html/2607.01433#bib.bib19)\)studied GPT\-3 on the AUT and concluded that humans exhibited greater creativity, with model responses showing weaker originality\. Lastly,Chuet al\.\([2024](https://arxiv.org/html/2607.01433#bib.bib36)\)found that model\-generated goals on the Task Task achieved similar creativity ratings as human\-generated goals, as assessed by a large panel of human raters\.

Improving the creativity of LLMs\.Most similar to this work,Olsonet al\.\([2024](https://arxiv.org/html/2607.01433#bib.bib13)\)proposed an activation steering method to amplify the creativity of LLMs, although improvements were only established for a single model, task, and human annotator\. Our study is the first to demonstrate a steering method to improve creative behavior and validate its effectiveness in a large\-scale human study\. Apart from steering, various other approaches to improving LLM creativity include prompting frameworks\(Nguyen and Singla,[2025](https://arxiv.org/html/2607.01433#bib.bib65); Morain and Ventura,[2025](https://arxiv.org/html/2607.01433#bib.bib32); Wanget al\.,[2025](https://arxiv.org/html/2607.01433#bib.bib1)\), varying decoding parameters such as temperature\(Peeperkornet al\.,[2024](https://arxiv.org/html/2607.01433#bib.bib66)\), and reinforcement learning \(RL\) on preference data\(Weiet al\.,[2025](https://arxiv.org/html/2607.01433#bib.bib73)\)\. Unlike steering and RL\-based approaches that typically require labeled behavioral data, our method operates entirely data\-free\.

Algorithm 1CreativityNeuro0:Model weights

\{Wℓ\}ℓ=1L\\\{W\_\{\\ell\}\\\}\_\{\\ell=1\}^\{L\}; creative prompts

𝒫cre\\mathcal\{P\}^\{\\text\{cre\}\}; non\-creative prompts

𝒫non\-cre\\mathcal\{P\}^\{\\text\{non\-cre\}\}; importance threshold

ρ\\rho; scaling factor

α\\alpha
0:Modified weights

\{Wℓ′\}ℓ=1L\\\{W^\{\\prime\}\_\{\\ell\}\\\}\_\{\\ell=1\}^\{L\}with amplified creativity weights

1:

2:foreach layer

ℓ\\elldo

3:Run forward passes on

𝒫cre,𝒫on\-cre\\mathcal\{P\}^\{\\text\{cre\}\},\\mathcal\{P\}^\{\\text\{on\-cre\}\}to obtain importance scores

Sℓ,i​jcre,Sℓ,i​jnon\-creS^\{\\text\{cre\}\}\_\{\\ell,ij\},S^\{\\text\{non\-cre\}\}\_\{\\ell,ij\}via:

Sℓ,i​j​\(𝒫\)=∑b=1\|𝒫\|∑t=1Tb\|Wℓ,i​j\|⋅‖𝐱ℓ,j\(b,t\)‖2Step 1: Compute weight importance scores\\displaystyle S\_\{\\ell,ij\}\(\\mathcal\{P\}\)=\\sum\_\{b=1\}^\{\|\\mathcal\{P\}\|\}\\sum\_\{t=1\}^\{T\_\{b\}\}\|W\_\{\\ell,ij\}\|\\cdot\\\|\\mathbf\{x\}^\{\(b,t\)\}\_\{\\ell,j\}\\\|\_\{2\}\\quad\\quad\\quad\\quad\\quad\\quad\\textbf\{Step 1: Compute weight importance scores\}for prompt

bband token position

ttwithin prompt

bb
4:endfor

5:

6:foreach layer

ℓ\\elldo

7:

Cℓ←C\_\{\\ell\}\\leftarrowtop\-

ρ\\rhoweights ranked by

Sℓ,i​jcreS^\{\\text\{cre\}\}\_\{\\ell,ij\}Step 2: Extract creative\-specific subspaces

8:

Nℓ←N\_\{\\ell\}\\leftarrowtop\-

ρ\\rhoweights ranked by

Sℓ,i​jnon\-creS^\{\\text\{non\-cre\}\}\_\{\\ell,ij\}
9:

Mℓ,i​jcre\-spec←𝕀​\[\(i,j\)∈Cℓ∖Nℓ\]M^\{\\text\{cre\-spec\}\}\_\{\\ell,ij\}\\leftarrow\\mathbb\{I\}\[\(i,j\)\\in C\_\{\\ell\}\\setminus N\_\{\\ell\}\]
10:endfor

11:

12:foreach layer

ℓ\\elldo

13:

Wℓ′←Wℓ⊙\(1\+α⋅Mℓcre\-spec\)W^\{\\prime\}\_\{\\ell\}\\leftarrow W\_\{\\ell\}\\odot\(1\+\\alpha\\cdot M^\{\\text\{cre\-spec\}\}\_\{\\ell\}\)Step 3: Creative parameter scaling

14:endfor

15:return

\{Wℓ′\}ℓ=1L\\\{W^\{\\prime\}\_\{\\ell\}\\\}\_\{\\ell=1\}^\{L\}

Table 1:Examples of contrastive prompt sets\.Each set contains creative \(𝒫cre\\mathcal\{P\}^\{\\text\{cre\}\}\) and non\-creative \(𝒫non\-cre\\mathcal\{P\}^\{\\text\{non\-cre\}\}\) prompts used for parameter importance scoring in[Algorithm1](https://arxiv.org/html/2607.01433#alg1)\.
## 3Method

Recently,Christet al\.\([2025](https://arxiv.org/html/2607.01433#bib.bib22)\)has shown that parameter importance methods can be used to identify and amplify weights involved in mathematical reasoning, improving scores on the MATH benchmark by 4–17%\(Hendryckset al\.,[2021](https://arxiv.org/html/2607.01433#bib.bib25)\)\. Unlike mathematical reasoning, which can be elicited and evaluated on structured benchmarks such as MATH and GSM8K, creativity is a property of*responses*, not of questions\. Open\-ended prompts, such as those inJianget al\.\([2025](https://arxiv.org/html/2607.01433#bib.bib37)\), admit a wide range of potentially creative completions, but novelty and usefulness are measured on the outputs\(Varshney,[2019](https://arxiv.org/html/2607.01433#bib.bib10); Maher,[2010](https://arxiv.org/html/2607.01433#bib.bib39); Boden,[2004](https://arxiv.org/html/2607.01433#bib.bib7)\)rather than the inputs themselves\. Therefore, our key methodological innovation is a framework for extending MathNeuro to a cognitive domain where the target behavior can be prompted but no structured dataset exists, making CreativityNeuro entirely data\-free\.

Contrastive Prompt SetsMathNeuro relies on questions drawn from MATH and GSM8K to obtain inputs for parameter importance scoring\. Because no analogous dataset exists for creativity, we instead construct*contrastive prompt sets*: short instructions that direct the model toward creative \(𝒫cre\\mathcal\{P\}^\{\\text\{cre\}\}\) versus non\-creative \(𝒫non\-cre\\mathcal\{P\}^\{\\text\{non\-cre\}\}\) behavior\. We use six such sets spanning a range of styles—dat,storytelling,ideation,problem solving,open\-ended, andminimal—whereminimalcontains only two\- to five\-word instructions \(e\.g\.,*Surprise me*vs*Be precise*\)\. Representative examples are given in[Table1](https://arxiv.org/html/2607.01433#S2.T1), and all six prompt sets are given in[Table2](https://arxiv.org/html/2607.01433#A1.T2)\. As a result, CreativityNeuro does not require datasets, behavioral generations, scored responses, or labeled examples, making it entirely data\-free, unlikeChristet al\.\([2025](https://arxiv.org/html/2607.01433#bib.bib22)\)\.

Parameter Importance ScoringWe use the same Wanda\-style\(Sunet al\.,[2023](https://arxiv.org/html/2607.01433#bib.bib62)\)parameter importance scoring asChristet al\.\([2025](https://arxiv.org/html/2607.01433#bib.bib22)\), restated here for completeness\. This is done by taking the product of weight magnitude and activation norm, summed across a set of promptsbband their token positionstt:Sℓ,i​j=∑b,t\|Wℓ,i​j\|⋅‖𝐱ℓ,j\(b,t\)‖2S\_\{\\ell,ij\}=\\sum\_\{b,t\}\|W\_\{\\ell,ij\}\|\\cdot\\\|\\mathbf\{x\}^\{\(b,t\)\}\_\{\\ell,j\}\\\|\_\{2\}, where𝐱ℓ,j\(b,t\)\\mathbf\{x\}^\{\(b,t\)\}\_\{\\ell,j\}is thejj\-th input activation at layerℓ\\ellfor tokenttin promptbb\. We compute importance scores on creative𝒫cre\\mathcal\{P\}^\{\\text\{cre\}\}and non\-creative prompts𝒫non\-cre\\mathcal\{P\}^\{\\text\{non\-cre\}\}\. Then, we isolate creativity\-specific parameters by selecting the topρ\\rhopercent of weights ranked by creative importance that do not also appear in the topρ\\rhopercent for non\-creative prompts\. This set difference operation \(Cℓ∖NℓC\_\{\\ell\}\\setminus N\_\{\\ell\}\) ensures we identify parameters uniquely associated with creative behavior\. At inference time, we multiply weights inCℓ∖NℓC\_\{\\ell\}\\setminus N\_\{\\ell\}by a scaling factor\(1\+α\)\(1\+\\alpha\)\. The hyperparametersρ\\rhoandα\\alphacontrol the importance threshold and scaling strength, respectively\. The full procedure is given in[Algorithm1](https://arxiv.org/html/2607.01433#alg1)\.

## 4Experiments on the Divergent Association Task

![Refer to caption](https://arxiv.org/html/2607.01433v1/figures/best_config_combined_6models.png)Figure 2:CreativityNeuro \(CN\) improves divergent thinking across models and prompt sets\.Given a human reference distribution\(Wanget al\.,[2025](https://arxiv.org/html/2607.01433#bib.bib1)\)\(N=9,297N=9\{,\}297,μ=78\.26\\mu=78\.26,σ=6\.73\\sigma=6\.73\), we report: \(a\) DAT human percentile \(±\\pmSEM\) averaged acrossT∈\{0\.9,1\.0,1\.2\}T\\in\\\{0\.9,1\.0,1\.2\\\}for CN, CAA, and the strongest sampling\-based baselines; dashed lines show cross\-model means for CN and CAA\. \(b\) Heatmap showing human percentile improvement \(Δ\\Delta%ile\) for CreativityNeuro models across prompt sets, with statistical significance \(p<0\.05p<0\.05\) at each of the temperatures tested \(0\.9, 1\.0, 1\.2\) denoted by an asterisk\. \(c\) CDF showing DAT scores for CreativityNeuro models on the best performing prompt set\.We test instruct\-tuned models across three open\-weight model families \(Phi, Llama, Qwen\), totaling six models at 3B, 4B, 7B, 8B, and 14B sizes: LLaMA \(3\.2\-3B\-Instruct, 3\.1\-8B\-Instruct\)\(Grattafioriet al\.,[2024](https://arxiv.org/html/2607.01433#bib.bib33)\), Qwen\-2\.5 \(7B\-Instruct, 14B\-Instruct\)\(Yanget al\.,[2025](https://arxiv.org/html/2607.01433#bib.bib35)\), and Phi \(3\.5\-mini\-Instruct \(4B\), 3\-medium\-4k\-Instruct \(14B\)\) from Microsoft\(Abdinet al\.,[2024](https://arxiv.org/html/2607.01433#bib.bib34)\)\. For each model we use the best\(ρ,α,prompt set\)\(\\rho,\\alpha,\\text\{prompt set\}\)configuration with≥120\\geq 120valid CN samples at all three temperaturesT∈\{0\.9,1\.0,1\.2\}T\\in\\\{0\.9,1\.0,1\.2\\\}\. Full hyperparameter sweep details are in[AppendixD](https://arxiv.org/html/2607.01433#A4)\.

TaskThe DAT asks participants to generateNNwords that are as semantically distant from each other as possible\(Olsonet al\.,[2021](https://arxiv.org/html/2607.01433#bib.bib9)\)\. Given a set ofNNwordsW:=\{w1,w2,…,wN\}W:=\\\{w\_\{1\},w\_\{2\},\\dots,w\_\{N\}\\\}with corresponding GloVe embeddingsV:=\{𝐯1,𝐯2,…,𝐯N\}⊆ℝ300V:=\\\{\\mathbf\{v\}\_\{1\},\\mathbf\{v\}\_\{2\},\\dots,\\mathbf\{v\}\_\{N\}\\\}\\subseteq\\mathbb\{R\}^\{300\}, the DAT score is the average pairwise semantic distance among all distinct pairs of thoseNNwords:

DAT​\(W\):=100N​\(N−1\)​∑i≠jN\(1−cos⁡\(𝐯i,𝐯j\)\)\\textrm\{DAT\}\(W\):=\\frac\{100\}\{N\(N\-1\)\}\\sum\_\{i\\neq j\}^\{N\}\(1\-\\cos\(\\mathbf\{v\}\_\{i\},\\mathbf\{v\}\_\{j\}\)\)\(1\)FollowingOlsonet al\.\([2021](https://arxiv.org/html/2607.01433#bib.bib9)\), we use the 840B\-token GloVe embeddings\(Penningtonet al\.,[2014](https://arxiv.org/html/2607.01433#bib.bib31)\)as our semantic space\. A participant is asked to nameN=10N=10words, and the first77valid words are kept\(Olsonet al\.,[2021](https://arxiv.org/html/2607.01433#bib.bib9)\)\. Full prompts are given in[AppendixE](https://arxiv.org/html/2607.01433#A5)\.

BaselinesExisting studies have found that temperature\-scaling and prompting can influence DAT scores\(Wanget al\.,[2025](https://arxiv.org/html/2607.01433#bib.bib1); Bellemare\-Pepinet al\.,[2024](https://arxiv.org/html/2607.01433#bib.bib23)\)\. To ensure the CN intervention leads to a meaningful improvement over such techniques, we compare against a broad set of baselines, including prompting; varying decoding parameters such as top\-p nucleus sampling, top\-k sampling, temperature, and repetition penalty; as well as activation steering via contrastive activation addition \(CAA;Panicksseryet al\.\([2024](https://arxiv.org/html/2607.01433#bib.bib49)\)\), which injects a steering vector𝐯ℓ=𝐡¯ℓ\+−𝐡¯ℓ−\\mathbf\{v\}\_\{\\ell\}=\\bar\{\\mathbf\{h\}\}\_\{\\ell\}^\{\+\}\-\\bar\{\\mathbf\{h\}\}\_\{\\ell\}^\{\-\}into the residual stream during decoding\. For activation steering, contrast pairs are obtained from top\- vs\. bottom\-quartile DAT responses by score, creating a “divergent thinking” direction in the residual stream\.111We also tested a prompt\-only CAA variant using forward\-pass activations from the same creative vs\. non\-creative prompt sets as CN, but it underperformed the behavioral\-data variant on all models and is omitted for clarity\.Full decoding strategy and activation steering hyperparameter settings are given in[AppendixB](https://arxiv.org/html/2607.01433#A2)\. All settings are evaluated across all six models atT∈\{0\.9,1\.0,1\.2\}T\\in\\\{0\.9,1\.0,1\.2\\\}untilN=120N\{=\}120valid DAT responses are obtained\.

### 4\.1Results

CreativityNeuro achieves robust performance improvements across prompt setsCN improves DAT performance across all six models and prompt sets \([Figure2](https://arxiv.org/html/2607.01433#S4.F2)\), outperforming all sampling\-based baselines\. Panel \(b\) of[Figure2](https://arxiv.org/html/2607.01433#S4.F2)showsΔ\\DeltaPercentile for the best\(ρ,α\)\(\\rho,\\alpha\)per model–prompt combination: while thedatprompt set produces the most consistent gains, non\-datprompt sets also yield statistically significant improvements, suggesting CN is able to identify weights controlling divergent thinking behavior, rather than localizing DAT\-specific task knowledge\.

CreativityNeuro outperforms activation steering without needing behavioral dataCN \(94\.1 avg\) slightly outperforms activation steering \(93\.9 avg\) on DAT percentile \([Figure2](https://arxiv.org/html/2607.01433#S4.F2)a\); however, activation steering requires scored DAT responses to construct its steering vector, while CN uses only creative and non\-creative prompts with no generation or scoring\. Prompt\-only activation steering \(omitted from the figure; see footnote in[Figure2](https://arxiv.org/html/2607.01433#S4.F2)\) averaged only 87\.8, comparable to prompting \(87\.9\), suggesting that the behavioral data is essential for activation steering to be competitive, whereas CN achieves stronger performance from prompts alone\.

CreativityNeuro improves DAT scores across all six models and prompt sets, outperforming prompting, sampling\-parameter, and activation steering baselines\.

## 5Experiments on the Alternative Uses Test and Task Task

Here, we evaluate CreativityNeuro on more complex divergent thinking tasks than the DAT\. The Alternative Uses Test \(AUT\), a standard instrument in the psychometrics literature, asks participants to generate creative uses for a common object\(Guilford,[1956](https://arxiv.org/html/2607.01433#bib.bib27)\)\. We administer the AUT using a standard set of objects in the creativity literature:*brick*,*paperclip*, and*fork*\. FollowingStevensonet al\.\([2022](https://arxiv.org/html/2607.01433#bib.bib19)\), uses are scored on*originality*,*surprise*, and*utility*\. We also evaluate CreativityNeuro on the Task Task \(TT\), which assesses the ability to generate novel challenges or goals that themselves expect novel solutions\(Chuet al\.,[2024](https://arxiv.org/html/2607.01433#bib.bib36)\)\. Participants design creative game show challenges that would be fun to attempt, entertaining to watch, and difficult enough to be interesting\. FollowingChuet al\.\([2024](https://arxiv.org/html/2607.01433#bib.bib36)\), we evaluate responses on creativity and originality\.222Chuet al\.\([2024](https://arxiv.org/html/2607.01433#bib.bib36)\)studies additional dimensions, such as difficulty, how fun the task is to do, and how fun it is to watch, but here we restrict focus to creativity and originality, as these are most relevant to the goal of evaluating divergent thinking\.Full prompts are in[AppendixE](https://arxiv.org/html/2607.01433#A5)\.

![Refer to caption](https://arxiv.org/html/2607.01433v1/x1.png)Figure 3:Cohen’sdd\(±\\pmSE\) from intra\-participantzz\-scored human ratings on the AUT and TT\.\(a–c\)AUT Originality, Surprise, Utility\.\(d–e\)TT Creativity, Originality\. Black outlines indicatep<\.05p<\.05, and green shading marks the positive\-effect region\.Models and InferenceFor each of the six models in[Section4](https://arxiv.org/html/2607.01433#S4), we select the CreativityNeuro configuration \(ρ\\rho,α\\alpha, prompt set\) that produced the largest statistically significant DAT improvement\. Then, for each task, we sample 40 stimuli \(20 baseline, 20 creative\) at temperatureT=1\.0T=1\.0,top\-​k=0\\text\{top\-\}k=0, andtop\-​p=1\.0\\text\{top\-\}p=1\.0\. We compare CreativityNeuro against activation steering, the strongest non\-CN baseline from[Section4](https://arxiv.org/html/2607.01433#S4)\.

Human Experiment DesignWe evaluate AUT and TT stimuli via human ratings on Prolific\. Human studies use a between\-subjects design, where every participant rates 10 stimuli \(5 baseline, 5 creative\) in randomized order with condition labels hidden\. We ensure balanced allocation via automated participant\-to\-slot assignment so that each stimulus receives exactly 10 independent ratings\. Ratings on the AUT and TT are given on continuous 0–100 sliders, and to control for individual differences in scale usage, we compute intra\-participantzz\-scores: for each participantiiand dimensiondd,zi,d,s=\(ri,d,s−r¯i,d\)/σi,dz\_\{i,d,s\}=\(r\_\{i,d,s\}\-\\bar\{r\}\_\{i,d\}\)\\,/\\,\\sigma\_\{i,d\}, wherer¯i,d\\bar\{r\}\_\{i,d\}andσi,d\\sigma\_\{i,d\}are computed across all stimuli that participant rated on that dimension\. We recruit 30 participants per \(model, task, method\) triple, reaching N=720 total human reviewers\. Effect sizes and significance \(tt\-tests333We confirm responses follow a normal distribution before applying thett\-tests\.\) are computed onzz\-scored ratings for baseline vs\. creative, where each participant is treated as an independent sample\.

\(a\)Alternative Uses Test— “List alternative uses for a\.”

CN\-enhanced \(Llama\-8B\),z¯=0\.67\\bar\{z\}=0\.67 Book:A book can be transformed into a giant marimba by hollowing out its pages, gluing wood strips to the hollowed\-out sections, and tuning metal or glass material to create a makeshift percussion instrument\.originality = 1\.12 surprise = 1\.32 utility =−\-0\.42

Baseline \(Phi\-4B\),z¯=0\.36\\bar\{z\}=0\.36 Book:A book can transform into a unique, eco\-friendly planter, nurturing a tiny ecosystem as it blossoms with flowers\.originality = 0\.58 surprise = 0\.70 utility =−\-0\.20

\(b\)Task Task— “Design a creative game show challenge\.”

CN\-enhanced \(Qwen\-14B\),z¯=1\.07\\bar\{z\}=1\.07 Your objective is to construct a functional periscope allowing clear line\-of\-sight around an obstacle, mimicking an espionage thrill; you must see a small target widget placed behind a bookcase \[…\] You may utilize a stack of encyclopedias, a roll of wrapping paper, a broomstick, two mirrors, safety goggles, and a length of garden hose…creativity = 1\.08 originality = 1\.06

Baseline \(Qwen\-14B\),z¯=0\.44\\bar\{z\}=0\.44 Your goal is to create a makeshift catapult using only a skateboard, a bicycle tire inner tube, a cooking pot lid, and six feet of string\. Using this device, you must launch a raw egg into an oversized wading pool filled with colorful foam noodles placed fifteen feet away\. Each successful catapult that lands the egg in the pool scores five points; however, the egg must remain intact upon impact to earn the full five points\.creativity = 0\.50 originality = 0\.37

Figure 4:Top\-rated CreativityNeuro vs\. baseline generations from the same model\.Intra\-participantzz\-scores averaged across raters \(N=30N\{=\}30per cell\)\. \(a\) CreativityNeuro responses on the AUT tend to score higher on originality and surprise\. \(b\) CreativityNeuro challenges on the Task Task\.### 5\.1Results

CreativityNeuro improves originality, surprise, and creativityWe report results in[Figure3](https://arxiv.org/html/2607.01433#S5.F3)\. On the AUT, CreativityNeuro achieves uniformly positive originality effects across all six models \(avg\.d=\+\.36d=\+\.36\), with four reaching significance\. The effects are even stronger for surprise \(avg\.d=\+\.43d=\+\.43\), with five models reaching significance\. On the Task Task, CreativityNeuro obtains strong originality gains \(avg\.d=\+\.40d=\+\.40\), with the strongest effects in Phi\-14B \(d=\+\.61d=\+\.61\) and Qwen\-14B \(d=\+\.56d=\+\.56\), and moderate improvements in overall creativity \(avg\.d=\+\.24d=\+\.24\)\. Although CreativityNeuro degrades AUT utility, this is a predictable consequence of the novelty–utility tradeoff already present in baseline responses\. See Appendix[AppendixF](https://arxiv.org/html/2607.01433#A6)for detailed analysis of this tradeoff\.

CreativityNeuro generalizes better to the AUT and TT than activation steeringAdditionally, while activation steering \(93\.9 avg\) performs comparably to CreativityNeuro \(94\.1 avg\) on the DAT \([Figure2](https://arxiv.org/html/2607.01433#S4.F2)a\), activation steering fails to generalize to the AUT and Task Task\. These results are consistent with recent work that has found weight\-space steering generalizes further out\-of\-distribution than activation steering on sycophancy and value alignment tasks\(Fierro and Roger,[2025](https://arxiv.org/html/2607.01433#bib.bib61)\)\. Moreover, activation steering effectiveness has been shown to vary significantly by behavior type, with more complex behaviors like embodying persona archetypes and public figures proving more difficult to steer\(Bas and Novak,[2025](https://arxiv.org/html/2607.01433#bib.bib70)\)\. Techniques such as context\-dependent\(Liet al\.,[2026](https://arxiv.org/html/2607.01433#bib.bib71); Leeet al\.,[2024](https://arxiv.org/html/2607.01433#bib.bib56)\)and learned activation steering\(Rodriguezet al\.,[2025](https://arxiv.org/html/2607.01433#bib.bib72)\)have been proposed to remediate such issues, and may be explored in future work\.

CreativityNeuro generalizes to open\-ended creative tasks judged by human raters, improving measures of originality and surprise, while activation steering does not exhibit reliable transfer\.

![Refer to caption](https://arxiv.org/html/2607.01433v1/x2.png)Figure 5:Measures of mode collapse across tasks\.Baseline / CAA / CN shown left\-to\-right \(teal / green / orange\)\.\(a\)DAT vocabulary entropy\.\(b\)DAT top 10 word share\.\(c\)AUT and TT embedding homogeneity\.\(d\)Cross\-family vocabulary overlap\.

## 6Evaluating Mode Collapse Across Tasks

Instruction\-tuned LLMs are known to suffer from mode collapse–the tendency to concentrate open\-ended outputs on a narrow set of semantic clusters\(Jianget al\.,[2025](https://arxiv.org/html/2607.01433#bib.bib37); Springeret al\.,[2026](https://arxiv.org/html/2607.01433#bib.bib74)\)\. In this section, we test whether CreativityNeuro reduces word\-level mode collapse on the DAT and response\-level collapse on the AUT and TT\.

MetricsOn the DAT, we pool outputs across temperaturesT∈\{0\.9,1\.0,1\.2\}T\\in\\\{0\.9,1\.0,1\.2\\\}and compute each model’s vocabulary entropyHHand probability mass assigned to each \(model, condition∈\{\\in\\\{baseline, CAA, CN\}\\\}\)’s own top 10 most\-frequent words\. On the AUT and TT, where responses span multiple sentences, we adopt the intra\-model repetition \(mean pairwise cosine similarity within a model’s responses\) and inter\-model homogeneity \(mean pairwise cosine similarity between responses from different models on the same query\) metrics fromJianget al\.\([2025](https://arxiv.org/html/2607.01433#bib.bib37)\), also usingopenai/text\-embedding\-3\-largefor response embedding\.

### 6\.1Results

Baseline models suffer significantly from mode collapse on the DAT\.Baseline models concentrate25\.5%25\.5\\%of generated tokens on average on each model’s top\-10 most frequent words \([Figure5](https://arxiv.org/html/2607.01433#S5.F5)b, averaged acrossT∈\{0\.9,1\.0,1\.2\}T\\in\\\{0\.9,1\.0,1\.2\\\}\)\. Furthermore,1919words appear in the top 30 vocabulary of at least three of the six tested models, as shown in[Figure5](https://arxiv.org/html/2607.01433#S5.F5)d\. Three of these words—*galaxy, quasar, xylophone*—appear in the baseline top 30 of*all three model families*\(LLaMA, Phi, Qwen\) simultaneously, and two more—*glacier, nebula*—appear in the baseline top\-30 of≥4\\geq 4of the six models\.

CreativityNeuro reduces word\-level mode collapse on the DAT\.As shown in[Figure5](https://arxiv.org/html/2607.01433#S5.F5)b, CreativityNeuro reduces the top 10 share by10\.210\.2pp on average and increases vocabulary entropy by0\.590\.59nats \(\+10%\+10\\%,[Figure5](https://arxiv.org/html/2607.01433#S5.F5)a\)\. Activation steering \(CAA\) produces a smaller but qualitatively similar effect: top\-10 share drops by6\.66\.6pp \(0\.255→0\.1890\.255\\to 0\.189\), and vocabulary entropy increases by0\.400\.40nats \(\+7%\+7\\%\)\.

CreativityNeuro and activation steering both reduce mode collapse on the AUT and TTIn[Figure5](https://arxiv.org/html/2607.01433#S5.F5)c, we report relative reductions vs\. baseline annotated above each CAA bar \(green\) and CN bar \(orange\)\. Both CN and CAA reduce homogeneity on each task\. On the AUT \(sentence\-length responses\), CAA reduces intra\-model repetition by6\.6%6\.6\\%and inter\-model homogeneity by4\.7%4\.7\\%, larger than CN’s2\.4%2\.4\\%and3\.3%3\.3\\%respectively\. On the TT \(paragraph\-length responses\), CN reduces intra\-model repetition by5\.5%5\.5\\%and inter\-model homogeneity by6\.3%6\.3\\%, larger than CAA’s3\.0%3\.0\\%and2\.5%2\.5\\%\.

CreativityNeuro reduces semantic mode collapse on both word\-level \(DAT\) and embedding\-level \(AUT, TT\) measures\. Activation steering produces a similar but smaller effect on the DAT and TT, while matching CreativityNeuro on the AUT\.

## 7Are Divergent Thinking and Factual Reasoning Separable in Weights?

Previously,Christet al\.\([2025](https://arxiv.org/html/2607.01433#bib.bib22)\)demonstrated that MathNeuro could improve MATH benchmark scores without degrading factual reasoning on the Massive Multi\-task Language Understanding \(MMLU\) benchmark\(Hendryckset al\.,[2021](https://arxiv.org/html/2607.01433#bib.bib25)\)\. Here, we study whether improved divergent thinking interferes with factual reasoning on MMLU\.

SetupFor each of the six models, we take the top performing\(ρ,α\(\\rho,\\alpha, prompt\) configuration from[Section4](https://arxiv.org/html/2607.01433#S4)and measure 5\-shot accuracy on 500 MMLU questions, across 5 seeds per model, under two mask construction techniques:

1. 1\.Default:Pcre∖Pnon\-creP^\{\\text\{cre\}\}\\setminus P^\{\\text\{non\-cre\}\}uses the default creative and non\-creative prompts from[Section4](https://arxiv.org/html/2607.01433#S4)\.
2. 2\.MMLU\-protected:Pcre∖\(Pnon\-cre∪PMMLU\)P^\{\\text\{cre\}\}\\setminus\(P^\{\\text\{non\-cre\}\}\\cup P^\{\\text\{MMLU\}\}\)adds 20 randomly sampled MMLU prompts𝒫MMLU\\mathcal\{P\}^\{\\text\{MMLU\}\}to the negative contrast set in[Algorithm1](https://arxiv.org/html/2607.01433#alg1), further removing any weight whose importance is above the1−ρ1\-\\rhopercentile for MMLU\.

In summary, the MMLU\-protected masks attempts to explicitly separate creative weights from MMLU weights\. If divergent thinking and factual reasoning are fully separable, the MMLU\-protected mask should preserve MMLU scores without degrading DAT scores\.

### 7\.1Results

Divergent thinking and factual reasoning are non\-separable in weight spaceDefault masks reduce MMLU accuracy by−3\.13\-3\.13pp on average \([Figure6](https://arxiv.org/html/2607.01433#S7.F6)\)\. Meanwhile, MMLU\-protected masks gain an additional\+1\.33\+1\.33percentileΔ\\DeltaDAT on top of default masks, but further reduce MMLU scores by−0\.71\-0\.71pp \([Figure6](https://arxiv.org/html/2607.01433#S7.F6)\)\. Surprisingly, adding MMLU prompts to the negative contrast set*further degrades*MMLU accuracy, despite reducing the size of the masks by∼2×\\sim 2\\times, providing evidence that divergent thinking and factual reasoning are functionally entangled and non\-separable in weight space\.

![Refer to caption](https://arxiv.org/html/2607.01433v1/x3.png)Figure 6:Parameter importance across masksatρ=0\.1\\rho\{=\}0\.1\. Thedefault mask𝒫cre∖𝒫noncre\\mathcal\{P\}^\{\\mathrm\{cre\}\}\\setminus\\mathcal\{P\}^\{\\mathrm\{noncre\}\}and theMMLU\-protected mask𝒫cre∖\(𝒫noncre∪𝒫MMLU\)\\mathcal\{P\}^\{\\mathrm\{cre\}\}\\setminus\(\\mathcal\{P\}^\{\\mathrm\{noncre\}\}\\cup\\mathcal\{P\}^\{\\mathrm\{MMLU\}\}\)are the dotted regions on the left\. AnnotatedΔ\\DeltaDAT \(percentile\) andΔ\\DeltaMMLU \(pp\) are cross\-model means±\\pmSEM\.Our results are consistent with broader findings in the mechanistic interpretability literature\. Namely, it has been shown that individual weights can be entangled in multiple distinct functions \(a phenomenon known as*polysemanticity*\), which supports the ability of neural models to represent more features than they have neurons \(*superposition*\)\(Elhageet al\.,[2022](https://arxiv.org/html/2607.01433#bib.bib51); Sharkeyet al\.,[2025](https://arxiv.org/html/2607.01433#bib.bib41)\)\. Creativity research more broadly suggests a need for separation between generation and selection\(Varshney,[2019](https://arxiv.org/html/2607.01433#bib.bib10)\)\. The success of multi\-agent creative systems\(Linet al\.,[2025](https://arxiv.org/html/2607.01433#bib.bib69)\)and multi\-stage prompting techniques that decouple creative exploration from constraint satisfaction\(Nguyen and Singla,[2025](https://arxiv.org/html/2607.01433#bib.bib65)\)can be interpreted within the context of these results: if weights controlling divergent and convergent thinking are entangled, simultaneously eliciting strong divergent*and*convergent abilities may be challenging or even impossible\. Therefore, separating these steps across agents or prompts can provide stronger overall performance\.

We find evidence, consistent with the broader mechanistic interpretability literature, that divergent thinking and factual reasoning are functionally entangled in model weights\.

## 8Limitations and Future Work

Our evaluation is restricted to a finite set of divergent thinking benchmarks and metrics, which capture only certain aspects of creativity\. AsRunco \([2008](https://arxiv.org/html/2607.01433#bib.bib8)\)notes, divergent thinking is not synonymous with creativity and should best be thought of as a measure of creative potential\. Our comparison to activation steering focuses on a standard CAA\-style method and does not exhaust the space of possible activation\-space interventions, such as context\-dependent or learned approaches\. Lastly, we found evidence suggesting divergent thinking and factual reasoning are non\-separable in model weights—understanding whether this entanglement reflects a fundamental architectural constraint or arises as an artifact of the Wanda\-style importance technique remains an important open question\. Designing architectures that learn unified factored representations\(Kumaret al\.,[2025](https://arxiv.org/html/2607.01433#bib.bib64)\)and disentangle divergent thinking from factual reasoning would be a valuable and timely direction for future work\.

## References

- M\. Abdin, J\. Aneja, H\. Awadalla, A\. Awadallah, A\. A\. Awan, N\. Bach, A\. Bahree, A\. Bakhtiari, J\. Bao, H\. Behl, A\. Benhaim, M\. Bilenko, J\. Bjorck, S\. Bubeck, M\. Cai, Q\. Cai, V\. Chaudhary, D\. Chen, D\. Chen, W\. Chen, Y\. Chen, Y\. Chen, H\. Cheng, P\. Chopra, X\. Dai, M\. Dixon, R\. Eldan, V\. Fragoso, J\. Gao, M\. Gao, M\. Gao, A\. Garg, A\. D\. Giorno, A\. Goswami, S\. Gunasekar, E\. Haider, J\. Hao, R\. J\. Hewett, W\. Hu, J\. Huynh, D\. Iter, S\. A\. Jacobs, M\. Javaheripi, X\. Jin, N\. Karampatziakis, P\. Kauffmann, M\. Khademi, D\. Kim, Y\. J\. Kim, L\. Kurilenko, J\. R\. Lee, Y\. T\. Lee, Y\. Li, Y\. Li, C\. Liang, L\. Liden, X\. Lin, Z\. Lin, C\. Liu, L\. Liu, M\. Liu, W\. Liu, X\. Liu, C\. Luo, P\. Madan, A\. Mahmoudzadeh, D\. Majercak, M\. Mazzola, C\. C\. T\. Mendes, A\. Mitra, H\. Modi, A\. Nguyen, B\. Norick, B\. Patra, D\. Perez\-Becker, T\. Portet, R\. Pryzant, H\. Qin, M\. Radmilac, L\. Ren, G\. de Rosa, C\. Rosset, S\. Roy, O\. Ruwase, O\. Saarikivi, A\. Saied, A\. Salim, M\. Santacroce, S\. Shah, N\. Shang, H\. Sharma, Y\. Shen, S\. Shukla, X\. Song, M\. Tanaka, A\. Tupini, P\. Vaddamanu, C\. Wang, G\. Wang, L\. Wang, S\. Wang, X\. Wang, Y\. Wang, R\. Ward, W\. Wen, P\. Witte, H\. Wu, X\. Wu, M\. Wyatt, B\. Xiao, C\. Xu, J\. Xu, W\. Xu, J\. Xue, S\. Yadav, F\. Yang, J\. Yang, Y\. Yang, Z\. Yang, D\. Yu, L\. Yuan, C\. Zhang, C\. Zhang, J\. Zhang, L\. L\. Zhang, Y\. Zhang, Y\. Zhang, Y\. Zhang, and X\. Zhou \(2024\)Phi\-3 technical report: a highly capable language model locally on your phone\.External Links:2404\.14219,[Link](https://arxiv.org/abs/2404.14219)Cited by:[§4](https://arxiv.org/html/2607.01433#S4.p1.3)\.
- T\. Bas and K\. Novak \(2025\)What can we actually steer? a multi\-behavior study of activation control\.arXiv preprint arXiv:2511\.18284\.Cited by:[§5\.1](https://arxiv.org/html/2607.01433#S5.SS1.p2.1)\.
- A\. Bellemare\-Pepin, F\. Lespinasse, P\. Thölke, Y\. Harel, K\. Mathewson, J\. A\. Olson, Y\. Bengio, and K\. Jerbi \(2024\)Divergent Creativity in Humans and LLMs\.Technical reportCited by:[§1](https://arxiv.org/html/2607.01433#S1.p1.1),[§2](https://arxiv.org/html/2607.01433#S2.p2.1),[§4](https://arxiv.org/html/2607.01433#S4.p3.3)\.
- M\. A\. Boden \(2004\)The Creative Mind: Myths and Mechanisms\.Routledge\.Cited by:[§1](https://arxiv.org/html/2607.01433#S1.p1.1),[§3](https://arxiv.org/html/2607.01433#S3.p1.1)\.
- Y\. Cao, B\. J\. Smucker, and T\. J\. Robinson \(2015\)On using the hypervolume indicator to compare pareto fronts: applications to multi\-criteria optimal experimental design\.Journal of Statistical Planning and Inference160,pp\. 60–74\.External Links:ISSN 0378\-3758,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/j.jspi.2014.12.004),[Link](https://www.sciencedirect.com/science/article/pii/S0378375814002006)Cited by:[Appendix F](https://arxiv.org/html/2607.01433#A6.p1.5)\.
- B\. R\. Christ, Z\. Gottesman, J\. Kropko, and T\. Hartvigsen \(2025\)Math Neurosurgery: Isolating Language Models’ Math Reasoning Abilities Using Only Forward Passes\.External Links:[Link](http://arxiv.org/abs/2410.16930)Cited by:[Appendix D](https://arxiv.org/html/2607.01433#A4.p1.11),[§3](https://arxiv.org/html/2607.01433#S3.p1.1),[§3](https://arxiv.org/html/2607.01433#S3.p2.2),[§3](https://arxiv.org/html/2607.01433#S3.p3.17),[§7](https://arxiv.org/html/2607.01433#S7.p1.1)\.
- J\. Chu, J\. Hu, and T\. D\. Ullman \(2024\)The task task: creative problem generation in humans and language models\.InProceedings of the Annual Meeting of the Cognitive Science Society,Vol\.46\.Cited by:[§E\.3](https://arxiv.org/html/2607.01433#A5.SS3.p1.1),[§2](https://arxiv.org/html/2607.01433#S2.p2.1),[§5](https://arxiv.org/html/2607.01433#S5.p1.1),[footnote 2](https://arxiv.org/html/2607.01433#footnote2)\.
- A\. Dietrich \(2019\)Types of creativity\.Psychonomic Bulletin and Review26\(1\),pp\. 1–12\.External Links:[Document](https://dx.doi.org/10.3758/s13423-018-1517-7),ISSN 15315320Cited by:[§1](https://arxiv.org/html/2607.01433#S1.p2.1)\.
- Dietrich, Arne \(2004\)The Cognitive Neuroscience of Creativity\.Psychonomic Bulletin & Review11 \(6\),pp\. 1011–1026\.Cited by:[§1](https://arxiv.org/html/2607.01433#S1.p1.1)\.
- N\. Elhage, T\. Hume, C\. Olsson, N\. Schiefer, T\. Henighan, S\. Kravec, Z\. Hatfield\-Dodds, R\. Lasenby, D\. Drain, C\. Chen, R\. Grosse, S\. McCandlish, J\. Kaplan, D\. Amodei, M\. Wattenberg, and C\. Olah \(2022\)Toy models of superposition\.Transformer Circuits Thread\.External Links:[Link](https://transformer-circuits.pub/2022/toy_model/index.html)Cited by:[§7\.1](https://arxiv.org/html/2607.01433#S7.SS1.p2.1)\.
- G\. Fauconnier and M\. Turner \(2008\)The Way We Think: Conceptual Blending and the Mind’s Hidden Complexities\.Basic Books\.Cited by:[§1](https://arxiv.org/html/2607.01433#S1.p1.1)\.
- C\. Fierro and F\. Roger \(2025\)Steering language models with weight arithmetic\.arXiv preprint arXiv:2511\.05408\.Cited by:[§5\.1](https://arxiv.org/html/2607.01433#S5.SS1.p2.1)\.
- F\. Galton \(1870\)Hereditary genius: an inquiry into its laws and consequences\.D\. Appleton & Company\.Cited by:[§1](https://arxiv.org/html/2607.01433#S1.p1.1)\.
- A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan, A\. Yang, A\. Fan, A\. Goyal, A\. Hartshorn, A\. Yang, A\. Mitra, A\. Sravankumar, A\. Korenev, A\. Hinsvark, A\. Rao, A\. Zhang, A\. Rodriguez, A\. Gregerson, A\. Spataru, B\. Roziere, B\. Biron, B\. Tang, B\. Chern, C\. Caucheteux, C\. Nayak, C\. Bi, C\. Marra, C\. McConnell, C\. Keller, C\. Touret, C\. Wu, C\. Wong, C\. C\. Ferrer, C\. Nikolaidis, D\. Allonsius, D\. Song, D\. Pintz, D\. Livshits, D\. Wyatt, D\. Esiobu, D\. Choudhary, D\. Mahajan, D\. Garcia\-Olano, D\. Perino, D\. Hupkes, E\. Lakomkin, E\. AlBadawy, E\. Lobanova, E\. Dinan, E\. M\. Smith, F\. Radenovic, F\. Guzmán, F\. Zhang, G\. Synnaeve, G\. Lee, G\. L\. Anderson, G\. Thattai, G\. Nail, G\. Mialon, G\. Pang, G\. Cucurell, H\. Nguyen, H\. Korevaar, H\. Xu, H\. Touvron, I\. Zarov, I\. A\. Ibarra, I\. Kloumann, I\. Misra, I\. Evtimov, J\. Zhang, J\. Copet, J\. Lee, J\. Geffert, J\. Vranes, J\. Park, J\. Mahadeokar, J\. Shah, J\. van der Linde, J\. Billock, J\. Hong, J\. Lee, J\. Fu, J\. Chi, J\. Huang, J\. Liu, J\. Wang, J\. Yu, J\. Bitton, J\. Spisak, J\. Park, J\. Rocca, J\. Johnstun, J\. Saxe, J\. Jia, K\. V\. Alwala, K\. Prasad, K\. Upasani, K\. Plawiak, K\. Li, K\. Heafield, K\. Stone, K\. El\-Arini, K\. Iyer, K\. Malik, K\. Chiu, K\. Bhalla, K\. Lakhotia, L\. Rantala\-Yeary, L\. van der Maaten, L\. Chen, L\. Tan, L\. Jenkins, L\. Martin, L\. Madaan, L\. Malo, L\. Blecher, L\. Landzaat, L\. de Oliveira, M\. Muzzi, M\. Pasupuleti, M\. Singh, M\. Paluri, M\. Kardas, M\. Tsimpoukelli, M\. Oldham, M\. Rita, M\. Pavlova, M\. Kambadur, M\. Lewis, M\. Si, M\. K\. Singh, M\. Hassan, N\. Goyal, N\. Torabi, N\. Bashlykov, N\. Bogoychev, N\. Chatterji, N\. Zhang, O\. Duchenne, O\. Çelebi, P\. Alrassy, P\. Zhang, P\. Li, P\. Vasic, P\. Weng, P\. Bhargava, P\. Dubal, P\. Krishnan, P\. S\. Koura, P\. Xu, Q\. He, Q\. Dong, R\. Srinivasan, R\. Ganapathy, R\. Calderer, R\. S\. Cabral, R\. Stojnic, R\. Raileanu, R\. Maheswari, R\. Girdhar, R\. Patel, R\. Sauvestre, R\. Polidoro, R\. Sumbaly, R\. Taylor, R\. Silva, R\. Hou, R\. Wang, S\. Hosseini, S\. Chennabasappa, S\. Singh, S\. Bell, S\. S\. Kim, S\. Edunov, S\. Nie, S\. Narang, S\. Raparthy, S\. Shen, S\. Wan, S\. Bhosale, S\. Zhang, S\. Vandenhende, S\. Batra, S\. Whitman, S\. Sootla, S\. Collot, S\. Gururangan, S\. Borodinsky, T\. Herman, T\. Fowler, T\. Sheasha, T\. Georgiou, T\. Scialom, T\. Speckbacher, T\. Mihaylov, T\. Xiao, U\. Karn, V\. Goswami, V\. Gupta, V\. Ramanathan, V\. Kerkez, V\. Gonguet, V\. Do, V\. Vogeti, V\. Albiero, V\. Petrovic, W\. Chu, W\. Xiong, W\. Fu, W\. Meers, X\. Martinet, X\. Wang, X\. Wang, X\. E\. Tan, X\. Xia, X\. Xie, X\. Jia, X\. Wang, Y\. Goldschlag, Y\. Gaur, Y\. Babaei, Y\. Wen, Y\. Song, Y\. Zhang, Y\. Li, Y\. Mao, Z\. D\. Coudert, Z\. Yan, Z\. Chen, Z\. Papakipos, A\. Singh, A\. Srivastava, A\. Jain, A\. Kelsey, A\. Shajnfeld, A\. Gangidi, A\. Victoria, A\. Goldstand, A\. Menon, A\. Sharma, A\. Boesenberg, A\. Baevski, A\. Feinstein, A\. Kallet, A\. Sangani, A\. Teo, A\. Yunus, A\. Lupu, A\. Alvarado, A\. Caples, A\. Gu, A\. Ho, A\. Poulton, A\. Ryan, A\. Ramchandani, A\. Dong, A\. Franco, A\. Goyal, A\. Saraf, A\. Chowdhury, A\. Gabriel, A\. Bharambe, A\. Eisenman, A\. Yazdan, B\. James, B\. Maurer, B\. Leonhardi, B\. Huang, B\. Loyd, B\. D\. Paola, B\. Paranjape, B\. Liu, B\. Wu, B\. Ni, B\. Hancock, B\. Wasti, B\. Spence, B\. Stojkovic, B\. Gamido, B\. Montalvo, C\. Parker, C\. Burton, C\. Mejia, C\. Liu, C\. Wang, C\. Kim, C\. Zhou, C\. Hu, C\. Chu, C\. Cai, C\. Tindal, C\. Feichtenhofer, C\. Gao, D\. Civin, D\. Beaty, D\. Kreymer, D\. Li, D\. Adkins, D\. Xu, D\. Testuggine, D\. David, D\. Parikh, D\. Liskovich, D\. Foss, D\. Wang, D\. Le, D\. Holland, E\. Dowling, E\. Jamil, E\. Montgomery, E\. Presani, E\. Hahn, E\. Wood, E\. Le, E\. Brinkman, E\. Arcaute, E\. Dunbar, E\. Smothers, F\. Sun, F\. Kreuk, F\. Tian, F\. Kokkinos, F\. Ozgenel, F\. Caggioni, F\. Kanayet, F\. Seide, G\. M\. Florez, G\. Schwarz, G\. Badeer, G\. Swee, G\. Halpern, G\. Herman, G\. Sizov, Guangyi, Zhang, G\. Lakshminarayanan, H\. Inan, H\. Shojanazeri, H\. Zou, H\. Wang, H\. Zha, H\. Habeeb, H\. Rudolph, H\. Suk, H\. Aspegren, H\. Goldman, H\. Zhan, I\. Damlaj, I\. Molybog, I\. Tufanov, I\. Leontiadis, I\. Veliche, I\. Gat, J\. Weissman, J\. Geboski, J\. Kohli, J\. Lam, J\. Asher, J\. Gaya, J\. Marcus, J\. Tang, J\. Chan, J\. Zhen, J\. Reizenstein, J\. Teboul, J\. Zhong, J\. Jin, J\. Yang, J\. Cummings, J\. Carvill, J\. Shepard, J\. McPhie, J\. Torres, J\. Ginsburg, J\. Wang, K\. Wu, K\. H\. U, K\. Saxena, K\. Khandelwal, K\. Zand, K\. Matosich, K\. Veeraraghavan, K\. Michelena, K\. Li, K\. Jagadeesh, K\. Huang, K\. Chawla, K\. Huang, L\. Chen, L\. Garg, L\. A, L\. Silva, L\. Bell, L\. Zhang, L\. Guo, L\. Yu, L\. Moshkovich, L\. Wehrstedt, M\. Khabsa, M\. Avalani, M\. Bhatt, M\. Mankus, M\. Hasson, M\. Lennie, M\. Reso, M\. Groshev, M\. Naumov, M\. Lathi, M\. Keneally, M\. Liu, M\. L\. Seltzer, M\. Valko, M\. Restrepo, M\. Patel, M\. Vyatskov, M\. Samvelyan, M\. Clark, M\. Macey, M\. Wang, M\. J\. Hermoso, M\. Metanat, M\. Rastegari, M\. Bansal, N\. Santhanam, N\. Parks, N\. White, N\. Bawa, N\. Singhal, N\. Egebo, N\. Usunier, N\. Mehta, N\. P\. Laptev, N\. Dong, N\. Cheng, O\. Chernoguz, O\. Hart, O\. Salpekar, O\. Kalinli, P\. Kent, P\. Parekh, P\. Saab, P\. Balaji, P\. Rittner, P\. Bontrager, P\. Roux, P\. Dollar, P\. Zvyagina, P\. Ratanchandani, P\. Yuvraj, Q\. Liang, R\. Alao, R\. Rodriguez, R\. Ayub, R\. Murthy, R\. Nayani, R\. Mitra, R\. Parthasarathy, R\. Li, R\. Hogan, R\. Battey, R\. Wang, R\. Howes, R\. Rinott, S\. Mehta, S\. Siby, S\. J\. Bondu, S\. Datta, S\. Chugh, S\. Hunt, S\. Dhillon, S\. Sidorov, S\. Pan, S\. Mahajan, S\. Verma, S\. Yamamoto, S\. Ramaswamy, S\. Lindsay, S\. Lindsay, S\. Feng, S\. Lin, S\. C\. Zha, S\. Patil, S\. Shankar, S\. Zhang, S\. Zhang, S\. Wang, S\. Agarwal, S\. Sajuyigbe, S\. Chintala, S\. Max, S\. Chen, S\. Kehoe, S\. Satterfield, S\. Govindaprasad, S\. Gupta, S\. Deng, S\. Cho, S\. Virk, S\. Subramanian, S\. Choudhury, S\. Goldman, T\. Remez, T\. Glaser, T\. Best, T\. Koehler, T\. Robinson, T\. Li, T\. Zhang, T\. Matthews, T\. Chou, T\. Shaked, V\. Vontimitta, V\. Ajayi, V\. Montanez, V\. Mohan, V\. S\. Kumar, V\. Mangla, V\. Ionescu, V\. Poenaru, V\. T\. Mihailescu, V\. Ivanov, W\. Li, W\. Wang, W\. Jiang, W\. Bouaziz, W\. Constable, X\. Tang, X\. Wu, X\. Wang, X\. Wu, X\. Gao, Y\. Kleinman, Y\. Chen, Y\. Hu, Y\. Jia, Y\. Qi, Y\. Li, Y\. Zhang, Y\. Zhang, Y\. Adi, Y\. Nam, Yu, Wang, Y\. Zhao, Y\. Hao, Y\. Qian, Y\. Li, Y\. He, Z\. Rait, Z\. DeVito, Z\. Rosnbrick, Z\. Wen, Z\. Yang, Z\. Zhao, and Z\. Ma \(2024\)The llama 3 herd of models\.External Links:2407\.21783,[Link](https://arxiv.org/abs/2407.21783)Cited by:[§4](https://arxiv.org/html/2607.01433#S4.p1.3)\.
- J\. P\. Guilford \(1956\)Psychological Bulletin THE STRUCTURE OF INTELLECT\.Technical reportTechnical Report4, Vol\.53\.Cited by:[§1](https://arxiv.org/html/2607.01433#S1.p1.1),[§1](https://arxiv.org/html/2607.01433#S1.p2.1),[§2](https://arxiv.org/html/2607.01433#S2.p2.1),[§5](https://arxiv.org/html/2607.01433#S5.p1.1)\.
- J\. Hadamard \(1954\)An Essay on the Psychology of Invention in the Mathematical Field\.Courier Corporation\.Cited by:[§1](https://arxiv.org/html/2607.01433#S1.p1.1)\.
- D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. Steinhardt \(2021\)Measuring Mathematical Problem Solving With the MATH Dataset\.External Links:[Link](http://arxiv.org/abs/2103.03874)Cited by:[§3](https://arxiv.org/html/2607.01433#S3.p1.1),[§7](https://arxiv.org/html/2607.01433#S7.p1.1)\.
- L\. Jiang, Y\. Chai, M\. Li, M\. Liu, R\. Fok, N\. Dziri, Y\. Tsvetkov, M\. Sap, A\. Albalak, and Y\. Choi \(2025\)Artificial hivemind: the open\-ended homogeneity of language models \(and beyond\)\.arXiv preprint arXiv:2510\.22954\.Cited by:[§1](https://arxiv.org/html/2607.01433#S1.p1.1),[§2](https://arxiv.org/html/2607.01433#S2.p2.1),[§3](https://arxiv.org/html/2607.01433#S3.p1.1),[§6](https://arxiv.org/html/2607.01433#S6.p1.1),[§6](https://arxiv.org/html/2607.01433#S6.p2.4)\.
- A\. Koestler \(1964\)The Act of Creation\.Macmillan\.Cited by:[§1](https://arxiv.org/html/2607.01433#S1.p1.1)\.
- A\. Kumar, J\. Clune, J\. Lehman, and K\. O\. Stanley \(2025\)Questioning representational optimism in deep learning: the fractured entangled representation hypothesis\.arXiv preprint arXiv:2505\.11581\.Cited by:[§8](https://arxiv.org/html/2607.01433#S8.p1.1)\.
- B\. W\. Lee, I\. Padhi, K\. N\. Ramamurthy, E\. Miehling, P\. Dognin, M\. Nagireddy, and A\. Dhurandhar \(2024\)Programming refusal with conditional activation steering\.arXiv preprint arXiv:2409\.05907\.Cited by:[§5\.1](https://arxiv.org/html/2607.01433#S5.SS1.p2.1)\.
- J\. Li, Y\. Li, and K\. Huang \(2026\)Steering vector fields for context\-aware inference\-time control in large language models\.arXiv preprint arXiv:2602\.01654\.Cited by:[§5\.1](https://arxiv.org/html/2607.01433#S5.SS1.p2.1)\.
- Y\. Lin, K\. Chen, Z\. Li, T\. Wu, T\. Wu, K\. Chen, H\. Lee, and Y\. Chen \(2025\)Creativity in llm\-based multi\-agent systems: a survey\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 27572–27595\.Cited by:[§7\.1](https://arxiv.org/html/2607.01433#S7.SS1.p2.1)\.
- J\. Lindsey, A\. Templeton, J\. Marcus, T\. Conerly, J\. Batson, and C\. Olah \(2024\)Sparse crosscoders for cross\-layer features and model diffing\.Technical reportAnthropic\.External Links:[Link](https://arxiv.org/html/%20//transformer-circuits.pub/2024/crosscoders/index.html.)Cited by:[§C\.1](https://arxiv.org/html/2607.01433#A3.SS1.SSS0.Px3.p1.1)\.
- M\. L\. Maher \(2010\)Evaluating creativity in humans, computers, and collectively intelligent systems\.InProceedings of the 1st DESIRE Network Conference on Creativity and Innovation in Design,DESIRE ’10,pp\. 22–28\.Cited by:[§1](https://arxiv.org/html/2607.01433#S1.p1.1),[§3](https://arxiv.org/html/2607.01433#S3.p1.1)\.
- S\. Mednick \(1962\)The associative basis of the creative process\.\.Psychological Review69\(3\),pp\. 220\.Cited by:[§1](https://arxiv.org/html/2607.01433#S1.p1.1)\.
- R\. Morain and D\. Ventura \(2025\)Is Prompt Engineering the Creativity Knob for Large Language Models?\.InProceedings of the 16th International Conference on Computational Creativity \(ICCC’25\),Cited by:[§2](https://arxiv.org/html/2607.01433#S2.p3.1)\.
- M\. H\. Nguyen and A\. Singla \(2025\)Divergent\-convergent thinking in large language models for creative problem generation\.arXiv preprint arXiv:2512\.23601\.Cited by:[§2](https://arxiv.org/html/2607.01433#S2.p3.1),[§7\.1](https://arxiv.org/html/2607.01433#S7.SS1.p2.1)\.
- J\. A\. Olson, J\. Nahas, D\. Chmoulevitch, S\. J\. Cropper, and M\. E\. Webb \(2021\)Naming unrelated words predicts creativity\.External Links:[Document](https://dx.doi.org/10.1073/pnas.2022340118/-/DCSupplemental.y)Cited by:[§2](https://arxiv.org/html/2607.01433#S2.p2.1),[§4](https://arxiv.org/html/2607.01433#S4.p2.5),[§4](https://arxiv.org/html/2607.01433#S4.p2.7)\.
- M\. L\. Olson, N\. Ratzlaff, M\. Hinck, S\. Tseng, and V\. Lal \(2024\)Steering Large Language Models to Evaluate and Amplify Creativity\.External Links:[Link](http://arxiv.org/abs/2412.06060)Cited by:[§2](https://arxiv.org/html/2607.01433#S2.p3.1)\.
- N\. Panickssery, N\. Gabrieli, J\. Schulz, M\. Tong, E\. Hubinger, and A\. M\. Turner \(2024\)Steering Llama 2 via contrastive activation addition\.arXiv preprint arXiv:2312\.06681\.Cited by:[§4](https://arxiv.org/html/2607.01433#S4.p3.3)\.
- M\. Peeperkorn, T\. Kouwenhoven, D\. Brown, and A\. Jordanous \(2024\)Is temperature the creativity parameter of large language models?\.Note:arXiv:2405\.00492Cited by:[§2](https://arxiv.org/html/2607.01433#S2.p3.1)\.
- J\. Pennington, R\. Socher, and C\. D\. Manning \(2014\)GloVe: Global Vectors for Word Representation\.InProceedings of the 2014 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 1532–1543\.External Links:[Document](https://dx.doi.org/10.3115/v1/D14-1162),[Link](https://nlp.stanford.edu/pubs/glove.pdf)Cited by:[§4](https://arxiv.org/html/2607.01433#S4.p2.7)\.
- L\. A\. Quetelet \(1842\)A treatise on man and the development of his faculties \(a facsimile reproduction of the english translation of 1842 with an introduction by solomon diamond\)\.\.Cited by:[§1](https://arxiv.org/html/2607.01433#S1.p1.1)\.
- P\. Rodriguez, M\. Klein, E\. Gualdoni, V\. Maiorca, A\. Blaas, L\. Zappella, M\. Cuturi, and X\. Suau \(2025\)LinEAS: end\-to\-end learning of activation steering with a distributional loss\.NeurIPS\.Cited by:[§5\.1](https://arxiv.org/html/2607.01433#S5.SS1.p2.1)\.
- A\. Rothenberg \(2014\)Flight from Wonder: An Investigation of Scientific Creativity\.Oxford University Press\.Cited by:[§1](https://arxiv.org/html/2607.01433#S1.p1.1)\.
- M\. A\. Runco \(2008\)Commentary: Divergent Thinking Is Not Synonymous With Creativity\.Psychology of Aesthetics, Creativity, and the Arts2\(2\),pp\. 93–96\.External Links:[Document](https://dx.doi.org/10.1037/1931-3896.2.2.93),ISSN 19313896Cited by:[§8](https://arxiv.org/html/2607.01433#S8.p1.1)\.
- A\. Sanyal, S\. Schapiro, S\. Shashidhar, R\. Moon, L\. R\. Varshney, and D\. Hakkani\-Tur \(2025\)Spark: A system for scientifically creative idea generation\.arXiv preprint arXiv:2504\.20090\.Cited by:[§1](https://arxiv.org/html/2607.01433#S1.p1.1)\.
- S\. Schapiro, J\. Black, and L\. R\. Varshney \(2025\)Transformational Creativity in Science: A Graphical Theory\.arXiv preprint arXiv:2504\.18687\.Cited by:[§1](https://arxiv.org/html/2607.01433#S1.p1.1)\.
- L\. Sharkey, B\. Chughtai, J\. Batson, J\. Lindsey, J\. Wu, L\. Bushnaq, N\. Goldowsky\-Dill, S\. Heimersheim, A\. Ortega, J\. Bloom, S\. Biderman, A\. Garriga\-Alonso, A\. Conmy, N\. Nanda, J\. Rumbelow, M\. Wattenberg, N\. Schoots, J\. Miller, E\. J\. Michaud, S\. Casper, M\. Tegmark, W\. Saunders, D\. Bau, E\. Todd, A\. Geiger, M\. Geva, J\. Hoogland, D\. Murfet, and T\. McGrath \(2025\)Open problems in mechanistic interpretability\.External Links:2501\.16496,[Link](https://arxiv.org/abs/2501.16496)Cited by:[§C\.1](https://arxiv.org/html/2607.01433#A3.SS1.SSS0.Px3.p1.1),[§7\.1](https://arxiv.org/html/2607.01433#S7.SS1.p2.1)\.
- C\. Si, T\. Hashimoto, and D\. Yang \(2025\)The Ideation\-Execution Gap: Execution Outcomes of LLM\-Generated versus Human Research Ideas\.External Links:[Link](http://arxiv.org/abs/2506.20803)Cited by:[§1](https://arxiv.org/html/2607.01433#S1.p1.1),[§2](https://arxiv.org/html/2607.01433#S2.p2.1)\.
- C\. Si, D\. Yang, and T\. Hashimoto \(2024\)Can LLMs Generate Novel Research Ideas? A Large\-Scale Human Study with 100\+ NLP Researchers\.External Links:[Link](http://arxiv.org/abs/2409.04109)Cited by:[§1](https://arxiv.org/html/2607.01433#S1.p1.1),[§2](https://arxiv.org/html/2607.01433#S2.p2.1)\.
- D\. K\. Simonton \(2004\)Creativity in Science: Chance, Logic, Genius, and Zeitgeist\.Cambridge University Press\.Cited by:[§1](https://arxiv.org/html/2607.01433#S1.p1.1)\.
- J\. M\. Springer, M\. Advani, L\. Aichberger, A\. Bradley, E\. Malach, O\. Saremi, S\. Williamson, P\. Nakkiran, E\. Littwin, and A\. Raghunathan \(2026\)Annotations mitigate post\-training mode collapse\.External Links:2605\.09995,[Link](https://arxiv.org/abs/2605.09995)Cited by:[§6](https://arxiv.org/html/2607.01433#S6.p1.1)\.
- C\. Stevenson, I\. Smal, M\. Baas, R\. Grasman, and H\. Van Der Maas \(2022\)Putting GPT\-3’s Creativity to the \(Alternative Uses\) Test\.InInternational Conference on Computational Creativity,External Links:[Link](http://osf.io/vmk3c/)Cited by:[§2](https://arxiv.org/html/2607.01433#S2.p2.1),[§5](https://arxiv.org/html/2607.01433#S5.p1.1)\.
- M\. Sun, Z\. Liu, A\. Bair, and J\. Z\. Kolter \(2023\)A simple and effective pruning approach for large language models\.arXiv preprint arXiv:2306\.11695\.Cited by:[§3](https://arxiv.org/html/2607.01433#S3.p3.17)\.
- L\. R\. Varshney \(2019\)Mathematical limit theorems for computational creativity\.IBM Journal of Research and Development63\(1\)\.External Links:[Document](https://dx.doi.org/10.1147/JRD.2019.2893907),ISSN 21518556Cited by:[Appendix F](https://arxiv.org/html/2607.01433#A6.p1.5),[§1](https://arxiv.org/html/2607.01433#S1.p1.1),[§3](https://arxiv.org/html/2607.01433#S3.p1.1),[§7\.1](https://arxiv.org/html/2607.01433#S7.SS1.p2.1)\.
- H\. Wang, P\. Bao, L\. Qiu, D\. Wu, N\. Yu, H\. Liu, and S\. Johnson \(2025\)A large\-scale comparison of divergent creativity in humans and large language models\.Nature Human Behaviour\.External Links:[Document](https://dx.doi.org/10.1038/s41562-025-02331-1)Cited by:[Appendix D](https://arxiv.org/html/2607.01433#A4.p2.6),[§1](https://arxiv.org/html/2607.01433#S1.p1.1),[§2](https://arxiv.org/html/2607.01433#S2.p2.1),[§2](https://arxiv.org/html/2607.01433#S2.p3.1),[Figure 2](https://arxiv.org/html/2607.01433#S4.F2),[§4](https://arxiv.org/html/2607.01433#S4.p3.3)\.
- Q\. Wang, D\. Downey, H\. Ji, and T\. Hope \(2024\)SciMON: Scientific Inspiration Machines Optimized for Novelty\.External Links:[Link](http://arxiv.org/abs/2305.14259%20http://dx.doi.org/10.18653/v1/2024.acl-long.18),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.18)Cited by:[§1](https://arxiv.org/html/2607.01433#S1.p1.1)\.
- X\. Wei, B\. Lu, X\. Zhang, Z\. Zhao, D\. Shen, L\. Xia, and D\. Yin \(2025\)Igniting creative writing in small language models: llm\-as\-a\-judge versus multi\-agent refined rewards\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 17171–17197\.Cited by:[§2](https://arxiv.org/html/2607.01433#S2.p3.1)\.
- A\. Yang, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Li, D\. Liu, F\. Huang, H\. Wei, H\. Lin, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Lin, K\. Dang, K\. Lu, K\. Bao, K\. Yang, L\. Yu, M\. Li, M\. Xue, P\. Zhang, Q\. Zhu, R\. Men, R\. Lin, T\. Li, T\. Tang, T\. Xia, X\. Ren, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Cui, Z\. Zhang, and Z\. Qiu \(2025\)Qwen2\.5 technical report\.External Links:2412\.15115,[Link](https://arxiv.org/abs/2412.15115)Cited by:[§4](https://arxiv.org/html/2607.01433#S4.p1.3)\.

## Appendix AFull Sets of Contrastive Prompts

Each of the six prompt sets contains 10 creative and 10 non\-creative exemplars\. Below we show 3 representative examples from each set\.

Table 2:Full contrastive prompt sets \(3 examples each\)\.Each prompt set contains 10 creative \(𝒫cre\\mathcal\{P\}^\{\\text\{cre\}\}\) and 10 non\-creative \(𝒫non\-cre\\mathcal\{P\}^\{\\text\{non\-cre\}\}\) exemplars used for parameter importance scoring in[Algorithm1](https://arxiv.org/html/2607.01433#alg1)\.
## Appendix BBaseline Sweep Settings

In[Section4](https://arxiv.org/html/2607.01433#S4), we compare CreativityNeuro against a set of baseline techniques\.

1. 1\.Prompting:We present creative exemplars from each of six prompt sets \([Table2](https://arxiv.org/html/2607.01433#A1.T2)\) as in\-context guides
2. 2\.Decoding Parameter Sweeps 1. \(a\)top\-ppnucleus sampling,p∈\{0\.8,0\.85,0\.9,0\.95,1\.0\}p\\in\\\{0\.8,0\.85,0\.9,0\.95,1\.0\\\} 2. \(b\)top\-kksampling,k∈\{10,25,50,100,disabled\}k\\in\\\{10,25,50,100,\\text\{disabled\}\\\} 3. \(c\)repetition penalty,θ∈\{1\.0,1\.1,1\.2,1\.5,2\.0,3\.0\}\\theta\\in\\\{1\.0,1\.1,1\.2,1\.5,2\.0,3\.0\\\}
3. 3\.Activation Steering: We sweep the injection layer suffix from single\-layer to the final 50% of layers and find that injecting into the final 30% works best\. We sweepα∈\{0\.1,0\.2,0\.3,0\.4,0\.5,1\.0,2\.0,4\.0\}\\alpha\\in\\\{0\.1,0\.2,0\.3,0\.4,0\.5,1\.0,2\.0,4\.0\\\}, selecting the bestα\\alphawhere all three temperatures yield≥120\\geq 120valid samples\.

## Appendix CLayerwise Ablation Studies

### C\.1Suffix vs\. Prefix

![Refer to caption](https://arxiv.org/html/2607.01433v1/figures/prefix_vs_suffix.png)Figure 7:Layerwise ablation: suffix vs\. prefix vs\. single\-layer\.Each panel shows the % of full CN DAT effect recovered as a function of the number of layerskkwith CN weights applied\. Solid lines: suffix \(lastkklayers\)\. Dashed lines: prefix \(firstkklayers\)\. Dotted lines: single\-layer \(one layer at a time, plotted by layer index\)\. Diamond markers indicate the fewest layers from the back achieving≥\\geq95% of the full effect\.To localize the CN effect across the network, we compare three layerwise interventions \([Figure7](https://arxiv.org/html/2607.01433#A3.F7)\): \(i\)*suffix*, applying CN weights to only the lastkklayers; \(ii\)*prefix*, applying CN weights to only the firstkklayers; and \(iii\)*single\-layer*, applying CN weights to one layer at a time\. For each condition, we generateN=120N=120valid DAT samples atT=1\.0T=1\.0and report the percentage of the full CN effect recovered in terms of DAT scores\.

#### Suffix\.

Applying CN weights to a suffix of layers \(layerskkthroughL−1L\{\-\}1\) recovers the full effect with roughly half the network\. On average, 51% of layers \(from the back\) are needed to reach 100% recovery, ranging from 29% for Qwen\-7B \(last 8/28\) to 81% for Llama\-8B \(last 26/32\)\. The remaining models fall in between: Llama\-3B \(last 14/28, 50%\), Phi\-14B \(last 20/40, 50%\), Qwen\-14B \(last 22/48, 46%\), and Phi\-4B \(last 17/32, 53%\)\. However, this analysis is only conducted on DAT scores, and it is unclear whether the last 51% of layers are sufficient to recover 100% of the scores on the AUT and Task Task\.

#### Prefix\.

Applying CN weights from the front \(layers0throughkk\) is far less efficient\. On average, 78% of all layers must be included before the prefix condition reaches 95% recovery\. For Qwen\-14B \(48 layers\), the prefix condition never reaches 95% even when all layers are included \(94% atk=48k=48\)\. This asymmetry provides evidence that the CN effect is concentrated in later layers of the residual stream\.

#### Single\-layer\.

No single layer is sufficient to recover the full CN effect\. The best individual layers recover 50–72% of the effect \(e\.g\., Q7B layer 19: 72%, L3B layer 15: 64%, P14B layer 20: 60%\), with top contributors generally appearing in middle\-to\-late layers\. The gap between the best single layer and the suffix provides evidence that the CN effect requires cooperation across multiple late layers, consistent with findings that representations of human\-interpretable concepts or behaviors may span multiple layers\(Lindseyet al\.,[2024](https://arxiv.org/html/2607.01433#bib.bib42); Sharkeyet al\.,[2025](https://arxiv.org/html/2607.01433#bib.bib41)\)\.

## Appendix DHyperparameter Sweeps

CreativityNeuro introduces two hyperparameters: the*importance threshold*ρ∈\(0,1\]\\rho\\in\(0,1\], which controls the fraction of parameters selected by the importance mask, and the*scaling factor*α\>0\\alpha\>0, which controls the magnitude of the weight perturbation applied to masked parameters\. To identify effective\(ρ,α\)\(\\rho,\\alpha\)configurations for each model, we conduct systematic grid sweeps evaluated on the DAT\. For each model, we generate CN weight masks using six different prompt sets:dat,story,ideation,problem,openended, andminimal\. Each prompt set produces a separate importance mask\. We then evaluate every combination of keep ratioρ∈\{0\.01,0\.05,0\.1,0\.2\}\\rho\\in\\\{0\.01,0\.05,0\.1,0\.2\\\}and scaling factorα∈\{0\.1,0\.5,1\.0,2\.0\}\\alpha\\in\\\{0\.1,0\.5,1\.0,2\.0\\\}at three sampling temperaturesT∈\{0\.9,1\.0,1\.2\}T\\in\\\{0\.9,1\.0,1\.2\\\}, withtop\-​k=0\\text\{top\-\}k=0andtop\-​p=1\.0\\text\{top\-\}p=1\.0\(i\.e\., untruncated sampling\)\. For LLaMA\-3\.1\-8B and Phi\-3\.5\-mini, after initial experiments revealed thatα\>0\.5\\alpha\>0\.5were too high, we additionally tested a finer alpha gridα∈\{0\.01,0\.05,0\.075,0\.1,0\.5,1\.0\}\\alpha\\in\\\{0\.01,0\.05,0\.075,0\.1,0\.5,1\.0\\\}to probe the conservative regime identified byChristet al\.\([2025](https://arxiv.org/html/2607.01433#bib.bib22)\)\. Each configuration generatesn=120n=120valid DAT samples \(10\-word lists with all words present in the GloVe vocabulary\)\.

For each\(ρ,α,T\)\(\\rho,\\alpha,T\)triple, we compute the mean DAT scores for the CreativityNeuro \(DAT¯CN\\overline\{\\text\{DAT\}\}\_\{\\text\{CN\}\}\) and baseline \(DAT¯base\\overline\{\\text\{DAT\}\}\_\{\\text\{base\}\}\) models, then convert each to a human percentile using the distribution fromWanget al\.\([2025](https://arxiv.org/html/2607.01433#bib.bib1)\)\.[Figures8](https://arxiv.org/html/2607.01433#A4.F8),[9](https://arxiv.org/html/2607.01433#A4.F9),[10](https://arxiv.org/html/2607.01433#A4.F10),[11](https://arxiv.org/html/2607.01433#A4.F11),[12](https://arxiv.org/html/2607.01433#A4.F12)and[13](https://arxiv.org/html/2607.01433#A4.F13)showΔ\\DeltaPercentile=P​\(DAT¯CN\)−P​\(DAT¯base\)=P\(\\overline\{\\text\{DAT\}\}\_\{\\text\{CN\}\}\)\-P\(\\overline\{\\text\{DAT\}\}\_\{\\text\{base\}\}\)\(averaged across temperatures\) as a function of\(α,ρ\)\(\\alpha,\\rho\)for each of the six prompt sets, across all six models\. Positive values indicate that CN moves the model’s DAT performance upward in the human score distribution\.

![Refer to caption](https://arxiv.org/html/2607.01433v1/figures/sensitivity_dat.png)Figure 8:Sensitivity to\(α,ρ\)\(\\alpha,\\rho\)— DAT prompt set\.Δ\\DeltaPercentile averaged across three temperatures \(T∈\{0\.9,1\.0,1\.2\}T\\in\\\{0\.9,1\.0,1\.2\\\}\) for each model\. Color scale is shared across panels and centered at zero\.![Refer to caption](https://arxiv.org/html/2607.01433v1/figures/sensitivity_story.png)Figure 9:Sensitivity to\(α,ρ\)\(\\alpha,\\rho\)— Story prompt set\.MeanΔ\\DeltaDAT averaged across three temperatures \(T∈\{0\.9,1\.0,1\.2\}T\\in\\\{0\.9,1\.0,1\.2\\\}\) for each model\. Color scale is shared across panels and centered at zero\.![Refer to caption](https://arxiv.org/html/2607.01433v1/figures/sensitivity_ideation.png)Figure 10:Sensitivity to\(α,ρ\)\(\\alpha,\\rho\)— Ideation prompt set\.MeanΔ\\DeltaDAT averaged across three temperatures \(T∈\{0\.9,1\.0,1\.2\}T\\in\\\{0\.9,1\.0,1\.2\\\}\) for each model\. Color scale is shared across panels and centered at zero\.![Refer to caption](https://arxiv.org/html/2607.01433v1/figures/sensitivity_problem.png)Figure 11:Sensitivity to\(α,ρ\)\(\\alpha,\\rho\)— Problem prompt set\.MeanΔ\\DeltaDAT averaged across three temperatures \(T∈\{0\.9,1\.0,1\.2\}T\\in\\\{0\.9,1\.0,1\.2\\\}\) for each model\. Color scale is shared across panels and centered at zero\.![Refer to caption](https://arxiv.org/html/2607.01433v1/figures/sensitivity_openended.png)Figure 12:Sensitivity to\(α,ρ\)\(\\alpha,\\rho\)— Open\-ended prompt set\.MeanΔ\\DeltaDAT averaged across three temperatures \(T∈\{0\.9,1\.0,1\.2\}T\\in\\\{0\.9,1\.0,1\.2\\\}\) for each model\. Color scale is shared across panels and centered at zero\.![Refer to caption](https://arxiv.org/html/2607.01433v1/figures/sensitivity_minimal.png)Figure 13:Sensitivity to\(α,ρ\)\(\\alpha,\\rho\)— Minimal prompt set\.MeanΔ\\DeltaDAT averaged across three temperatures \(T∈\{0\.9,1\.0,1\.2\}T\\in\\\{0\.9,1\.0,1\.2\\\}\) for each model\. Color scale is shared across panels and centered at zero\.
## Appendix EEvaluation Prompts

In this section, we share the full prompts used to generate and/or post\-process the stimuli for the DAT, AUT, and TT\.

### E\.1Divergent Association Task \(DAT\)

The DAT prompt instructs models to generate 10 maximally dissimilar nouns:

DAT PromptYou will be asked to name exactly 10 English nouns\. Output exactly 10 words, separated by commas, and nothing else\. Return ONLY: word1, word2, \.\.\., word10 DO NOT write explanations\. DO NOT think step by step\.Be as original and unusual as possible\. Avoid common or closely related words\. Now name 10 English nouns that are as different from each other as possible\.

### E\.2Alternative Uses Test \(AUT\)

The AUT uses a system prompt and a user prompt, with\{object\}replaced by one of 3 standard objects:*brick*,*paperclip**fork*\.

AUT System PromptYou are participating in a creativity test\. Your task is to generate creative, unusual, and original uses for common objects\. Be imaginative and think outside the box\.

AUT User PromptList 5 creative and unusual alternative uses for a \{object\}\. Be specific and creative\. List each use on a new line, numbered 1 through 5\. Only list the uses, no explanations\.

### E\.3Task Task \(TT\)

The Task Task uses a system prompt and a user prompt with three in\-context examples fromChuet al\.\([2024](https://arxiv.org/html/2607.01433#bib.bib36)\)\.

TT System PromptYou are a creative game show designer\. Your task is to invent fun, original, and entertaining challenges that would be exciting for contestants to attempt and audiences to watch\.

TT User PromptYou’ve been recruited to help design challenge tasks for a new game show\! Your job is to come up with a new creative, silly, and fun task for humans to solve\.Here are a few example tasks: 1\. Your goal is to: Throw a teabag into a mug from the farthest distance\. You can use: Anything you can reasonably expect to find in a house, garage, and garden shed\. 2\. Someone has squeezed all of the toothpaste out of the toothpaste tube\. Your goal is to: Get as much of the original toothpaste back into the empty toothpaste tube as possible\. You can use: Anything you can reasonably expect to find in a bathroom\. 3\. Your goal is to: Transfer water between two fishbowls using only the supplied items\. You cannot move the fishbowls\. You can use: a chocolate bar, a rubber glove, a baguette, a snorkel, a cardboard tube, and a plate of pasta\. Now it’s your turn\! Create your own creative, silly, and fun task for future participants to solve\. Specify the goal, scoring criteria, and any materials or constraints\. Describe it in exactly 3\-\-4 sentences as a short paragraph \-\-\- do not use lists or bullet points\. Do not comment on how entertaining, creative, or fun the task would be\.

### E\.4Task Task Post\-Processing

Raw Task Task generations are post\-processed using GPT\-4o to normalize formatting before human evaluation\.

TT Post\-Processing System PromptYou are an editor cleaning game show challenge descriptions for a human evaluation study\.Make these MINIMAL edits: 1\. REMOVE any task title at the start \(e\.g\., ‘In "Baking Bonanza,"’\)\. After removing, capitalize the first word of the remaining text\. 2\. Convert ALL third\-person references to SECOND PERSON \(e\.g\., ‘‘contestants must’’ →\\rightarrow‘‘you must’’, ‘‘the player’’→\\rightarrow‘‘you’’\)\.3\. REMOVE any metacommentary sentences \(e\.g\., ‘‘This creative juggling act combines laughs with a dash of physics comedy\.’’\)\. CRITICAL RULES: \- Do NOT summarize, condense, or shorten the task description \- Do NOT change the creative content or game mechanics \- Do NOT add any text \-\-\- only edit or remove \- Output ONLY the cleaned description with no preamble or explanation

TT Post\-Processing User PromptClean this game show challenge description: \{text\}

## Appendix FNovelty–Utility Tradeoff Analysis

Among baseline AUT stimuli, originality and utility are negatively correlated \(r=−\.49r=\-\.49,p<10−7p<10^\{\-7\}\), as are surprise and utility \(r=−\.64r=\-\.64,p<10−13p<10^\{\-13\}\)\. As predicted by fundamental novelty–utility tradeoffs established in the creativity literature\(Varshney,[2019](https://arxiv.org/html/2607.01433#bib.bib10)\), increases in subjective ratings of novelty tend to be accompanied by decreases in perceived utility\. Therefore, CN’s utility reduction is an expected byproduct of steering towards high\-novelty responses, rather than an independent failure mode\. To confirm this, we perform a hypervolume \(HV\) analysis\(Caoet al\.,[2015](https://arxiv.org/html/2607.01433#bib.bib63)\)over all responses in the three\-dimensional rating space \(originality, surprise, utility\) and find that the CN HV indicator is 15\.2% larger than baseline \(p=\.18p=\.18\), suggesting CN obtains a moderate \(but non\-significant\) gain in Pareto efficiency as well\.

## Appendix GDisclosure of Large Language Model Usage

In this paper, large language models \(LLMs\) were used to assist in the code implementation, plotting and figure generation, reporting \(but not analysis\) of experiment results, and for comprehensive surveys of related work\.