Authority Bias in Conversational Search Engines for Academic Paper Recommendation

# Authority Bias in Conversational Search Engines for Academic Paper Recommendation
Source: [https://arxiv.org/html/2609.00248](https://arxiv.org/html/2609.00248)
Parsa GhazvinianAffiliation:Georgia State UniversityEmail:[bampel@gsu\.edu](mailto:)Anjila BudathokiAffiliation:University of Tennessee, KnoxvilleEmail:[rsunderraman@gsu\.edu](mailto:)Benjamin M\. AmpelAffiliation:Georgia State UniversityEmail:[pghazvinian1@student\.gsu\.edu](mailto:)Rajshekhar SunderramanAffiliation:Georgia State UniversityEmail:[abudatho@vols\.utk\.edu](mailto:)Yi DingAffiliation:University of Tennessee, KnoxvilleEmail:[yding@utk\.edu](mailto:)

###### Abstract

Large Language Models \(LLMs\) are increasingly used as conversational search engines for academic literature, yet whether they judge papers on content or on authority signals has not been tested causally\. We investigate*authority bias*: systematic preference for papers based on author prestige, venue, and citations rather than content\. Holding title and abstract constant, we vary authority metadata across three counterfactual conditions \(original, flipped, boosted\) over eight LLMs \(five open\-weight and three frontier closed\-weight\) in an in\-context, single\-turn, top\-1 recommendation setting\. Our experiments show that authority bias is substantial and directional, varies markedly across models, and is only partially addressable through prompt\-level debiasing\. We further document a*say\-do gap*: debiasing instructions suppress authority*mentions*far faster than authority\-driven*flips*, so surface auditing systematically underestimates behavioral bias\.111Code:[https://github\.com/jinaduuthman/Authority\-Bias\-In\-Conversational\-Search\-Engine](https://github.com/jinaduuthman/Authority-Bias-In-Conversational-Search-Engine) Data:[https://huggingface\.co/datasets/uthmanjinadu/authority\-bias\-paper\-recommendation](https://huggingface.co/datasets/uthmanjinadu/authority-bias-paper-recommendation)

††∗Corresponding author\.## 1Introduction

Large Language Models \(LLMs\) are rapidly becoming a primary interface for academic literature discovery\. Tools built on systems such as GPT\-4\([Achiam et al\., 2023](https://arxiv.org/html/2609.00248#bib.bib2)\)\(including ChatGPT, Perplexity, Elicit, and Semantic Scholar’s AI features\([Ammar et al\., 2018](https://arxiv.org/html/2609.00248#bib.bib5)\)\) accept natural\-language research queries and return synthesized recommendations with justifications\. Unlike keyword\-based retrieval engines such as Google Scholar or PubMed\([Gusenbauer and Haddaway, 2020](https://arxiv.org/html/2609.00248#bib.bib24)\), these systems act as*recommenders*\([Wu et al\., 2024](https://arxiv.org/html/2609.00248#bib.bib58)\): they internally score candidate papers, select one or more to highlight, and articulate reasons for their choices\. This shift delegates a substantial portion of relevance judgment from the researcher to the model, on the implicit assumption that the model evaluates papers on*content*\([Manning et al\., 2008](https://arxiv.org/html/2609.00248#bib.bib39)\)rather than on*authority signals*– author h\-indices, venue prestige, citation counts, and institutional affiliations, which are pervasive in web\-crawled training data\([Bender et al\., 2021](https://arxiv.org/html/2609.00248#bib.bib9);[Dodge et al\., 2021](https://arxiv.org/html/2609.00248#bib.bib14)\)\.

That assumption is fragile\.[Algaba et al\. \(2025\)](https://arxiv.org/html/2609.00248#bib.bib4)show that LLMs reproduce human citation patterns with a*heightened*citation bias when generating references;[Barolo et al\. \(2025\)](https://arxiv.org/html/2609.00248#bib.bib8)find that LLMs identifying top experts in physics favor senior, highly\-cited researchers; and[Howell et al\. \(2025\)](https://arxiv.org/html/2609.00248#bib.bib28)report a similar prestige\-over\-merit pattern in LLM\-driven peer review\. These findings span distinct LLM\-mediated academic tasks and echo the Matthew Effect\([Merton, 1968](https://arxiv.org/html/2609.00248#bib.bib43)\)and the experimental peer\-review evidence of[Tomkins et al\. \(2017\)](https://arxiv.org/html/2609.00248#bib.bib54)for human reviewers\. None of them, however, targets paper recommendation in conversational search, and none establishes the*causal*effect of authority metadata: correlational designs cannot rule out confounding between authority and content\([Pearl, 2009](https://arxiv.org/html/2609.00248#bib.bib48)\), and single\-dimension manipulations leave open whether the effect generalizes across signals or operates through one cue \(e\.g\., affiliation\) in particular\.

We close this gap with a content\-controlled counterfactual audit\. Following the correspondence\-testing logic of labor\-market audit studies\([Bertrand and Mullainathan, 2004](https://arxiv.org/html/2609.00248#bib.bib10)\)and the counterfactual bias framework formalized for LLMs\([Huang et al\., 2025](https://arxiv.org/html/2609.00248#bib.bib29)\), we hold paper content \(title and abstract\) constant and vary only the authority metadata\. Any recommendation change is then attributable to authority signals by construction\. To decompose authority into measurable signals rather than predetermined weights\([Ioannidis et al\., 2019](https://arxiv.org/html/2609.00248#bib.bib30), which scientometric indicators rely on; e\.g\.,\), we run a 15,000\-run pilot across three models in which one paper’s content is paired with each of the 10 candidate papers’ metadata in turn \(the 1:N flip design, formally introduced in §[4\.3](https://arxiv.org/html/2609.00248#S4.SS3)\)\. Logistic regression with z\-score standardized predictors\([Hosmer and Lemeshow, 2000](https://arxiv.org/html/2609.00248#bib.bib27);[Menard, 2004](https://arxiv.org/html/2609.00248#bib.bib42)\)and dominance analysis\([Budescu, 1993](https://arxiv.org/html/2609.00248#bib.bib11);[Azen and Budescu, 2003](https://arxiv.org/html/2609.00248#bib.bib7)\)converge on a stable rank ordering of five authority dimensions – venue prestige, median author h\-index, maximum author h\-index, citation count, and institutional affiliation, which together define the composite authority score used downstream\.

We then construct three content\-identical conditions that test distinct mechanisms:*original*\(authentic metadata\),*flipped*\(high↔\\leftrightarrowlow authority swap, testing whether models follow metadata over content\), and*boosted*\(mid\-tier papers paired with elite metadata, testing attraction to inflated prestige\)\. Combining these with eight LLMs – five open\-weight models served via Ollama and three frontier closed\-weight models \(gpt\-5\.4,gemini\-3\-flash\-preview,claude\-sonnet\-4\-6\) accessed via providers’ APIs – three instruction variants \(neutral, mild debiasing, strong debiasing\), and 250 queries spanning 25 computer\-science topics yields 17,898 parsed observations\. Throughout, the task is in\-context, single\-turn, top\-1 recommendation: papers are supplied in the prompt rather than retrieved, each query is a single stateless request, and the model returns one paper\. This factorial design addresses three research questions \(§[4\.1](https://arxiv.org/html/2609.00248#S4.SS1)\): whether authority bias exists and is dose\-responsive \(RQ1\), whether changes systematically move toward higher authority \(RQ2\), and whether prompt\-level debiasing mitigates the effect \(RQ3\)\.

#### Our contributions are as follows:

1. 1\.A content\-controlled counterfactual methodology for causally measuring authority bias in LLM paper recommendation, in which authority weights are derived empirically from a flip pilot run via logistic regression and dominance analysis rather than set arbitrarily\.
2. 2\.A factorial benchmark spanning open\- and closed\-weight LLMs: 17,898 paper\-recommendation evaluations across eight models \(five open\-weight and three frontier closed\-weight, from seven developer organizations\), three counterfactual conditions, three instruction variants, and 25 CS topics, publicly released for reuse\.
3. 3\.Two diagnostic phenomena not previously isolated for LLM recommenders: \(i\) a*say\-do gap*, in which debiasing instructions reduce authority language in justifications substantially more than they reduce authority\-driven flips in decisions; and \(ii\) a*frontier\-tier backfire*, in which mild anti\-authority prompting*increases*flip rate across three frontier closed\-weight models from three independent vendors while reducing it for a smaller\-tier ablation and most open\-weight models\.
4. 4\.An empirical bias profile across models, signals, and topics that identifies venue prestige \(not institutional affiliation\) as the dominant authority signal, an open–closed\-weight susceptibility gap, and substantial cross\-topic heterogeneity – negating common assumptions about which prestige dimension drives recommendation bias\([Howell et al\., 2025](https://arxiv.org/html/2609.00248#bib.bib28);[Tomkins et al\., 2017](https://arxiv.org/html/2609.00248#bib.bib54)\)\.

## 2Related Work

#### Generative search engines\.

[Aggarwal et al\. \(2024\)](https://arxiv.org/html/2609.00248#bib.bib3)introduce Generative Engine Optimization \(GEO\), a black\-box framework in which content creators inject SEO\-style cues – added citations, statistics, quotations, or authoritative phrasing – into source pages to enhance their visibility in LLM\-synthesized answers, reporting up to 40% visibility gains on a multi\-domain benchmark\.[Puerto et al\. \(2025\)](https://arxiv.org/html/2609.00248#bib.bib50)\(C\-SEO Bench\) extend this question to conversational settings across question\-answering and product recommendation, finding that dedicated C\-SEO methods are largely ineffective compared to classical SEO and that gains become zero\-sum as more adopters compete\. Both lines target the manipulability of generative search from a content\-creator perspective\. We instead measure the bias the system carries intrinsically, with no external manipulation\.

#### LLM prestige and citation bias\.

Recent work documents prestige effects across distinct LLM\-mediated academic tasks: heightened citation reproduction\([Algaba et al\., 2025](https://arxiv.org/html/2609.00248#bib.bib4)\), senior/high\-citation favoritism in expert naming\([Barolo et al\., 2025](https://arxiv.org/html/2609.00248#bib.bib8)\), affiliation\-driven peer\-review scoring\([Howell et al\., 2025](https://arxiv.org/html/2609.00248#bib.bib28)\), and latent venue preferences across news, e\-commerce, and paper selection\([Khan et al\., 2026](https://arxiv.org/html/2609.00248#bib.bib33)\)\. We instead target*paper recommendation*with a content\-controlled counterfactual that decomposes authority into five empirically weighted signals, and pair it with prompt\-level debiasing across open\- and closed\-weight models\.

#### Counterfactual bias frameworks\.

Counterfactual perturbation has been applied to LLM code generation\([Huang et al\., 2025](https://arxiv.org/html/2609.00248#bib.bib29)\), clinical reasoning\([Ghosh et al\., 2025](https://arxiv.org/html/2609.00248#bib.bib18)\), and reference selection with prompt\-based mitigation\([He, 2025](https://arxiv.org/html/2609.00248#bib.bib25)\); the broader fairness literature\([Gallegos et al\., 2024](https://arxiv.org/html/2609.00248#bib.bib15)\)concentrates on social\-identity attributes\. We adapt the framework to*epistemic authority bias*– bias toward academic prestige – and vary multiple authority signals jointly rather than one at a time\.

#### Position and popularity bias in LLM rankers\.

Authority bias is structurally analogous to popularity bias in collaborative filtering\([Abdollahpouri et al\., 2020](https://arxiv.org/html/2609.00248#bib.bib1)\)and position bias in listwise LLM ranking\([Wang et al\., 2024](https://arxiv.org/html/2609.00248#bib.bib56)\);[Lichtenberg et al\. \(2024\)](https://arxiv.org/html/2609.00248#bib.bib36)additionally report*lower*popularity bias for LLM recommenders than traditional systems in a movie domain, suggesting domain\-dependence that motivates a focus on academic search, where authority signals are multi\-dimensional and fairness implications shape which research gets read and cited\.

#### Bias in LLM\-as\-a\-judge\.

A parallel line studies bias when LLMs evaluate answer quality: audits and surveys catalogue judgement biases including position, verbosity, and authority effects\([Ye et al\., 2025](https://arxiv.org/html/2609.00248#bib.bib59);[Gu et al\., 2024](https://arxiv.org/html/2609.00248#bib.bib22)\), and[Chen et al\. \(2024\)](https://arxiv.org/html/2609.00248#bib.bib12)compare human and LLM judges and find both susceptible\. That work perturbs a candidate answer and asks whether the quality verdict is robust; we instead study recommendation, holding paper content fixed and varying only authority metadata decomposed into five empirically weighted signals\.

## 3Problem Formulation

#### Task formalization\.

We formalize LLM\-based academic paper recommendation as listwise top\-1 selection\([Sun et al\., 2023](https://arxiv.org/html/2609.00248#bib.bib52)\): given an unordered candidate set, the model selects one item\. Equivalently, this is a multi\-class classification problem in which each candidate paper is a class\. Letϕ\\phidenote a large language model\. Given a research queryq∈𝒬q\\in\\mathcal\{Q\}and a candidate set ofnnacademic papers𝒫q=\{p1,…,pn\}\\mathcal\{P\}\_\{q\}=\\\{p\_\{1\},\\ldots,p\_\{n\}\\\}, the model produces a recommendationrq=ϕ⁡\(q,𝒫q,ℐ\)∈𝒫qr\_\{q\}=\\phi\(q,\\mathcal\{P\}\_\{q\},\\mathcal\{I\}\)\\in\\mathcal\{P\}\_\{q\}, whereℐ\\mathcal\{I\}is an instruction variant specifying the recommendation criteria\.

#### Paper representation\.

Each paper decomposes aspi=\(Ci,Mi\)p\_\{i\}=\(C\_\{i\},M\_\{i\}\), whereCi=\(titlei,abstracti\)C\_\{i\}=\(\\text\{title\}\_\{i\},\\text\{abstract\}\_\{i\}\)is thecontentandMi=\(𝐚i,vi,si\)M\_\{i\}=\(\\mathbf\{a\}\_\{i\},v\_\{i\},s\_\{i\}\)is theauthority metadata, comprising author profiles𝐚i=\{\(namej,hj,affj\)\}j=1ki\\mathbf\{a\}\_\{i\}=\\\{\(\\text\{name\}\_\{j\},h\_\{j\},\\text\{aff\}\_\{j\}\)\\\}\_\{j=1\}^\{k\_\{i\}\}\(full name, h\-index, and institutional affiliation of author j\), publication venueviv\_\{i\}, and citation countsis\_\{i\}\.

#### Authority score\.

We define a composite authority scoreα⁡\(pi\)∈\[0,1\]\\alpha\(p\_\{i\}\)\\in\[0,1\]as a weighted sum over five dimensions – venue prestige, median author h\-index, maximum author h\-index, citation count, and institutional affiliation prestige:

α⁡\(pi\)=∑d=15wd⋅x^i,d\\alpha\(p\_\{i\}\)=\\sum\_\{d=1\}^\{5\}w\_\{d\}\\cdot\\hat\{x\}\_\{i,d\}\(1\)wherex^i,d\\hat\{x\}\_\{i,d\}is the min\-max normalized value of dimensionddwithin the paper’s research topic, and the weightswdw\_\{d\}are derived empirically in §[4\.3](https://arxiv.org/html/2609.00248#S4.SS3)\.

#### Authority bias\.

We defineauthority biasas the sensitivity of model recommendations to authority metadata, holding content constant: a recommendation that changes when only the metadata changes is, by construction, evidence of bias\. This sensitivity counts as bias because the pick changes on authority alone, against a content\-relevance request and without disclosure; it would not be bias if the user explicitly requested authority\-aware results such as highly cited work, a case we do not study\. Throughout the paper, we use*susceptibility*as the canonical term for this property and*resistance*as its antonym; both refer to behavior measured by flip rate and boost rate together \(§[4\.7](https://arxiv.org/html/2609.00248#S4.SS7)\):

AuthorityBias​\(ϕ\)=Prq∼𝒬\[ϕ\(q,𝒫qorig,ℐ\)≠ϕ\(q,𝒫qmanip,ℐ\)\]\\begin\{split\}\\text\{AuthorityBias\}\(\\phi\)=\{\}&\\Pr\_\{q\\sim\\mathcal\{Q\}\}\\bigl\[\\phi\(q,\\mathcal\{P\}\_\{q\}^\{\\text\{orig\}\},\\mathcal\{I\}\)\\\\ &\\neq\\phi\(q,\\mathcal\{P\}\_\{q\}^\{\\text\{manip\}\},\\mathcal\{I\}\)\\bigr\]\\end\{split\}\(2\)where𝒫qmanip∈\{𝒫qflip,𝒫qboost\}\\mathcal\{P\}\_\{q\}^\{\\text\{manip\}\}\\in\\\{\\mathcal\{P\}\_\{q\}^\{\\text\{flip\}\},\\mathcal\{P\}\_\{q\}^\{\\text\{boost\}\}\\\}denotes candidate sets with manipulated metadata \(§[4\.4](https://arxiv.org/html/2609.00248#S4.SS4)\)\. This follows the counterfactual bias framework of[Huang et al\. \(2025\)](https://arxiv.org/html/2609.00248#bib.bib29): a system exhibits bias if changing a protected attribute \(here, authority metadata\) while holding task\-relevant features constant \(here, paper content\) changes the output\.

## 4Experimental Setup

### 4\.1Research Questions

RQ1: Do LLM\-based conversational search engines exhibit authority bias when recommending academic papers?We operationalize this through the*flipped*and*boosted*conditions \(§[4\.4](https://arxiv.org/html/2609.00248#S4.SS4)\), and refine it with two sub\-questions: \(1\.1\) whether recommendation probability shows a dose\-response relationship with authority signals \(§[5\.5](https://arxiv.org/html/2609.00248#S5.SS5)\); \(1\.2\) whether different LLMs vary in susceptibility \(§[5\.3](https://arxiv.org/html/2609.00248#S5.SS3)\)\.

RQ2: When recommendations change, do they systematically move toward higher\-authority papers?A high flip rate alone does not establish directionality; we measure whether flips and boosts move toward the higher\-authority paper \(§[5\.2](https://arxiv.org/html/2609.00248#S5.SS2)\)\.

RQ3: Can explicit debiasing instructions mitigate authority bias?We compare three instruction variants \(neutral, mild, strong; §[4\.5](https://arxiv.org/html/2609.00248#S4.SS5)\) and additionally test whether behavioral change tracks rhetorical change \(the say\-do gap; §[5\.7](https://arxiv.org/html/2609.00248#S5.SS7)–§[5\.8](https://arxiv.org/html/2609.00248#S5.SS8)\)\.

### 4\.2Dataset Construction

We collected 1,250 papers across 25 computer\-science research topics using the Semantic Scholar API\([Ammar et al\., 2018](https://arxiv.org/html/2609.00248#bib.bib5)\), with 50 papers per topic \(with publication date range of 2018–2026\), stratified by citation tier \(15 high, 15 mid, 10 low, 10 emerging from 2024–2026\); the rationale for tier\-stratified sampling is to ensure coverage across the authority spectrum \(further details in Appendix[B](https://arxiv.org/html/2609.00248#A2)\)\. Author affiliations are resolved via a three\-phase pipeline: Semantic Scholar paper search, Semantic Scholar batch author lookups, and OpenAlex\([Priem et al\., 2022](https://arxiv.org/html/2609.00248#bib.bib49)\)backfill via DOI matching\. Titles and abstracts are used verbatim, so the content under evaluation is not altered by any LLM rewriting\.

### 4\.3Authority Signal Scoring and Weight Derivation

We score each paper on five authority dimensions: venue prestige \(vv, on a 5\-tier scale from CORE 2026 A∗conferences and top journals down to arXiv preprints\), median author h\-index \(h~\\tilde\{h\}, robust to author\-count effects unlike the mean;[Hirsch, 2005](https://arxiv.org/html/2609.00248#bib.bib26)\), maximum author h\-index \(hmaxh\_\{\\max\}, capturing star\-author effects\), citation count \(ss\), and institutional affiliation prestige \(aa, derived from 4icu\.org / CSRankings plus curated industry\-lab tiers\)\. Each dimension is mapped to\[0,1\]\[0,1\]and then min–max normalized within its research topic so that authority reflects relative standing within a field rather than across fields\. The full tier thresholds are shown in Appendix[A\.6](https://arxiv.org/html/2609.00248#A1.SS6)\.

#### 1:N flip pilot\.

To derive empirical weights, we ran a pilot following the audit\-study methodology of[Bertrand and Mullainathan \(2004\)](https://arxiv.org/html/2609.00248#bib.bib10)\. For 5 of the 25 topics introduced in §[4\.2](https://arxiv.org/html/2609.00248#S4.SS2), we authored 10 natural\-language research queries per topic, each paired with 10 candidate papers from which the model is asked to recommend one\. Within each \(query, candidate set\), we fixed one paper’s content and paired it in turn with each candidate’s authority metadata, generating 15,000 runs across three models \(Gemma 2:9b, Llama 3\.1:8b, Mistral:7b\) under the*Baseline*\(neutral, no\-debiasing\) instruction variant\. We fit a logistic regression\([Hosmer and Lemeshow, 2000](https://arxiv.org/html/2609.00248#bib.bib27)\)with z\-score standardized predictors:

P⁡\(rec=1\)=σ⁡\(CLOSEβ0\+β1​hmax\+β2​h~OPEN\+β3​s\+β4​v\+β5​a\)\\begin\{split\}P\(\\text\{rec\}=1\)=\\sigma\\bigl\(&\\beta\_\{0\}\+\\beta\_\{1\}h\_\{\\max\}\+\\beta\_\{2\}\\tilde\{h\}\\\\ &\{\}\+\\beta\_\{3\}s\+\\beta\_\{4\}v\+\\beta\_\{5\}a\\bigr\)\\end\{split\}\(3\)whereσ\\sigmais the sigmoid and standardization makes\|βk\|\|\\beta\_\{k\}\|directly comparable across signals on different scales\. The final weights average standardized coefficients with dominance\-analysisR2R^\{2\}contributions\([Budescu, 1993](https://arxiv.org/html/2609.00248#bib.bib11);[Azen and Budescu, 2003](https://arxiv.org/html/2609.00248#bib.bib7)\)and are normalized to sum to 1 \(Table[1](https://arxiv.org/html/2609.00248#S4.T1)\)\. The rank ordering \(venue\>\>median h\>\>max h\>\>citations\>\>affiliation\) is stable across single\-model subsets\. Standardization, multicollinearity diagnostics, McFaddenR2R^\{2\}, per\-predictor estimates, and pilot prompt construction are reported in Appendix[A\.7](https://arxiv.org/html/2609.00248#A1.SS7); an author\-identity robustness check is in Appendix[D](https://arxiv.org/html/2609.00248#A4)\.

Table 1:Empirically derived authority signal weights\.

### 4\.4Experimental Conditions

Three conditions are defined, all preserving content \(title \+ abstract\) identically\. \(See Table[2](https://arxiv.org/html/2609.00248#S4.T2)\) Because content is fixed within each item, the*original*condition serves as that item’s own control, so a flip is a within\-item change and requires no external relevance gold standard\. A worked example on a real candidate set is given in Appendix[A\.3](https://arxiv.org/html/2609.00248#A1.SS3)\.

Table 2:Experimental conditions\. Any recommendation change is attributable to authority signals, not content\.
### 4\.5Models and Instruction Variants

We test five open\-weight 7B–9B LLMs locally via Ollama: Llama 3\.1:8b\([Grattafiori et al\., 2024](https://arxiv.org/html/2609.00248#bib.bib20)\), Mistral:7b\([Jiang et al\., 2023](https://arxiv.org/html/2609.00248#bib.bib31)\), Gemma 2:9b\([Gemma Team, 2024](https://arxiv.org/html/2609.00248#bib.bib17)\), Qwen 2\.5:7b\([Qwen Team, 2024](https://arxiv.org/html/2609.00248#bib.bib51)\), and DeepSeek\-R1:8b\([Guo et al\., 2025](https://arxiv.org/html/2609.00248#bib.bib23)\), complemented with three API\-served frontier closed\-weight models from independent vendors: OpenAI’sgpt\-5\.4\([OpenAI, 2026](https://arxiv.org/html/2609.00248#bib.bib46)\), Google’sgemini\-3\-flash\-preview\([Google DeepMind, 2025](https://arxiv.org/html/2609.00248#bib.bib19)\), and Anthropic’sclaude\-sonnet\-4\-6\([Anthropic, 2026](https://arxiv.org/html/2609.00248#bib.bib6)\)\. All eight are queried with provider\-default decoding to measure out\-of\-the\-box behavior\. Endpoints, decoding flags, the reasoning\-toggle handling for closed\-weight models, statelessness across cells, and the smaller\-tiergpt\-4o\-miniablation are in Appendices[A\.4](https://arxiv.org/html/2609.00248#A1.SS4)and[E](https://arxiv.org/html/2609.00248#A5)\.

We construct three instruction variants following the escalating\-instruction pattern of[Tamkin et al\. \(2023\)](https://arxiv.org/html/2609.00248#bib.bib53)and[Ganguli et al\. \(2023\)](https://arxiv.org/html/2609.00248#bib.bib16): a neutral baseline, a*mild*intervention naming the protected authority attributes, and a*strong*content\-only directive \(Table[3](https://arxiv.org/html/2609.00248#S4.T3); Appendix[A\.1](https://arxiv.org/html/2609.00248#A1.SS1)reproduces them verbatim\)\.

Table 3:Instruction variants and debiasing strategies\.
### 4\.6Query Design and Candidate Set Construction

#### Queries and candidate sets\.

For each of the 25 topics we manually authored 10 natural\-language research queries \(250 total\), each targeting a distinct sub\-question; query phrasing mirrors GEO\([Aggarwal et al\., 2024](https://arxiv.org/html/2609.00248#bib.bib3)\)and C\-SEO Bench\([Puerto et al\., 2025](https://arxiv.org/html/2609.00248#bib.bib50)\)\. Each query is paired with 10 candidate papers \(3 top\-tier, 3 mid\-tier, 2 low\-tier, 2 emerging\), presented listwise with full metadata \(title, venue, year, citation count, authors with h\-indices and affiliations, and abstract\) following the listwise protocol of[Sun et al\. \(2023\)](https://arxiv.org/html/2609.00248#bib.bib52)\. To control for position bias\([Wang et al\., 2024](https://arxiv.org/html/2609.00248#bib.bib56)\), candidate papers’ order is seeded once per query and held constant across all three conditions, so any inter\-condition change is attributable to metadata, not order\. Topic list with example queries and the paper\-card template are further discussed in Appendices[B](https://arxiv.org/html/2609.00248#A2),[A\.2](https://arxiv.org/html/2609.00248#A1.SS2), and[A](https://arxiv.org/html/2609.00248#A1)\.

#### Experiment matrix\.

The full matrix,8​models×3​instructions×3​conditions×250​queries=18,0008\\text\{ models\}\\times 3\\text\{ instructions\}\\times 3\\text\{ conditions\}\\times 250\\text\{ queries\}=18\{,\}000target runs, yields 17,898 parsed responses \(99\.4% overall parse rate\)\. Generation settings, the per\-cohort parse\-rate breakdown, and the response\-parsing rule are in Appendices[A\.4](https://arxiv.org/html/2609.00248#A1.SS4)and[A\.5](https://arxiv.org/html/2609.00248#A1.SS5)\.

### 4\.7Evaluation Metrics

We evaluate authority bias along six metrics, each tied to a research question and paired with an appropriate test \(Table[4](https://arxiv.org/html/2609.00248#S4.T4)\)\. The instruction effect uses McNemar’s test\([McNemar, 1947](https://arxiv.org/html/2609.00248#bib.bib41)\)since flip outcomes are paired across query–condition cells; aggregate rates report Wilson confidence intervals\([Wilson, 1927](https://arxiv.org/html/2609.00248#bib.bib57)\)alongside binomial tests\. Throughout, differences between rates are reported in*percentage points*\(pp\), the additive difference between two percentages \(e\.g\., a change from 40% to 28% is−12\-12pp\)\.

Table 4:Evaluation metrics and statistical tests\.

## 5Results

### 5\.1Authority Bias Exists and Is Substantial \(RQ1\)

Table 5:Aggregate recommendation flip rates against the*original*condition, computed across the 8\-model headline set\.*Changed*is the count of \(query, instruction, model\) cells where the recommendation differed from the original;*Rate*is the corresponding percentage\. Both rates are significant atp<0\.001p<0\.001\. CI bounds in percent\.As shown in Table[5](https://arxiv.org/html/2609.00248#S5.T5), 39\.2% of recommendations change when authority metadata is swapped between high\- and low\-prestige papers, even though content is identical; a lower but still substantial 21\.7% change under the*boosted*condition\. The 17\.5pp gap is itself significant \(χ2=427\.5\\chi^\{2\}=427\.5,p<0\.001p<0\.001\): the metadata swap perturbs both ends of the candidate set, while inflation alters only one paper and leaves the remaining authority landscape intact for the model to anchor on\.

### 5\.2Bias Is Directional: Models Follow Authority \(RQ2\)

Table 6:Flip direction analysis\.*Flips*= recommendations that changed from the original;*Higher*= how many of those flips landed on a paper with a larger composite authority score under the manipulated condition \(boosted papers carry their inflated elite score; swapped papers carry their displaced\-tier score\)\. Both directional rates differ from 50/50 atp<0\.001p<0\.001\.As shown in Table[6](https://arxiv.org/html/2609.00248#S5.T6), the two manipulations diverge directionally\. Under*flipped*, only 40\.9% of flips move toward higher composite authority; the metadata swap scrambles the prestige landscape and disperses flips across the candidate set\. Under*boosted*, the pull reverses sharply: 68\.2% of flips favor the higher\-authority side \(p<0\.001p<0\.001\)\. Because the boosted paper is the only candidate with inflated metadata, this skew is direct evidence of authority attraction\. This directional result is robust to the choice of authority weights \(Appendix[A\.8](https://arxiv.org/html/2609.00248#A1.SS8)\)\. Flip rate and directional pull are independent dimensions of susceptibility; per\-model patterns \(including Gemma 2’s low\-flip / high\-pull asymmetry\) are further discussed in Appendix[G](https://arxiv.org/html/2609.00248#A7)\.

### 5\.3Models Differ Significantly in Susceptibility \(Sub\-RQ1\.2\)

Table 7:Per\-model authority bias susceptibility, sorted by flip rate \(*Susc\.*= mean of flip and boost rates\)\. Developers and model citations are given in §[4\.5](https://arxiv.org/html/2609.00248#S4.SS5)\. Chi\-squared test of homogeneity:χ2=479\.4\\chi^\{2\}=479\.4,p<0\.001p<0\.001\.As shown in Table[7](https://arxiv.org/html/2609.00248#S5.T7), susceptibility varies2\.83×2\.83\\times, from 17\.4% \(gpt\-5\.4\) to 49\.2% \(Llama 3\.1\); non\-overlapping confidence intervals confirm the gap is robust\. All three frontier closed\-weight models sit at or below the open\-weight band, with Gemma 2 the only open\-weight model whose interval overlaps theirs; this suggests that frontier closed\-weight models’ post\-training \(instruction tuning, RLHF\([Ouyang et al\., 2022](https://arxiv.org/html/2609.00248#bib.bib47)\), safety fine\-tuning\) reduces authority bias more than the smaller\-scale post\-training the 7B–9B cohort receives\. It does not eliminate the bias: more than one in five closed\-weight models’ recommendations still flip, and Llama 3\.1 exceeds the coin\-flip threshold at 63\.2%\. Models also reach similar susceptibility through different mechanisms \(Mistral 49\.7% flip / 22\.0% boost; DeepSeek\-R1 41\.9% / 35\.4%\)\. The smaller\-tiergpt\-4o\-miniablation in Appendix[E](https://arxiv.org/html/2609.00248#A5)points to frontier\-tier post\-training as the likely lever; per\-model mechanism profiles and our broader training\-data hypothesis are in Appendix[G](https://arxiv.org/html/2609.00248#A7)\.

### 5\.4Cross\-Model Agreement: Shared vs\. Idiosyncratic Bias

We compute pairwise agreement on which paper each model selects and on whether each model flips, using Cohen’sκ\\kappa\([Cohen, 1960](https://arxiv.org/html/2609.00248#bib.bib13)\)for the binary flip indicator across matched cells\.

Table 8:Cross\-model agreement across the 8\-model headline set\. Selection agreement reflects how often two models pick the same paper for an identical \(query, condition, instruction\) cell\. Flip\-agreementκ\\kappacorrects for chance\.As shown in Table[8](https://arxiv.org/html/2609.00248#S5.T8), selection agreement \(43\.9%\) sits far above the∼\\sim10% chance baseline expected when two independent models pick uniformly from 10 candidates\. Authority bias is*partially shared*under flip condition \(κ=0\.143\\kappa=0\.143\) but*essentially uncorrelated*under boosted \(κ=0\.062\\kappa=0\.062\): models flip on the same queries more often than chance, but*which*inflated paper attracts each model is largely model\-specific\. The three frontier closed\-weight models cluster tightly, forming the only cross\-vendor pairs that exceed 60% selection agreement\. Per\-pair numbers and a frontier post\-training interpretation are discussed further in Appendix[G](https://arxiv.org/html/2609.00248#A7)\.

### 5\.5Dose\-Response Relationship \(Sub\-RQ1\.1\)

Table 9:Distribution of recommended papers by underlying authority tier across the three conditions, computed across the 8\-model headline set\. The Share column shows the fixed proportion of each tier in the candidate set\. Top\-tier dominance under*original*and the near\-equalization of top/mid/low under*flipped*are the headline dose\-response signatures\.As shown in Table[9](https://arxiv.org/html/2609.00248#S5.T9), top\-tier papers receive 52\.1% of*original*recommendations despite being only 30% of candidates, a strong baseline preference for high\-authority work\. Under*flipped*, top\-tier preference drops to 32\.5% while low\-tier picks nearly double \(11\.8%→\\to20\.2%\) and mid\-tier rises to 29\.0%, substantially flattening the authority gradient\. Under*boosted*the top\-tier share moves by only 3pp because only the targeted mid\-tier paper carries inflated metadata\.

Aggregated, boosted papers are selected at 27\.1% versus 25\.6% expected by chance \(1\.06×1\.06\\timeslift,p<0\.001p<0\.001\)\. Per\-model lifts span0\.92×0\.92\\times\(gpt\-5\.4, claude\-sonnet\-4\-6\) to1\.43×1\.43\\times\(Qwen 2\.5\); the four least\-susceptible models on flip rate are also the four with the lowest boost lifts, so resistance to inflation tracks resistance to swap\. Per\-model boost lifts and baseline\-pick authority profiles are in Appendix[G](https://arxiv.org/html/2609.00248#A7)\.

### 5\.6Topic\-Level Variation

As shown in Table[10](https://arxiv.org/html/2609.00248#S5.T10), flip rates vary from 15\.7% \(Attention Mechanisms\) to 57\.8% \(Text Summarization\), a 42\.1pp spread\. Cross\-topic interactions are also substantial: Generative Adversarial Networks pairs a low flip rate \(28\.4%\) with the second\-highest boost rate \(29\.8%\), and Explainable AI tops the boost ranking \(33\.6%\) while sitting mid\-rank on flips, so swap\- and inflation\-susceptibility are partially independent\. The full 25\-topic ranking is in Appendix[C](https://arxiv.org/html/2609.00248#A3)\.

Table 10:Top\-5 most susceptible and bottom\-5 least susceptible research topics on the 8\-model headline set, ordered by flip rate\. The full 25\-topic ranking appears in Appendix[C](https://arxiv.org/html/2609.00248#A3)\.
### 5\.7Debiasing Instructions Help but Do Not Solve \(RQ3\)

Table 11:Aggregate instruction effect on flip rates across the 8\-model headline set\.As shown in Table[11](https://arxiv.org/html/2609.00248#S5.T11), the strong*content\-first*instruction reduces flip rate by 12\.9pp, but a 31\.4% residual persists; prompt\-based debiasing only partially reduces prestige bias\([He, 2025](https://arxiv.org/html/2609.00248#bib.bib25);[Tamkin et al\., 2023](https://arxiv.org/html/2609.00248#bib.bib53)\)\. The mild*anti\-authority*aggregate effect is much smaller \(−2\.3\-2\.3pp; McNemarp=0\.059p=0\.059\) for a reason visible per\-model\.

Table 12:Per\-model effect of the*mild*anti\-authority instruction \(baseline→\\toanti\-authority flip rate\)\. Bolded rows: the three frontier closed\-weight models, all of which backfire \(flip rate*rises*under the instruction\)\. All three frontier closed\-weight models backfire, replicating across three independent labs \(OpenAI, Google DeepMind, Anthropic\)\. Mistral:7b shows a smaller backfire \(\+2\.0\+2\.0pp\); the four remaining open\-weight models reduce flip rate\.As shown in Table[12](https://arxiv.org/html/2609.00248#S5.T12), all three frontier closed\-weight models*backfire*under the mild instruction \(Table[3](https://arxiv.org/html/2609.00248#S4.T3)\) \(\+2\.4\+2\.4to\+4\.8\+4\.8pp\), while the smaller\-tiergpt\-4o\-miniablation \(Appendix[E](https://arxiv.org/html/2609.00248#A5)\) reduces as expected \(−3\.6\-3\.6pp\) and four open\-weight models also reduce\. The strong content\-first instruction recovers reductions across all eight models\. Pooled by tier, the split is significant in opposite directions: the three frontier models rise together \(\+3\.9\+3\.9pp, McNemarp=\.02p=\.02\) while the open\-weight cohort falls in aggregate \(−5\.7\-5\.7pp,p<\.001p<\.001\); no single frontier model is individually significant \(per\-modelp=\.10p=\.10–\.45\.45\), so we make the backfire claim at the group level\. A second asymmetry: instructions barely move boost rate \(21\.8%→\\to21\.0%\) even when flip rate falls 12\.9pp, so prompting helps models resist*swapped*but not*inflated*authority\. Cross\-vendor independence of the backfire and the citation\-gaming implications are discussed further in Appendix[G](https://arxiv.org/html/2609.00248#A7)\.

### 5\.8Justification Analysis: The Say\-Do Gap

Table 13:The say\-do gap on the 8\-model headline set\. Authority*mentions*\(what the model says\) collapse far more under debiasing instructions than authority\-driven*flips*\(what the model does\)\. The 11\.1pp gap between the−24\.0\-24\.0pp drop in mentions and the−12\.9\-12\.9pp drop in flip rate is the say\-do gap\.As shown in Table[13](https://arxiv.org/html/2609.00248#S5.T13), debiasing instructions suppress authority*mentions*far faster than authority\-driven*flips*: an 11\.1pp gap between the−24\.0\-24\.0pp drop in mentions and the−12\.9\-12\.9pp drop in flip rate\. Across all three counterfactual conditions,∼\\sim18% of justifications mention authority markers \(16\.6–18\.7%\)\. Authority\-marker categories, per\-model mention rates and their loose coupling to behavior, and the interpretation in terms of RLHF surface effects\([Ouyang et al\., 2022](https://arxiv.org/html/2609.00248#bib.bib47)\)are in Appendix[G](https://arxiv.org/html/2609.00248#A7)\.

A consolidated statistical summary across all RQs is provided in Appendix[F](https://arxiv.org/html/2609.00248#A6); nine of ten primary tests reachp<0\.001p<0\.001, with the sole exception being the mild anti\-authority instruction’s aggregate effect \(p=0\.059p=0\.059\), offset by the cross\-vendor frontier backfire reported in §[5\.7](https://arxiv.org/html/2609.00248#S5.SS7)\.

## 6Discussion

#### Venue prestige is the dominant signal\.

The pilot 1:N flip runs \(Table[1](https://arxiv.org/html/2609.00248#S4.T1)\) identify venue prestige \(weight = 0\.353\) as the strongest single authority signal, followed by median h\-index \(0\.292\) and max h\-index \(0\.187\), with institutional affiliation near zero \(0\.031\) after controlling for the others\. This negates the common assumption that institutional prestige drives LLM bias\([Howell et al\., 2025](https://arxiv.org/html/2609.00248#bib.bib28)\)and extends[Tomkins et al\. \(2017\)](https://arxiv.org/html/2609.00248#bib.bib54)’s peer\-review finding to LLM\-based evaluation\. The effect is plausibly amplified by surface properties of the signal: venue names are short, high\-frequency tokens \(“NeurIPS”, “ICLR”\) that co\-occur explicitly with quality judgments throughout academic web text, while h\-indices and citations appear as raw numerics and affiliations are long\-tail\.

#### Implicit bias and the say\-do gap\.

The most consequential finding for practitioners is the say\-do gap \(Table[13](https://arxiv.org/html/2609.00248#S5.T13)\): instructions reduce authority*mentions*by∼\\sim24\.0pp but reduce authority*behavior*by only 12\.9pp\. Surface\-level auditing therefore systematically underestimates behavioral bias, analogous to the gap between stated attitudes and revealed behavior studied in implicit\-bias measurement\([Greenwald and Banaji, 1995](https://arxiv.org/html/2609.00248#bib.bib21)\); we intend this only as an analogy to that measurement gap, not a claim that models hold attitudes or mental states\. Prompt\-level debiasing alone is insufficient; architectural or training\-level interventions are likely necessary\.

#### Frontier\-tier closed\-weight models: reduce but do not eliminate, and can backfire\.

The three frontier closed\-weight models all sit below the open\-weight band \(§[5\.3](https://arxiv.org/html/2609.00248#S5.SS3)\), and thegpt\-4o\-miniablation \(Appendix[E](https://arxiv.org/html/2609.00248#A5)\) is consistent with this being a property of frontier\-tier post\-training rather than closed\-weightness per se, though we do not test the mechanism directly\. The bias is not eliminated, however, and all three frontier models exhibit a cross\-vendor backfire under mild anti\-authority prompting \(§[5\.7](https://arxiv.org/html/2609.00248#S5.SS7); Table[12](https://arxiv.org/html/2609.00248#S5.T12)\): partial debiasing prompts can be worse than no prompt on the most capable models, so debiasing language for production environments should be empirically validated rather than assumed to act monotonically with its explicitness\. The strongest in\-cohort mitigation is frontier\-tier selection combined with the content\-first instruction\.

#### Topic susceptibility, equity, and gameability\.

The 42\.1pp topic spread \(§[5\.6](https://arxiv.org/html/2609.00248#S5.SS6); Table[10](https://arxiv.org/html/2609.00248#S5.T10)\) means a single corpus\-wide bias rate misrepresents what users encounter; audits should disaggregate by research area\. The dominance of venue prestige implies a compounding equity risk: under\-recommendation of work from emerging researchers, smaller institutions, and newer publications amplifies the Matthew Effect through LLM\-mediated discovery\. The 68\.2% directional pull under*boosted*\(Table[6](https://arxiv.org/html/2609.00248#S5.T6)\) also has a gameability reading: the model\-side property we measure intrinsically and the GEO literature\([Aggarwal et al\., 2024](https://arxiv.org/html/2609.00248#bib.bib3);[Puerto et al\., 2025](https://arxiv.org/html/2609.00248#bib.bib50)\)on creator\-side manipulation describe two sides of the same weakness\.

## 7Conclusion

Using a content\-controlled counterfactual audit across eight LLMs, we show that 39\.2% of academic\-paper recommendations change when only authority metadata changes, with bias directionally pulled toward higher prestige and susceptibility varying2\.83×2\.83\\timesacross models\. Frontier closed\-weight models sit below the open\-weight band but exhibit a cross\-vendor backfire under mild anti\-authority prompts, and a say\-do gap persists where instructions reduce authority*language*faster than authority*behavior*\. Venue prestige, not institutional affiliation, is the dominant signal\. These findings argue for architectural and training\-level interventions beyond prompt engineering, and for adding academic prestige as a protected dimension in LLM fairness audits\.

## 8Limitations

Our model coverage spans eight LLMs across both deployment tiers, but is bounded along several dimensions\. The open\-weight cohort is in the 7–9B band, so larger open\-weight models \(70B\+\) are not covered, and the three frontier closed\-weight models are run in non\-reasoning mode \(Gemini’sthinkingBudgetis set to 0;gpt\-5\.4andclaude\-sonnet\-4\-6use their default chat / messages mode\) to keep the listwise direct\-answer protocol comparable across models\. Whether explicit reasoning would improve resistance to authority bias is left to future work, and quantitative open\-vs\.\-closed comparisons reflect*deployed*model behavior rather than mechanistic differences, since the closed\-weight post\-training mixture is not public\.

The experimental task is also bounded\. We prompt the model for a single top\-1 recommendation per query under a single\-turn stateless request, with papers presented in\-context rather than through a retrieval\-augmented pipeline\([Lewis et al\., 2020](https://arxiv.org/html/2609.00248#bib.bib35)\); top\-kkranking, multi\-turn dialogue, and an actual retrieval stage may all expose different patterns\. Queries are hand\-crafted natural\-language research questions rather than samples from real search logs, all 25 topics are in computer science, and citation counts correlate with paper age, a recency effect we mitigate through stratified sampling \(including a 2024–2026 “emerging” band\) but cannot eliminate\. Authority dynamics in other disciplines \(medicine, social sciences\) plausibly differ in both signal availability and signal weighting\. A human relevance or quality check on a subset of queries would further separate authority effects from cases where the model’s original pick was already weak; because such relevance judgments are themselves subjective and noisy, they would benefit from noise\-correction methods for subjective labels\([Jinadu and Ding, 2024](https://arxiv.org/html/2609.00248#bib.bib32)\), which we leave to future work\.

We also use the title and abstract as the paper’s content, which matches how conversational search tools like Perplexity, Elicit, and Semantic Scholar’s AI features actually work: they rank candidates using title, abstract, and metadata, not the full paper text\([Sun et al\., 2023](https://arxiv.org/html/2609.00248#bib.bib52)\)\. The original author’s writing style stays the same across all three conditions, so any flips we observe come from the metadata change rather than from differences in how the title or abstract is written\. A version that gave models the full paper text could reduce the bias we measure by giving them more content to anchor on; our numbers should therefore be read as the bias visible under the typical recommendation setup, not as an upper bound on what a full\-text\-aware model would do\.

## 9Ethical Considerations

This study uses only publicly available paper metadata from Semantic Scholar and OpenAlex\. No human subjects are involved\. The metadata manipulations are performed for experimental purposes only and are not published or disseminated as real paper information\. We acknowledge that our findings could theoretically be used to game LLM\-based recommendation systems; however, we believe the greater benefit lies in exposing these biases so that system designers can mitigate them\. Our code is publicly available at[https://github\.com/jinaduuthman/Authority\-Bias\-In\-Conversational\-Search\-Engine](https://github.com/jinaduuthman/Authority-Bias-In-Conversational-Search-Engine), and the dataset \(papers, queries, candidate sets, and model responses\) at[https://huggingface\.co/datasets/uthmanjinadu/authority\-bias\-paper\-recommendation](https://huggingface.co/datasets/uthmanjinadu/authority-bias-paper-recommendation)\.

## 10Acknowledgements

Research was sponsored by the Army Research Laboratory and was accomplished under Cooperative Agreement Number W911NF\-23\-2\-0224\. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Laboratory or the U\.S\. Government\. The U\.S\. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein\.

## References

- Abdollahpouri et al\. \(2020\)Hamed Abdollahpouri, Masoud Mansoury, Robin Burke, and Bamshad Mobasher\. 2020\.The connection between popularity bias, calibration, and fairness in recommendation\.In*Proceedings of the 14th ACM Conference on Recommender Systems*, pages 726–731\.
- Achiam et al\. \(2023\)Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al\. 2023\.GPT\-4 technical report\.*arXiv preprint arXiv:2303\.08774*\.
- Aggarwal et al\. \(2024\)Pranjal Aggarwal, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, Karthik R\. Narasimhan, and Ameet Deshpande\. 2024\.GEO: Generative engine optimization\.In*Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining*, pages 5–16\.ArXiv:2311\.09735\.
- Algaba et al\. \(2025\)Andres Algaba, Carmen Mazijn, Vincent Holst, Floriano Tori, Sylvia Wenmackers, and Vincent Ginis\. 2025\.Large language models reflect human citation patterns with a heightened citation bias\.In*Findings of the Association for Computational Linguistics: NAACL 2025*\.ArXiv:2405\.15739\.
- Ammar et al\. \(2018\)Waleed Ammar, Dirk Groeneveld, Chandra Bhagavatula, Iz Beltagy, Miles Crawford, Doug Downey, Jason Dunkelberger, Ahmed Elgohary, Sergey Feldman, Vu Ha, Rodney Kinney, Sebastian Kohlmeier, Kyle Lo, Tyler Murray, Hsu\-Han Ooi, Matthew Peters, Joanna Power, Sam Skjonsberg, Lucy Wang, Chris Wilhelm, Zheng Yuan, Madeleine van Zuylen, and Oren Etzioni\. 2018\.Construction of the literature graph in Semantic Scholar\.In*Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 3 \(Industry Papers\)*, pages 84–91\.
- Anthropic \(2026\)Anthropic\. 2026\.Claude Sonnet 4\.6\.Anthropic API model card\.[https://docs\.anthropic\.com/en/docs/about\-claude/models](https://docs.anthropic.com/en/docs/about-claude/models)\.Accessed via the Anthropic Messages API asclaude\-sonnet\-4\-6; provider\-default decoding \(no temperature, top\-pp, or seed override\)\.
- Azen and Budescu \(2003\)Razia Azen and David V\. Budescu\. 2003\.The dominance analysis approach for comparing predictors in multiple regression\.*Psychological Methods*, 8\(2\):129–148\.
- Barolo et al\. \(2025\)Daniele Barolo, Chiara Valentin, Fariba Karimi, Luis Galárraga, Gonzalo G\. Méndez, and Lisette Espín\-Noboa\. 2025\.Whose name comes up? Auditing LLM\-based scholar recommendations\.*arXiv preprint arXiv:2506\.00074*\.
- Bender et al\. \(2021\)Emily M\. Bender, Timnit Gebru, Angelina McMillan\-Major, and Shmargaret Shmitchell\. 2021\.On the dangers of stochastic parrots: Can language models be too big?In*Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency \(FAccT ’21\)*, pages 610–623\.
- Bertrand and Mullainathan \(2004\)Marianne Bertrand and Sendhil Mullainathan\. 2004\.Are Emily and Greg more employable than Lakisha and Jamal? A field experiment on labor market discrimination\.*American Economic Review*, 94\(4\):991–1013\.
- Budescu \(1993\)David V\. Budescu\. 1993\.Dominance analysis: A new approach to the problem of relative importance of predictors in multiple regression\.*Psychological Bulletin*, 114\(3\):542–551\.
- Chen et al\. \(2024\)Guiming Hardy Chen, Shunian Chen, Ziche Liu, Feng Jiang, and Benyou Wang\. 2024\.Humans or LLMs as the judge? a study on judgement bias\.In*Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing \(EMNLP\)*, pages 8301–8327\.
- Cohen \(1960\)Jacob Cohen\. 1960\.A coefficient of agreement for nominal scales\.*Educational and Psychological Measurement*, 20\(1\):37–46\.
- Dodge et al\. \(2021\)Jesse Dodge, Maarten Sap, Ana Marasović, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Margaret Mitchell, and Matt Gardner\. 2021\.Documenting large webtext corpora: A case study on the Colossal Clean Crawled Corpus\.In*Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing*, pages 1286–1305\.
- Gallegos et al\. \(2024\)Isabel O\. Gallegos, Ryan A\. Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K\. Ahmed\. 2024\.Bias and fairness in large language models: A survey\.*Computational Linguistics*, 50\(3\):1097–1179\.
- Ganguli et al\. \(2023\)Deep Ganguli, Amanda Askell, Nicholas Schiefer, Thomas I\. Liao, Kamilė Lukošiūtė, Anna Chen, Anna Goldie, Azalia Mirhoseini, Catherine Olsson, Danny Hernandez, et al\. 2023\.The capacity for moral self\-correction in large language models\.*arXiv preprint arXiv:2302\.07459*\.
- Gemma Team \(2024\)Gemma Team\. 2024\.Gemma 2: Improving open language models at a practical size\.*arXiv preprint arXiv:2408\.00118*\.
- Ghosh et al\. \(2025\)Rajarshi Ghosh, Abhay Gupta, Hudson McBride, Anurag Jayant Vaidya, and Faisal Mahmood\. 2025\.Medequalqa: Evaluating biases in llms with counterfactual reasoning\.In*Proceedings of The First Workshop on Human–LLM Collaboration for Ethical and Responsible Science Production \(SciProdLLM\)*, pages 25–37\.
- Google DeepMind \(2025\)Google DeepMind\. 2025\.Gemini 3 Flash: Preview release\.Google AI for Developers documentation\.[https://ai\.google\.dev/gemini\-api/docs/models](https://ai.google.dev/gemini-api/docs/models)\.Accessed via the Gemini API asgemini\-3\-flash\-previewwiththinkingConfig\.thinkingBudget = 0to disable internal reasoning, December 2025\.
- Grattafiori et al\. \(2024\)Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, et al\. 2024\.The Llama 3 herd of models\.*arXiv preprint arXiv:2407\.21783*\.
- Greenwald and Banaji \(1995\)Anthony G\. Greenwald and Mahzarin R\. Banaji\. 1995\.Implicit social cognition: Attitudes, self\-esteem, and stereotypes\.*Psychological Review*, 102\(1\):4–27\.
- Gu et al\. \(2024\)Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, Yuanzhuo Wang, and Jian Guo\. 2024\.A survey on LLM\-as\-a\-judge\.*arXiv preprint arXiv:2411\.15594*\.
- Guo et al\. \(2025\)Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al\. 2025\.[DeepSeek\-R1 incentivizes reasoning in LLMs through reinforcement learning](https://doi.org/10.1038/s41586-025-09422-z)\.*Nature*, 645:633–638\.
- Gusenbauer and Haddaway \(2020\)Michael Gusenbauer and Neal R\. Haddaway\. 2020\.Which academic search systems are suitable for systematic reviews or meta\-analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources\.*Research Synthesis Methods*, 11\(2\):181–217\.
- He \(2025\)Jiangen He\. 2025\.Who gets cited? Gender\- and majority\-bias in LLM\-driven reference selection\.*arXiv preprint arXiv:2508\.02740*\.
- Hirsch \(2005\)Jorge E\. Hirsch\. 2005\.An index to quantify an individual’s scientific research output\.*Proceedings of the National Academy of Sciences*, 102\(46\):16569–16572\.
- Hosmer and Lemeshow \(2000\)David W\. Hosmer and Stanley Lemeshow\. 2000\.*Applied Logistic Regression*, 2 edition\.Wiley\.
- Howell et al\. \(2025\)Anthony Howell, Jieshu Wang, Luyu Du, Julia Melkers, and Varshil Shah\. 2025\.Prestige over merit: An adapted audit of LLM bias in peer review\.*arXiv preprint arXiv:2509\.15122*\.
- Huang et al\. \(2025\)Dong Huang, Jie M\. Zhang, Qingwen Bu, Xiaofei Xie, Junjie Chen, and Heming Cui\. 2025\.Bias testing and mitigation in llm\-based code generation\.*ACM Transactions on Software Engineering and Methodology*, 35\(1\):1–31\.
- Ioannidis et al\. \(2019\)John P\. A\. Ioannidis, Jeroen Baas, Richard Klavans, and Kevin W\. Boyack\. 2019\.A standardized citation metrics author database annotated for scientific field\.*PLOS Biology*, 17\(8\):e3000384\.
- Jiang et al\. \(2023\)Albert Q\. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie\-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed\. 2023\.Mistral 7B\.*arXiv preprint arXiv:2310\.06825*\.
- Jinadu and Ding \(2024\)Uthman Jinadu and Yi Ding\. 2024\.Noise correction on subjective datasets\.In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 5385–5395\.
- Khan et al\. \(2026\)Mohammad Aflah Khan, Mahsa Amani, Soumi Das, Bishwamittra Ghosh, Qinyuan Wu, Krishna P\. Gummadi, Manish Gupta, and Abhilasha Ravichander\. 2026\.In agents we trust, but who do agents trust? Latent source preferences steer LLM generations\.In*International Conference on Learning Representations \(ICLR\)*\.ArXiv:2602\.15456\.
- Landis and Koch \(1977\)J\. Richard Landis and Gary G\. Koch\. 1977\.The measurement of observer agreement for categorical data\.*Biometrics*, 33\(1\):159–174\.
- Lewis et al\. \(2020\)Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen\-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela\. 2020\.Retrieval\-augmented generation for knowledge\-intensive NLP tasks\.In*Advances in Neural Information Processing Systems 33 \(NeurIPS 2020\)*\.ArXiv:2005\.11401\.
- Lichtenberg et al\. \(2024\)Jan Malte Lichtenberg, Alexander Buchholz, and Pola Schwöbel\. 2024\.Large language models as recommender systems: A study of popularity bias\.In*Proceedings of the Gen\-IR Workshop at SIGIR 2024*\.ArXiv:2406\.01285\.
- Liu et al\. \(2024\)Nelson F\. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang\. 2024\.Lost in the middle: How language models use long contexts\.*Transactions of the Association for Computational Linguistics*, 12:157–173\.
- Mann and Whitney \(1947\)Henry B\. Mann and Donald R\. Whitney\. 1947\.On a test of whether one of two random variables is stochastically larger than the other\.*The Annals of Mathematical Statistics*, 18\(1\):50–60\.
- Manning et al\. \(2008\)Christopher D\. Manning, Prabhakar Raghavan, and Hinrich Schütze\. 2008\.*Introduction to Information Retrieval*\.Cambridge University Press\.
- McFadden \(1974\)Daniel McFadden\. 1974\.Conditional logit analysis of qualitative choice behavior\.In Paul Zarembka, editor,*Frontiers in Econometrics*, pages 105–142\. Academic Press, New York\.
- McNemar \(1947\)Quinn McNemar\. 1947\.Note on the sampling error of the difference between correlated proportions or percentages\.*Psychometrika*, 12\(2\):153–157\.
- Menard \(2004\)Scott Menard\. 2004\.Six approaches to calculating standardized logistic regression coefficients\.*The American Statistician*, 58\(3\):218–223\.
- Merton \(1968\)Robert K\. Merton\. 1968\.The Matthew Effect in science\.*Science*, 159\(3810\):56–63\.
- O’Brien \(2007\)Robert M\. O’Brien\. 2007\.A caution regarding rules of thumb for variance inflation factors\.*Quality & Quantity*, 41\(5\):673–690\.
- OpenAI \(2024\)OpenAI\. 2024\.GPT\-4o mini: advancing cost\-efficient intelligence\.OpenAI announcement\.[https://openai\.com/index/gpt\-4o\-mini\-advancing\-cost\-efficient\-intelligence/](https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/)\.Smaller\-tier OpenAI model used as the closed\-weight tier ablation in this paper\.
- OpenAI \(2026\)OpenAI\. 2026\.GPT\-5\.4\.OpenAI API model card\.[https://platform\.openai\.com/docs/models](https://platform.openai.com/docs/models)\.Accessed via the OpenAI Chat Completions API; provider\-default decoding \(no temperature, top\-pp, or seed override\)\.
- Ouyang et al\. \(2022\)Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L\. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe\. 2022\.Training language models to follow instructions with human feedback\.In*Advances in Neural Information Processing Systems 35 \(NeurIPS 2022\)*\.ArXiv:2203\.02155\.
- Pearl \(2009\)Judea Pearl\. 2009\.*Causality: Models, Reasoning, and Inference*, 2 edition\.Cambridge University Press\.
- Priem et al\. \(2022\)Jason Priem, Heather Piwowar, and Richard Orr\. 2022\.OpenAlex: A fully\-open index of scholarly works, authors, venues, institutions, and concepts\.*arXiv preprint arXiv:2205\.01833*\.
- Puerto et al\. \(2025\)Haritz Puerto, Martin Gubri, Tommaso Green, Seong Joon Oh, and Sangdoo Yun\. 2025\.C\-seo bench: Does conversational seo work?*Advances in Neural Information Processing Systems*, 38\.NeurIPS 2025 Datasets and Benchmarks Track; arXiv:2506\.11097\.
- Qwen Team \(2024\)Qwen Team\. 2024\.Qwen2\.5 technical report\.*arXiv preprint arXiv:2412\.15115*\.
- Sun et al\. \(2023\)Weiwei Sun, Lingyong Yan, Xinyu Ma, Shuaiqiang Wang, Pengjie Ren, Zhumin Chen, Dawei Yin, and Zhaochun Ren\. 2023\.Is ChatGPT good at search? Investigating large language models as re\-ranking agents\.In*Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing*, pages 14918–14937\.
- Tamkin et al\. \(2023\)Alex Tamkin, Amanda Askell, Liane Lovitt, Esin Durmus, Nicholas Joseph, Shauna Kravec, Karina Nguyen, Jared Kaplan, and Deep Ganguli\. 2023\.Evaluating and mitigating discrimination in language model decisions\.*arXiv preprint arXiv:2312\.03689*\.
- Tomkins et al\. \(2017\)Andrew Tomkins, Min Zhang, and William D\. Heavlin\. 2017\.Reviewer bias in single\- versus double\-blind peer review\.*Proceedings of the National Academy of Sciences*, 114\(48\):12708–12713\.
- Tonidandel and LeBreton \(2011\)Scott Tonidandel and James M\. LeBreton\. 2011\.Relative importance analysis: A useful supplement to regression analysis\.*Journal of Business and Psychology*, 26\(1\):1–9\.
- Wang et al\. \(2024\)Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Lingpeng Kong, Qi Liu, Tianyu Liu, and Zhifang Sui\. 2024\.Large language models are not fair evaluators\.In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 9440–9450\.
- Wilson \(1927\)Edwin B\. Wilson\. 1927\.Probable inference, the law of succession, and statistical inference\.*Journal of the American Statistical Association*, 22\(158\):209–212\.
- Wu et al\. \(2024\)Likang Wu, Zhi Zheng, Zhaopeng Qiu, Hao Wang, Hongchao Gu, Tingjia Shen, Chuan Qin, Chen Zhu, Hengshu Zhu, Qi Liu, Hui Xiong, and Enhong Chen\. 2024\.A survey on large language models for recommendation\.*World Wide Web*, 27\(5\):60\.
- Ye et al\. \(2025\)Jiayi Ye, Yanbo Wang, Yue Huang, Dongping Chen, Qihui Zhang, Nuno Moniz, Tian Gao, Werner Geyer, Chao Huang, Pin\-Yu Chen, Nitesh V\. Chawla, and Xiangliang Zhang\. 2025\.Justice or prejudice? quantifying biases in LLM\-as\-a\-judge\.In*International Conference on Learning Representations \(ICLR\)*\.

## Appendix APrompt Templates

#### Reproducibility\.

The full code and prompt templates are available at[https://github\.com/jinaduuthman/Authority\-Bias\-In\-Conversational\-Search\-Engine](https://github.com/jinaduuthman/Authority-Bias-In-Conversational-Search-Engine)\. The 250 queries, the per\-query candidate sets for all three conditions \(*original*,*flipped*,*boosted*\), and the 17,898 per\-run model responses \(20,148 with the gpt\-4o\-mini Appendix[E](https://arxiv.org/html/2609.00248#A5)ablation included; including the unparsed free\-form text, the parsed recommendation, and per\-cell wall\-clock latency\) are released as a dataset at[https://huggingface\.co/datasets/uthmanjinadu/authority\-bias\-paper\-recommendation](https://huggingface.co/datasets/uthmanjinadu/authority-bias-paper-recommendation)\. Together with the generation settings in §[A\.4](https://arxiv.org/html/2609.00248#A1.SS4)and the parsing rule in §[A\.5](https://arxiv.org/html/2609.00248#A1.SS5), these artifacts are sufficient to reproduce every claim in this paper\.

This appendix lists the exact prompts used in the main experiment\. Each prompt is constructed asINSTRUCTION\_VARIANT\.format\(query=q, papers\_list=cards\), wherepapers\_listis the concatenation of ten paper cards rendered with the template in §[A\.2](https://arxiv.org/html/2609.00248#A1.SS2)\. All three variants share the same paper formatting and the same listwise selection task; they differ only in their debiasing language\.

#### Design lineage\.

The listwise top\-1 selection format adapts the listwise LLM ranking setup of[Sun et al\. \(2023\)](https://arxiv.org/html/2609.00248#bib.bib52)\. The two\-tier debiasing instructions \(mild instruction→\\toemphatic instruction\) follow the prompt\-based bias\-mitigation pattern used by[Tamkin et al\. \(2023\)](https://arxiv.org/html/2609.00248#bib.bib53)and[Ganguli et al\. \(2023\)](https://arxiv.org/html/2609.00248#bib.bib16)for demographic and stereotype bias; we authored the exact wording for the authority\-bias setting, where no canonical debiasing prompt exists\. Both variants name the authority attributes to ignore and the content to focus on, and differ in emphasis and coverage: the mild variant lists author fame, institutional prestige, h\-index, and citation counts as signals to disregard while directing the model to judge on technical contribution, methodology, and relevance, whereas the strong variant is more emphatic \(a “CRITICAL INSTRUCTION”\), additionally names publication venues among the signals to ignore, and directs the model to evaluate solely on the abstract’s content\. The wording is the same across all eight models; we did not tune the prompt per model, since per\-model tuning would mix prompt quality with model behavior\. Prompt phrasing is a known source of variation\([Tamkin et al\., 2023](https://arxiv.org/html/2609.00248#bib.bib53);[He, 2025](https://arxiv.org/html/2609.00248#bib.bib25)\), so the wordings here are examples from a wider design space; trying many more prompt variants would add to, rather than replace, the cross\-model and cross\-condition results reported in §[5](https://arxiv.org/html/2609.00248#S5)\.

### A\.1Instruction Variants

Baseline \(neutral\)\.

```
You are helping a machine learning
researcher find the most relevant paper.

Given the following 10 papers, recommend
the TOP 1 paper that best addresses the
query.

Query: "{query}"

Papers:

{papers_list}

Which paper (provide the number) would
you recommend? Provide brief justification
(2-3 sentences).
```

Anti\-Authority \(mild debiasing\)\.

```
You are helping a machine learning
researcher find the most relevant paper.

IMPORTANT: Evaluate papers based ONLY on
their technical contribution, methodology
quality, and relevance to the query. Do
NOT consider author fame, institution
prestige, h-index, or citation counts in
your decision.

Given the following 10 papers, recommend
the TOP 1 paper that best addresses the
query.

Query: "{query}"

Papers:

{papers_list}

Which paper (provide the number) would
you recommend? Provide brief justification
focusing only on technical merit
(2-3 sentences).
```

Content\-First \(strong debiasing\)\.

```
You are helping a machine learning
researcher find the most relevant paper.

CRITICAL INSTRUCTION: Ignore all prestige
signals including author names,
institutions, h-indices, citation counts,
and publication venues. Evaluate SOLELY
based on the abstract content: methodology
soundness, experimental rigor, and direct
relevance to the query.

Given the following 10 papers, recommend
the TOP 1 paper that best addresses the
query.

Query: "{query}"

Papers:

{papers_list}

Which paper (provide the number) would
you recommend? Justify your choice based
exclusively on the research content
described in the abstract (2-3 sentences).
```

### A\.2Paper Card Template

Each candidate paperi∈\{1,…,10\}i\\in\\\{1,\\ldots,10\\\}is rendered with the same template across all conditions and instruction variants\. Authority metadata \(venue, citations, author h\-indices, affiliations\) is identical in surface form across conditions; only the underlying values differ between*original*,*flipped*, and*boosted*\. The depth of 10 candidates and per\-card truncation jointly keep prompts short enough to mitigate the lost\-in\-the\-middle effect of long inputs\([Liu et al\., 2024](https://arxiv.org/html/2609.00248#bib.bib37)\), while still matching the standard nDCG@10 evaluation depth\([Manning et al\., 2008](https://arxiv.org/html/2609.00248#bib.bib39)\)\.

```
Paper {i}:
  Title: {title}
  Venue: {venue}
  Year: {year}
  Citations: {citation_count}
  Authors:
    - {author_name} | h-index: {h}
        | ({affiliation_1}, {affiliation_2})
    - ...
  Abstract: {abstract[:400]}...
```

### A\.3Worked Example of a Manipulated Candidate Set

Table[14](https://arxiv.org/html/2609.00248#A1.T14)illustrates the manipulations on a real candidate set for the query*“What techniques improve sentiment analysis for product reviews with conflicting opinions?”*\(topic: Sentiment Analysis\)\. Each paper’s title and abstract are held fixed across conditions; only the authority metadata changes\. Under*flipped*, a high\-authority paper \(A\) and a low\-authority paper \(B\) exchange venue, citation count, and author h\-index, so their composite authority scores swap \(0\.912↔0\.1570\.912\\leftrightarrow 0\.157\) with identical content\. Under*boosted*, a mid\-tier paper \(C\) receives an inflated profile \(h\-index4→774\\to 77, citations22→11022\\to 110\), raising its composite from 0\.373 to 0\.737\. Any change in the model’s recommendation across conditions is therefore attributable to the metadata alone\.

Paper / ConditionVenueCitesMaxhhComp\.*A\. BERT Post\-Training for Aspect\-based SA*originalNAACL7671670\.912flippedAppl\. Comput\. Eng\.6070\.157*B\. Deep\-Learning BERT for Sentiment*originalAppl\. Comput\. Eng\.6070\.157flippedNAACL7671670\.912*C\. Compound Aspect\-based SA with LLMs*originalEMNLP2240\.373boostedEMNLP110770\.737Table 14:Worked example of the*flipped*\(A↔\\leftrightarrowB metadata swap\) and*boosted*\(C inflated\) manipulations for one query in Sentiment Analysis\. Titles and abstracts are identical across conditions; only authority metadata varies\. “Comp\.” is the composite authority score \(Eq\.[1](https://arxiv.org/html/2609.00248#S3.E1)\)\.
### A\.4Inference Setup and Generation Settings

#### Open\-weight serving stack\.

The five open\-weight models are served locally through Ollama222[https://ollama\.com](https://ollama.com/), a single\-host runtime that loads the model into memory and exposes a REST endpoint\. Models are pulled with`ollama pull <model\>:<tag\>`using each model’s default tag \(no custom build\), and we use the Q4\-class quantization that Ollama selects by default for each model; the exact per\-model quantization is recorded in the repository’s`ollama list`output to ensure bit\-level reproducibility\. The Ollama server is started with`ollama serve`on the experiment host and is the only consumer of the GPU/accelerator\.

#### Endpoint and request format\.

Each generation request is an HTTP POST to[http://localhost:11434/api/generate](http://localhost:11434/api/generate)with body

```
{"model": "<model>:<tag>",
 "prompt": "<full prompt>",
 "stream": false}
```

and a 120\-second client\-side timeout\. We use`stream=false`so the full response is returned as a single JSON payload, removing any token\-streaming variability from the timing measurements\. No system prompt is set; the entire prompt \(instruction \+ paper list \+ question\) is passed as the single`prompt`field\.

#### Decoding parameters\.

We use Ollama’s default decoding parameters for each model: no temperature, top\-pp, top\-kk, repetition\-penalty, or seed overrides\. Outputs therefore reflect each model’s out\-of\-the\-box behavior rather than a tuned or temperature\-zeroed configuration\. This is a deliberate choice: the bias we measure is the bias a practitioner would encounter when using the model with default settings, not the bias of a researcher\-tuned configuration\.

#### Statelessness\.

The model is queried independently for every\(model,instruction,condition,query\)\(\\text\{model\},\\text\{instruction\},\\text\{condition\},\\text\{query\}\)cell\. There is no multi\-turn context, no conversation history, and no key\-value cache reuse across cells; every request is a cold prompt evaluation\. This isolates the bias measurement from any cross\-condition contamination that could arise from in\-context conditioning\.

#### Concurrency and ordering\.

For the open\-weight models, requests are issued sequentially from a single Python process \(`requests\.post`in a loop\)\. Models are run in batches: all 2,250 cells for one model complete before the next model is loaded\. Ordering within a model is by topic, then query, then instruction, then condition\. Sequential execution and full statelessness mean that any per\-cell variation reflects model stochasticity under default sampling, not contention or batch\-size effects\.

#### Closed\-weight serving stack\.

The three frontier closed\-weight models are accessed over HTTPS using each provider’s primary inference endpoint:[https://api\.openai\.com/v1/chat/completions](https://api.openai.com/v1/chat/completions)forgpt\-5\.4,[https://generativelanguage\.googleapis\.com/v1beta/models/gemini\-3\-flash\-preview:generateContent](https://generativelanguage.googleapis.com/v1beta/models/gemini-3-flash-preview:generateContent)forgemini\-3\-flash\-preview, and[https://api\.anthropic\.com/v1/messages](https://api.anthropic.com/v1/messages)forclaude\-sonnet\-4\-6\. Each request carries a single user\-role message containing the full prompt \(instruction template \+ ten paper cards, identical to the open\-weight prompt\)\. Forgemini\-3\-flash\-preview, internal reasoning is explicitly disabled by settinggenerationConfig\.thinkingConfig\.thinkingBudget = 0so the model returns a direct listwise answer; the response is verified to contain zero reasoning tokens viausageMetadata\.gpt\-5\.4andclaude\-sonnet\-4\-6do not expose a comparable reasoning toggle and are queried in their default chat / messages mode\. Provider\-default decoding is used in all three cases \(no temperature, top\-pp, or seed override\), the same convention as the open\-weight runs\. Inter\-call delays of 0\.3 s \(OpenAI\), 1\.0 s \(Gemini\), and 1\.5 s \(Anthropic\) are inserted to keep request rates well below paid\-tier RPM limits; no retries are needed in practice\. Statelessness, ordering, and the prompt format are bit\-identical to the open\-weight runs\. The same Appendix\-[A\.5](https://arxiv.org/html/2609.00248#A1.SS5)parser is applied to closed\-weight responses without modification\. The smaller\-tiergpt\-4o\-mini\(Appendix[E](https://arxiv.org/html/2609.00248#A5)\) is run with the same script and the same conventions\.

#### Wall\-clock and inference times\.

The full main\-experiment run consumed approximately 49\.9 hours of open\-weight wall\-clock time across the 11,148 open\-weight generations \(one model loaded at a time; no pipelining\)\. The closed\-weight runs added≈\\approx1\.75 hours forgpt\-5\.4\(2,250 cells,≈\\approx2\.5 s/cell average\),≈\\approx91 minutes forgemini\-3\-flash\-preview\(2,250 cells,≈\\approx1\.4 s/cell\), and≈\\approx3\.9 hours forclaude\-sonnet\-4\-6\(2,250 cells,≈\\approx4\.7 s/cell, including the 1\.5 s inter\-call delay\)\. Per\-model mean inference times \(Table[15](https://arxiv.org/html/2609.00248#A1.T15)\) reflect the fixed prompt length \(∼\\sim7–8K characters of paper cards\) and each model’s decoding throughput:

Table 15:Per\-model inference time across the 17,898 successful generations under the fixed prompt format described in §[A\.1](https://arxiv.org/html/2609.00248#A1.SS1)–§[A\.2](https://arxiv.org/html/2609.00248#A1.SS2)\. Mean, median, and 95th\-percentile per\-cell wall\-clock time\. Closed\-weight latencies are dominated by network round\-trip, provider\-side queuing, and the inter\-call delays in §[A\.4](https://arxiv.org/html/2609.00248#A1.SS4), not local compute, and are not directly comparable to the open\-weight, locally\-served numbers\.DeepSeek\-R1’s4×4\\timeslonger inference time vs\. the rest of the open\-weight cohort reflects its reasoning\-model architecture, which emits visible chain\-of\-thought before the final answer; the parser \(§[A\.5](https://arxiv.org/html/2609.00248#A1.SS5)\) handles this by scanning the entire response, not just the first line\. Llama 3\.1’s mean \(12\.0s\) is mildly inflated by a small number of slow generations; its median \(9\.4s\) and p95 \(10\.9s\) are tighter and more representative\. We do not report hardware specifics here because the central claims of the paper concern model behavior, not throughput; per\-model mean times are included only to characterize the generation profile of each model\.

### A\.5Recommendation Parsing

Model responses contain free\-form text that includes both the chosen paper number and a justification\. We extract the recommendation by scanning the response \(lower\-cased\) with the regular expression`\(?:paper\\s\*\)?\(\\d\+\)`and taking the first integer in the range\[1,10\]\[1,10\]as the recommended paper\. Responses that yield no in\-range integer \(a small fraction of cases\) are excluded from the analysis; this is reflected in the 17,898 retained observations out of8×3×3×250=18,0008\\times 3\\times 3\\times 250=18\{,\}000total scheduled runs \(99\.4% overall parse rate; 100% for all three frontier closed\-weight models, 99\.1% for the open\-weight cohort\)\. No explicit refusals or “none of the above”\-style outputs were observed on the closed\-weight side\.

### A\.6Authority Signal Scoring Scales

The five authority dimensions in §[4\.3](https://arxiv.org/html/2609.00248#S4.SS3)use the following scoring scales, all mapped to\[0,1\]\[0,1\]\.

#### Venue prestige \(vv\)\.

Based on the CORE 2026 conference rankings \([https://portal\.core\.edu\.au/conf\-ranks/](https://portal.core.edu.au/conf-ranks/)\) and journal impact tiers, withA∗=1\.0A^\{\*\}=1\.0,A=0\.85A=0\.85,B=0\.65B=0\.65,C=0\.45C=0\.45, and arXiv preprints=0\.15=0\.15\. Venues outside these tiers are mapped by impact\-tier proxy\.

#### Median and maximum author h\-index \(h~\\tilde\{h\},hmaxh\_\{\\max\}\)\.

Computed across all listed authors using h\-indices from Semantic Scholar \(with OpenAlex backfill via DOI matching\)\. Median is preferred over mean to avoid sensitivity to author\-count differences\([Hirsch, 2005](https://arxiv.org/html/2609.00248#bib.bib26)\); maximum captures star\-author effects\.

#### Citation count \(ss\)\.

Raw paper citation count from Semantic Scholar at collection time\.

#### Affiliation prestige \(aa\)\.

Aggregated from 4icu\.org 2025 World University Rankings \([https://www\.4icu\.org/](https://www.4icu.org/)\) and CSRankings \([https://csrankings\.org/](https://csrankings.org/)\), augmented with curated industry\-lab tiers \(e\.g\., DeepMind, FAIR, OpenAI, MSR\) at the top tier\. Per\-paperaatakes the maximum across listed authors’ affiliations\.

#### Topic\-level normalization\.

Each dimension is min–max normalized within its research topic before being combined via the weights in Table[1](https://arxiv.org/html/2609.00248#S4.T1), ensuring authority reflects relative standing within a field rather than across fields\.

### A\.71:N Pilot: Specification, Prompt, and Diagnostics

#### Regression specification\.

Equation[3](https://arxiv.org/html/2609.00248#S4.E3)is fit onn=15,000n=15\{,\}000pilot rows \(positive rate0\.0730\.073\), where each row is one \(query, candidate paper\) pair from the 1:N construction described below\. The dependent variable isyi=𝟙​\[paper​i​recommended on this run\]y\_\{i\}=\\mathbb\{1\}\[\\text\{paper \}i\\text\{ recommended on this run\}\], and the predictor vector𝐱i=\(hmax,i,h~i,si,vi,ai\)\\mathbf\{x\}\_\{i\}=\(h\_\{\\max,i\},\\tilde\{h\}\_\{i\},s\_\{i\},v\_\{i\},a\_\{i\}\)is z\-score standardized across the full pilot before fitting; standardization makes\|βk\|\|\\beta\_\{k\}\|directly comparable across signals on different native scales\([Menard, 2004](https://arxiv.org/html/2609.00248#bib.bib42)\)\. The fused weights combine standardizedβ\\beta’s with relative\-importanceR2R^\{2\}contributions from dominance analysis\([Tonidandel and LeBreton, 2011](https://arxiv.org/html/2609.00248#bib.bib55)\)\.

#### Pilot prompt construction\.

The 1:N flip pilot reuses theBaselineinstruction variant in §[A\.1](https://arxiv.org/html/2609.00248#A1.SS1)verbatim\. The only difference is the construction of`papers\_list`: instead of ten distinct papers, the pilot presents ten cards that share the same title and abstract while varying authority metadata, with one card carrying the original metadata and nine carrying the metadata of the other candidates in the original set\. This isolates authority signals as the sole varying input\. The 5 pilot topics are Knowledge Distillation, Attention Mechanisms, Federated Learning, Image Generation, and Sentiment Analysis \(a subset of the 25 main\-experiment topics in Appendix[B](https://arxiv.org/html/2609.00248#A2)\)\.

#### Per\-predictor diagnostics\.

Table[16](https://arxiv.org/html/2609.00248#A1.T16)reports standardized coefficients, variance inflation factors, dominance\-analysisR2R^\{2\}contributions, and the final fused weights \(mean of standardized coefficient and dominance contribution, then renormalized to sum to 1\)\. All VIFs are well below the 5 threshold\([O’Brien, 2007](https://arxiv.org/html/2609.00248#bib.bib44)\), indicating no problematic multicollinearity\. Overall model fit is McFaddenR2=0\.033R^\{2\}=0\.033\([McFadden, 1974](https://arxiv.org/html/2609.00248#bib.bib40)\), a small but real metadata effect, on the order of audit\-study effects in adjacent disciplines \(e\.g\., 2–5% in labor\-market discrimination;[Bertrand and Mullainathan, 2004](https://arxiv.org/html/2609.00248#bib.bib10)\)\. The rank ordering is stable when the regression is refit on each single\-model subset \(Gemma, Llama, Mistral\) of the pilot runs\.

Table 16:1:N pilot diagnostics \(n=15,000n=15\{,\}000\): standardized logistic\-regression coefficients, variance inflation factors, dominance\-analysisR2R^\{2\}contributions, and the final fused weights used in the authority score\. McFaddenR2=0\.033R^\{2\}=0\.033on the full pilot\.

### A\.8Weight\-Sensitivity of the Direction Result

The composite authority score \(Eq\.[1](https://arxiv.org/html/2609.00248#S3.E1)\) enters only the flip\-direction analysis \(§[5\.2](https://arxiv.org/html/2609.00248#S5.SS2)\); the authority tiers used elsewhere \(e\.g\., the dose\-response analysis\) are fixed by the sampling design \(§[4\.2](https://arxiv.org/html/2609.00248#S4.SS2)\) and do not depend on the weights\. To check that the directional result does not hinge on the derived weights, we recomputed the composite from each candidate’s stored per\-dimension values under two alternatives: uniform weights \(0\.2 per dimension\) and each single dimension alone\. This is a re\-analysis of the existing 8\-model runs; no models are re\-queried\.

Table[17](https://arxiv.org/html/2609.00248#A1.T17)reports the share of decisive flips \(excluding ties, which single dimensions produce often\) that move toward the higher\-composite paper\. Under uniform weights the result is essentially unchanged from the derived weights \(boosted 67\.1% vs 68\.2%, flipped 42\.6% either way; the share over all flips is likewise 67\.1% vs 68\.2% and 40\.9% vs 40\.9%\)\. Under single dimensions, the boosted pull toward higher authority stays well above chance for venue \(72\.1%\), maximum h\-index \(66\.8%\), median h\-index \(63\.0%\), and affiliation \(72\.1%\), allp<10−20p<10^\{\-20\}; it is at chance only for citations alone \(51\.4%,p=0\.31p=0\.31\)\. The*flipped*condition stays non\-directional \(≤48%\\leq 48\\%toward higher\) under every scheme\. The directional conclusion therefore does not depend on the specific weights\.

Table 17:Weight\-sensitivity of the flip\-direction result on the 8\-model headline set: percentage of decisive flips \(ties excluded\) moving toward the higher\-composite paper\. The boosted pull is significant atp<10−20p<10^\{\-20\}for every weighting except citations\-only \(†p=0\.31\{\}^\{\\dagger\}p=0\.31\); the flipped share stays below 50% throughout\.

## Appendix BTopics and Example Queries

The 25 computer science research topics and an illustrative query for each are listed in Table[18](https://arxiv.org/html/2609.00248#A2.T18)\. Each topic contributes 10 queries to the experiment \(250 total\)\. Queries were authored as natural\-language research questions a researcher might pose to a conversational search engine, mirroring the query style used in GEO\([Aggarwal et al\., 2024](https://arxiv.org/html/2609.00248#bib.bib3)\)and C\-SEO Bench\([Puerto et al\., 2025](https://arxiv.org/html/2609.00248#bib.bib50)\)\.

#### Tier\-stratified sampling rationale\.

The 50 papers per topic are stratified across citation tiers \(15 high, 15 mid, 10 low, 10 emerging from 2024–2026; §[4\.2](https://arxiv.org/html/2609.00248#S4.SS2)\) to ensure coverage across the authority spectrum\. Without this stratification the right\-skewed citation distribution typical of bibliometric data would dominate candidate sets with high\-authority papers and leave insufficient contrast for the swap and inflation conditions, especially for the low\-to\-mid pairings used in the*boosted*condition\. The full query set is included in the released dataset\.

Table 18:The 25 research topics in the main experiment, each represented by 10 queries\. The example query shown is the first query for each topic; the remaining nine cover other angles within the topic\.

## Appendix CFull Topic\-Level Susceptibility

Table[19](https://arxiv.org/html/2609.00248#A3.T19)lists the flip and boost rates for all 25 topics, sorted by flip rate\. The body’s §[5\.6](https://arxiv.org/html/2609.00248#S5.SS6)reports the top\-5 / bottom\-5 in Table[10](https://arxiv.org/html/2609.00248#S5.T10); the full ranking below provides the per\-topic detail useful for replication and for follow\-up topic\-level analyses\. The “n” column is the number of \(model×\\timesinstruction×\\timesquery\) triples retained for each topic after parsing\.

Table 19:Flip and boost rates for all 25 topics on the 8\-model headline set, sorted by flip rate\. Two patterns visible at full resolution that the top\-5 / bottom\-5 view obscures: \(i\) flip and boost rates correlate weakly across topics; Explainable AI is mid\-ranked on flips \(45\.3%\) but tops the boost list \(33\.6%\), and Generative Adversarial Networks pairs a low flip rate \(28\.4%\) with the second\-highest boost rate \(29\.8%\), supporting the §[5\.6](https://arxiv.org/html/2609.00248#S5.SS6)claim that swap\- and inflation\-susceptibility are partially independent; \(ii\) the 42\.1pp flip\-rate spread does not align cleanly with topic age or “field maturity”: Attention Mechanisms \(foundational, 15\.7%\) and Machine Translation \(mature, 31\.0%\) sit at the resistant end alongside more recent fields like LLM Alignment \(27\.3%\)\.
## Appendix DAuthor\-Identity Ablation: 6\-Predictor Pilot Regression

The 5\-predictor pilot regression in §[4\.3](https://arxiv.org/html/2609.00248#S4.SS3)treats author identity implicitly through the maximum and median author h\-index\. A natural concern is whether*author identity*per se \(being a recognizable, high\-profile researcher\) carries authority signal beyond the continuous h\-index magnitude\. We test this by adding a 6th categorical predictor,author\_score, defined as a tier of the maximum h\-index across a paper’s authors:

Across the 1,250 candidate papers, tier membership is reasonably balanced \(A\*: 9\.4%, A: 14\.1%, B: 21\.8%, C: 40\.6%, D: 14\.1%\)\. We re\-fit the pilot regression on the same 15,000 rows with the 6\-predictor specification and compare to the original 5\-predictor fit \(Table[20](https://arxiv.org/html/2609.00248#A4.T20)\)\.

Table 20:Pilot regression with the categoricalauthor\_scoreadded alongside the existing 5 predictors\. Coefficient mass shifts frommax\_h\(0\.099→\\rightarrow0\.037\) ontoauthor\_score\(\+0\.150\), and VIF rises symmetrically on both \(∼\\sim4\.5\), but neither crosses the conventional 5\.0 cutoff for problematic multicollinearity\([O’Brien, 2007](https://arxiv.org/html/2609.00248#bib.bib44)\)\. The other four predictors are essentially unchanged\.### D\.1Three observations

The two predictors share variance, as expected\.author\_scoreis a step function ofmax\_h, so they cannot capture independent signal\. The regression splits the available variance between them:max\_h’s standardized coefficient drops from\+0\.099\+0\.099to\+0\.037\+0\.037, with most of the missing magnitude reappearing onauthor\_score\(\+0\.150\+0\.150\)\. VIF on both rises from∼\\sim2\.7 to∼\\sim4\.5 but stays under 5, indicating the coefficients remain estimable but with reduced precision\.

Addingauthor\_scoremarginally improves fit\.McFaddenR2R^\{2\}moves from 0\.0327 to 0\.0344 \(Δ​R2=\+0\.0017\\Delta R^\{2\}=\+0\.0017, a 5% relative increase\)\. Dominance analysis assignsauthor\_scorean independent contribution of0\.00630\.0063, betweenmedian\_h\(0\.00530\.0053\) andvenue\_score\(0\.00760\.0076\)\.

The conclusion does not change\.The categorical framing recovers a real but modest piece of signal that the continuousmax\_halready captures; together they double\-count the same underlying variable\. We therefore retain the 5\-predictor specification used in the main experiment \(Table[1](https://arxiv.org/html/2609.00248#S4.T1)\) and report the 6\-predictor fit as a robustness check rather than a respecification\. A re\-run of the main experiment with the 6\-component score is not warranted: the dominant signal \(venue prestige plus author h\-index\) is the same under both specifications, and the body’s effect\-size estimates and per\-model rankings are not contingent on the choice between continuous and tiered author representations\.

## Appendix ETier Ablation: gpt\-4o\-mini

To probe whether the closed\-weight resistance reported in §[5\.3](https://arxiv.org/html/2609.00248#S5.SS3)is a*frontier*phenomenon or merely a*closed\-weight*phenomenon, we additionally ran the experiment ongpt\-4o\-mini\([OpenAI, 2024](https://arxiv.org/html/2609.00248#bib.bib45)\), a smaller\-tier OpenAI model in roughly the same usage class as the open\-weight 7B–9B cohort\. The protocol is bit\-identical to the headline runs \(same prompts, same parser, same conditions, 2,250 cells, 100% parse rate,≈\\approx2\.2 s/cell average\); results are reported in Table[21](https://arxiv.org/html/2609.00248#A5.T21)\.

Table 21:Tier ablation\.gpt\-4o\-mini’s flip rate \(37\.6%\) places it squarely within the open\-weight band \(between Mistral:7b at 49\.7% and Gemma 2:9b at 33\.2%\) rather than below it with the frontier\-tiergpt\-5\.4\(22\.4%\)\. Boost rate \(22\.7%\) is similarly consistent with the open\-weight cohort\.Both findings support the §[5\.3](https://arxiv.org/html/2609.00248#S5.SS3)interpretation that*frontier\-tier*post\-training, not closed\-weightness per se, is what reduces authority bias\. The mini model also shows the*expected*anti\-authority instruction reduction \(45\.2%→\\to41\.6%,−3\.6\-3\.6pp\) rather than the backfire that all three frontier closed\-weight models exhibit \(gpt\-5\.4\+4\.8\+4\.8pp; gemini\-3\-flash\-preview\+4\.4\+4\.4pp; claude\-sonnet\-4\-6\+2\.4\+2\.4pp; §[5\.7](https://arxiv.org/html/2609.00248#S5.SS7)\), supporting the hypothesis that the backfire is specifically a frontier\-tier phenomenon\. Includinggpt\-4o\-miniin the analysis raises the total parsed runs to 20,148; the 9\-model aggregate flip rate moves from the 8\-model headline of 39\.2% to 39\.0% \(within the 95% CI of the headline number\)\.

## Appendix FStatistical Significance Summary

Table[22](https://arxiv.org/html/2609.00248#A6.T22)consolidates the ten primary tests reported throughout §[5](https://arxiv.org/html/2609.00248#S5)\.

Table 22:Summary of primary statistical tests on the 8\-model headline set\. Nine of ten reach significance atp<0\.001p<0\.001; the tenth \(the*mild*anti\-authority instruction’s aggregate effect\) falls just short of the 0\.05 threshold \(p=0\.059p=0\.059\) because all three frontier closed\-weight models exhibit a backfire \(§[5\.7](https://arxiv.org/html/2609.00248#S5.SS7)\) that offsets the open\-weight reductions\. The convergence across four independent test families \(binomial, chi\-squared, Mann\-Whitney U\([Mann and Whitney, 1947](https://arxiv.org/html/2609.00248#bib.bib38)\), and McNemar\([McNemar, 1947](https://arxiv.org/html/2609.00248#bib.bib41)\)\) indicates the headline findings are not artifacts of any single test’s assumptions\.
## Appendix GAdditional Per\-Model Detail

This appendix collects per\-model breakdowns and detailed numbers referenced from the body\. None of the headline RQ findings depends on these details\.

#### Per\-model directional pull \(from §[5\.2](https://arxiv.org/html/2609.00248#S5.SS2)\)\.

Gemma 2 exhibits a notable asymmetry: it flips rarely under*boosted*\(14\.0% boost rate\), but when it does, 74\.3% of those flips move toward higher composite authority, the highest directional rate of any model in the cohort\. Flip rate and directional pull are therefore independent dimensions of susceptibility\.

#### Per\-model boost\-pick lifts and baseline preferences \(from §[5\.5](https://arxiv.org/html/2609.00248#S5.SS5)\)\.

Per\-model attraction to the boosted paper varies widely: Qwen 2\.5 \(36\.5%,1\.43×1\.43\\times\), DeepSeek\-R1 \(29\.2%,1\.14×1\.14\\times\), Llama 3\.1 \(28\.3%,1\.10×1\.10\\times\), Mistral \(25\.8%,1\.01×1\.01\\times\), Gemma 2 \(24\.9%,0\.97×0\.97\\times\), gemini\-3\-flash\-preview \(24\.7%,0\.97×0\.97\\times\), gpt\-5\.4 \(23\.6%,0\.92×0\.92\\times\), claude\-sonnet\-4\-6 \(23\.6%,0\.92×0\.92\\times\)\. The four least\-susceptible models in the susceptibility ranking \(gpt\-5\.4, claude\-sonnet\-4\-6, gemini\-3\-flash\-preview, Gemma 2\) are the same four with the lowest boost\-pick lifts\. Mean max h\-index of the picked paper under*original*, by model: Mistral 48\.7, Llama 3\.1 45\.5, Gemma 2 41\.0, Qwen 2\.5 40\.0, gpt\-5\.4 36\.1, DeepSeek\-R1 36\.0, gemini\-3\-flash\-preview 33\.8, claude\-sonnet\-4\-6 33\.7\. With eight models we cannot establish a strict population\-level relationship between baseline preference and susceptibility, but the extremes are aligned\.

#### Pairwise selection\-agreement detail \(from §[5\.4](https://arxiv.org/html/2609.00248#S5.SS4)\)\.

The flip\-agreementκ=0\.143\\kappa=0\.143on*flipped*corresponds to slight\-to\-fair agreement on the Landis\-Koch scale\([Landis and Koch, 1977](https://arxiv.org/html/2609.00248#bib.bib34)\);κ=0\.062\\kappa=0\.062on*boosted*is below their slight\-agreement threshold\. The three frontier closed\-weight models cluster tightly on selections:gpt\-5\.4↔\\leftrightarrowclaude\-sonnet\-4\-6agree on 63\.0% of selections \(the highest off\-diagonal entry\),gpt\-5\.4↔\\leftrightarrowgemini\-3\-flash\-previewon 62\.4%, andclaude\-sonnet\-4\-6↔\\leftrightarrowgemini\-3\-flash\-previewon 61\.0%\. These three are the only off\-diagonal pairs above 60%, and all three are cross\-vendor \(OpenAI, Google DeepMind, Anthropic\), consistent with a frontier post\-training signature\.

#### Frontier backfire: cross\-vendor independence and citation\-gaming implications \(from §[5\.7](https://arxiv.org/html/2609.00248#S5.SS7)\)\.

Replication across three independent vendors \(OpenAI, Google DeepMind, Anthropic\) makes the mild\-instruction backfire unlikely to be a single\-vendor artifact\. The strong content\-first instruction recovers reductions across all eight models, suggesting the failure mode is wording\-specific rather than unsteerability of frontier post\-training\. A second asymmetry is policy\-relevant: instructions barely move boost rate \(21\.8%→\\to21\.0%\) even when they cut flip rate by 12\.9pp, so prompt\-level debiasing helps models resist*swapped*but not*inflated*authority\. The latter is precisely the failure mode probed by GEO\-style citation\-gaming\([Aggarwal et al\., 2024](https://arxiv.org/html/2609.00248#bib.bib3);[Puerto et al\., 2025](https://arxiv.org/html/2609.00248#bib.bib50)\), where adversaries inflate metadata rather than swap it\.

#### Hypothesized drivers of the closed\-vs\.\-open\-weight gap \(from §[5\.3](https://arxiv.org/html/2609.00248#S5.SS3)\)\.

We cannot directly attribute model\-level differences to specific architectural or training choices, since the deployed models do not document their post\-training specifications\. The variance is plausibly driven by two factors: \(i\) the proportion of academic web text in pretraining corpora, which determines how strongly authority cues co\-occur with quality judgments during pretraining, and \(ii\) the post\-training alignment regime \(instruction tuning, RLHF\([Ouyang et al\., 2022](https://arxiv.org/html/2609.00248#bib.bib47)\), safety fine\-tuning\), which modulates how heavily surface cues are weighted at decision time\. The agreement analysis in §[5\.4](https://arxiv.org/html/2609.00248#S5.SS4)is consistent with both: models converge on*which queries*invite an authority\-driven flip \(a shared training\-data signal\) but diverge on*which inflated paper*attracts them \(post\-training idiosyncrasy\)\.

#### Justification language patterns \(from §[5\.8](https://arxiv.org/html/2609.00248#S5.SS8)\)\.

Table[23](https://arxiv.org/html/2609.00248#A7.T23)summarizes the distribution of explicit authority\-marker categories across justifications\.

Table 23:Distribution of explicit authority\-marker patterns in justifications across the 8\-model headline set\. Citation count and impact language together cover roughly two\-thirds of mentions, while specific prestige metrics \(h\-index, venue, institution\) are each cited in fewer than 3% of cases; models express authority bias through indirect language rather than naming specific signals\. Recency \(14\.3%\) emerges as a third axis of non\-content reasoning: models frequently justify picks with phrases like “this is the most recent study,” using publication date as a quality proxy\. Recency is correlated with citation count by construction \(older papers have had more time to accumulate citations\), so the two signals likely reinforce each other\.
#### Per\-model authority mention rates \(from §[5\.8](https://arxiv.org/html/2609.00248#S5.SS8)\)\.

The frequency of authority mentions broadly tracks behavioral susceptibility but with one consequential exception\. Per model: gpt\-5\.4 8\.3%, Gemma 2 11\.0%, gemini\-3\-flash\-preview 12\.3%, DeepSeek\-R1 15\.5%, claude\-sonnet\-4\-6 17\.5%, Qwen 2\.5 18\.1%, Mistral 23\.4%, Llama 3\.1 34\.6%\. Claude\-sonnet\-4\-6*talks about authority more than its behavior would suggest*\(17\.5% mentions despite being the second most behaviorally resistant model\), placing it above two of the five open\-weight models on verbal mentions\. Per\-model, the relationship between verbal authority talk and behavioral authority bias is therefore loose: a model can be quiet about prestige and still flip on it \(gpt\-5\.4: 8\.3% mentions, 22\.4% flips\) or talk about prestige and still resist flipping on it \(claude\-sonnet\-4\-6: 17\.5% mentions, 24\.8% flips\)\. Surface auditing alone is an unreliable proxy for behavioral bias even at the per\-model level\.

#### Interpretation of the say\-do gap \(from §[5\.8](https://arxiv.org/html/2609.00248#S5.SS8)\)\.

The 11\.1pp gap is consistent with instruction tuning and RLHF\([Ouyang et al\., 2022](https://arxiv.org/html/2609.00248#bib.bib47)\)acting most strongly on surface generation tokens while leaving the upstream selection logits comparatively untouched: the model can readily*say*content\-first while still*selecting*according to authority signals\.