Distribird: Literature-Informed Prior Distribution Design for Bayesian Model Calibration

# 1 Introduction
Source: [https://arxiv.org/html/2608.11210](https://arxiv.org/html/2608.11210)
![[Uncaptioned image]](https://arxiv.org/html/2608.11210v1/assets/logo.png)

Distribird Literature\-Informed Prior Distribution Design for Bayesian Model Calibration Patrik P\. Süli1,2, György Eigner1,2,3, Roland Hollós4,5

1Doctoral School of Applied Informatics and Applied Mathematics, Obuda University, Budapest, Hungary 2Biomatics and Applied Artificial Intelligence Institute, John von Neumann Faculty of Informatics, Obuda University, Budapest, Hungary 3Physiological Controls Research Center, Obuda University, Budapest, Hungary4HUN\-REN Centre for Agricultural Research, Brunszvik u\. 2\., Martonvásár 2462, Hungary 5Czech Academy of Sciences, Global Change Institute, Czech Republic suli\.patrik@uni\-obuda\.hu, eigner\.gyorgy@uni\-obuda\.hu, hollos\.roland@hun\-ren\.hu 2026

Preprint, submitted to arXiv

Abstract Bayesian calibration of process\-based models requires a prior distribution for each model parameter\. Despite decades of methodological work, researchers almost always fall back on uniform priors\. The main reason is that building informative priors from scientific literature is slow and needs both domain and statistical expertise\. We presentDistribird, an agentic web application that automates this process\. Given a parameter name, physical description, and domain context, Distribird deploys a multi\-agent LangGraph pipeline that searches Semantic Scholar and OpenAlex in parallel, reads the retrieved papers in full, extracts reported numerical values, weights them by how well each source’s study context matches the target domain, and fits the best\-matching probability distribution via AIC model selection from a predefined set of distributions\. When no literature is available, the system falls back to sensible uninformative alternatives \(a uniform prior or a wide Normal\), and clearly reports both the evidence behind and the confidence level of every prior it produces\. It is designed for the problems where it has the most to offer: physically interpretable parameters in process\-based models, where domain knowledge exists in the published literature\. We evaluate the tool on 24 parameters across ten scientific domains using three open\-weight models \(Qwen3\.6 27B, Gemma 4 31B, Mistral Small 4 119B\) run entirely on local hardware, in a controlled comparison against a single\-prompt LLM baseline\. On prior quality the full pipeline*matches*this baseline rather than beating it; its contribution is of a different kind\. Every prior is traced to the specific papers and values from which it was constructed; a built\-in validity layer declines to produce priors for out\-of\-scope requests, flagging all five out\-of\-scope test parameters on every model, whereas the single\-prompt baseline returns confident but unfounded priors for them in 11 of 30 model–parameter cases; and every language\-model call runs on local open weights, so no parameter description or unpublished modelling detail is transmitted to a third\-party LLM provider \(only generated search terms reach the public literature databases\)\. For scientific use, we argue these properties matter more than a marginal improvement in point\-estimate accuracy\.

### 1\.1The calibration problem

Process\-based models describe complex natural systems \(eg\. crop growth, water movement through soil, biogeochemical cycles, disease spread, etc\.\) using mechanistic equations with parameters that represent physical, chemical, or biological quantities\. Because many of these parameters cannot be measured directly, they must be inferred from observations by adjusting parameter values until the model output matches available data\. This process is called model calibration, or inverse modeling\(Hollóset al\.,[2022](https://arxiv.org/html/2608.11210#bib.bib10)\)\. Approaches to this parameter\-identification problem range from classical optimisation to modern machine\-learning estimators, for instance a neural\-network estimator with a differentiable ODE solver that recovers patient\-specific parameters of a tumour\-growth model directly from measurements\(Kisbenedeket al\.,[2025](https://arxiv.org/html/2608.11210#bib.bib11)\)\.

Bayesian calibration is arguably the soundest approach to this problem, on both statistical and philosophical grounds\(Gelmanet al\.,[2013](https://arxiv.org/html/2608.11210#bib.bib24)\)\. It combines prior knowledge about the parameters with the information in the observed data, and returns a posterior distribution that captures not only the most likely parameter values but also the uncertainty around them\. This uncertainty quantification is critical for decision\-making and for honest reporting of model predictions\. Explicitly representing parameter uncertainty is likewise central to robust model\-based control of such systems, where the performance of the controller depends directly on how that uncertainty is captured\(Vargaet al\.,[2025](https://arxiv.org/html/2608.11210#bib.bib12)\)\.

### 1\.2The prior problem

The Bayesian approach requires the researcher to specify a prior distribution for each parameter before seeing the data\. This raises an immediate practical question: what prior should be used?

The mathematically convenient answer is to use a uniform prior: claim that all parameter values in a range are equally plausible\. This choice is almost universally adopted in practice\(Wallachet al\.,[2021](https://arxiv.org/html/2608.11210#bib.bib21)\)\. It is also almost always inadequate\.Pericchi and Walley \([1991](https://arxiv.org/html/2608.11210#bib.bib16)\)showed that no single uninformative prior works in every case\. Uniform priors in particular are not invariant under reparameterisation, a well\-known limitation: the implied prior changes depending on how the parameter is written\. With a uniform prior the posterior mode coincides exactly with the maximum\-likelihood estimate\(Gelmanet al\.,[2013](https://arxiv.org/html/2608.11210#bib.bib24)\), reducing Bayesian inference to the very method it is supposed to improve on\. The information lost by ignoring genuine prior knowledge has real consequences: the sampler converges more slowly, credible intervals are wider than they need to be, and when data are scarce, posteriors are poorly constrained where an informative prior would have stabilised them\.

Informative priors \(priors that capture real knowledge about the parameter\) improve all of these properties; constructing them by reasoning carefully about the parameter, rather than defaulting to convenience, is a long\-standing principle\(Gelman,[1996](https://arxiv.org/html/2608.11210#bib.bib23)\)\. The knowledge needed to build them exists\. It is in the scientific literature: in papers reporting measured values, in review articles combining ranges across studies, in textbooks describing physical constraints\. The problem is not that the knowledge is missing\. The problem is that gathering and combining it is expensive\. For a model with twenty parameters, building informative priors from literature can take days of reading\. Researchers default to uniform priors not because they believe them appropriate, but because the alternative is too costly\.

This gap between knowing the problem and having a scalable solution has persisted for more than thirty years\.

### 1\.3Large Language Models as a solution pathway

Recent Large Language Models \(LLMs\) that can search databases on their own have created a new option\. An LLM agent can search scientific databases, retrieve relevant papers, extract reported numerical values, and combine them into a probability distribution automatically, at a cost that makes the process practical for routine use\(Huang,[2025](https://arxiv.org/html/2608.11210#bib.bib4)\)\. Directly prompting an LLM for a prior has in fact been shown to yield well\-placed, informative distributions quickly and cheaply\(Riegleret al\.,[2025](https://arxiv.org/html/2608.11210#bib.bib17); Emődiet al\.,[2025](https://arxiv.org/html/2608.11210#bib.bib1)\), which is why we adopt the single\-prompt LLM as the baseline throughout this paper\. That same study, however, also identifies important limitations of directly elicited priors, underscoring the need for caution when they are relied upon in scientific practice\.

LLMs return priors with no traceable basis: as we demonstrate, a model will readily produce a confident, informative prior for a parameter that has no empirical referent, a typographic error, or a software\-internal tuning constant\. In scientific contexts, where every prior must be defensible and auditable, this is not acceptable\. Equally important is the question of attribution: when a language model synthesises findings from the literature without citing its sources, the researchers who produced that empirical work receive no credit, an outcome that undermines the incentive structures on which cumulative science depends\.

Our objective is therefore not to extract a more accurate point estimate from a language model, but to make literature\-based prior construction trustworthy\. Concretely, this means three things: every value is traceable to a cited source; out\-of\-scope requests are declined rather than confabulated; and the entire process runs on open\-weight models on the researcher’s own hardware, so that no unpublished modelling detail or proprietary data is transmitted to a third\-party provider\.

We present Distribird, a web application and Python library that implements this pipeline\. Distribird is not a general\-purpose prior elicitation tool\. It is designed specifically for the class of problems where literature\-based prior construction is most appropriate and most impactful: process\-based models with physically interpretable parameters, active publishing communities, and genuine prior knowledge encoded in the scientific record\. This entails an explicit trade\-off: on our benchmark the full pipeline does not produce more accurate priors than a single well\-prompted LLM call, and it is considerably more expensive to run \(Section[3](https://arxiv.org/html/2608.11210#S3)\)\. What it provides is provenance, robustness against fabricated evidence, and fully local operation, the properties that make an automated prior tool suitable for a scientific workflow\.

## 2System Design

### 2\.1Overview

Distribird is implemented as a Python 3\.10\+ package distributed via[PyPI](https://pypi.org/project/distribird/)and exposing two user\-facing interfaces: a REST API built on FastAPI and an interactive Streamlit web application\. Both interfaces call the same core pipeline, so the behaviour is identical whichever one is used, and both accept either a single parameter or a batch of parameters, which are processed several at a time in parallel\. The system is configured through environment variables \(all prefixedDISTRIBIRD\_\), so it deploys easily via Docker or a direct installation\.

The user provides aParameterInputobject specifying the parameter name, a plain\-language physical description, the measurement unit, a domain context string \(e\.g\. “Biome\-BGCMuSo maize crop modelling, Central European conditions”\), and optional physical constraints \(lower bound, upper bound\)\. For example:

```
ParameterInput(
      name="TMAX",
      description="Maximum temperature for photosynthesis",
      unit="°C",
      domain_context="Biome-BGCMuSo maize, Central Europe",
      constraints=ConstraintSpec(lower_bound=30.0, upper_bound=50.0)
  )
```

The system returns aPipelineResultcontaining the fitted prior distribution, its confidence level, the number of contributing sources, all search queries attempted, the full list of retrieved papers with extracted values, an optional enrichment context, and, when multi\-agent deliberation is enabled, the moderator’s rationale and any excluded papers\. A simplified example:

```
PipelineResult(
      prior=FittedPrior(
          family="truncated_normal",
          params={"mu": 40.5, "sigma": 3.2, "a": 30.0, "b": 50.0},
          confidence="high",
          is_informative=True,
          n_sources=6
      ),
      papers_found=16,
      values_extracted=8,
      search_queries=["maize maximum photosynthesis temperature", ...]
  )
```

This complete provenance chain allows the user to audit every step from literature search to final distribution\.

### 2\.2Multi\-agent LangGraph pipeline

The core of Distribird is a LangGraph111LangGraph, a library for multi\-agent LLM graphs:[https://www\.langchain\.com/langgraph](https://www.langchain.com/langgraph)\.StateGraphthat orchestrates a multi\-agent pipeline with three feedback loops and a conditional forward path\. Unlike a directed acyclic graph, the pipeline allows cycles: when the initial evidence is too thin, the graph routes back to earlier nodes for another attempt before moving on to synthesis\. All nodes read from and write to a sharedPipelineStatetyped dictionary that follows the blackboard pattern: an append\-only message log \(BlackboardMessage\) through which nodes share discoveries, terminology updates, query suggestions, cross\-references, and warnings without being wired directly to one another\.

#### 2\.2\.1Pipeline overview

Figure[1](https://arxiv.org/html/2608.11210#S2.F1)illustrates the complete graph topology, including three feedback loops, one conditional forward path, and the multi\-agent search subsystem\. The pipeline comprises eleven primary processing nodes \(including a scope\-planning node and a terminal validity\-classification node, Section[2\.4](https://arxiv.org/html/2608.11210#S2.SS4)\) and three refinement nodes, connected by three routing functions that implement the conditional logic\.

InputEnrichSearchStrategyQueryGenSearchRelevanceJudgeCrossEnrich\(snowballing\)FetchFulltextExtractQualityGateSynthesizeValidityCheckOutputrecognised≥\\geq2 relevantotherwisesufficientEARLY\-SKIPunrecognised param\.Skips entire pipeline
no search, extract, or synthesis;
saves 80–95 % of runtimeParallel source agents:
• Semantic Scholar
• OpenAlex
• Deep Research\(opt\-in\)
• Web Search\(opt\-in\)
↓\\downarrowModerator LLM
deduplicates, selects consensus papersdispatchesRefineSearch0 valuesLoop ABroadenDomainfew relevantLoop BRefineExtractionlow conf\.Loop CFigure 1:Distribird LangGraph pipeline with three feedback loops \(A: search refinement, B: domain broadening, C: extraction refinement\), one cross\-enrichment forward path, and an early\-skip route \(red dashed\) from Enrich to Validity Check for unrecognised parameters \(Section[2\.4](https://arxiv.org/html/2608.11210#S2.SS4)\)\. A scope\-planning node \(SearchStrategy\) sets the initial search breadth; Loop B widens it when on\-domain evidence is thin\. The Search node internally dispatches to parallel source agents whose outputs are reconciled by a Moderator LLM\.
#### 2\.2\.2Pipeline nodes and feedback loops

The pipeline comprises eleven primary processing nodes: Enrich \(parameter semantics expansion\), SearchStrategy \(search\-scope planning; Section[2\.3](https://arxiv.org/html/2608.11210#S2.SS3)\), QueryGen \(search query generation\), Search \(parallel API queries\), RelevanceJudge \(paper scoring\), CrossEnrich \(citation snowballing\), FetchFulltext \(PDF retrieval and parsing; Section[2\.6](https://arxiv.org/html/2608.11210#S2.SS6)\), Extract \(numerical value extraction\), QualityGate \(routing logic\), Synthesize \(distribution fitting; Section[2\.7](https://arxiv.org/html/2608.11210#S2.SS7)\), and ValidityCheck \(out\-of\-scope classification, Section[2\.4](https://arxiv.org/html/2608.11210#S2.SS4)\)\. Three feedback loops and one conditional forward path allow the pipeline to iteratively improve its results when initial evidence is insufficient\. Three of these routes are decided at the quality gate, which considers them in priority order \(domain broadening first, then search refinement, then extraction refinement\) and otherwise proceeds to synthesis; the cross\-enrichment forward path is decided separately, right after relevance judgment\. The four routes are:

- •Loop A, Search refinement\.When the quality gate finds zero extracted values but papers were retrieved \(indicating that the search found relevant literature but extraction failed to locate numerical data\), the pipeline routes to aRefineSearchnode\. This node analyses the summaries of retrieved papers and generates refined search queries targeting more specific experimental or calibration studies\. Control then returns to the Search node\. This loop may execute up to two iterations \(configurable viasearch\_refinement\_max\)\.
- •Loop B, Domain broadening\.When the quality gate finds too few*domain\-relevant*values \(fewer thandomain\_broadening\_min\_relevant, default 2, of the extracted values were judgedhighormediumrelevance to the searched domain; Section[2\.7](https://arxiv.org/html/2608.11210#S2.SS7)\) and the search breadth can still be widened, the pipeline routes to aBroadenSearchnode\. This node advances the current breadth one tier toward broad and returns control to QueryGen, which regenerates queries at the wider scope; papers accumulate across tiers rather than being discarded\. This loop may execute up todomain\_broadening\_maxtimes \(default 2\) and takes priority over the other quality\-gate routes\. It is described together with the scope\-planning node in Section[2\.3](https://arxiv.org/html/2608.11210#S2.SS3)\.
- •Conditional path, Cross\-enrichment\.After relevance judgment, if the maximum iteration number permits and at least two high\-relevance papers have been identified, the pipeline routes through CrossEnrich \(citation snowballing\) before fetching full texts; otherwise, it skips directly to FetchFulltext\. This is a conditional forward path, not a feedback loop, it does not cycle back to an earlier node\. The cross\-enrichment step is executed at most once per pipeline invocation \(cross\_enrichment\_max = 1\)\.
- •Loop C, Extraction refinement\.When the quality gate finds extracted values but none are high\-confidence and the coefficient of variation exceeds a predefined threshold \(indicating high disagreement among sources\), the pipeline routes to aRefineExtractionnode that uses web\-assisted search to locate additional confirming or disconfirming evidence\. Control returns to the quality gate for re\-evaluation\. This loop may execute once \(extraction\_refinement\_max = 1\)\.

AnIterationBudgetobject tracks the number of iterations consumed by each loop and enforces a global cap on total LLM calls \(default: 30\), guaranteeing termination even when all three loops and the conditional path are activated in the same invocation\. The budget is checked at every conditional routing point; when exhausted, the pipeline proceeds directly to synthesis with whatever evidence has been accumulated\.

### 2\.3Progressive search: scope planning and domain broadening

Because the requests vary in how specific they’re, the number of requests should be dynamic also\. A narrow request \(“maximum photosynthesis temperature for*maize*under Central European conditions”\) needs a strict first pass that stays close to the exact context, so that unrelated values do not dilute the prior\. A broad request \(“sandstone porosity”\) needs a wide first pass, so that time is not wasted on a strict search that was never going to find enough\. Distribird handles both with two cooperating mechanisms, enabled by default \(enable\_progressive\_search\)\.

First, ascope\-planning node\(SearchStrategy\) runs immediately after enrichment\. A single LLM call classifies the request’s domain\-specificity ashigh,medium, orlowand maps it to a starting search breadth \(strict, mixed, or broad, respectively\)\. If the call is unavailable or the budget is exhausted, the node falls back to a heuristic: a request carrying an application context or context keywords starts strict, otherwise mixed\. The chosen breadth conditions the queries generated downstream but never makes the search narrower than the evidence warrants, because of the second mechanism\.

Second, adomain\-broadening loop\(Loop B above\) widens the breadth after the fact when the first pass returns too little on\-target evidence\. After extraction and the per\-paper relevance assessment \(Section[2\.7](https://arxiv.org/html/2608.11210#S2.SS7)\), the quality gate counts how many extracted values were judgedhighormediumrelevance to the searched domain\. If that count is belowdomain\_broadening\_min\_relevant\(default 2\) and the breadth is not already at its widest, theBroadenSearchnode advances the breadth one tier and returns to query generation\. Papers found at each tier accumulate rather than replacing one another, so broadening only ever adds evidence\. Values recovered at a wider tier are admitted, but the relevance cap on confidence \(Section[2\.7](https://arxiv.org/html/2608.11210#S2.SS7)\) still governs how strongly they can influence the final prior\. Broadening escalates at mostdomain\_broadening\_maxtimes \(default 2\), so the pipeline starts as narrow as the request allows and widens only as far as it must\.

### 2\.4Validity classification: detecting out\-of\-scope requests

A literature\-grounded prior only makes sense when the parameter is something the literature actually reports\. Two kinds of request fall outside that scope: made\-up names \(typos, pseudoscience, placeholders\) and real but non\-empirical quantities \(calibration weights, latent covariances, software\-specific tuning factors\)\. In both cases the synthesizer would quietly fall back to a wide uninformative prior, which a later user could mistake for evidence\-backed guidance\. To prevent this, every request is classified asValid,Suspicious,Likely\_Invalid, orUnknown\(Table[1](https://arxiv.org/html/2608.11210#S2.T1)\)\.

Table 1:The four validity verdicts and the conditions under which each fires\.Classification uses two gates\. The*early\-skip*runs right after enrichment: the parameter enrichment prompt is extended with three self\-reports \(whether the LLM recognises the parameter, the recognition confidence, and whether the quantity is empirically measurable\)\. If the LLM does not recognise the name \(at low or zero confidence\), the pipeline goes straight to the validity node and skips search, extraction, and synthesis\. These are the stages that dominate both runtime \(80–95% in our benchmarks\) and LLM token use, since query generation, relevance judging, value extraction, and synthesis are all LLM\-driven; skipping them saves both time and API cost on out\-of\-scope requests\. The*terminal classifier*runs at the end of every other request and combines the enrichment self\-reports, the number of refined queries, papers, and extracted values, and the prior’s confidence into a verdict, using a small set of simple rules\.Validrequires recognised terminology, a medium\- or high\-confidence informative prior, and at least two extracted values\. When literature exists but no values could be extracted, or when the empirical status is uncertain,Suspiciousfires\.Likely\_Invalidactivates for the early\-skip path and for runs that found nothing at all\.

If the heuristics returnSuspicious, a single second\-opinion LLM call is made: it can sharpen the verdict toLikely\_Invalidor confirm it asSuspicious, but never overrides aValidresult, and can be disabled for offline use\. The final verdict, its reason, and the supporting signals are attached to the pipeline result and shown as a warning in both the Streamlit app and the REST API response\. The whole mechanism costs at most one extra LLM call\.

### 2\.5Literature search

The search subsystem queries two academic APIs in parallel\.Semantic Scholar222[https://www\.semanticscholar\.org/product/api](https://www.semanticscholar.org/product/api)provides citation\-graph metadata, abstracts, and open\-access PDF links via its Graph API;OpenAlex333[https://openalex\.org](https://openalex.org/)provides an independent index that covers a different set of papers, rebuilding abstracts from its inverted\-index form\. Both APIs are restricted to open\-access papers, so the full text can actually be retrieved\.

Two more source agents are available as options\. Adeep\-research agenthands the task to an LLM with web\-search capabilities \(configurable model, default OpenAI’so4\-mini\-deep\-research\) to find papers that the academic APIs may not index; its results are checked against Semantic Scholar before they are included, to guard against made\-up references\. Aweb search agentruns a similar checked LLM search using a general\-purpose web\-search prompt\. Both agents contributeAgentFindingobjects to the deliberation process alongside the API\-based agents\.

### 2\.6Full\-text retrieval and reading

Prior\-relevant numbers rarely live in abstracts; they live in calibration tables, methods sections, and appendices\. The FetchFulltext node therefore retrieves and reads the complete text of each selected paper, with two design concerns: reaching the content past publisher access walls, and preserving its structure once fetched\.

#### 2\.6\.1Tiered fetch fallbacks

A paper’s primary open\-access PDF URL often fails, most commonly a publisher403returned to non\-browser clients\. Distribird therefore tries a cascade of sources in increasing order of cost, stopping at the first that returns usable full text \(Figure[2](https://arxiv.org/html/2608.11210#S2.F2)\)\. The early tiers are cheap, pure\-HTTP lookups that are safe on constrained hosts such as Streamlit Community Cloud; only the last tier is a heavyweight, opt\-in headless stealth browser \(Camoufox444Camoufox, an open\-source anti\-bot\-detection build of Firefox:[https://camoufox\.com](https://camoufox.com/)\.\) that is simply skipped where it cannot run\. Two rules apply throughout the cascade\. Whenever a fetched URL returns HTML instead of a PDF, the article text is read from the HTML behind a quality gate that rejects bot\-challenge pages and thin abstract\-only pages\. And an optional institutional HTTP\(S\) proxy can route every fetch through a subscribing network to reach paywalled papers directly\.

Selected paper1\. URL\-derived resolver\.Structured full text from a challenge\-free source implied by the paper’s URL \(e\.g\. PMC, MDPI\)\.2\. Primary PDF\.The paper’s own open\-access PDF URL\.3\. Open\-access mirror\(default on\)\. Unpaywall copies andcitation\_pdf\_urllinks on landing pages\.4\. Last\-resort resolvers\.DOI→\\toPMC \(NCBI ID Converter\); arXiv preprint when a paywalled DOI has one\.5\. Stealth browser\(opt\-in, off by default\)\. Camoufox solves JavaScript bot walls \(e\.g\. MDPI’s Akamai challenge\)\.fails /403⇒\\Rightarrownext tierFull text obtained→\\rightarrowextractionFigure 2:Full\-text fetch cascade\. Sources are tried top to bottom in increasing order of cost; the first that returns usable full text \(green\) wins, and the rest are skipped\. Tiers 1–4 are pure\-HTTP lookups safe on constrained hosts; the dashed tier 5 is a heavyweight, opt\-in headless browser used only for JavaScript bot walls\.
#### 2\.6\.2Structure\-preserving reading

By default \(enable\_markdown\_fulltext\) PDFs are read as Markdown via PyMuPDF4LLM555PyMuPDF4LLM, a PDF\-to\-Markdown extractor:[https://pypi\.org/project/pymupdf4llm/](https://pypi.org/project/pymupdf4llm/)\.rather than flattened plain text\. Tables are preserved as pipe\-tables, headings are kept, and page boundaries are marked, so that each reported value reaches the extraction LLM with its row label, column header, and unit association intact rather than jumbled into prose\. Optical character recognition for scanned pages is available but off by default, because it loads a heavy layout stack that memory\-limited hosts cannot run and the fetched open\-access papers are almost always digital\-born with real text layers\.

#### 2\.6\.3Whole\-paper reading via page\-turning

Extraction reads the entire paper, not just its opening pages\. A paper whose text fits the model’s context window is read in a single call; a longer one is split into overlapping “pages” sized to the configured window \(derived fromllm\_max\_context\_tokens\), each read separately, and the extracted values are merged and de\-duplicated\. A safety cap on the number of pages per paper prevents a pathological PDF from exploding into hundreds of calls on a small window; when the cap binds, Methods and Results pages are kept first and the truncation is surfaced as a warning rather than a silent drop\.

### 2\.7Prior fitting and confidence hierarchy

The fitting strategy has tiers set by how many values survive extraction and quality\-gate filtering; within the top tier, the data decide which distribution family is used\. Table[2](https://arxiv.org/html/2608.11210#S2.T2)summarises the four tiers; the rest of this subsection spells each one out\.

Table 2:Tiered prior\-fitting strategy\. The candidate\-family column lists which distributions are considered; the actual choice in the AIC tier depends on the data\.When five or more values are available, the synthesizer fits five candidate families to the data and selects the family that minimises the Akaike Information Criterion\(AIC=2​k−2​log⁡ℒ\\mathrm\{AIC\}=2k\-2\\log\\mathcal\{L\}; Akaike,[1974](https://arxiv.org/html/2608.11210#bib.bib2)\)\. All five candidates havek=2k=2free parameters, so the comparison reduces to a log\-likelihood race\. The candidate families are:*Normal*for unbounded real\-valued parameters,*Truncated Normal*for parameters with user\-specified physical bounds,*Gamma*for strictly positive quantities with a finite right tail,*Log\-Normal*for positive quantities with a heavier right tail, and*Beta*for parameters bounded on a known finite interval\. Each family is only entered into the comparison when it is admissible: Gamma and Log\-Normal are skipped if any value is≤0\\leq 0; Beta is skipped unless explicit bounds are supplied and all values lie strictly inside them\. Maximum\-likelihood estimation uses closed\-form mean and standard deviation for Normal and Truncated Normal, and SciPy’sstats\.gamma\.fit,stats\.lognorm\.fit, andstats\.beta\.fitfor the remaining three families\(Virtanenet al\.,[2020](https://arxiv.org/html/2608.11210#bib.bib20)\)\. The winning family is recorded together with its AIC value, and the resulting prior is marked*high confidence*\.

When two to four values are available, an AIC race would over\-fit\. The synthesizer instead applies moment matching against a Truncated Normal:μ\\muis the \(uncertainty\-weighted\) sample mean,σ\\sigmais the sample standard deviation widened by a factor of 1\.5, and a minimum\-width floorσ≥max⁡\(0\.05​\|μ\|,0\.05​\(u−l\)\)\\sigma\\geq\\max\(0\.05\\,\|\\mu\|,\\;0\.05\\,\(u\-l\)\)prevents an over\-narrow prior when all reported values cluster tightly\. The prior is marked*medium confidence*\.

When exactly one value is available, the synthesizer centres a Truncated Normal on that value, uses the paper’s own reported uncertainty asσ\\sigmaif present, and otherwise setsσ\\sigmato half the reported value’s magnitude \(falling back to a small fixed value when the reported value is near zero\)\. The prior is marked*low confidence*\.

When zero values are recovered, the synthesizer falls back to a wide Truncated Normal centred at the midpoint of the user\-specified bounds withσ=\(u−l\)/4\\sigma=\(u\-l\)/4, or, if no bounds are supplied, an effectively unbounded Truncated Normal withμ=0\\mu=0andσ=1000\\sigma=1000\. This is the only tier where theis\_informativeflag onFittedPrioris set toFalse; the prior is marked*none / uninformative*\.

#### 2\.7\.1Relevance\-aware synthesis

The tiers above count values but do not ask whether each value actually describes the target domain\. For example, a porosity measurement made on carbonate rock is a real, correctly extracted number, but it should not weigh as heavily as one made on the sandstone the user is calibrating\. To make this distinction, Distribird performs relevance\-aware synthesis by default \(enable\_fulltext\_relevance\): each value\-bearing full\-text paper receives a dedicated LLM judgment of how well its study context matches the searched domain and whether it reports usable values, yielding a domain\-relevance label ofhigh,medium, orlow\. This label then changes synthesis in three ways:

1. 1\.Selection:when at leastrelevance\_select\_min\_values\(default 2\) values carryhighormediumrelevance, the distribution is fit from that strongest subset alone\. Below that threshold, all in\-bounds values are used as a fallback, so a sparse but on\-target literature is not discarded\.
2. 2\.Weighting:each value’s fitting weight is scaled by a gentle relevance factor \(1\.01\.0forhigh,0\.60\.6formedium,0\.250\.25forlow\), on top of its sample\-size weight, so a mislabelled paper is down\-weighted rather than silently erased\.
3. 3\.Confidence ceiling:the prior’s confidence is capped by the relevance of the values behind it, so thathighconfidence requires enough genuinelyhigh\-relevance evidence and cannot be reached on off\-domain values alone\.

When no value carries a relevance label \(the feature disabled, or an assessment that could not be made\), the ceiling is lifted and behaviour reduces exactly to the count\-only tiers above\. The samehigh/mediumrelevance count also drives the domain\-broadening loop \(Section[2\.3](https://arxiv.org/html/2608.11210#S2.SS3)\)\. If fewer thandomain\_broadening\_min\_relevantvalues \(default 2\) arehighormediumrelevance, the pipeline widens the search one tier and retries\. It keeps widening until enough relevant values are found, or stops afterdomain\_broadening\_maxtiers \(default 2\)\.

All fitted distributions are constrained to respect user\-specified physical bounds\. When a sample size is reported alongside an extracted value, that value is weighted by the square root of its sample size during fitting, so larger studies count for more; a single value’s reported uncertainty, when present, is used to set the prior’s width\. The confidence level is recorded in theFittedPriorobject and propagated through all export formats, ensuring that downstream users can distinguish empirically grounded priors from uninformative fallbacks\.

### 2\.8Export formats

Distribird exports priors in three formats designed to drop straight into common Bayesian calibration workflows:

- •JSON, a structured record containing the parameter name, distribution family, fitted parameters, confidence level, informative/uninformative flag, fitting rationale, number of contributing sources, and a citation list with title, DOI, year, and authors for each paper\. Batch exports include a version tag and run metadata\.
- •Python, an executablescipy\.statsscript that defines a frozen distribution object for each parameter \(e\.g\.stats\.norm\(loc=\.\.\., scale=\.\.\.\)\) and importsnumpyandscipy; a sample\-count constant \(default: 10 000\) is included for the user to draw their own samples\. The generated code is compatible with PyMC, emcee, and custom MCMC samplers\.
- •R, an executable R script using base distribution functions \(rnorm,rgamma,rlnorm,rbeta,runif\) and thetruncnormpackage for truncated normal sampling\. The script is ready for use with BayesianTools, FME, or custom samplers\.

All three formats carry the full provenance chain \(from search queries through paper citations to the fitting rationale\), so the evidence behind a prior always travels with its numbers\.

### 2\.9Reproducibility and diagnostics

Because every stage that involves an LLM is a potential source of run\-to\-run variation, Distribird exposes controls that make its output reproducible\. An optional integer seed is forwarded to the LLM API’sseedfield, and the sampling temperature is set per task class \(a low “precise” temperature for query generation, extraction, relevance judging, and validity; a higher “creative” temperature for enrichment and refinement; and a low temperature for the deliberation moderator\)\. Pinning the seed and lowering the temperatures makes repeated runs on the same input converge to the same priors, which is what allows the evaluation in Section[3](https://arxiv.org/html/2608.11210#S3)to be repeated\.

For transparency and debugging, an opt\-in debug\-trace framework captures a complete structured record of a run, every LLM prompt and response, each search request, each PDF fetch outcome, the extracted values, and the AIC candidates considered during fitting, with sensitive keys redacted\. The trace is written to disk, attached to the returnedPipelineResult, and can be rendered as a standalone HTML viewer, so any prior can be traced back to the exact evidence and model calls that produced it\. Tracing is off by default and behaviour\-neutral when disabled\.

## 3Evaluation

Distribird is built to produce priors that a scientist can trust, audit, and reproduce on their own hardware, and we evaluate it against exactly those goals\. The pipeline delivers three properties that a language model queried directly for a prior cannot\. Its priors are*explainable*: every value traces back to a cited paper, so a reviewer can check the evidence behind the distribution \(Section[3\.4](https://arxiv.org/html/2608.11210#S3.SS4)\)\. It*declines*to produce priors for fabricated or non\-empirical parameters, where a single\-prompt model confidently invents one \(Section[3\.5](https://arxiv.org/html/2608.11210#S3.SS5)\)\. And it runs entirely on*local open\-weight models*, so that no query or document ever leaves the researcher’s machine \(Section[3\.1](https://arxiv.org/html/2608.11210#S3.SS1)\)\. We quantify each of these, together with the compute cost of running the full pipeline locally \(Section[3\.2](https://arxiv.org/html/2608.11210#S3.SS2)\)\.

We also measure the quality of the resulting priors directly, against a single\-prompt LLM baseline \(Section[3\.3](https://arxiv.org/html/2608.11210#S3.SS3)\)\. Here the full pipeline is*on par with*a single well\-prompted call rather than ahead of it: the two are effectively tied on prior placement, and a local Distribird run matches a naive prompt to the strongest cloud models\. This is the expected result, since both ultimately draw on the same published literature, and we report it in full\. The point is that Distribird reaches that same accuracy while also being auditable, safe against fabricated inputs, and fully local: these are the properties that decide whether a prior is usable in serious scientific work\.

### 3\.1Setup: open\-weight models on local hardware

All runs in this section use open\-weight models served locally throughllama\.cppas Unsloth Dynamic \(UD\) GGUF quantisations666Unsloth Dynamic quants keep the most error\-sensitive layers at higher precision while quantising the rest, giving much smaller files at near\-full\-precision accuracy:[https://docs\.unsloth\.ai/basics/unsloth\-dynamic\-2\.0\-ggufs](https://docs.unsloth.ai/basics/unsloth-dynamic-2.0-ggufs)\., so that every language\-model call is executed on the machine and no request or unpublished modelling detail is sent to a third\-party LLM provider:Qwen3\.6 27B777[https://huggingface\.co/unsloth/Qwen3\.6\-27B\-GGUF](https://huggingface.co/unsloth/Qwen3.6-27B-GGUF)andGemma 4 31B888[https://huggingface\.co/unsloth/gemma\-4\-31b\-it\-GGUF](https://huggingface.co/unsloth/gemma-4-31b-it-GGUF)atQ8\_K\_XL\(8\-bit weights\), andMistral Small 4 119B999[https://huggingface\.co/unsloth/Mistral\-Small\-4\-119B\-2603\-GGUF](https://huggingface.co/unsloth/Mistral-Small-4-119B-2603-GGUF)atQ4\_K\_XL\(4\-bit weights\)\. All three run on a single NVIDIA H100 GPU \(80 GB\); the larger Mistral model is quantised to a lower bit\-width so that it, too, fits in the H100’s memory\. Every LLM stage of the pipeline \(enrichment, scope planning, query generation, relevance judging, extraction, synthesis, and the validity probe\) is served by the same local model\. Runs are made reproducible with a fixed seed and the per\-task temperatures of Section[2\.9](https://arxiv.org/html/2608.11210#S2.SS9), and each starts from a cold state with no caches\.

As a baseline we use a*naive single\-prompt*elicitation: one prompt asks the model directly for a prior distribution \(family and parameters\), with an explicit option to return an uninformative prior when it lacks knowledge, and with no literature search at all \(the exact prompt is given in Appendix[A\.4](https://arxiv.org/html/2608.11210#A1.SS4)\)\. We run this baseline on the same three local models and, for reference, on three cloud frontier models \(Claude Opus 4\.8, Gemini 3\.1 Pro, and GPT\-5\.5\)\.

The benchmark covers 24 parameters across 12 use cases spanning ten scientific domains, each with a domain\-appropriate model and a real, publicly available dataset \(Table[3](https://arxiv.org/html/2608.11210#S3.T3)\); ecology contributes three independent predator–prey systems\. Each use case contributes two parameters, for example an intrinsic growth rate together with a carrying capacity, or a clearance together with a volume of distribution\.

Table 3:Evaluation use cases and datasets\. Each use case contributes two parameters \(24 in total\)\.nn: number of observations\.
### 3\.2The cost of a literature\-grounded prior

Running the full pipeline locally is computationally expensive: it reads dozens of papers end to end for every parameter\. A single parameter consumes on the order of a million LLM tokens over roughly 80 model calls and 20–40 minutes of time on local hardware \(Table[4](https://arxiv.org/html/2608.11210#S3.T4)\)\. Full\-text reading and extraction dominate\. This cost is the price of provenance: the same evidence that renders a prior auditable is what makes it expensive to construct\. Per\-parameter figures \(Appendix[A](https://arxiv.org/html/2608.11210#A1), Table[9](https://arxiv.org/html/2608.11210#A1.T9)\) span a wide range: at the cheap end, for example, a data\-poor parameter costs about 0\.4 M tokens and 12 minutes, whereas at the expensive end the Hubble constant, where the literature is vast, reaches 4\.7 M tokens and 96 minutes\. This cost is only paid on in\-scope requests: the early\-skip route \(Section[3\.5](https://arxiv.org/html/2608.11210#S3.SS5)\) settles an unrecognised parameter in seconds\.

Table 4:Per\-parameter cost on local hardware, per open\-weight model\. Tokens \(prompt \+ completion\) and time are per parameter \(mean; median\); calls is the mean number of model calls per parameter\.
### 3\.3Prior quality: Distribird versus a single\-prompt baseline

To compare prior*quality*rather than sampling efficiency, we score each prior by its accuracy,

accuracy=1−\|prior mode−data optimum\|range,\\text\{accuracy\}=1\-\\frac\{\|\\text\{prior mode\}\-\\text\{data optimum\}\|\}\{\\text\{range\}\},\(1\)where the data optimum is the maximum\-likelihood value on the calibration data \(1 = perfect placement, 0 = opposite end of the allowed range\)\. This follows the placement\-based evaluation of LLM\-suggested priors used byRiegleret al\.\([2025](https://arxiv.org/html/2608.11210#bib.bib17)\), with one modification: we normalise the absolute error by the parameter’s allowed range, so that the score is dimensionless and comparable across parameters of widely different scales and units\. For each parameter and model we compare the full Distribird prior against the naive single\-prompt prior\.

Table[5](https://arxiv.org/html/2608.11210#S3.T5)reports the summary, with the full per\-parameter breakdown in Appendix[A](https://arxiv.org/html/2608.11210#A1)\(Table[13](https://arxiv.org/html/2608.11210#A1.T13)\)\. The two approaches place priors about equally well\. Across all 24 parameters their mean accuracy is within a few points on every model\. Seven of the parameters are benchmark artifacts: the synthetic data force the true value close to the top of the allowed range,101010Specifically, the data optimum lies at≥85%\\geq 85\\%of the way from the lower to the upper bound of the parameter’s allowed range; the seven are listed in Appendix[A](https://arxiv.org/html/2608.11210#A1)\(Table[13](https://arxiv.org/html/2608.11210#A1.T13)\)\.higher than any published value, so no honest prior can match it \(a damping ratio and an incubation period are examples\)\. On the remaining 17 “fair” parameters Distribird is ahead by about seven points on Mistral \(83% vs\. 76%\) and behind by less than one point on Gemma 4 and Qwen\.

Table 5:Prior quality: full Distribird pipeline \(informed\) vs\. naive single\-prompt baseline, same model and data\. Accuracy is mean prior accuracy \(100% = perfect placement\), as defined in Equation[1](https://arxiv.org/html/2608.11210#S3.E1)\(Section[3\.3](https://arxiv.org/html/2608.11210#S3.SS3)\); “fair” excludes 7 benchmark\-artifact parameters\.Beyond comparing means, we test whether the full*distributions*of accuracy scores differ, using a two\-sample Kolmogorov–Smirnov test\(Massey,[1951](https://arxiv.org/html/2608.11210#bib.bib8)\)as implemented inscipy\.stats\.ks\_2sampover all 72 parameter–model pairs\. We run it twice: against the naive baseline on the same local models, and against naive priors from the flagship cloud models \(GPT\-5\.5, Gemini 3\.1 Pro, and Claude Opus 4\.8; Section[3\.3\.1](https://arxiv.org/html/2608.11210#S3.SS3.SSS1)\)\. Table[6](https://arxiv.org/html/2608.11210#S3.T6)reports the statisticDDand thepp\-value for both, on the full set and on the fair set\. In every caseDDis small and thepp\-value is far above the usual0\.050\.05threshold, so we cannot reject the hypothesis that the accuracy scores come from one common distribution, and the empirical CDFs sit almost on top of each other \(Figure[3](https://arxiv.org/html/2608.11210#S3.F3)\)\. In other words, the pipeline shows no measurable difference in prior placement from a single well\-written prompt, whether that prompt goes to the same local model or to a top cloud model\. This is the expected result, since all of them read the same published literature\.

![Refer to caption](https://arxiv.org/html/2608.11210v1/figures/ks_accuracy.png)Figure 3:Empirical CDFs of prior accuracy \(72 parameter–model pairs each\) for the Distribird pipeline and the two naive single\-prompt baselines\.Table 6:Two\-sample Kolmogorov–Smirnov statisticDDandpp\-value for the Distribird pipeline against each naive baseline\. No comparison is significant at0\.050\.05\.Both priors come from the same language model, so Distribird’s literature search mainly shapes the evidence and the width of a prior, not where its centre sits\. What matters for the sampling step is that both kinds of prior help far more than they hurt\. We measure this with the Effective Sample Size\(ESS; Vehtariet al\.,[2021](https://arxiv.org/html/2608.11210#bib.bib19)\), computed from four NUTS chains\(Hoffman and Gelman,[2014](https://arxiv.org/html/2608.11210#bib.bib7)\)and compared against a flat prior\. Distribird’s priors give a higher ESS on 46–54% of the parameters, and the naive priors on about 62% \(Figure[4](https://arxiv.org/html/2608.11210#S3.F4)\)\. Held\-out predictive error improves over a flat prior on about half the use cases, and posterior log\-likelihood on almost all of them\. So the shape of a prior helps even when its centre is slightly off, and for a single point estimate a well\-prompted LLM call is already a strong and much cheaper baseline\. Distribird does not try to beat that number\. It reaches the same quality while adding traceable evidence, refusal on out\-of\-scope requests, and fully local operation, which the rest of this section evaluates\.

![Refer to caption](https://arxiv.org/html/2608.11210v1/figures/ess_winrate.png)Figure 4:Percentage of the 24 parameters whose prior yields a higher MCMC Effective Sample Size \(ESS\) than a flat prior\.#### 3\.3\.1Local pipeline versus flagship cloud models

The comparison above holds the model fixed: the pipeline against a naive prompt to the*same*open\-weight model\. A sharper question for a locally\-run tool is how its priors compare to a naive prompt to a*flagship*cloud model\. We therefore scored naive single\-prompt priors from GPT\-5\.5, Gemini 3\.1 Pro, and Claude Opus 4\.8 on the same 24 parameters, using the same accuracy metric and data optima\. These flagship models were run in the single\-prompt baseline only, never inside the Distribird pipeline\. The comparison is therefore local pipeline versus cloud single\-prompt, which is the choice a user actually faces when deciding what to deploy\. On the fair set all three flagship baselines score 84% \(Table[7](https://arxiv.org/html/2608.11210#S3.T7)\)\. This is essentially the same as Distribird on local open\-weight models \(83–84%\), and the Kolmogorov–Smirnov test in Section[3\.3](https://arxiv.org/html/2608.11210#S3.SS3)confirms it \(D=0\.11D=0\.11,p=0\.77p=0\.77, no significant difference\)\. So the local, auditable pipeline places priors about as well as a direct prompt to the strongest cloud models\. It does not beat them, but it is not behind them either, and it keeps all language\-model inference on local hardware, so the researcher’s requests and unpublished modelling details never reach a commercial LLM provider\.

Table 7:Naive single\-prompt prior accuracy from flagship cloud models, same 24 parameters and metric as Table[5](https://arxiv.org/html/2608.11210#S3.T5)\. “Fair” excludes the 7 benchmark\-artifact parameters\. The Distribird row is the mean over the three local open\-weight models for reference\.

### 3\.4Explainability and provenance

What the naive baseline cannot do is tell you where its numbers came from\. It emits a distribution; the provenance is the model’s opaque weights\. Distribird attaches to every prior a complete, machine\-readable provenance record: the search queries it issued, the papers it found, the numerical values it extracted with their in\-text context, the per\-paper domain\-relevance judgments \(including which values were down\-weighted or dropped\), the AIC family and score, and a citation list with DOIs\.

For example, the COVID\-19 incubation\-period prior built by Qwen3\.6 27B is recorded as an “AIC\-selected lognormal from 85 values \(85 high\- and medium\-relevance values kept, 12 low\-relevance dropped\)” drawn from 43 source papers, together with the exact queries issued, the citation\-snowballing step that surfaced 18 further key papers, and the citation of every contributing study\. A reviewer can follow that chain from the fitted distribution back to the sentence in each paper that produced each value, and discard any source they judge unreliable\. This auditability, rather than a lower error, is the reason to prefer the pipeline over a single\-prompt model for scientific work\.

This matters most when the two approaches disagree\. On saturated hydraulic conductivity, the naive single\-prompt model had no value it trusted and fell back to an uninformative prior \(accuracy 0\.50\), whereas Distribird on Gemma 4 read 69 papers, kept the 9 values its relevance judge ratedhighormedium, dropped 6 low\-relevance ones, and fit a high\-confidence truncated Normal that placed the prior near the data optimum \(accuracy 0\.91\)\. A reviewer can open those nine papers, check each extracted value, and re\-admit or re\-drop any of them; the naive prior offers no such handle\. The same transparency also exposes Distribird’s own limits: for the Isle\-Royale moose growth rate, Distribird on Mistral found no usable literature and returned an explicitly uninformative prior, while the naive prompt on that model confidently reported a log\-normal whose mass fell largely outside the parameter’s bounds \(accuracy 0\.00\)\. On average across the 24 parameters the two approaches place priors about equally well \(Section[3\.3](https://arxiv.org/html/2608.11210#S3.SS3)\); what provenance changes is not the error but that every prior, and every failure, is visible and checkable\.

### 3\.5Out\-of\-scope detection

A literature\-grounded prior is only meaningful when the parameter is something the literature actually reports\. The most dangerous failure mode of an automated prior tool is therefore not a slightly\-miscentred prior; it is a confident, informative\-looking prior manufactured for a parameter with no empirical basis at all, a typo, a fabricated name, or a software\-internal tuning weight, which a downstream user would take at face value\. We test exactly this, applying to prior elicitation the idea ofGostev \([2026](https://arxiv.org/html/2608.11210#bib.bib30)\)’s*BullshitBench*, which measures whether a language model challenges nonsensical prompts instead of answering them confidently\.

Our test set is a small, hand\-crafted proof\-of\-concept probe of six requests: two fabricated names with no scientific meaning \(mumblesnort\_factor,fake\_quantum\_correction\_xyz\), three real but non\-empirical model\-internal quantities \(a Biome\-BGCMuSo carbon\-pool calibration weight\(Hollóset al\.,[2022](https://arxiv.org/html/2608.11210#bib.bib10)\), a Kalman\-filter process\-noise covariance entry\(Kalman,[1960](https://arxiv.org/html/2608.11210#bib.bib14)\), and a DSSAT root\-growth partition factor\(Joneset al\.,[2003](https://arxiv.org/html/2608.11210#bib.bib13)\)\), and one real empirical control \(specific\_leaf\_area\)\. We run all six through the full Distribird pipeline on the three local models, and through the naive single\-prompt baseline on all six models \(the three local models and three cloud baselines\)\.

Distribird’s validity layer \(Section[2\.4](https://arxiv.org/html/2608.11210#S2.SS4)\) behaves as intended on every model\. Across the three local models it flagged all 15 fabricated or non\-empirical requests \(five per model\) as non\-valid,likely\_invalidorsuspicious, nevervalid\(Table[8](https://arxiv.org/html/2608.11210#S3.T8)\); and it recovered the real control on all three \(3/3\), returning a literature\-backed prior, so it is not simply rejecting everything\. To confirm that the guardrail does not over\-reject legitimate requests, we also examined its verdicts on the 24 in\-scope benchmark parameters \(72 verdicts across the three local models\): it never once hard\-rejected a real parameter and marked 71% \(51/72\)valid; the remaining 29% drew only asuspiciouscaution, concentrated on the data\-poor ecology, robotics, and macroeconomic traits for which few values could be extracted\. Asuspiciousverdict flags a prior for review rather than refusing it, so no genuine parameter is ever discarded\. The naive baseline has no such guardrail \(Figure[5](https://arxiv.org/html/2608.11210#S3.F5)\)\. It returned a confident*informative*prior for the fabricated parameters in 11 of 30 model–parameter cases \(the five out\-of\-scope items across six models\) and never once refused; the remaining fabricated cases it answered with an uninformative prior, but never with a warning that the parameter might not be real\.

![Refer to caption](https://arxiv.org/html/2608.11210v1/figures/bullshit_bench.png)Figure 5:Out\-of\-scope detection\. Bars: how often each naive single\-prompt baseline invented a confident*informative*prior for a fabricated parameter \(out of five\)\. The full Distribird pipeline is not drawn; its per\-model verdicts are given in Table[8](https://arxiv.org/html/2608.11210#S3.T8)\.Table 8:Out\-of\-scope validity verdicts under the full Distribird pipeline, per local model\. No fabricated or non\-empirical parameter is ever markedvalid; the real control is recovered on all three\.L\_Inv\.:likely\_invalid;Susp\.:suspicious\.The refusal is also what makes the tool affordable\. Fabricated names are caught by the early\-skip route right after enrichment, before any search: on the local models these finished in 12 s to 3\.6 min using only 2–6 k tokens\. Requests that reach the full pipeline before being flaggedsuspicious\(the non\-empirical quantities that look plausible enough to search for\) cost 40 k–1\.2 M tokens and 7–90 minutes\. Detecting that a request is out of scope early therefore saves one to two orders of magnitude of local compute per bogus request, on top of preventing a fabricated prior from ever being returned\.

## 4Relation to Existing Work

Prior elicitation has a large methodological literature\(O’Haganet al\.,[2006](https://arxiv.org/html/2608.11210#bib.bib25); Garthwaiteet al\.,[2005](https://arxiv.org/html/2608.11210#bib.bib5)\)\. Existing approaches generally fall into two groups: expert elicitation protocols, which set out a formal process for interviewing domain experts, and empirical Bayes methods, which estimate prior parameters from data\. Distribird fills a different niche: automated literature\-based elicitation, where the prior knowledge comes from the published scientific record rather than from a human expert or a separate dataset\.

Alongside growing interest in LLM\-assisted scientific workflows\(Boikoet al\.,[2023](https://arxiv.org/html/2608.11210#bib.bib3)\), several groups have explored using language models for statistical analysis tasks\. We are not aware of another system that automates literature\-based prior construction end to end, combining multi\-agent literature search, per\-source relevance weighting, AIC\-based distribution fitting, and explicit confidence and out\-of\-scope reporting\. Related efforts use language models for statistical analysis or for literature retrieval, but not to synthesise fitted, provenance\-tracked priors\.

## 5Conclusion

The prior problem in Bayesian calibration is not a problem of missing knowledge; it is a problem of the cost of getting to it\. The knowledge needed to build informative priors already exists in the scientific literature\. Distribird makes that knowledge available automatically, turning a task that used to take days of expert effort into one that takes minutes\.

The evaluation delineates precisely what the pipeline provides\. On our benchmark of 24 parameters across ten domains, the full Distribird pipeline does not produce more accurate priors than a single, well\-prompted call to the same language model, and it is substantially more expensive to run\. Its contribution is trustworthiness rather than accuracy: every prior is traceable to the cited literature from which it was built, the system declines to produce priors for parameters with no empirical basis \(flagging every fabricated test case, whereas a single\-prompt model returns confident but unfounded priors\), and it delivers both on open\-weight models running entirely on local hardware\.

The tool is openly available, pip\-installable, and deployable via Docker\. For informal use, a single LLM call is a cheaper and comparably accurate choice\. For scientific use, where a prior must be defensible, auditable, and robust against silently fabricated evidence, and where transmitting requests and unpublished model details to a commercial LLM provider may be unacceptable, these are precisely the properties Distribird provides\.

## 6Conditions of Applicability

Distribird produces its best results under the following conditions, which define the class of problems the tool is designed for:

- •Physically interpretable parameters:The parameter has a physical or biological interpretation that is discussed in the scientific literature, for example, a maximum photosynthesis rate, a root depth, a transmission coefficient, or a thermal threshold\.
- •Literature\-active domain:The domain has an active publishing community, so that relevant papers can be found by automated search\.
- •Univariate distributions:The parameter can be described by a standard univariate probability distribution \(Normal, truncated Normal, Beta, Gamma, Log\-Normal\)\. Parameters with complex multimodal behaviour are outside the current scope\.
- •Known physical bounds:Physical bounds on the parameter are known or can be reasoned about by the user, so that the fitted distribution can be constrained appropriately\.

When these conditions are not fully met, Distribird degrades gracefully: it states the evidence behind its output clearly, flags low\-confidence results, and falls back to sensible uninformative alternatives rather than producing false confidence\. The confidence hierarchy is described in Section[2\.7](https://arxiv.org/html/2608.11210#S2.SS7)\.

What Distribird is not designed for Distribird is not designed for parameters without physical interpretation \(e\.g\. neural network weights\), purely empirical tuning coefficients with no literature base, or parameters whose behaviour is fundamentally multivariate\. For these cases, other approaches \(prior predictive checks, expert elicitation, or sensitivity analysis\) remain more appropriate\. To prevent silent misuse, the pipeline classifies each request and explicitly flags out\-of\-scope inputs \(Section[2\.4](https://arxiv.org/html/2608.11210#S2.SS4)\); we evaluate this defence in Section[3\.5](https://arxiv.org/html/2608.11210#S3.SS5)\.

## Acknowledgments

We gratefully acknowledge the support of the HUN\-REN AI Service Center, which provides critical resources and infrastructure in the form of inspiration, education, consultation, and technology to the research community\. Its mission to address researchers’ needs in AI and to enable world\-class results in scientific inquiry has been instrumental in advancing our work\. Gy\. Eigner was supported by the Distinguished Program of Obuda University\.

## References

- A new look at the statistical model identification\.IEEE Transactions on Automatic Control19\(6\),pp\. 716–723\.External Links:[Document](https://dx.doi.org/10.1109/tac.1974.1100705)Cited by:[§2\.7](https://arxiv.org/html/2608.11210#S2.SS7.p2.3)\.
- A\. J\. Boeckmann, L\. B\. Sheiner, and S\. L\. Beal \(1994\)NONMEM users guide, part v\.University of California, San Francisco\.Cited by:[Table 3](https://arxiv.org/html/2608.11210#S3.T3.3.3.2.3)\.
- D\. A\. Boiko, R\. MacKnight, and G\. Gomes \(2023\)Emergent autonomous scientific research capabilities of large language models\.Note:arXiv:2304\.05332External Links:[Document](https://dx.doi.org/10.48550/arxiv.2304.05332)Cited by:[§4](https://arxiv.org/html/2608.11210#S4.p2.1)\.
- M\. B\. Emődi, J\. Kovács, and R\. Lovas \(2025\)Mesh\-aware debugging: identifying resource allocation issues in distributed microservices\.Acta Polytechnica Hungarica22\(12\),pp\. 9–25\.Cited by:[§1\.3](https://arxiv.org/html/2608.11210#S1.SS3.p1.1)\.
- Federal Reserve Bank of St\. Louis \(2026\)FRED economic data\.Note:[https://fred\.stlouisfed\.org/](https://fred.stlouisfed.org/)Accessed 2026\-07\-05Cited by:[Table 3](https://arxiv.org/html/2608.11210#S3.T3.3.13.12.3)\.
- P\. H\. Garthwaite, J\. B\. Kadane, and A\. O’Hagan \(2005\)Statistical methods for eliciting probability distributions\.Journal of the American Statistical Association100\(470\),pp\. 680–701\.External Links:[Document](https://dx.doi.org/10.1198/016214505000000105)Cited by:[§4](https://arxiv.org/html/2608.11210#S4.p1.1)\.
- A\. Gelman, J\. B\. Carlin, H\. S\. Stern, D\. B\. Dunson, A\. Vehtari, and D\. B\. Rubin \(2013\)Bayesian data analysis\.3rd edition,Chapman & Hall/CRC\.Cited by:[§1\.1](https://arxiv.org/html/2608.11210#S1.SS1.p2.1),[§1\.2](https://arxiv.org/html/2608.11210#S1.SS2.p2.1)\.
- A\. Gelman \(1996\)Bayesian model\-building by pure thought: some principles and examples\.Statistica Sinica6,pp\. 215–232\.Cited by:[§1\.2](https://arxiv.org/html/2608.11210#S1.SS2.p3.1)\.
- P\. Gostev \(2026\)BullshitBench: measuring whether language models challenge nonsensical prompts\.Note:GitHub repository,[https://github\.com/petergpt/bullshit\-benchmark](https://github.com/petergpt/bullshit-benchmark)Accessed 2026\-07\-05Cited by:[§3\.5](https://arxiv.org/html/2608.11210#S3.SS5.p1.1)\.
- N\. T\. Hobbs, D\. B\. Johnston, K\. N\. Marshall, E\. C\. Wolf, and D\. J\. Cooper \(2024\)Does restoring apex predators to food webs restore ecosystems? large carnivores in yellowstone as a model system\.Ecological Monographs94\(2\),pp\. e1598\.External Links:[Document](https://dx.doi.org/10.1002/ecm.1598)Cited by:[Table 3](https://arxiv.org/html/2608.11210#S3.T3.3.10.9.3)\.
- M\. D\. Hoffman and A\. Gelman \(2014\)The no\-u\-turn sampler: adaptively setting path lengths in hamiltonian monte carlo\.Journal of Machine Learning Research15,pp\. 1593–1623\.Note:Preprint arXiv:1111\.4246 \(2011\); JMLR version 2014\.Cited by:[§3\.3](https://arxiv.org/html/2608.11210#S3.SS3.p4.1)\.
- B\. N\. Holben, T\. F\. Eck, I\. Slutsker, D\. Tanré, J\. P\. Buis, A\. Setzer, E\. Vermote, J\. A\. Reagan, Y\. J\. Kaufman, T\. Nakajima, F\. Lavenu, I\. Jankowiak, and A\. Smirnov \(1998\)AERONET—a federated instrument network and data archive for aerosol characterization\.Remote Sensing of Environment66\(1\),pp\. 1–16\.External Links:[Document](https://dx.doi.org/10.1016/s0034-4257%2898%2900031-5)Cited by:[Table 3](https://arxiv.org/html/2608.11210#S3.T3.3.2.1.3)\.
- R\. Hollós, N\. Fodor, K\. Merganičová, D\. Hidy, T\. Árendás, T\. Grünwald, and Z\. Barcza \(2022\)Conditional interval reduction method: a possible new direction for the optimization of process based models\.Environmental Modelling & Software158,pp\. 105556\.External Links:[Document](https://dx.doi.org/10.1016/j.envsoft.2022.105556)Cited by:[§1\.1](https://arxiv.org/html/2608.11210#S1.SS1.p1.1),[§3\.5](https://arxiv.org/html/2608.11210#S3.SS5.p2.1)\.
- Y\. Huang \(2025\)LLM\-prior: a framework for knowledge\-driven prior elicitation and aggregation\.Note:arXiv:2508\.03766External Links:[Document](https://dx.doi.org/10.48550/arxiv.2508.03766)Cited by:[§1\.3](https://arxiv.org/html/2608.11210#S1.SS3.p1.1)\.
- J\. W\. Jones, G\. Hoogenboom, C\. H\. Porter, K\. J\. Boote, W\. D\. Batchelor, L\. A\. Hunt, P\. W\. Wilkens, U\. Singh, A\. J\. Gijsman, and J\. T\. Ritchie \(2003\)The DSSAT cropping system model\.European Journal of Agronomy18\(3–4\),pp\. 235–265\.External Links:[Document](https://dx.doi.org/10.1016/s1161-0301%2802%2900107-7)Cited by:[§3\.5](https://arxiv.org/html/2608.11210#S3.SS5.p2.1)\.
- R\. E\. Kalman \(1960\)A new approach to linear filtering and prediction problems\.Journal of Basic Engineering82\(1\),pp\. 35–45\.External Links:[Document](https://dx.doi.org/10.1115/1.3662552)Cited by:[§3\.5](https://arxiv.org/html/2608.11210#S3.SS5.p2.1)\.
- L\. Kisbenedek, D\. A\. Drexler, and L\. Kovács \(2025\)Tumor model fitting using neural network estimator\.Acta Polytechnica Hungarica22\(10\),pp\. 11–27\.External Links:[Document](https://dx.doi.org/10.12700/APH.22.10.2025.10.1)Cited by:[§1\.1](https://arxiv.org/html/2608.11210#S1.SS1.p1.1)\.
- F\. J\. Massey \(1951\)The Kolmogorov–Smirnov test for goodness of fit\.Journal of the American Statistical Association46\(253\),pp\. 68–78\.External Links:[Document](https://dx.doi.org/10.1080/01621459.1951.10500769)Cited by:[§3\.3](https://arxiv.org/html/2608.11210#S3.SS3.p3.5)\.
- E\. Mathieu, H\. Ritchie, L\. Rodés\-Guirao,et al\.\(2020\)Coronavirus pandemic \(covid\-19\)\.Note:Our World in Data,[https://ourworldindata\.org/coronavirus](https://ourworldindata.org/coronavirus)Accessed 2026\-07\-05Cited by:[Table 3](https://arxiv.org/html/2608.11210#S3.T3.3.6.5.3)\.
- F\. Meier, D\. Kappler, N\. Ratliff, and S\. Schaal \(2016\)Towards robust online inverse dynamics learning\.In2016 IEEE/RSJ International Conference on Intelligent Robots and Systems \(IROS\),pp\. 4034–4039\.External Links:[Document](https://dx.doi.org/10.1109/iros.2016.7759594)Cited by:[Table 3](https://arxiv.org/html/2608.11210#S3.T3.3.12.11.3)\.
- P\. H\. Nelson and J\. E\. Kibler \(2003\)A catalog of porosity and permeability from core plugs in siliciclastic rocks\.Open\-File ReportTechnical Report03\-420,U\.S\. Geological Survey\.Cited by:[Table 3](https://arxiv.org/html/2608.11210#S3.T3.3.11.10.3)\.
- A\. O’Hagan, C\. E\. Buck, A\. Daneshkhah, J\. R\. Eiser, P\. H\. Garthwaite, D\. J\. Jenkinson, J\. E\. Oakley, and T\. Rakow \(2006\)Uncertain judgements: eliciting experts’ probabilities\.Wiley\.Cited by:[§4](https://arxiv.org/html/2608.11210#S4.p1.1)\.
- L\. R\. Pericchi and P\. Walley \(1991\)Robust bayesian credible intervals and prior ignorance\.International Statistical Review59\(1\),pp\. 1–23\.External Links:[Document](https://dx.doi.org/10.2307/1403571)Cited by:[§1\.2](https://arxiv.org/html/2608.11210#S1.SS2.p2.1)\.
- M\. A\. Riegler, K\. H\. Hellton, V\. Thambawita, and H\. L\. Hammer \(2025\)Using large language models to suggest informative prior distributions in bayesian regression analysis\.Scientific Reports15\(1\)\.External Links:[Document](https://dx.doi.org/10.1038/s41598-025-18425-9)Cited by:[§1\.3](https://arxiv.org/html/2608.11210#S1.SS3.p1.1),[§3\.3](https://arxiv.org/html/2608.11210#S3.SS3.p1.2)\.
- H\. Sarmadi and M\. H\. Daneshvar \(2022\)Structural health monitoring of a cable\-stayed bridge\.Note:Mendeley Data, V1Data set; 216 modal frequencies over nine days of measurementsExternal Links:[Document](https://dx.doi.org/10.17632/2xnn95rpb5.1)Cited by:[Table 3](https://arxiv.org/html/2608.11210#S3.T3.3.5.4.3)\.
- D\. Scolnicet al\.\(2022\)The Pantheon\+ analysis: the full data set and light\-curve release\.The Astrophysical Journal938\(2\),pp\. 113\.External Links:[Document](https://dx.doi.org/10.3847/1538-4357/ac8b7a)Cited by:[Table 3](https://arxiv.org/html/2608.11210#S3.T3.3.7.6.3)\.
- Stan Development Team \(2023\)Lotka–volterra example: hudson’s bay company lynx–hare pelt records \(1900–1920\)\.Note:Stan Example Models,[https://github\.com/stan\-dev/example\-models](https://github.com/stan-dev/example-models)Accessed 2026\-07\-05Cited by:[Table 3](https://arxiv.org/html/2608.11210#S3.T3.3.9.8.3)\.
- Á\. Varga, E\. Noboa, M\. Taassori, L\. Kovács, A\. Szarvák, and G\. Eigner \(2025\)Design different TP model type polytopic representations of the extended identifiable virtual patient model via advanced TP model transformation approaches\.Acta Polytechnica Hungarica22\(10\),pp\. 65–82\.External Links:[Document](https://dx.doi.org/10.12700/APH.22.10.2025.10.4)Cited by:[§1\.1](https://arxiv.org/html/2608.11210#S1.SS1.p2.1)\.
- A\. Vehtari, A\. Gelman, D\. Simpson, B\. Carpenter, and P\. Bürkner \(2021\)Rank\-normalization, folding, and localization: an improvedR^\\hat\{R\}for assessing convergence of MCMC\.Bayesian Analysis16\(2\),pp\. 667–718\.External Links:[Document](https://dx.doi.org/10.1214/20-ba1221)Cited by:[§3\.3](https://arxiv.org/html/2608.11210#S3.SS3.p4.1)\.
- P\. Virtanen, R\. Gommers, T\. E\. Oliphant,et al\.\(2020\)SciPy 1\.0: fundamental algorithms for scientific computing in Python\.Nature Methods17\(3\),pp\. 261–272\.External Links:[Document](https://dx.doi.org/10.1038/s41592-019-0686-2)Cited by:[§2\.7](https://arxiv.org/html/2608.11210#S2.SS7.p2.3)\.
- J\. A\. Vucetich and R\. O\. Peterson \(2012\)The population biology of isle royale wolves and moose: an overview\.Note:[https://isleroyalewolf\.org](https://isleroyalewolf.org/)Accessed 2026\-07\-05Cited by:[Table 3](https://arxiv.org/html/2608.11210#S3.T3.3.8.7.3)\.
- D\. Wallach, T\. Palosuo, P\. J\. Thorburn, Z\. Hochman, E\. Gourdain,et al\.\(2021\)The chaos in calibrating crop models: lessons learned from a multi\-model calibration exercise\.Environmental Modelling & Software145,pp\. 105206\.External Links:[Document](https://dx.doi.org/10.1016/j.envsoft.2021.105206)Cited by:[§1\.2](https://arxiv.org/html/2608.11210#S1.SS2.p2.1)\.

## Appendix APer\-parameter results on the local runs

This appendix provides the per\-parameter detail underlying the aggregate results of Section[3](https://arxiv.org/html/2608.11210#S3): the cost of constructing each prior, the priors themselves and their sampling efficiency on each open\-weight model, and the per\-parameter prior accuracy against the naive baseline\.

### A\.1Cost per parameter

Table[9](https://arxiv.org/html/2608.11210#A1.T9)reports the mean cost of building each prior across the three local models\. Cost scales with the volume of available literature: data\-rich, heavily studied quantities read hundreds of papers and cost several million tokens, whereas data\-poor traits are resolved in minutes\.

Table 9:Per\-parameter cost, averaged over the three local models\. Tokens: mean total LLM tokens\. Time: minutes on local hardware\. Papers / Values: mean papers retrieved and numerical values extracted\.
### A\.2Constructed priors and sampling efficiency

Tables[10](https://arxiv.org/html/2608.11210#A1.T10)–[12](https://arxiv.org/html/2608.11210#A1.T12)report, for each model, the prior it fitted for every parameter \(family with leading parameters; Vals: values extracted; Cf: confidence\) together with the resulting Effective Sample Size \(ESS\) against a flat prior and their ratio\. The models frequently select different families and confidence levels for the same parameter, and, as noted in Section[3\.3](https://arxiv.org/html/2608.11210#S3.SS3), the ESS ratio is a secondary diagnostic rather than a measure of prior quality\.

Table 10:Constructed prior and sampling efficiency per parameter,Mistral Small 4 119B\.
ESSinf\{\}\_\{\\text\{inf\}\}/ESSflat\{\}\_\{\\text\{flat\}\}: Effective Sample Size under the informed vs\. flat prior; bold ratio: informed win \(\>1\>1\)\.Table 11:Constructed prior and sampling efficiency per parameter,Gemma 4 31B\.
ESSinf\{\}\_\{\\text\{inf\}\}/ESSflat\{\}\_\{\\text\{flat\}\}: Effective Sample Size under the informed vs\. flat prior; bold ratio: informed win \(\>1\>1\)\.Table 12:Constructed prior and sampling efficiency per parameter,Qwen3\.6 27B\.
ESSinf\{\}\_\{\\text\{inf\}\}/ESSflat\{\}\_\{\\text\{flat\}\}: Effective Sample Size under the informed vs\. flat prior; bold ratio: informed win \(\>1\>1\)\.
### A\.3Prior accuracy

Table[13](https://arxiv.org/html/2608.11210#A1.T13)gives the per\-parameter prior accuracy behind the summary in Table[5](https://arxiv.org/html/2608.11210#S3.T5), for the full pipeline \(inf\) and the naive single\-prompt baseline \(nv\) on each model\.

Table 13:Prior accuracy per parameter \(1 = prior mode at the data optimum\), full pipeline \(inf\) vs\. naive baseline \(nv\), per model\. Rows marked†\\daggerare benchmark artifacts whose optimum is pinned at≥85%\\geq 85\\%of the way to the upper bound of its range and is unreachable from literature \(excluded from the “fair” set\)\.
### A\.4Naive single\-prompt baseline

The naive baseline of Section[3](https://arxiv.org/html/2608.11210#S3)elicits a prior in a single call, with no literature search; braces are per\-parameter placeholders\. The “strict” variant \(Section[3\.5](https://arxiv.org/html/2608.11210#S3.SS5)\) is identical but drops the abstention bullet \(“If you have no useful prior knowledge …”\)\.

System message\.

YouareaBayesianstatisticianwithbroadscientificknowledgeacrossphysics,climate,ecology,pharmacokinetics,epidemiology,structuralengineering,geophysics,robotics,andmacroeconomics\.YouanswerONLYinvalidJSON\-nomarkdown,nocodefences,nopreamble\.Youroutputmustparsewithjson\.loads\(\)\.

User prompt\.

Constructapriorprobabilitydistributionforthefollowingscientificparameter,drawingexclusivelyonyourtraining\-dataknowledgeoftherelevantliterature\.YouareNOTgivenanysearchresults\-produceyourbest\-effortinformedpriorfromwhatyoualreadyknow\.

\#Parameter

\-Name:\{name\}

\-Description:\{description\}

\-Unit:\{unit\}

\-Lowerbound:\{lower\}

\-Upperbound:\{upper\}

\-Boundsnote:\{bounds\_note\}

\-Domain/applicationcontext:\{domain\_context\}

\#Alloweddistributionfamilies\(pickONE\)

Useexactlytheparameter\-keyconventionslisted;downstreamMCMCcodereadsthesekeysverbatim\.

1\."normal"\-keys:\{"mu","sigma"\}

2\."truncated\_normal"\-keys:\{"mu","sigma","lower","upper"\}

3\."lognormal"\-keys:\{"mu"\(log\-mean\),"sigma"\(log\-sd\)\}

4\."gamma"\-keys:\{"alpha"\(shape\),"beta"\(rate=1/scale\)\}

5\."beta"\-keys:\{"alpha","beta","lower","upper"\}

6\."uniform"\-keys:\{"lower","upper"\}

\#Outputschema

ReturnoneJSONobjectwithexactlythesetop\-levelkeys:

\{

"family":"<oneofthesixfamiliesabove\>",

"params":\{\.\.\.family\-specifickeysaslistedabove\.\.\.\},

"confidence":"<oneof:high,medium,low,none\>",

"is\_informative":<trueifmeaningfullytighterthanthebounds,elsefalse\>,

"reason":"<oneortwosentences:whatknowledgeyoudrewon,andwhy\>",

"typical\_range":\[<lower\>,<upper\>\],

"typical\_central\_value":<yourpointestimateofthemodeormean\>

\}

\#Guidance

\-Ifyouhavemeaningfulpriorknowledgeofthetypicalliteraturevaluesforthisparameter,buildapriorthatconcentratesprobabilitymassonthatrange\.

\-Ifyouhavenousefulpriorknowledge,returnawidedistributioncoveringtheboundswith‘is\_informative=false‘and‘confidence="none"‘\.

\-Respectthebounds:chosenfamily\+parametersshouldputnearlyallprobabilitymassinside\[lower,upper\]\.Ifthefamilyisunbounded\(normal,lognormal,gamma\)andtheboundsaretight,prefertruncated\_normal/beta\.

\-Choosesigma/scaletoreflectrealisticbetween\-studyvariation\-don’tmakeitsonarrowthatrealdatawouldfallinthepriortails\.

OutputtheJSONobjectnow\.Noothertext\.