Tag
This paper benchmarks LLM-simulated human survey responses across two large-scale datasets, finding that no model beats simple baselines at the individual level and that models systematically over-determine demographics, distorting segment differences. The failures persist across model scales and families, raising concerns about using synthetic users for decision support.
Anamnesis is an open-source platform for large-scale backstory-conditioned survey simulation using LLMs, enabling demographically controllable virtual surveys. It outperforms standard persona-prompting in replicating real-world survey distributions.
Proposes cross-survey transfer as a rigorous evaluation framework for LLM-based human survey simulation, finding that zero-shot LLMs achieve 52% accuracy on unseen items.
This paper introduces a three-axis fidelity framework (structural, marginal, individual) to evaluate how well LLMs can simulate survey responses from small pilot data. Using a COVID-19 misinformation survey, it compares prompting, rectification, and fine-tuning approaches, finding that fine-tuning offers balanced fidelity but with variation across subsamples.