Tag
This paper compares Structural Topic Models (STM) and BERTopic for analyzing short, open-ended survey responses, finding that BERTopic with contextual augmentation yields better topic coherence and interpretability, while STM offers stronger support for inferential covariate analysis.
This paper presents QuestBench, a benchmark built by students to evaluate deep research systems across humanities and social science domains. Results show that even advanced systems like GPT-5.5 pass only 57.58% of questions, highlighting failures in trustworthiness.
Introduces 'personality engineering,' a methodology using AI agents to parameterize, manipulate, and evaluate negotiator personality based on the interpersonal circumplex, enabling controlled experiments in negotiation theory.
This paper presents a five-stage framework integrating large language models into survey research, addressing declining response rates, sample bias, and fraudulent completions. Using 2024 Hurricane Milton survey data, the authors propose a theory-informed LLM (A-TLM) that outperforms classical imputation methods in missing-data scenarios and demonstrates manageable hallucination risk through grounded refusal.
This paper uses large language models to analyze persuasion dynamics and polarization in Reddit's r/ChangeMyView, finding that empathetic alignment increases belief change while frontal refutation diminishes it.
This paper introduces Synthetic Discussion Generation (SDG), a novel NLP framework for creating simulated discussions to enable cost-effective pilot experiments in social science research. The authors demonstrate that smaller quantized models (7B-8B parameters) can produce effective simulations at 44x lower cost than proprietary models like GPT, and apply this framework to evaluate LLM facilitators in online discussions.
OpenAI releases GABRIEL, an open-source toolkit that uses GPT to convert unstructured qualitative data (text, images) into quantitative measurements for social scientists and economists. The tool enables researchers to analyze large-scale qualitative datasets more efficiently by automating repetitive labeling tasks while preserving the richness of human data.
OpenAI argues that AI safety research on value alignment requires social scientists to help address how human cognitive biases and inconsistencies affect the data used to train AI systems. The organization proposes human-only experiments as a method to uncover alignment problems before deploying machine learning solutions.