Legal Guidance in the Wild: How Users Seek Legal Help in Real-World LLM Conversations
Summary
The paper studies how users seek legal guidance in real-world LLM conversations, introducing a taxonomy, annotating 2,122 WildChat conversations, and releasing a 200-conversation evaluation set along with trained classifiers and public resources.
View Cached Full Text
Cached at: 10/21/26, 07:05 AM
Paper page - Legal Guidance in the Wild: How Users Seek Legal Help in Real-World LLM Conversations
Source: https://huggingface.co/papers/2610.25159 We study how people seek legal guidance in real-world LLM conversations. We introduce a taxonomy, annotate 2,122 WildChat conversations, and release a human-reviewed evaluation set of 200 conversations. We also train and evaluate smaller classifiers to detect guidance-seeking requests and classify their legal topics.
The datasets, model checkpoints, and evaluation code are publicly available. We hope these resources support further research into legal guidance seeking in conversational AI.
Happy to discuss the work and hear your thoughts!
Similar Articles
Security and Privacy Prompts in the Wild: What Users Ask LLMs and How LLMs Respond
This paper analyzes real-world user queries about digital security and privacy asked to LLMs, categorizing them into nine topics and evaluating response quality and consistency across commercial and open-weight models.
Adopt $\neq$ Adapt: Longitudinal Analyses of LLM Conversations in the Wild
This paper analyzes longitudinal conversational trajectories of Bing Copilot users and compares them with WildChat data, finding that individual user habits are sticky and that WildChat overrepresents power users, challenging static views of user-LLM interactions.
DLawBench: Evaluating LLMs Through Multi-Turn Legal Consultation
DLawBench is a new benchmark for evaluating large language models in multi-turn legal consultation, covering Chinese and US law with four client types. Experiments show significant room for improvement, with the best model achieving only 0.562 on legal reasoning.
We’ve been analyzing how people are using LLMs for legal and compliance tasks (GDPR, AI Act, etc.).
Analysis of LLM usage in legal and compliance tasks reveals that models often produce confident but unverifiable citations, raising questions about reliable legal grounding for AI outputs.
LegalWorld: A Life-Cycle Interactive Environment for Legal Agents
LegalWorld is a life-cycle interactive environment that models Chinese civil litigation as a causally connected state chain across five stages, paired with LongJud-Bench for evaluating legal agents across the full process.