llm-audit

This paper introduces a Voice Retention/Representation Ratio metric to audit representational bias in LLM summaries of employee feedback, finding that summarization pipelines filter by prevalence rather than sentiment — rarely voiced concerns are dropped 86% of the time, and short or German-only content is systematically lost.

This paper audits a multilingual affective generation benchmark and reveals that system rankings are driven by measurement artifacts rather than genuine performance differences, with annotator variability playing a key role.

This paper conducts a systematic audit of six open-weight LLMs, revealing that job-posting language triggers gender and racial biases in recruitment tasks, with implications for compliance under the EU AI Act and U.S. EEOC guidelines.

This paper introduces Anchor, a synthetic audit framework to evaluate long-horizon persona stability and trajectory recall in AI companions, finding that current models fail to reliably preserve identity and memory across extended interactions.

This paper audits how LLMs default to US or Chinese legal frameworks based on input language, finding a systematic bias where English prompts favor US answers and Chinese prompts favor China answers. The authors propose mitigations.