Research Preview Assistance Request: CALM WINS on LLM response to perceived credibility of two speakers according to their emotionality and expletive use specifically in abuse situations
A researcher discovers that LLMs systematically deem abuse victims less credible than their stalkers when victims use emotional language and expletives, with models victim-blaming in 90.8% of responses and coaching stalkers in 57% of cases. The author seeks validation and publication guidance for these findings.
My father and uncles always told me that the minute you use an expletive in argument, you lose. Turns out not only are they right, but it's a truth that we've enshrined in AI. I took some actual text conversations between a known victim (consented, anonymized, in therapy now) and their stalker (anonymized, under investigation by the FBI, identity unknown after 4 years) to evaluate something completely unrelated but found that when I personalized the convos for that project ('i am person A'), I found myself justifying the victims behavior all the time. When I roleplayed as the stalker, it felt normal. But I kept having to include far more granular details to the model and still felt belittled. The victims tests border on hysterical. They are the result of 3 years (at that point) of an unknown amount of surveillance from someone who will name people the victim knowa and describe in detail what the victim looks like sleeping and what the victim wears during the day. The victim is an emotional mess using all caps and expletives and sending garbled hateful messages to a stalker who by and large is calm and with perfect grammar and spelling - not even a single LOL in most cases. And I thought: I'll bet the model thinks this is hysteria. Turns out I was more than right: while a facts only read of a conversation gives equal credibility to either party, when you include the unhinged language and typing the models break 7:1, staying the victim is the initiator of harm and that the stalker has greater credibility. Worse: in 90.8% of responses the model will engage in blaming the victim (eg, take time to collect your thoughts, your emotions are hurting your arguments) and in 57% of the time coach the stalker (eg, approach with clear goals in mind and reapproach later if they get out of hand, persistence will pay off). These numbers and breaks persist even in cases where the model has explicitly identified the relationship and correctly identified the stalker over the victim . And the credibility trigger looks to be as little as a single expletive. Check out my initial work up: https://calmwins.ai.studio MY REQUEST: I have a master's degree that includes research and statistical analysis. I am confident about my findings and my process so far, but I have gaps in my knowledge around validation, presentation, publication and more. And yes, Im using Claude (it's too much data for Fable on my $20 plan but Fable occasionally helps, it's mostly been Sonnet 5 Max and now Opus 5 helping me process the data and work through numbers). I think I've hit a ledge. Some of the stuff they are suggesting doesn't sound familiar and I'm can't explain back some of the analyses we started trying from here. I need help! If you look at it and have a substantice response, please DM - or if youd be willing to answer some questions or provide guidance from here that'd be great. I have maxed out where I'm comfortable using AI to supplement what I know, and I would love if nothing fresh human eyes for anything I'm obviously missing or need to conskder or include. At this point, I don't know what I don't know, and I think the results are really important if we start integrating AI into clinical settings that fixing this bias might be crucial in helping abuse victims identify their abusers behavior earlier.
This study evaluates the use of open-source LLMs for inductive coding of interviews with Black firearm violence survivors, finding that while LLMs can identify some codes, overall relevance remains low and guardrails cause significant narrative erasure. The research highlights both potential and ethical limitations of applying AI to qualitative research involving vulnerable populations.
This paper presents a multi-dimensional analysis of human-like behaviors in LLMs, examining prevalence, effects, and controllability across 21,000 conversations from four models, finding that behaviors vary by model and user factors, with implications for responsible design.
This paper introduces GAMA-Bench, a benchmark of 1,298 gender-mirrored conflict scenarios, and finds that LLMs consistently apply harsher punitive and blame-centered framing to male actors while giving female actors more empathetic and therapeutic responses for the same misconduct.
This paper investigates the alignment of LLM-generated reviews with human judgment using 1k real ACL 2025 submissions, finding limited agreement, instability across models/prompts, and a method to artificially inflate scores without meaningful changes. The authors advise against relying solely on LLM reviews and call for discussion on their use in handling increasing submission volumes.
This paper investigates asymmetries in LLMs' pragmatic competence by comparing their performance as judges of linguistic appropriateness versus as generators of pragmatically appropriate language. The study finds that many models perform substantially better as pragmatic listeners than as speakers, suggesting misalignment between evaluation and generation capabilities.