More Context, Larger Models, or Moral Knowledge? A Systematic Study of Schwartz Value Detection in Political Texts

Hugging Face Daily Papers Papers
schwartz-values political-text value-detection context retrieval-augmented moral-knowledge natural-language-processing

Summary

A systematic study on detecting Schwartz values in political text, comparing context lengths, model sizes, and retrieval-augmented generation methods. Results show that full-document context improves supervised models but not zero-shot LLMs, while retrieved moral knowledge consistently helps via early fusion.

Detecting Schwartz values in political text is difficult because implicit cues often depend on surrounding arguments and fine-grained distinctions between neighboring values. We study when context and explicit moral knowledge help sentence-level value detection. Using the ValuesML/Touch{é} ValueEval format, we compare sentence, window, and full-document inputs; no-RAG and retrieval-augmented settings with a curated moral knowledge base; supervised DeBERTa-v3-base/large encoders; and zero-shot LLMs from 12B to 123B parameters. The results show that more context is not uniformly better: full-document context improves supervised DeBERTa encoders by 3.8--4.8 macro-F1 points over sentence-only input, but does not consistently help zero-shot LLMs. Retrieved moral knowledge is more consistently useful in matched comparisons, improving each tested model family and context condition under early fusion. However, scaling from DeBERTa-v3-base to large and from 12B to larger LLMs does not guarantee gains, and simple early fusion outperforms the tested late-fusion and cross-attention RAG variants for encoders. Per-value analyses show that context and retrieval help most for socially situated or conceptually confusable values. These findings suggest that value-sensitive NLP should evaluate context, knowledge, and model family jointly rather than treating longer inputs or larger models as universal improvements.
Original Article
View Cached Full Text

Cached at: 05/22/26, 10:19 AM

Paper page - More Context, Larger Models, or Moral Knowledge? A Systematic Study of Schwartz Value Detection in Political Texts

Source: https://huggingface.co/papers/2605.22641

Abstract

Context and moral knowledge enhance sentence-level value detection differently across model types, with full-document context benefiting supervised models but not zero-shot LLMs, and retrieved knowledge consistently improving performance through early fusion.

Detecting Schwartz values in political text is difficult because implicit cues often depend on surrounding arguments and fine-grained distinctions between neighboring values. We study when context and explicit moral knowledge help sentence-level value detection. Using theValuesML/Touch{é} ValueEval format, we compare sentence, window, and full-document inputs; no-RAG and retrieval-augmented settings with a curated moral knowledge base; supervisedDeBERTa-v3-base/large encoders; andzero-shot LLMsfrom 12B to 123B parameters. The results show that more context is not uniformly better: full-document context improves supervised DeBERTa encoders by 3.8--4.8macro-F1points over sentence-only input, but does not consistently helpzero-shot LLMs. Retrieved moral knowledge is more consistently useful in matched comparisons, improving each tested model family and context condition underearly fusion. However, scaling fromDeBERTa-v3-base to large and from 12B to larger LLMs does not guarantee gains, and simpleearly fusionoutperforms the tested late-fusion andcross-attentionRAG variants for encoders. Per-value analyses show that context and retrieval help most for socially situated orconceptually confusable values. These findings suggest that value-sensitive NLP should evaluate context, knowledge, and model family jointly rather than treating longer inputs or larger models as universal improvements.

View arXiv pageView PDFGitHub0Add to collection

Get this paper in your agent:

hf papers read 2605\.22641

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper1

#### VictorYeste/value-context-rag-deberta-v3-base-doc-rag Text Classification• 0.2B• Updatedabout 3 hours ago • 28

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2605.22641 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2605.22641 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

LLMs Can Better Capture Human Judgments--With the Right Prompts

arXiv cs.CL

This paper presents simple prompting strategies that help large language models better capture the full distribution of human judgments, improving alignment on moral scenarios and beliefs. The authors show that asking models to report standard deviations and response proportions, along with ensuring scenario clarity, yields better agreement with human responses.

PolitNuggets: Benchmarking Agentic Discovery of Long-Tail Political Facts

arXiv cs.AI

PolitNuggets is a multilingual benchmark for evaluating large reasoning models within agentic frameworks on their ability to discover and synthesize long-tail political facts by constructing biographies for 400 global elites. The benchmark introduces evaluation protocols like FactNet and reveals that current systems struggle with fine-grained details and efficiency.

Evaluation Awareness in Language Models Has Limited Effect on Behaviour

arXiv cs.CL

This paper investigates whether verbalized evaluation awareness (VEA) in large reasoning models causally affects their behavior on safety, alignment, moral reasoning, and political opinion benchmarks. The authors find that VEA has limited behavioral impact, with near-zero effects from injecting VEA and small shifts from removing it, suggesting that high VEA rates should not be taken as strong evidence of strategic behavior or alignment tampering.