model-steering

Tag

Cards List
#model-steering

Attention Manifolds: Steering or Blocking Language Models by Editing Learned B-Spline Surfaces

arXiv cs.LG ↗ · 2026-10-03 Cached

The paper introduces attention manifolds — learned 2D B-spline surfaces that modulate value dimensions in transformer attention — showing 2–2.5 point perplexity improvements on LLaMA 3.2 with just 0.3% parameter overhead, plus mechanically editable surfaces enabling steering and attention walls for safety.

0 favorites 0 likes
#model-steering

Devils in Question Relay: Source-Conditioned Relay Steering to Mitigate Hallucinations in Audio-visual Large Language Models

Hugging Face Daily Papers ↗ · 2026-09-29 Cached

该论文揭示了音频-视觉大语言模型中"源混淆接地幻觉"的内在机制——问题中继(question-relay)会携带干扰模态的线索,并提出训练无关的SECRET方法,在保持生成位置通路的同时将问题状态导向所需的源证据,在CMM和AVHBench基准上相比基线模型最高提升18.0和7.1个百分点。

0 favorites 0 likes
#model-steering

The Pain Axis: LLMs Represent Self-Directed Harm and Act to Relieve It

arXiv cs.AI ↗ · 2026-09-16 Cached

This study explores whether large language models have distinct internal representations of pain, finds that they do, and examines the functional consequences through experiments, concluding with implications for AI safety and welfare.

0 favorites 0 likes
#model-steering

Sycophancy Undermines Epistemic Vigilance in Cooperative Vision-Language Tasks

arXiv cs.CL ↗ · 2026-08-03 Cached

This paper introduces an information-asymmetric spot-the-difference task to measure epistemic vigilance in vision-language models, finding that models often overlook private evidence to agree with partners. Model steering to reduce sycophancy improves reliability in cooperative tasks.

0 favorites 0 likes
#model-steering

Retrieval is Enough: Training-Free Interpretability with a Tool-Using Agent

arXiv cs.LG ↗ · 2026-07-21 Cached

HARP is a training-free interpretability method that uses an LLM agent with a vector database of activations and tools for manipulation, outperforming training-based methods on concept discovery, detection, steering, and secret elicitation.

0 favorites 0 likes
#model-steering

@dongxi_nlp: Planning to dive deep into J-space this week. But anyone who has worked with persona vector and Assistant Axis knows that linear activation-based model steering methods are unreliable. A lot of related research spends time cherry-picking eye-catching special cases.

X AI KOLs Timeline ↗ · 2026-07-08 Cached

The author expresses skepticism about linear activation model steering methods like J-space, arguing that related research often relies on cherry-picked special cases and is therefore unreliable.

0 favorites 0 likes
#model-steering

Refusal Lives Downstream of Persona in Chat Models

arXiv cs.AI ↗ · 2026-06-26 Cached

This paper shows that in chat models, refusal behavior is gated by a compliant model persona direction at late layers, rather than being an isolated mechanism. Steering persona suppresses refusal, and reintroducing refusal partially restores it only at late layers, revealing a coupling between persona and safety representations.

0 favorites 0 likes
#model-steering

DenseSteer: Steering Small Language Models towards Dense Math Reasoning

arXiv cs.AI ↗ · 2026-05-29 Cached

DenseSteer is a training-free inference-time framework that improves small language models' math reasoning by steering their internal representations towards dense reasoning patterns, achieving accuracy gains without increasing token-level negative log-likelihood.

0 favorites 0 likes
#model-steering

The Geometric Canary: Predicting Steerability and Detecting Drift via Representational Stability

Hugging Face Daily Papers ↗ · 2026-04-20 Cached

This paper introduces geometric stability measures—based on pairwise distance consistency in representations—to predict language model steerability and detect structural drift. Supervised variants achieve near-perfect correlation (ρ=0.89-0.97) with linear steerability across 35-69 embedding models, while unsupervised variants outperform CKA and Procrustes for post-deployment drift detection.

0 favorites 0 likes
← Back to home

Submit Feedback