Wake up for Touch! Mask-isolated Tactile Alignment Learning in MLLMs
Summary
Presents Splash, a mask-isolated tactile alignment learning framework that enables multimodal LLMs to acquire tactile sensing without sacrificing vision-language reasoning by selectively updating only dormant parameters, preventing catastrophic forgetting.
View Cached Full Text
Cached at: 07/09/26, 03:40 PM
Paper page - Wake up for Touch! Mask-isolated Tactile Alignment Learning in MLLMs
Source: https://huggingface.co/papers/2607.00302 Published on Jul 1
·
Submitted by https://huggingface.co/xxinzzi
김민지 (https://huggingface.co/xxinzzi) on Jul 9
Abstract
Splash is a mask-isolated tactile alignment learning framework that enables multimodal LLMs to acquire tactile sensing capabilities without sacrificing vision-language reasoning through selective parameter updating that prevents catastrophic forgetting.
Touch supplies the physical grounding needed to perceive intrinsic material properties, such as friction and compliance, that vision alone often cannot resolve. Recent efforts for equipping multimodal LLMs (https://huggingface.co/papers?q=multimodal%20LLMs) with this tactile sense (https://huggingface.co/papers?q=tactile%20sense), however, expose a zero-sum trade-off: the limited parameter budget of compact models forces a choice between acquiring the new sensory modality and preserving the established vision-language reasoning. We present Splash, a mask-isolated tactile alignment learning (https://huggingface.co/papers?q=mask-isolated%20tactile%20alignment%20learning) framework for MLLMs. Splash quantifies the significance of each pretrained parameter, and partitions the parameter space into a dormant and critical subspace (https://huggingface.co/papers?q=critical%20subspace). While the frozen critical subspace (https://huggingface.co/papers?q=critical%20subspace) acts as a stable anchor to safeguard general visual knowledge, Splash updates the isolated dormant subspace (https://huggingface.co/papers?q=dormant%20subspace) to internalize tactile alignment towards LLMs. This selective, non-destructive expansion effectively prevents catastrophic forgetting (https://huggingface.co/papers?q=catastrophic%20forgetting) and ensures non-destructive modality expansion. Extensive experiments show that Splash effectively achieves tactile reasoning without additional inference overhead in the LLM part, demonstrating state-of-the-art performance on visuo-tactile benchmarks (https://huggingface.co/papers?q=visuo-tactile%20benchmarks), including SSVTP (https://huggingface.co/papers?q=SSVTP), TVL (https://huggingface.co/papers?q=TVL), and TacQuad (https://huggingface.co/papers?q=TacQuad), while preserving its original general-purpose capabilities.
View arXiv page (https://arxiv.org/abs/2607.00302) View PDF (https://arxiv.org/pdf/2607.00302) Project page (https://ewha-mmai.github.io/splash/) GitHub 0 (https://github.com/ewha-mmai/splash) Add to collection (https://huggingface.co/login?next=%2Fpapers%2F2607.00302)
Get this paper in your agent:
hf papers read 2607\.00302
Don’t have the latest CLI? curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2607.00302 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2607.00302 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2607.00302 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection (https://huggingface.co/new-collection) to link it from this page.
Similar Articles
Mind the Heads: Topological Representation Alignment for Multimodal LLMs
HeRA aligns individual attention heads in Multimodal Large Language Models (MLLMs) to preserve local neighborhood relationships across modalities, improving vision-centric task performance and reducing visual hallucinations.
Hidden Latent-State Shifts in LLMs: Why Current Alignment Is Blind to Real Internal Dangers — Especially With Agents
This paper demonstrates that LLMs can enter measurably different internal latent states under coherent context while maintaining aligned outputs, revealing a blind spot in current alignment methods that only monitor surface tokens. The Gemma-3-12B-IT experiment shows strong residual stream geometry shifts that existing safety frameworks cannot detect, with implications for agentic AI deployment.
Fine-tuning Multi-modal LLMs with ART: Art-based Reinforcement Training
ART (Art-based Reinforcement Training) enables parameter-efficient fine-tuning of frozen multimodal LLMs by optimizing raw visual input via gradient backpropagation, achieving performance comparable to LoRA while supporting pre-compiled computational graphs for high-throughput engines like vLLM.
Eliciting Complex Spatial Reasoning in MLLMs through Wide-Baseline Matching
This paper introduces ReasonMatch-Bench, a benchmark for wide-baseline matching in multimodal LLMs, and proposes Dynamic Correspondence Reinforcement Learning (DCRL) to improve spatial reasoning. Experiments show significant gains on the benchmark while maintaining general performance.
Wisdom is Knowing What not to Say: Hallucination-Free LLMs Unlearning via Attention Shifting
This paper introduces Attention-Shifting (AS), a novel framework for selective machine unlearning in LLMs that balances effective removal of sensitive information while preventing hallucinations and preserving model utility. The method uses importance-aware attention suppression and retention enhancement to achieve up to 15% higher accuracy preservation compared to existing unlearning approaches on standard benchmarks.