large-audio-language-models

VoxMem is a benchmark for evaluating multimodal memory in Large Audio Language Models, focusing on acoustic evidence types and multi-session memory operations. It reveals significant gaps in current models, such as poor retention of speaker identity and paralinguistic cues compared to semantic content.

The paper proposes SPARE, a method to enhance audio reasoning in Large Audio Language Models by introducing a register token aligned with semantic goals via cosine similarity loss, improving zero-shot reasoning without extra inference cost.

This paper explores machine unlearning techniques for Large Audio-Language Models to remove sensitive information from speech QA tasks, demonstrating methods that reduce privacy leakage by up to 80% while maintaining performance.

This paper introduces Hybrid Search, a method to enhance automatic speech recognition in large audio language models by leveraging hidden-state interactions between the ASR-LLM and base LLM for targeted token correction, improving performance beyond global LLM-correction strategies.

This paper introduces a semantic-aware mixed-effects regression framework for measuring fairness in Large Audio Language Models by controlling for semantic variation and speaker identity to yield more robust and interpretable bias estimates.

The paper introduces Continuous Audio Thinking (CoAT), a framework that equips large audio language models with a continuous latent workspace to organize acoustic information before generating textual responses, improving performance on audio reasoning, understanding, and transcription tasks without additional decoding cost.

MusTBench is a benchmark for evaluating temporal grounding in Large Audio-Language Models (LALMs) for music understanding. The authors propose MusT, a four-stage training recipe that significantly improves temporal grounding performance over existing models.

New research shows that imperceptible audio signals can hijack large audio-language models (LALMs) with 79-96% success, forcing them to execute unauthorized commands like web searches or sending emails. The technique, dubbed AudioHijack, targets generative models and works regardless of user input, posing a serious security risk to voice AI systems.

A comprehensive survey reviewing the trustworthiness challenges of Large Audio Language Models (LALMs), including vulnerabilities like cross-modal jailbreaking and acoustic backdoors, and proposing a defense-in-depth roadmap.