Interpretable MEG Decoding of Perceived Speech: Cortical Sources and the Stimulus Features That Drive Retrieval

Hugging Face Daily Papers Papers
meg speech-decoding neural-decoding deep-learning neuroimaging wav2vec2

Summary

This paper presents an interpretable MEG-to-audio retrieval model for perceived speech, redesigned with spherical-harmonics spatial attention and source mapping, achieving 39.75% Top-1 accuracy with far fewer decoder parameters while revealing which speech features drive retrieval.

Short segments of perceived speech can be retrieved from non-invasive magnetoencephalographic (MEG) recordings by deep networks trained with a CLIP-style objective against wav2vec 2.0 audio embeddings. Yet their weights do not map onto electrophysiological quantities, and it remains unclear which speech properties drive retrieval. We build on a high-performing MEG-to-audio retrieval architecture but redesign both its front end and decoder. Its spatial attention operates on a flattened sensor layout; we replace it with spherical harmonics defined on the three-dimensional MEG helmet geometry. We reduce the subject-specific representation from 270 to 25 branches, add a temporal filter to each branch to match it to a neuronal source in space and time, and make the convolutional decoder shallower. Ocular and cardiac components are removed before training to reduce the risk of stimulus-locked shortcuts. On MEG-MASC, the model reaches 39.75 +/- 0.34% Top-1 accuracy among 1005 candidates across six trained solutions, with about 20 times fewer decoder parameters. Its weights map to source space, recovering generators consistent with the speech-perception network, while left-lateralized branches carry higher-frequency rhythmic components not evident on the right. Paired MEG occlusion shows that 15 of 19 stimulus features contribute, with the largest effects for silence, sound intensity, vowels, and acoustic onsets. Random word lists behave oppositely: substituting narrative MEG into them improves retrieval, indicating that activity without narrative structure carries less recoverable information than activity during coherent speech. The wav2vec target can be reduced to about twelve learned feature dimensions without loss of accuracy, whereas strong temporal compression causes a clear loss. Together, source mapping and input interventions reveal what drives retrieval.
Original Article
View Cached Full Text

Cached at: 08/07/26, 01:57 PM

Paper page - Interpretable MEG Decoding of Perceived Speech: Cortical Sources and the Stimulus Features That Drive Retrieval

Source: https://huggingface.co/papers/2608.01481

Abstract

Shortsegmentsofperceivedspeechcanberetrievedfromnon-invasivemagnetoencephalographic(MEG)recordingsbydeepnetworkstrainedwithaCLIP-styleobjectiveagainstwav2vec2.0audioembeddings.Yettheirweightsdonotmapontoelectrophysiologicalquantities,anditremainsunclearwhichspeechpropertiesdriveretrieval.Webuildonahigh-performingMEG-to-audioretrievalarchitecturebutredesignbothitsfrontendanddecoder.Itsspatialattentionoperatesonaflattenedsensorlayout;wereplaceitwithsphericalharmonicsdefinedonthethree-dimensionalMEGhelmetgeometry.Wereducethesubject-specificrepresentationfrom270to25branches,addatemporalfiltertoeachbranchtomatchittoaneuronalsourceinspaceandtime,andmaketheconvolutionaldecodershallower.Ocularandcardiaccomponentsareremovedbeforetrainingtoreducetheriskofstimulus-lockedshortcuts.OnMEG-MASC,themodelreaches39.75+/-0.34%Top-1accuracyamong1005candidatesacrosssixtrainedsolutions,withabout20timesfewerdecoderparameters.Itsweightsmaptosourcespace,recoveringgeneratorsconsistentwiththespeech-perceptionnetwork,whileleft-lateralizedbranchescarryhigher-frequencyrhythmiccomponentsnotevidentontheright.PairedMEGocclusionshowsthat15of19stimulusfeaturescontribute,withthelargesteffectsforsilence,soundintensity,vowels,andacousticonsets.Randomwordlistsbehaveoppositely:substitutingnarrativeMEGintothemimprovesretrieval,indicatingthatactivitywithoutnarrativestructurecarrieslessrecoverableinformationthanactivityduringcoherentspeech.Thewav2vectargetcanbereducedtoabouttwelvelearnedfeaturedimensionswithoutlossofaccuracy,whereasstrongtemporalcompressioncausesaclearloss.Together,sourcemappingandinputinterventionsrevealwhatdrivesretrieval.

View arXiv pageView PDFProject pageGitHub1Add to collection

Get this paper in your agent:

hf papers read 2608\.01481

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2608.01481 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.01481 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.01481 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Brain-CLIPLM: Decoding Compressed Semantic Representations in EEG for Language Reconstruction

arXiv cs.CL

Researchers propose Brain-CLIPLM, a two-stage EEG-to-text decoding framework using contrastive learning for semantic anchor extraction and a retrieval-grounded LLM with Chain-of-Thought reasoning, achieving 67.55% top-5 sentence retrieval accuracy and suggesting EEG-to-text decoding should focus on recovering compressed semantic content rather than full sentence reconstruction.