CADENCE: A Cardiac Atom Dictionary for Interpretable Neural Concept Extraction from ECG Foundation Models

# CADENCE: A Cardiac Atom Dictionary for Interpretable Neural Concept Extraction from ECG Foundation Models
Source: [https://arxiv.org/html/2607.25244](https://arxiv.org/html/2607.25244)
Yixuan Duan,Arjun NaikDepartment of Computer ScienceRice UniversityHoustonTexasUSA[an182@rice\.edu](https://arxiv.org/html/2607.25244v1/mailto:[email protected]),Sadeer Al\-KindiDepartment of CardiologyHouston Methodist HospitalHoustonTexasUSA[sal\-kindi@houstonmethodist\.org](https://arxiv.org/html/2607.25244v1/mailto:[email protected])andWei QiuDepartment of Electrical and Computer EngineeringRice UniversityHoustonTexasUSA[wq8@rice\.edu](https://arxiv.org/html/2607.25244v1/mailto:[email protected])

\(2027\)

###### Abstract\.

Foundation models for 12\-lead electrocardiograms \(ECGs\) demonstrate strong transferability across clinical tasks, yet the physiological knowledge encoded in their representations remains opaque, limiting their interpretability, auditability, and scientific utility\. We presentCADENCE\(*Cardiac Atom Dictionary for Explainable Neural Concept Extraction*\), a framework that decomposes the latent representations of an ECG foundation model into a human\-interpretable and queryable dictionary of physiological concepts\. Using a BatchTopK sparse autoencoder, CADENCE factorizes Layer\-6 embeddings extracted from more than nine million ECG tokens into8,1928\{,\}192sparse cardiac*atoms*\. Compared with individual dimensions of the original dense embedding space, the learned atoms align substantially better with both clinical phenotypes and waveform morphology\. CADENCE recovers a broad spectrum of cardiological concepts, including arrhythmias, conduction abnormalities, infarction and repolarization patterns, chamber and electrical\-axis findings, and lead\- and beat\-phase\-specific waveform primitives\. At Layer 6, the best\-performing atoms achieve mean AUROCs of0\.880\.88for clinical phenotypes and0\.900\.90for waveform morphology, compared with0\.780\.78and0\.830\.83, respectively, for the best dense embedding dimensions\. Sparse atom\-based probes match or outperform dense embedding probes for phenotype, morphology, and age prediction while attributing each prediction to a small set of interpretable atoms\. For phenotype prediction, the test AUROC improves from0\.930\.93in the dense embedding space to0\.950\.95in the sparse atom space\. Moreover, atom\-space geometry recovers physiologically coherent relationships, while targeted ablation of phenotype\-associated atoms produces selective changes in frozen downstream model outputs\. An automated large language model pipeline generates atom descriptions and quantitatively validates their descriptions by predicting held\-out atom activations\. Across independent external ECG datasets, CADENCE recovers overlapping physiological concepts and maintains consistent phenotype\-prediction performance\. Together, these findings establish CADENCE as a scalable framework for discovering and auditing the physiological knowledge encoded by ECG foundation models, providing a transparent foundation for scientific analysis and trustworthy clinical AI\.

Sparse autoencoders, Mechanistic interpretability, Foundation models, Electrocardiogram \(ECG\), Representation learning, Concept discovery, Interpretable AI in healthcare

††copyright:none††journalyear:2027††conference:Proceedings of the 33rd ACM SIGKDD Conference on Knowledge Discovery and Data Mining; August 1–5, 2027; San Jose, CA, USA††isbn:978\-1\-4503\-XXXX\-X/2027/08††ccs:Computing methodologies Learning latent representations††ccs:Applied computing Health informatics††ccs:Computing methodologies Neural networks††ccs:Information systems Data mining## 1\.Introduction

![Refer to caption](https://arxiv.org/html/2607.25244v1/pipline.png)Figure 1\.Overview of CADENCE\.\(a\) Dictionary construction\.A frozen ECG foundation model encodes a 12\-lead ECG into patch tokens; a BatchTopK sparse autoencoder*decomposes*a layer’s Layer\-6 representationxℓx\_\{\\ell\}into sparse*cardiac atoms*and*reconstructs*it asx^ℓ\\hat\{x\}\_\{\\ell\}\.\(b\) Clinical knowledge discovery\.The atom dictionary drives three analyses:*concept discovery*,*relationship analysis*, and*concept\-based prediction*\.\(c\) LLM\-assisted atom annotation\.An LLM describes an atom from its top\-activating ECG crops and geometry, then validates the description by predicting*held\-out*activations\.Three\-part schematic: \(a\) a frozen ECG foundation model encodes a 12\-lead ECG; a BatchTopK sparse autoencoder decomposes a layer’s representation into cardiac atoms and reconstructs it; \(b\) the atom dictionary supports concept discovery, relationship analysis as a network, and attributable concept\-based prediction; \(c\) an LLM describes an atom and validates it by predicting held\-out activations\.Electrocardiography \(ECG\) is the most widely used cardiac diagnostic test, yet reading one is itself a difficult, expertise\-intensive skill: clinically decisive information is carried by subtle, millisecond\-scale morphological details distributed across twelve simultaneous leads, competent interpretation takes years of training\(Salernoet al\.,[2003](https://arxiv.org/html/2607.25244#bib.bib45)\), and even physicians interpret ECGs with substantial and variable error rates\(Cooket al\.,[2020](https://arxiv.org/html/2607.25244#bib.bib46)\)\. Precisely because the raw signal is hard for humans to fully parse, self\-supervised ECG foundation models \(FMs\) that learn general\-purpose representations and transfer to dozens of downstream tasks—arrhythmia detection, risk prediction, and screening—often from a single frozen backbone\(Guet al\.,[2026](https://arxiv.org/html/2607.25244#bib.bib1); Naet al\.,[2024](https://arxiv.org/html/2607.25244#bib.bib2); McKeenet al\.,[2024](https://arxiv.org/html/2607.25244#bib.bib3)\)are attractive, but they also make interpretability more important, not less: understanding what such a model encodes is valuable both for trusting and auditing its predictions and as a route to surfacing the physiological structure hidden in the signal\. Yet these gains come from high\-dimensional embeddings whose internal structure is opaque: a practitioner can see that a model predicts left bundle branch block, but not what physiological evidence the representation encodes\. In a clinical setting this opacity is a barrier to trust, auditing, and scientific use—we cannot easily ask an ECG FM what it has learned about the heart\.

Interpreting these models is hard because individual neurons are*polysemantic*: a network can represent more features than it has dimensions, packing them into*superposition*, so each neuron responds to several unrelated features rather than mapping cleanly onto one clinical concept\. Sparse autoencoders \(SAEs\) can undo this superposition, decomposing activations into a large dictionary of sparse, approximately monosemantic features, and have exposed interpretable structure in language\(Brickenet al\.,[2023](https://arxiv.org/html/2607.25244#bib.bib5)\), protein\(Simon and Zou,[2025](https://arxiv.org/html/2607.25244#bib.bib9)\), and, more recently, genomic and single\-cell foundation models—small gene language models\(Guanet al\.,[2025](https://arxiv.org/html/2607.25244#bib.bib47)\), single\-cell transcriptomic models\(Kendiukhov,[2026](https://arxiv.org/html/2607.25244#bib.bib48)\), and computational\-pathology foundation models\(Kimet al\.,[2026](https://arxiv.org/html/2607.25244#bib.bib49)\)\. Whether the same holds for physiological time\-series FMs—and whether the recovered features map onto real cardiology—remains open\.

We answer this for 12\-lead ECG with CADENCE \(*Cardiac Atom Dictionary for Explainable Neural Concept Extraction*\)\. As illustrated in[Figure1](https://arxiv.org/html/2607.25244#S1.F1)a, CADENCE trains a BatchTopK SAE on the residual stream of an ECG foundation model \(CSFM\) and factorizes its 6 layers patch\-token embeddings into a dictionary of∼8,000\{\\sim\}8\{,\}000sparse atoms: the encoder decomposes each embedding into a few active atoms and the decoder reconstructs it, so the dictionary is faithful to the model yet far more interpretable than raw neurons\. This turns a black\-box backbone into a queryable substrate for knowledge discovery\.

Building on this dictionary, CADENCE drives three complementary analyses \([Figure1](https://arxiv.org/html/2607.25244#S1.F1)b\)\.*Concept discovery*associates each atom with clinical phenotypes and waveform morphology, showing that atoms align with cardiology far better than raw neurons\.*Relationship analysis*reads off physiologically coherent relations between concepts directly from atom\-space geometry, without relational supervision\.*Concept\-based prediction*uses sparse atom probes that match or exceed the dense embedding for phenotype, morphology, and age while attributing each prediction to a small set of named atoms\. An automated LLM pipeline \([Figure1](https://arxiv.org/html/2607.25244#S1.F1)c\) then describes each atom from its top\-activating ECG crops and activation geometry and then validates the description by predicting held\-out atom activations, yielding human\-readable, quantitatively checked annotations at scale\. Finally, we show that the frozen MIMIC\-trained dictionary transfers to an independent, cardiologist\-labeled external cohort \(PTB\-XL\) without retraining, recovering the same physiological concepts and maintaining phenotype\-prediction performance\. Together, these findings establish CADENCE as a scalable framework that turns an opaque ECG foundation model into a transparent, queryable dictionary of physiological concepts—discovered, related, predictive, automatically described, and externally validated—providing a foundation for scientific discovery and trustworthy clinical AI\.

Our contributions are:

- •To our knowledge we are the first to open up a cardiac physiological\-signal foundation model: we show that its opaque, high\-dimensional embeddings can be pulled out of superposition into a dictionary of sparse, approximately monosemantic features that align with clinical cardiac concepts far better than the model’s raw neurons\.
- •We introduce CADENCE, which trains a BatchTopK sparse autoencoder on the residual stream of an ECG foundation model to factorize it into∼8,000\{\\sim\}8\{,\}000sparse cardiac atoms that reconstruct the representation faithfully yet are far more interpretable than raw neurons\.
- •Across phenotype and morphology prediction, CADENCE probes match or exceed probes on the dense embedding while attributing each prediction to a small set of named atoms—atoms that an automated LLM pipeline describes and validates at scale\.
- •CADENCE turns an opaque ECG foundation model into an auditable dictionary of physiologically grounded concepts: practitioners can inspect what a representation encodes, decompose each prediction into named atoms, and surface confounded or shortcut features\.

## 2\.Related Work

#### ECG foundation models\.

Self\-supervised pre\-training on large unlabeled ECG corpora now yields general\-purpose encoders that transfer across arrhythmia detection, risk stratification, and screening tasks\(Hannunet al\.,[2019](https://arxiv.org/html/2607.25244#bib.bib16); Ribeiroet al\.,[2020](https://arxiv.org/html/2607.25244#bib.bib17); Attiaet al\.,[2019](https://arxiv.org/html/2607.25244#bib.bib18)\), typically via masked\-signal\(Heet al\.,[2022](https://arxiv.org/html/2607.25244#bib.bib33)\)or contrastive objectives\(Kiyassehet al\.,[2021](https://arxiv.org/html/2607.25244#bib.bib34); Mehari and Strodthoff,[2022](https://arxiv.org/html/2607.25244#bib.bib35)\)over 12\-lead recordings\(Guet al\.,[2026](https://arxiv.org/html/2607.25244#bib.bib1); Naet al\.,[2024](https://arxiv.org/html/2607.25244#bib.bib2); McKeenet al\.,[2024](https://arxiv.org/html/2607.25244#bib.bib3)\)\. These models deliver strong downstream accuracy from a single frozen backbone, but they are used as black boxes: their embeddings are high\-dimensional and their learned structure is not directly inspectable\. Our work takes such a model as given and asks a complementary question: how to*read out*the physiological concepts it has already learned\.

#### Sparse autoencoders and mechanistic interpretability\.

A central obstacle to interpreting neural representations is*superposition*: networks encode more features than they have dimensions, so individual neurons are polysemantic and do not align with human concepts\(Elhageet al\.,[2022](https://arxiv.org/html/2607.25244#bib.bib4)\)\. Sparse autoencoders \(SAEs\) address this by learning an overcomplete, sparsely activating dictionary that reconstructs the model’s activations\(Olshausen and Field,[1996](https://arxiv.org/html/2607.25244#bib.bib19)\), yielding features that are far more monosemantic than neurons\(Brickenet al\.,[2023](https://arxiv.org/html/2607.25244#bib.bib5); Cunninghamet al\.,[2023](https://arxiv.org/html/2607.25244#bib.bib6); Templetonet al\.,[2024](https://arxiv.org/html/2607.25244#bib.bib23)\)\. Subsequent work has scaled SAEs to frontier language models and improved the sparsity mechanism, including top\-kkand BatchTopK formulations that fix the active\-code budget and stabilize training\(Makhzani and Frey,[2013](https://arxiv.org/html/2607.25244#bib.bib20); Gaoet al\.,[2024](https://arxiv.org/html/2607.25244#bib.bib7); Bussmannet al\.,[2024](https://arxiv.org/html/2607.25244#bib.bib8)\), alongside Gated and JumpReLU variants that reduce activation shrinkage\(Rajamanoharanet al\.,[2024a](https://arxiv.org/html/2607.25244#bib.bib21),[b](https://arxiv.org/html/2607.25244#bib.bib22)\)\. We adopt a BatchTopK SAE but apply it, for the first time to our knowledge, to a physiological time\-series foundation model rather than to language\.

#### Interpreting scientific foundation models with SAEs\.

Beyond language, SAEs have recently been used to interpret scientific sequence models\. InterPLM trains SAEs on protein language model embeddings and recovers thousands of interpretable features that map onto binding sites, structural motifs, and functional domains\(Simon and Zou,[2025](https://arxiv.org/html/2607.25244#bib.bib9)\)\. Related work trains and evaluates SAEs on protein language models to surface both universal and family\-specific features and to propose mechanistic hypotheses\(Adamset al\.,[2025](https://arxiv.org/html/2607.25244#bib.bib10)\)\. Beyond proteins, SAEs have begun to open up other biological foundation models: single\-cell transcriptomic models such as Geneformer and scGPT\(Kendiukhov,[2026](https://arxiv.org/html/2607.25244#bib.bib48)\), gene language models\(Guanet al\.,[2025](https://arxiv.org/html/2607.25244#bib.bib47)\), and computational\-pathology image models\(Kimet al\.,[2026](https://arxiv.org/html/2607.25244#bib.bib49)\), in each case recovering biologically meaningful features from otherwise opaque embeddings\. These studies establish SAEs as a route from representation to biological insight for scientific foundation models; we extend this paradigm to*physiological signals*, where features additionally carry lead\-specific spatial and beat\-phase temporal structure, and where a generative decoder lets us relate atoms back to waveforms\.

#### Interpretability for medical and ECG models\.

Prior interpretability for ECG classifiers has largely relied on post\-hoc saliency and attribution maps\(Selvarajuet al\.,[2017](https://arxiv.org/html/2607.25244#bib.bib27); Sundararajanet al\.,[2017](https://arxiv.org/html/2607.25244#bib.bib28); Lundberg and Lee,[2017](https://arxiv.org/html/2607.25244#bib.bib29); Zhanget al\.,[2021](https://arxiv.org/html/2607.25244#bib.bib11)\), linear probing of hidden layers\(Alain and Bengio,[2016](https://arxiv.org/html/2607.25244#bib.bib26)\), or concept\-based and concept\-bottleneck models\(Kimet al\.,[2018](https://arxiv.org/html/2607.25244#bib.bib30); Kohet al\.,[2020](https://arxiv.org/html/2607.25244#bib.bib31)\)\. Saliency highlights*where*a specific prediction looks but not*what*reusable concepts a representation encodes; probing requires a predefined label for every concept of interest and cannot discover unnamed features\. By decomposing the backbone into a concept dictionary, our approach is unsupervised at the feature level, discovers sub\-diagnostic waveform primitives that have no standard label, and quantifies concept structure directly in the model’s own representation\. Taken together, these gaps motivate a concept\-discovery framework specifically designed for physiological foundation models\.

![Refer to caption](https://arxiv.org/html/2607.25244v1/figure2_combined.png)Figure 2\.What the dictionary encodes, how well it predicts, and where it fires\.\(a,b\)Phenotype, morphology, and age across layers, solid*SAE atoms*vs\. dotted*CSFM dense*with shaded SAE\-over\-dense margin:\(a\)best single\-atom AUROC,\(b\)linear\-probe test AUROC\.\(c,d\)One atom’s red activated region on two 10 s example ECGs, each shown on a MIMIC \(top\) and an external PTB\-XL \(bottom\) record:\(c\)a phenotype atom for PVC and\(d\)a rhythm atom for atrial fibrillation firing on the fibrillatory baseline\. A morphology \(ST\-segment\) atom is in[FigureA9](https://arxiv.org/html/2607.25244#A1.F9)and a rhythm atom in[FigureA8](https://arxiv.org/html/2607.25244#A1.F8)\.

## 3\.Methods

### 3\.1\.Datasets

We build the dictionary onMIMIC\-IV\-ECG\(Gowet al\.,[2023](https://arxiv.org/html/2607.25244#bib.bib12); Johnsonet al\.,[2023](https://arxiv.org/html/2607.25244#bib.bib13); Goldbergeret al\.,[2000](https://arxiv.org/html/2607.25244#bib.bib14)\), a large public collection of de\-identified 12\-lead ECGs: a subject\-disjoint sample of150,000150\{,\}000ECGs for SAE training and a held\-out∼15,000\{\\sim\}15\{,\}000for all downstream analyses \(cohort details in[AppendixA\.10](https://arxiv.org/html/2607.25244#A1.SS10)\)\. For external validation we apply the frozen, MIMIC\-trained SAEs toPTB\-XL\(Wagneret al\.,[2020](https://arxiv.org/html/2607.25244#bib.bib44)\), an independent, cardiologist\-labeled 12\-lead cohort \([AppendixA\.6](https://arxiv.org/html/2607.25244#A1.SS6)\)\.

### 3\.2\.ECG foundation model backbone

We interpret a pre\-trained 12\-lead ECG foundation model \(CSFM\)\(Guet al\.,[2026](https://arxiv.org/html/2607.25244#bib.bib1)\), a transformer encoder\(Vaswaniet al\.,[2017](https://arxiv.org/html/2607.25244#bib.bib32)\)with six layers and hidden widthd=768d\{=\}768\. Each recording is preprocessed identically to pre\-training: signals are resampled to 250 Hz, cleaned per channel, andzz\-normalized per lead\. The model tokenizes each lead into non\-overlapping 100 ms patches \(25 samples\), yielding12​leads×100​patches=120012\\ \\text\{leads\}\\times 100\\ \\text\{patches\}=1200patch tokens per ECG\. We extract the residual\-stream activations at 6 layers,H∈ℝ1200×dH\\in\\mathbb\{R\}^\{1200\\times d\}, as the target for dictionary learning; the backbone is frozen throughout\.

### 3\.3\.Sparse autoencoder

We factorize the 6 layers activations with a BatchTopK sparse autoencoder \(SAE\)\(Gaoet al\.,[2024](https://arxiv.org/html/2607.25244#bib.bib7); Bussmannet al\.,[2024](https://arxiv.org/html/2607.25244#bib.bib8)\)\. For a standardized tokenx∈ℝdx\\in\\mathbb\{R\}^\{d\}\(per\-feature mean/scale estimated once on a random token sample\), the encoder produces latent codes and the decoder reconstructs the input,

\(1\)z=BatchTopKk​\(Wenc​\(x−bdec\)\+benc\),x^=Wdec​z\+bdec\.z=\\mathrm\{BatchTopK\}\_\{k\}\\\!\\big\(W\_\{\\mathrm\{enc\}\}\(x\-b\_\{\\mathrm\{dec\}\}\)\+b\_\{\\mathrm\{enc\}\}\\big\),\\qquad\\hat\{x\}=W\_\{\\mathrm\{dec\}\}\\,z\+b\_\{\\mathrm\{dec\}\}\.where BatchTopK retains, on average, thekklargest activations per token across the batch, giving an overcomplete dictionary ofFF*atoms*\. We choose BatchTopK for its fixed average activation budget—which lets the number of active atoms vary across tokens—and its stable training on large overcomplete dictionaries\. We useF=8192F\{=\}8192andk=128k\{=\}128, chosen by a grid search over dictionary size and sparsity that trades off reconstruction explained variance, dead\-atom fraction, and interpretability yield \([App\.A\.1](https://arxiv.org/html/2607.25244#A1.SS1)\)\. The training objective is reconstruction error with an auxiliary term that lets the most\-dead latents explain the residual, mitigating dead atoms, and the decoder columns are unit\-normalized after each step:

\(2\)ℒ=∥x−x^∥22\+α​∥\(x−x^\)−x^aux∥22\.\\mathcal\{L\}=\\lVert x\-\\hat\{x\}\\rVert\_\{2\}^\{2\}\+\\alpha\\,\\lVert\(x\-\\hat\{x\}\)\-\\hat\{x\}\_\{\\mathrm\{aux\}\}\\rVert\_\{2\}^\{2\}\.The SAE is trained on 6 layers tokens sampled from∼\\sim150k ECGs\. Each recording contributes multiple tokens balanced across leads \(60 per recording\), for a training corpus of roughly9×1069\\times 10^\{6\}\(9M\) tokens\. We report reconstruction explained variance, the average active\-code countL0L\_\{0\}, and the number of dead atoms as dictionary\-quality diagnostics\.

### 3\.4\.Atom–concept association and fingerprints

To relate atoms to clinical concepts we use per\-ECG atom activations, defined as the maximum activation of each atom over the ECG’s tokens\. We use the maximum activation because many ECG abnormalities are temporally localized \(e\.g\., PVCs, pacing spikes, ST changes\), and averaging would dilute these sparse events\. For a binary conceptcc\(e\.g\., a phenotype label\) and atomjj, the association is the rank\-based AUROC\(Hanley and McNeil,[1982](https://arxiv.org/html/2607.25244#bib.bib37)\)ofaja\_\{j\}againstcc, folded to\[0\.5,1\]\[0\.5,1\]to be sign\-agnostic\. The*best\-atom*score of a concept at a layer is the largest such association over the dictionary, evaluated on the held\-out test set\. We compare concepts by their best\-atom AUROC—the single most concept\-selective atom—because interpretability hinges on whether a concept is captured by one nameable unit\. We summarize each concept by an*atom\-space fingerprint*: the length\-FFvector of signed AUROCs of every atom againstcc\. Concept–concept similarity is the cosine of these fingerprints; as an activation\-based alternative we also use a*prototype*fingerprint, the meanzz\-scored atom activation over the concept’s positive ECGs\. We additionally derive continuous and measurement\-based concepts \(heart rate, PR/QRS/QT intervals, electrical axis, and thresholded morphology such as wide\-QRS or prolonged QTc, using standard diagnostic thresholds\(Kligfieldet al\.,[2007](https://arxiv.org/html/2607.25244#bib.bib38)\)\) from the recordings’ machine measurements\. The complete set of concepts CADENCE is evaluated against—diagnostic phenotypes, continuous measurements, derived interval concepts, demographics, and morphology primitives, with their definitions and cohort prevalences—is listed in[TableA4](https://arxiv.org/html/2607.25244#A1.T4)\.

### 3\.5\.Sparse probing for downstream prediction

To test whether atoms are predictive as well as interpretable, we fitℓ2\\ell\_\{2\}\-regularized logistic\-regression probes\(Pedregosaet al\.,[2011](https://arxiv.org/html/2607.25244#bib.bib43)\)on the per\-ECG atom activations—a sparse\-probing analysis over named features\(Gurneeet al\.,[2023](https://arxiv.org/html/2607.25244#bib.bib36)\)—and compare them to probes on the dense 6 layers embedding, using a subject\-disjoint train/test split\. Because atom activations are sparse and named, each probe’s prediction decomposes into a small set of per\-atom coefficient contributions, which we report as interpretable attributions\. For a given diagnosis we rank atoms strictly by probe coefficient—with no manual reselection—and inspect the top few; atoms that carry a large weight but rarely fire on the test set are kept and explicitly marked as non\-visualizable rather than dropped\.

### 3\.6\.Automated LLM interpretation and validation

We scale interpretation from hand\-reading to the dictionary with a geometry\-first LLM pipeline, following an InterPLM\-style describe\-then\-validate protocol in which a language model explains each feature from its top activations\(Billset al\.,[2023](https://arxiv.org/html/2607.25244#bib.bib24)\)\([Figure1](https://arxiv.org/html/2607.25244#S1.F1)c\)\.

We auto\-interpret only atoms whose activation evidence can be constructed and fairly validated, filtering out*dead*,*rare*,*near\-constant*,*weak*, and*lead\-diffuse*atoms from pre\-hoc activation and geometry statistics\. For each remaining atom the model produces a structured, geometry\-first description from its top\-activating waveform crops and activation\-geometry summaries alone, with no machine\-report text\. We then validate each description by having the model predict held\-out activations from the text alone, scored by a rank correlation; inclusion thresholds, prompting, and validation details are in[AppendixA\.7](https://arxiv.org/html/2607.25244#A1.SS7)\.

## 4\.Results

### 4\.1\.Clinical concepts across network depth

We first ask*where*in the network different clinical concepts become linearly accessible\. For each layerℓ∈\{1,…,6\}\\ell\\in\\\{1,\\dots,6\\\}we train a separate SAE, compute per\-ECG feature activations, and score each concept by its held\-out best\-atom AUROC \([Sec\.3](https://arxiv.org/html/2607.25244#S3)\), taking, for the per\-feature comparison, the best value over concepts as each feature’s concept\-association strength\.

Across layers, SAE atoms are also far cleaner concept detectors than the raw CSFM dimensions, reaching substantially higher per\-feature association than the dense embedding at every layer; we defer this atoms\-vs\-neurons comparison to[AppendixA\.2](https://arxiv.org/html/2607.25244#A1.SS2)\([FigureA4](https://arxiv.org/html/2607.25244#A1.F4)\)\.

#### Concepts become more decodable with depth, and atoms beat the dense embedding\.

Grouping concepts into the three downstream categories used later for prediction—phenotype \(16 diagnoses\), morphology \(QRS/QTc/PR abnormality\), and age—and plotting the mean best\-atom AUROC per category against depth \([Figure2](https://arxiv.org/html/2607.25244#S2.F2)a\), two patterns emerge\. First,every category becomes more decodable with depth: best\-atom AUROC rises monotonically from Layer 1 to Layer 6 \(phenotype0\.73→0\.880\.73\\\!\\to\\\!0\.88, morphology0\.72→0\.900\.72\\\!\\to\\\!0\.90, age0\.71→0\.830\.71\\\!\\to\\\!0\.83\), confirming that the backbone progressively concentrates clinical concepts into individual directions the SAE can isolate\. Second,the best SAE atom beats the best dense dimension, increasingly so with depth: for phenotype and morphology the SAE atom exceeds the CSFM dense embedding at every layer \(at Layer 6, phenotype0\.880\.88vs\.0\.780\.78, morphology0\.900\.90vs\.0\.830\.83\), with the margin widening from shallow to deep layers; the age signal, stronger in the dense embedding early on, is matched and then exceeded by atoms by the deepest layers\. Together these results explain our choice of Layer 6 for the main dictionary: clinical concepts are most linearly separable, and most cleanly captured by single atoms, there\.

#### Sparse atom probes match or exceed the dense embedding\.

We test whether imposing sparsity and interpretability costs predictive accuracy by comparing, at each layer, a linear probe fit on the dense CSFM embedding \(mean\-pooled,768768\-d\) against one fit on the SAE atom activations \(max\-pooled,81928192\-d\), for three downstream task families \([Figure2](https://arxiv.org/html/2607.25244#S2.F2)b\)\. Probes are elastic\-net logistic regressions trained on the training split and evaluated on a subject\-disjoint test set; for the phenotype and morphology panels we report the mean test AUROC over the tasks in each family\.

Across all three task families and all six layers,the sparse atom probe matches or exceeds the dense\-embedding probe\([Figure2](https://arxiv.org/html/2607.25244#S2.F2)b\)—interpretability comes at no measurable accuracy cost\. At the deepest layer it improves mean test AUROC by\+0\.017\+0\.017on phenotype \(0\.929→0\.9460\.929\\\!\\to\\\!0\.946\),\+0\.011\+0\.011on age, and\+0\.005\+0\.005on morphology, a significant gain for phenotype \(disjoint confidence bands at every layer\) and parity for morphology \([FigureA4](https://arxiv.org/html/2607.25244#A1.F4)\)\. The advantage is largest in the*shallow*layers—at Layer 1 the atom probe reaches0\.8430\.843vs\.0\.8050\.805on phenotype and0\.8610\.861vs\.0\.8370\.837on age—then narrows by Layers 5–6 as the backbone concentrates each concept into more linearly separable directions, consistent with the SAE disentangling features still superposed in the early\-layer dense embedding\. Because each atom is named and each prediction decomposes into a handful of per\-atom contributions, the resulting probe is as accurate as the dense baseline yet directly attributable\.

### 4\.2\.Clinical concept discovery

Overlaying an atom’s activation onto the raw waveform reveals not only*that*an atom is concept\-associated but*what*it detects and*where*\. Doing so for atoms of different kinds shows that the dictionary is organized hierarchically, spanning three levels of temporal and spatial abstraction \([Figure2](https://arxiv.org/html/2607.25244#S2.F2)\); the full set of phenotype and morphology features CADENCE recovers, grouped by category, is listed in[TableA4](https://arxiv.org/html/2607.25244#A1.T4)\.

#### Rhythm level \(multi\-beat timing\)\.

A sinus\-bradycardia atom \(L6/3082, best\-atom AUROC0\.980\.98\) activates on the*elongated diastolic baseline between beats*—the prolonged TP/RR interval \([FigureA8](https://arxiv.org/html/2607.25244#A1.F8),[App\.A\.4](https://arxiv.org/html/2607.25244#A1.SS4)\)\. Across both example records it fires on every inter\-beat segment in lead II and stays silent through the QRS–T complexes, so its “feature” is a property of the*spacing*of beats\. This shows that atoms can encode temporal structure that only exists*across*several beats: the SAE represents heart rate as a distributed, beat\-locked baseline detector\. A single named atom captures such multi\-beat features directly, firing on the spacing between beats\.

#### Phenotype level \(a discrete pathological event\)\.

A premature\-ventricular\-complex atom \(L6/5530, AUROC0\.900\.90\) behaves as an*event detector*: within an otherwise regular sinus rhythm it fires on a*single*premature, wide, bizarre QRS and its following pause, while remaining silent on all surrounding normal beats \([Figure2](https://arxiv.org/html/2607.25244#S2.F2)c\)\. The atom thus isolates one clinically meaningful, temporally sparse abnormality per record\. The same event morphology is picked out by the corresponding PVC atom on an external PTB\-XL record \([Figure2](https://arxiv.org/html/2607.25244#S2.F2)c, bottom\), so the localization holds across datasets\. A second, cross\-dataset phenotype example is atrial fibrillation: an AF atom \(L6/5715\) fires on the irregular fibrillatory baseline between QRS complexes on both a MIMIC and an external PTB\-XL record \([Figure2](https://arxiv.org/html/2607.25244#S2.F2)d\)\. On the surface ECG, atrial fibrillation replaces organized P waves with continuous low\-amplitude fibrillatory activity and an irregularly irregular ventricular response\(Kligfieldet al\.,[2007](https://arxiv.org/html/2607.25244#bib.bib38)\); the atom fires on exactly this fibrillatory baseline in the inter\-QRS segments, localizing the arrhythmia’s diagnostic signature on the trace\. Further atom localizations—sinus bradycardia, atrial flutter, paced rhythm, and prolonged PR—are shown in the appendix \([FigureA8](https://arxiv.org/html/2607.25244#A1.F8)–[FigureA8](https://arxiv.org/html/2607.25244#A1.F8)\)\.

#### Morphology level \(a sub\-beat waveform primitive\)\.

At the finest scale, atoms localize sub\-beat waveform*primitives*: a specific segment of*every*beat in a specific lead\. They are reusable geometric building blocks that recur on each cardiac cycle and from which higher\-level ischemia, repolarization, and conduction concepts are composed\. We show two such localizations—a prolonged PR segment and an ST\-segment primitive—in the appendix \([FigureA8](https://arxiv.org/html/2607.25244#A1.F8),[FigureA9](https://arxiv.org/html/2607.25244#A1.F9)\)\.

Taken together, the three atoms span timescales from multi\-beat rhythm, through a single\-beat pathological event, down to a sub\-beat morphological segment, and range from lead\-agnostic to strictly lead\-specific, and each one is made visible the same way—by highlighting where the atom fires on the raw ECG\. The SAE dictionary is therefore a layered vocabulary, from waveform primitives up to rhythm, each element localizable on the signal\.

### 4\.3\.Concept relationship analysis

![Refer to caption](https://arxiv.org/html/2607.25244v1/figure_relations_cosine.png)Figure 3\.Phenotype relationships in atom space\.Concept–concept cosine similarity of prototype fingerprints \(yellow stars: known cardiology pairs\)\.Beyond how well concepts are encoded, we ask whether the*relations*among concepts are recovered\. We represent each concept by an atom\-space*prototype*fingerprint—its meanzz\-scored atom activation over the concept’s positive ECGs—and measure concept–concept similarity as the cosine of these fingerprints\. Without any relational supervision, the curated concepts self\-organize into two physiologically coherent blocks \([Figure3](https://arxiv.org/html/2607.25244#S4.F3)\)\. The first groups LBBB, RBBB, IVCD, and paced rhythm together with the derived wide\-QRS measurement \(pairwise cosine0\.800\.80–0\.870\.87\), and places prolonged QTc adjacent to it \(0\.640\.64–0\.790\.79\)—recovering the interval\-containment relationship that a wider QRS mechanically lengthens the QT\(Kligfieldet al\.,[2007](https://arxiv.org/html/2607.25244#bib.bib38); Rautaharjuet al\.,[2009](https://arxiv.org/html/2607.25244#bib.bib39)\)\. Within this block, LBBB and RBBB are the least similar pair \(0\.350\.35\), consistent with their opposite ventricular activation\(Kligfieldet al\.,[2007](https://arxiv.org/html/2607.25244#bib.bib38)\)despite a shared wide QRS\. A second block groups the atrial tachyarrhythmias—atrial fibrillation and flutter \(cosine0\.810\.81\)—apart from the conduction concepts, matching their distinct atrial origin\(Hindrickset al\.,[2021](https://arxiv.org/html/2607.25244#bib.bib40)\)\. The recovered structure is robust to the fingerprint definition: a signed\-AUROC\-fingerprint variant yields the same structure and agrees essentially perfectly with the prototype cosine \([AppendixA](https://arxiv.org/html/2607.25244#A1),[FigureA13](https://arxiv.org/html/2607.25244#A1.F13)\)\.

### 4\.4\.Concept\-based prediction

![Refer to caption](https://arxiv.org/html/2607.25244v1/figure_steer_four.png)\(a\)

![Refer to caption](https://arxiv.org/html/2607.25244v1/figure_pvc_paced_probe.png)\(b\)

Figure 4\.Concept\-level steering and sparse probing\.\(a\)For four targets—LBBB, wide QRS, age, one\-year mortality—the change in a frozen predictor’s target logit as one Layer\-6 atom is scaled from dose11to0onN=80N\{=\}80positives: the orange*target*atom drives the logit down, while a frequency\-matched green*random*atom and a blue*other\-concept*atom do not; bands are 95% bootstrap CIs\.\(b\)Sparse probing decomposes two diagnosis predictions—premature ventricular contractions \(PVC, probe AUROC0\.9470\.947; top\) and paced rhythm \(0\.9510\.951; bottom\)—into their five highest\-weight atoms, each bar labeled by its most over\-represented report statement \(auxiliary\)\.#### Single atoms selectively steer the downstream readout\.

If atoms are genuinely the units in which the model represents a concept, then removing a single concept\-atom should selectively change that concept’s downstream prediction—an atom\-level analogue of activation steering\(Turneret al\.,[2023](https://arxiv.org/html/2607.25244#bib.bib25); Templetonet al\.,[2024](https://arxiv.org/html/2607.25244#bib.bib23)\)\. We test this with an ablation audit \([Figure4](https://arxiv.org/html/2607.25244#S4.F4)a\)\. For each target we train a frozen linear predictor on the CSFM Layer\-6 mean\-pooled embedding—which never sees the SAE—and, using an exact integrated\-gradients attribution\(Sundararajanet al\.,[2017](https://arxiv.org/html/2607.25244#bib.bib28)\)in atom space, identify the single atom that most contributes to that target’s logit\. We then scale that atom’s activation from full to zero, decode the edited codes back to the embedding, and read the frozen predictor, onN=80N=80high\-confidence target\-positive ECGs\. Two matched controls ablate instead a frequency/magnitude\-matched*random*atom and a*wrong*atom that strongly encodes a different concept \(atrial fibrillation\) but is orthogonal to the target readout\.

Across all four targets, ablating the target atom drives the target logit down monotonically with dose and by a large margin at full ablation \(−0\.62\-0\.62for LBBB,−0\.40\-0\.40for wide\-QRS,−0\.36\-0\.36for age,−0\.27\-0\.27for mortality\), whereas both the random and the wrong atom leave the target logit unchanged\. A single named atom therefore acts as a selective control knob on the corresponding readout, for targets as different as a conduction diagnosis, a QRS\-width measurement, patient age, and prognosis\.

#### Sparse probing decomposes a diagnosis into named atoms\.

Sparse probing makes a diagnosis prediction legible by decomposing it into a handful of atoms and showing, for each, the clinical statement it fires on\. We illustrate this for premature ventricular contractions \(PVCs\), where the atom probe reaches test AUROC0\.9470\.947\([Figure4](https://arxiv.org/html/2607.25244#S4.F4)b\)\. Strictly by probe weight, the top\-five visualizable atoms are each report\-enriched for a ventricular\-ectopy statement\. The dominant\-weight atom \(rank 1, coefficient0\.730\.73, roughly three times the next\) is∼90×\\sim\\\!90\\timesover\-represented among ECGs the machine reads say have “frequent premature ventricular contractions” \(lift91\.591\.5\); ranks 2–4 carry the same frequent\-PVC and ventricular\-premature\-complex statements\. Thus the probe’s accuracy is not carried by an opaque combination of features but by a small stack of atoms that each concentrate on exactly the ECGs a clinician would call PVCs\. Read out independently, the same atoms’ activation geometry is that of the ectopic beat—abnormal QRS\-onset spikes and secondary ST–T deflections \([Sec\.4\.5](https://arxiv.org/html/2607.25244#S4.SS5)\)—so the report enrichment says*which*diagnosis and the geometry says*what*on the waveform\.

The same decomposition holds for other diagnoses and is often anatomically specific\. For right bundle branch block \(RBBB; probe AUROC0\.9880\.988\), the top visualizable atoms are each report\-enriched for \(incomplete\) RBBB and rSR′\(V1\) statements, with lifts up to∼25×\\sim\\\!25\\times\([FigureA1](https://arxiv.org/html/2607.25244#A1.F1)b\)\. Their activation geometry localizes to the terminal QRS and repolarization of the right precordial leads \(V1/V2\)—for example a V2 terminal\-QRS notch, the classic rSR′signature—exactly the diagnostic territory of RBBB\. The same decomposition recurs across other diagnoses—for example sinus bradycardia, whose probe likewise concentrates on a handful of atoms whose enriched statements match the target \([FigureA1](https://arxiv.org/html/2607.25244#A1.F1)a\); the same procedure applies to the morphology, age, and mortality probes \([AppendixA](https://arxiv.org/html/2607.25244#A1)\)\.

### 4\.5\.LLM\-assisted atom annotation

\(a\)

![Refer to caption](https://arxiv.org/html/2607.25244v1/figure_llm_interp.png)

\(b\)

![Refer to caption](https://arxiv.org/html/2607.25244v1/figure_three_atoms_desc.png)

Figure 5\.Automated LLM atom descriptions are quantitatively faithful\.\(a\)Per\-atom Spearmanρ\\rhobetween the LLM’s held\-out activation predictions \(from the text description alone\) and measured activations, for10001000random atoms \(gpt\-5\.4\-mini\); medianρ=0\.65\\rho=0\.65, with92%92\\%of atoms above0\.30\.3\.\(b\)Three example atoms: each atom’s top\-activating ECG \(red marks where it fires\) next to the model’s geometry\-based name and description\.We test whether the geometry\-first descriptions of the SAE’s concepts are not merely plausible but*quantitatively faithful*\. Running the describe\-then\-validate pipeline \([Figure1](https://arxiv.org/html/2607.25244#S1.F1)c\) on a random10001000of the29912991interpretable atoms \([Figure5](https://arxiv.org/html/2607.25244#S4.F5)a\), the LLM’s activation predictions—made from the text description alone, with no access to activations—track the measured activation with a median Spearmanρ\\rhoof0\.650\.65;92%92\\%of atoms exceed0\.30\.3and only2%2\\%fail \(ρ≤0\\rho\\leq 0\)\. Automated interpretation therefore extends from hand\-read exemplars to the dictionary at large: an LLM can read an atom’s activation evidence and write a description faithful enough to predict where that atom fires on held\-out ECGs, at a median rank correlation of0\.650\.65over a thousand atoms\. As concrete examples,[Figure5](https://arxiv.org/html/2607.25244#S4.F5)b pairs three atoms with their top\-activating ECG alongside the LLM’s geometry\-based name and description\.

## 5\.Scientific and Clinical Contributions

#### Scientific contributions\.

CADENCE is, to our knowledge, the first application of sparse\-autoencoder dictionary learning to a*physiological time\-series*foundation model, giving direct evidence that ECG FM embeddings are monosemantically decomposable and store concepts in superposition—extending a phenomenon so far documented mainly in language, vision, and protein models to a new, clinically consequential modality\. Beyond showing that such atoms*exist*, we characterize*how*an ECG backbone organizes them: clinical concepts become progressively more linearly accessible with depth, and single atoms increasingly outperform raw embedding dimensions\. The recovered atoms span three levels of temporal and spatial abstraction—multi\-beat rhythm, single\-beat phenotype, and sub\-beat morphology—so one dictionary captures features from millisecond waveform primitives to whole\-recording timing\. Their arrangement in atom space also reproduces physiologically coherent relationships between concepts*without any relational supervision*, indicating that clinical structure is written into the geometry of the representation itself; this structure, and the concepts themselves, replicate on an independent external cohort, indicating that they are properties of the representation that generalize across datasets\. Targeted ablation of a single named atom selectively changes the corresponding frozen readout, showing atoms behave as functional units of the model’s representation, and an automated describe\-then\-validate pipeline demonstrates that this interpretation can be scaled to the whole dictionary and*quantitatively*checked\.

#### Clinical contributions\.

CADENCE turns an opaque ECG FM into an auditable, queryable dictionary of physiologically grounded concepts, with several implications for how such models could be used and trusted in cardiology\. First, it makes a model’s evidence*inspectable*: instead of accepting a prediction on faith, a practitioner can ask which atoms it rests on and see the exact waveform behavior each atom responds to—a premature\-ventricular\-complex atom firing on the ectopic beat, or a right\-bundle\-branch\-block atom on the terminal\-QRS notch in V1—evidence a cardiologist can check against their own reading\. Second, prediction becomes*attributable*: sparse atom probes match or exceed the dense embedding while decomposing each diagnosis into a small, named set of contributing atoms, so accuracy no longer comes at the price of an opaque score\. Third, the same transparency surfaces*failure modes*—confounded, shortcut, or demographic features become visible for audit rather than hiding inside a vector, a prerequisite for safe adoption\. Fourth, because the recovered concepts and their relationships replicate on an independent, cardiologist\-labeled external cohort without retraining, the dictionary behaves as a cross\-site auditing substrate, and it can double as a hypothesis\-generating tool—exposing which physiological structure a model has actually learned about the heart\.

## 6\.Conclusion

We introduced CADENCE, which decomposes a frozen 12\-lead ECG foundation model into a sparse dictionary of physiological*atoms*that are far more concept\-aligned than raw neurons\. The dictionary recovers a broad cardiology spectrum and reusable sub\-diagnostic waveform primitives; sparse atom probes are accurate and interpretable; atom\-space geometry reconstructs known physiological relationships; and an automated LLM pipeline describes and validates every atom at scale, generalizing to an external cohort\. Together these establish sparse dictionary learning as a practical route to auditing what physiological foundation models encode\. Extending CADENCE to other backbones, layers, and multi\-center cohorts is a natural next step toward trustworthy, auditable clinical AI\.

## 7\.Limitations and Ethical Considerations

#### Limitations\.

Our study has several limitations\. First, atom naming is bounded by current cardiology knowledge and fixed geometric heuristics, so some interpretations may be incomplete\. Second, we focus on standard resting 12\-lead ECGs; single\-lead, ambulatory, and longer continuous recordings are outside the present scope\.

#### Ethical considerations\.

All experiments use publicly available, de\-identified ECG data—the MIMIC\-IV\-ECG resource\(Gowet al\.,[2023](https://arxiv.org/html/2607.25244#bib.bib12); Johnsonet al\.,[2023](https://arxiv.org/html/2607.25244#bib.bib13); Goldbergeret al\.,[2000](https://arxiv.org/html/2607.25244#bib.bib14)\)for training and the PTB\-XL cohort\(Wagneret al\.,[2020](https://arxiv.org/html/2607.25244#bib.bib44)\)for external validation—accessed under their respective PhysioNet data\-use agreements\. MIMIC\-IV was reviewed by the Beth Israel Deaconess Medical Center Institutional Review Board, which granted a waiver of informed consent and approved the data\-sharing initiative\. For PTB\-XL, the relevant institutional ethics committee approved publication of the anonymized data in an open\-access database \(PTB\-2020\-1\)\. We add no identifiers, do not attempt re\-identification, and report only de\-identified waveform examples and aggregate, non\-identifying analyses\. The atoms are not diagnostic devices: not regulatorily cleared and not for clinical decisions\.

## 8\.Generative AI Usage

Generative AI is used at two levels\. As a studied component of the method, our interpretation pipeline \([Figure1](https://arxiv.org/html/2607.25244#S1.F1)c,[Sec\.4\.5](https://arxiv.org/html/2607.25244#S4.SS5)\) uses large language models to describe and validate atoms\. As an authoring aid, AI assistants helped copy\-edit and polish the writing; they were not used to design the study, alter data or results, or draw conclusions\. All AI\-assisted outputs were reviewed and verified by the authors, who take full responsibility for the content\.

## References

- E\. Adams, L\. Bai, M\. Lee, Y\. Yu, and M\. AlQuraishi \(2025\)From mechanistic interpretability to mechanistic biology: training, evaluating, and interpreting sparse autoencoders on protein language models\.InProceedings of the 42nd International Conference on Machine Learning \(ICML\),PMLR, Vol\.267,pp\. 460–476\.External Links:[Document](https://dx.doi.org/10.1101/2025.02.06.636901)Cited by:[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px3.p1.1)\.
- G\. Alain and Y\. Bengio \(2016\)Understanding intermediate layers using linear classifier probes\.arXiv preprint arXiv:1610\.01644\.External Links:[Link](https://arxiv.org/abs/1610.01644)Cited by:[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px4.p1.1)\.
- Z\. I\. Attia, S\. Kapa, F\. Lopez\-Jimenez, P\. M\. McKie, D\. J\. Ladewig, G\. Satam, P\. A\. Pellikka, M\. Enriquez\-Sarano, P\. A\. Noseworthy, T\. M\. Munger, S\. J\. Asirvatham, C\. G\. Scott, R\. E\. Carter, and P\. A\. Friedman \(2019\)Screening for cardiac contractile dysfunction using an artificial intelligence\-enabled electrocardiogram\.Nature Medicine25\(1\),pp\. 70–74\.External Links:[Document](https://dx.doi.org/10.1038/s41591-018-0240-2)Cited by:[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px1.p1.1)\.
- S\. Bills, N\. Cammarata, D\. Mossing, H\. Tillman, L\. Gao, G\. Goh, I\. Sutskever, J\. Leike, J\. Wu, and W\. Saunders \(2023\)Language models can explain neurons in language models\.OpenAI\.External Links:[Link](https://openaipublic.blob.core.windows.net/neuron-explainer/paper/index.html)Cited by:[§3\.6](https://arxiv.org/html/2607.25244#S3.SS6.p1.1)\.
- T\. Bricken, A\. Templeton, J\. Batson, B\. Chen, A\. Jermyn, T\. Conerly, N\. Turner, C\. Anil, C\. Denison, A\. Askell,et al\.\(2023\)Towards monosemanticity: decomposing language models with dictionary learning\.Transformer Circuits Thread\.External Links:[Link](https://transformer-circuits.pub/2023/monosemantic-features/index.html)Cited by:[§1](https://arxiv.org/html/2607.25244#S1.p2.1),[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px2.p1.1)\.
- B\. Bussmann, P\. Leask, and N\. Nanda \(2024\)BatchTopK sparse autoencoders\.arXiv preprint arXiv:2412\.06410\.External Links:[Link](https://arxiv.org/abs/2412.06410)Cited by:[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px2.p1.1),[§3\.3](https://arxiv.org/html/2607.25244#S3.SS3.p1.1)\.
- D\. A\. Cook, S\. Oh, and M\. V\. Pusic \(2020\)Accuracy of physicians’ electrocardiogram interpretations: a systematic review and meta\-analysis\.JAMA Internal Medicine180\(11\),pp\. 1461–1471\.External Links:[Document](https://dx.doi.org/10.1001/jamainternmed.2020.3989)Cited by:[§1](https://arxiv.org/html/2607.25244#S1.p1.1)\.
- H\. Cunningham, A\. Ewart, L\. Riggs, R\. Huben, and L\. Sharkey \(2023\)Sparse autoencoders find highly interpretable features in language models\.arXiv preprint arXiv:2309\.08600\.Note:Published at ICLR 2024External Links:[Link](https://arxiv.org/abs/2309.08600)Cited by:[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px2.p1.1)\.
- N\. Elhage, T\. Hume, C\. Olsson, N\. Schiefer, T\. Henighan, S\. Kravec, Z\. Hatfield\-Dodds, R\. Lasenby, D\. Drain, C\. Chen,et al\.\(2022\)Toy models of superposition\.Transformer Circuits Thread\.External Links:[Link](https://transformer-circuits.pub/2022/toy_model/index.html)Cited by:[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px2.p1.1)\.
- L\. Gao, T\. D\. la Tour, H\. Tillman, G\. Goh, R\. Troll, A\. Radford, I\. Sutskever, J\. Leike, and J\. Wu \(2024\)Scaling and evaluating sparse autoencoders\.arXiv preprint arXiv:2406\.04093\.External Links:[Link](https://arxiv.org/abs/2406.04093)Cited by:[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px2.p1.1),[§3\.3](https://arxiv.org/html/2607.25244#S3.SS3.p1.1)\.
- A\. L\. Goldberger, L\. A\. N\. Amaral, L\. Glass, J\. M\. Hausdorff, P\. Ch\. Ivanov, R\. G\. Mark, J\. E\. Mietus, G\. B\. Moody, C\. Peng, and H\. E\. Stanley \(2000\)PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals\.Circulation101\(23\),pp\. e215–e220\.External Links:[Document](https://dx.doi.org/10.1161/01.CIR.101.23.e215)Cited by:[§A\.10](https://arxiv.org/html/2607.25244#A1.SS10.p1.11),[§3\.1](https://arxiv.org/html/2607.25244#S3.SS1.p1.2),[§7](https://arxiv.org/html/2607.25244#S7.SS0.SSS0.Px2.p1.1)\.
- B\. Gow, T\. Pollard, L\. A\. Nathanson, A\. Johnson, B\. Moody, C\. Fernandes, N\. Greenbaum, J\. W\. Waks, P\. Eslami, T\. Carbonati, A\. Chaudhari, E\. Herbst, D\. Moukheiber, S\. Berkowitz, R\. Mark, and S\. Horng \(2023\)MIMIC\-IV\-ECG: diagnostic electrocardiogram matched subset \(version 1\.0\)\.PhysioNet\.External Links:[Document](https://dx.doi.org/10.13026/4nqg-sb35)Cited by:[§A\.10](https://arxiv.org/html/2607.25244#A1.SS10.p1.11),[§3\.1](https://arxiv.org/html/2607.25244#S3.SS1.p1.2),[§7](https://arxiv.org/html/2607.25244#S7.SS0.SSS0.Px2.p1.1)\.
- X\. Gu, W\. Tang, J\. Han, V\. Sangha, F\. Liu, S\. N\. Gowda, A\. H\. Ribeiro, P\. Schwab, K\. Branson, L\. Clifton, A\. L\. P\. Ribeiro, Z\. Liu, and D\. A\. Clifton \(2026\)Cardiac health assessment across scenarios and devices using a multimodal foundation model pretrained on data from 1\.7 million individuals\.Nature Machine Intelligence8\(2\),pp\. 220–233\.External Links:[Document](https://dx.doi.org/10.1038/s42256-026-01180-5)Cited by:[§1](https://arxiv.org/html/2607.25244#S1.p1.1),[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px1.p1.1),[§3\.2](https://arxiv.org/html/2607.25244#S3.SS2.p1.4)\.
- H\. Guan, J\. He, and J\. Zhang \(2025\)Sparse autoencoders reveal interpretable structure in small gene language models\.arXiv preprint arXiv:2507\.07486\.External Links:[Link](https://arxiv.org/abs/2507.07486)Cited by:[§1](https://arxiv.org/html/2607.25244#S1.p2.1),[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px3.p1.1)\.
- W\. Gurnee, N\. Nanda, M\. Pauly, K\. Harvey, D\. Troitskii, and D\. Bertsimas \(2023\)Finding neurons in a haystack: case studies with sparse probing\.Transactions on Machine Learning Research \(TMLR\)\.External Links:[Link](https://arxiv.org/abs/2305.01610)Cited by:[§3\.5](https://arxiv.org/html/2607.25244#S3.SS5.p1.1)\.
- J\. A\. Hanley and B\. J\. McNeil \(1982\)The meaning and use of the area under a receiver operating characteristic \(roc\) curve\.Radiology143\(1\),pp\. 29–36\.External Links:[Document](https://dx.doi.org/10.1148/radiology.143.1.7063747)Cited by:[§3\.4](https://arxiv.org/html/2607.25244#S3.SS4.p1.8)\.
- A\. Y\. Hannun, P\. Rajpurkar, M\. Haghpanahi, G\. H\. Tison, C\. Bourn, M\. P\. Turakhia, and A\. Y\. Ng \(2019\)Cardiologist\-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network\.Nature Medicine25\(1\),pp\. 65–69\.External Links:[Document](https://dx.doi.org/10.1038/s41591-018-0268-3)Cited by:[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px1.p1.1)\.
- K\. He, X\. Chen, S\. Xie, Y\. Li, P\. Dollár, and R\. Girshick \(2022\)Masked autoencoders are scalable vision learners\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 16000–16009\.External Links:[Document](https://dx.doi.org/10.1109/CVPR52688.2022.01553)Cited by:[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px1.p1.1)\.
- G\. Hindricks, T\. Potpara, N\. Dagres,et al\.\(2021\)2020 ESC guidelines for the diagnosis and management of atrial fibrillation developed in collaboration with the European Association for Cardio\-Thoracic Surgery \(EACTS\)\.European Heart Journal42\(5\),pp\. 373–498\.External Links:[Document](https://dx.doi.org/10.1093/eurheartj/ehaa612)Cited by:[§4\.3](https://arxiv.org/html/2607.25244#S4.SS3.p1.7)\.
- A\. E\. W\. Johnson, L\. Bulgarelli, L\. Shen, A\. Gayles, A\. Shammout, S\. Horng, T\. J\. Pollard, S\. Hao, B\. Moody, B\. Gow, L\. H\. Lehman, L\. A\. Celi, and R\. G\. Mark \(2023\)MIMIC\-IV, a freely accessible electronic health record dataset\.Scientific Data10\(1\),pp\. 1\.External Links:[Document](https://dx.doi.org/10.1038/s41597-022-01899-x)Cited by:[§A\.10](https://arxiv.org/html/2607.25244#A1.SS10.p1.11),[§3\.1](https://arxiv.org/html/2607.25244#S3.SS1.p1.2),[§7](https://arxiv.org/html/2607.25244#S7.SS0.SSS0.Px2.p1.1)\.
- I\. Kendiukhov \(2026\)Sparse autoencoders reveal organized biological knowledge but minimal regulatory logic in single\-cell foundation models: a comparative atlas of Geneformer and scGPT\.arXiv preprint arXiv:2603\.02952\.External Links:[Link](https://arxiv.org/abs/2603.02952)Cited by:[§1](https://arxiv.org/html/2607.25244#S1.p2.1),[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px3.p1.1)\.
- B\. Kim, M\. Wattenberg, J\. Gilmer, C\. Cai, J\. Wexler, F\. Viégas, and R\. Sayres \(2018\)Interpretability beyond feature attribution: quantitative testing with concept activation vectors \(TCAV\)\.InProceedings of the 35th International Conference on Machine Learning \(ICML\),pp\. 2668–2677\.External Links:[Link](https://arxiv.org/abs/1711.11279)Cited by:[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px4.p1.1)\.
- C\. Kim, J\. Kaczmarzyk, D\. Savant, Z\. Zhao, P\. K\. Koo, and S\. Lee \(2026\)Dissecting and directing pathology foundation models\.bioRxiv\.External Links:[Document](https://dx.doi.org/10.64898/2026.06.12.731496)Cited by:[§1](https://arxiv.org/html/2607.25244#S1.p2.1),[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px3.p1.1)\.
- D\. P\. Kingma and J\. Ba \(2015\)Adam: a method for stochastic optimization\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://arxiv.org/abs/1412.6980)Cited by:[Table A6](https://arxiv.org/html/2607.25244#A1.T6.8.8.1)\.
- D\. Kiyasseh, T\. Zhu, and D\. A\. Clifton \(2021\)CLOCS: contrastive learning of cardiac signals across space, time, and patients\.InProceedings of the 38th International Conference on Machine Learning \(ICML\),PMLR, Vol\.139,pp\. 5606–5615\.External Links:[Link](https://arxiv.org/abs/2005.13249)Cited by:[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px1.p1.1)\.
- P\. Kligfield, L\. S\. Gettes, J\. J\. Bailey, R\. Childers, B\. J\. Deal, E\. W\. Hancock, G\. van Herpen, J\. A\. Kors, P\. Macfarlane, D\. M\. Mirvis, O\. Pahlm, P\. Rautaharju, and G\. S\. Wagner \(2007\)Recommendations for the standardization and interpretation of the electrocardiogram: part I\.Circulation115\(10\),pp\. 1306–1324\.External Links:[Document](https://dx.doi.org/10.1161/CIRCULATIONAHA.106.180200)Cited by:[§3\.4](https://arxiv.org/html/2607.25244#S3.SS4.p1.8),[§4\.2](https://arxiv.org/html/2607.25244#S4.SS2.SSS0.Px2.p1.1),[§4\.3](https://arxiv.org/html/2607.25244#S4.SS3.p1.7)\.
- P\. W\. Koh, T\. Nguyen, Y\. S\. Tang, S\. Mussmann, E\. Pierson, B\. Kim, and P\. Liang \(2020\)Concept bottleneck models\.InProceedings of the 37th International Conference on Machine Learning \(ICML\),pp\. 5338–5348\.External Links:[Link](https://arxiv.org/abs/2007.04612)Cited by:[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px4.p1.1)\.
- S\. M\. Lundberg and S\. Lee \(2017\)A unified approach to interpreting model predictions\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.30,pp\. 4765–4774\.External Links:[Link](https://arxiv.org/abs/1705.07874)Cited by:[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px4.p1.1)\.
- A\. Makhzani and B\. Frey \(2013\)K\-sparse autoencoders\.arXiv preprint arXiv:1312\.5663\.External Links:[Link](https://arxiv.org/abs/1312.5663)Cited by:[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px2.p1.1)\.
- D\. Makowski, T\. Pham, Z\. J\. Lau, J\. C\. Brammer, F\. Lespinasse, H\. Pham, C\. Schölzel, and S\. H\. A\. Chen \(2021\)NeuroKit2: a python toolbox for neurophysiological signal processing\.Behavior Research Methods53\(4\),pp\. 1689–1696\.External Links:[Document](https://dx.doi.org/10.3758/s13428-020-01516-y)Cited by:[§A\.11](https://arxiv.org/html/2607.25244#A1.SS11.p1.1)\.
- K\. McKeen, L\. Oliva, S\. Masood,et al\.\(2024\)ECG\-fm: an open electrocardiogram foundation model\.arXiv preprint arXiv:2408\.05178\.Note:Published in JAMIA Open, 2025External Links:[Link](https://arxiv.org/abs/2408.05178)Cited by:[§1](https://arxiv.org/html/2607.25244#S1.p1.1),[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px1.p1.1)\.
- T\. Mehari and N\. Strodthoff \(2022\)Self\-supervised representation learning from 12\-lead ECG data\.Computers in Biology and Medicine141,pp\. 105114\.External Links:[Document](https://dx.doi.org/10.1016/j.compbiomed.2021.105114)Cited by:[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px1.p1.1)\.
- Y\. Na, M\. Park, Y\. Tae, and S\. Joo \(2024\)Guiding masked representation learning to capture spatio\-temporal relationship of electrocardiogram\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://arxiv.org/abs/2402.09450)Cited by:[§1](https://arxiv.org/html/2607.25244#S1.p1.1),[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px1.p1.1)\.
- B\. A\. Olshausen and D\. J\. Field \(1996\)Emergence of simple\-cell receptive field properties by learning a sparse code for natural images\.Nature381\(6583\),pp\. 607–609\.External Links:[Document](https://dx.doi.org/10.1038/381607a0)Cited by:[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px2.p1.1)\.
- A\. Paszke, S\. Gross, F\. Massa, A\. Lerer, J\. Bradbury, G\. Chanan, T\. Killeen, Z\. Lin, N\. Gimelshein, L\. Antiga,et al\.\(2019\)PyTorch: an imperative style, high\-performance deep learning library\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.32,pp\. 8024–8035\.External Links:[Link](https://arxiv.org/abs/1912.01703)Cited by:[§A\.9](https://arxiv.org/html/2607.25244#A1.SS9.p1.1)\.
- F\. Pedregosa, G\. Varoquaux, A\. Gramfort, V\. Michel, B\. Thirion, O\. Grisel, M\. Blondel, P\. Prettenhofer, R\. Weiss, V\. Dubourg, J\. Vanderplas, A\. Passos, D\. Cournapeau, M\. Brucher, M\. Perrot, and É\. Duchesnay \(2011\)Scikit\-learn: machine learning in Python\.Journal of Machine Learning Research12,pp\. 2825–2830\.External Links:[Link](https://jmlr.org/papers/v12/pedregosa11a.html)Cited by:[§A\.9](https://arxiv.org/html/2607.25244#A1.SS9.p1.1),[§3\.5](https://arxiv.org/html/2607.25244#S3.SS5.p1.1)\.
- S\. Rajamanoharan, A\. Conmy, L\. Smith, T\. Lieberum, V\. Varma, J\. Kramár, R\. Shah, and N\. Nanda \(2024a\)Improving dictionary learning with gated sparse autoencoders\.arXiv preprint arXiv:2404\.16014\.External Links:[Link](https://arxiv.org/abs/2404.16014)Cited by:[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px2.p1.1)\.
- S\. Rajamanoharan, T\. Lieberum, N\. Sonnerat, A\. Conmy, V\. Varma, J\. Kramár, and N\. Nanda \(2024b\)Jumping ahead: improving reconstruction fidelity with JumpReLU sparse autoencoders\.arXiv preprint arXiv:2407\.14435\.External Links:[Link](https://arxiv.org/abs/2407.14435)Cited by:[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px2.p1.1)\.
- P\. M\. Rautaharju, B\. Surawicz, and L\. S\. Gettes \(2009\)AHA/ACCF/HRS recommendations for the standardization and interpretation of the electrocardiogram: part IV: the ST segment, T and U waves, and the QT interval\.Journal of the American College of Cardiology53\(11\),pp\. 982–991\.External Links:[Document](https://dx.doi.org/10.1016/j.jacc.2008.12.014)Cited by:[§4\.3](https://arxiv.org/html/2607.25244#S4.SS3.p1.7)\.
- A\. H\. Ribeiro, M\. H\. Ribeiro, G\. M\. M\. Paixão, D\. M\. Oliveira, P\. R\. Gomes, J\. A\. Canazart, M\. P\. S\. Ferreira, C\. R\. Andersson, P\. W\. Macfarlane, W\. Meira Jr, T\. B\. Schön, and A\. L\. P\. Ribeiro \(2020\)Automatic diagnosis of the 12\-lead ECG using a deep neural network\.Nature Communications11\(1\),pp\. 1760\.External Links:[Document](https://dx.doi.org/10.1038/s41467-020-15432-4)Cited by:[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px1.p1.1)\.
- S\. M\. Salerno, P\. C\. Alguire, and H\. S\. Waxman \(2003\)Competency in interpretation of 12\-lead electrocardiograms: a summary and appraisal of published evidence\.Annals of Internal Medicine138\(9\),pp\. 751–760\.External Links:[Document](https://dx.doi.org/10.7326/0003-4819-138-9-200305060-00013)Cited by:[§1](https://arxiv.org/html/2607.25244#S1.p1.1)\.
- R\. R\. Selvaraju, M\. Cogswell, A\. Das, R\. Vedantam, D\. Parikh, and D\. Batra \(2017\)Grad\-CAM: visual explanations from deep networks via gradient\-based localization\.InProceedings of the IEEE International Conference on Computer Vision \(ICCV\),pp\. 618–626\.External Links:[Document](https://dx.doi.org/10.1109/ICCV.2017.74)Cited by:[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px4.p1.1)\.
- E\. Simon and J\. Zou \(2025\)InterPLM: discovering interpretable features in protein language models via sparse autoencoders\.Nature Methods22,pp\. 2107–2117\.External Links:[Document](https://dx.doi.org/10.1038/s41592-025-02836-7)Cited by:[§1](https://arxiv.org/html/2607.25244#S1.p2.1),[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px3.p1.1)\.
- M\. Sundararajan, A\. Taly, and Q\. Yan \(2017\)Axiomatic attribution for deep networks\.InProceedings of the 34th International Conference on Machine Learning \(ICML\),pp\. 3319–3328\.External Links:[Link](https://arxiv.org/abs/1703.01365)Cited by:[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px4.p1.1),[§4\.4](https://arxiv.org/html/2607.25244#S4.SS4.SSS0.Px1.p1.1)\.
- A\. Templeton, T\. Conerly, J\. Marcus, J\. Lindsey, T\. Bricken, B\. Chen, A\. Pearce, C\. Citro, E\. Ameisen, A\. Jones,et al\.\(2024\)Scaling monosemanticity: extracting interpretable features from Claude 3 Sonnet\.Transformer Circuits Thread\.External Links:[Link](https://transformer-circuits.pub/2024/scaling-monosemanticity/)Cited by:[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px2.p1.1),[§4\.4](https://arxiv.org/html/2607.25244#S4.SS4.SSS0.Px1.p1.1)\.
- A\. M\. Turner, L\. Thiergart, D\. Udell, G\. Leech, U\. Mini, and M\. MacDiarmid \(2023\)Activation addition: steering language models without optimization\.arXiv preprint arXiv:2308\.10248\.External Links:[Link](https://arxiv.org/abs/2308.10248)Cited by:[§4\.4](https://arxiv.org/html/2607.25244#S4.SS4.SSS0.Px1.p1.1)\.
- A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, Ł\. Kaiser, and I\. Polosukhin \(2017\)Attention is all you need\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.30,pp\. 5998–6008\.External Links:[Link](https://arxiv.org/abs/1706.03762)Cited by:[§3\.2](https://arxiv.org/html/2607.25244#S3.SS2.p1.4)\.
- P\. Wagner, N\. Strodthoff, R\. Bousseljot, D\. Kreiseler, F\. I\. Lunze, W\. Samek, and T\. Schaeffter \(2020\)PTB\-XL, a large publicly available electrocardiography dataset\.Scientific Data7\(1\),pp\. 154\.External Links:[Document](https://dx.doi.org/10.1038/s41597-020-0495-6)Cited by:[§A\.6](https://arxiv.org/html/2607.25244#A1.SS6.p1.10),[§3\.1](https://arxiv.org/html/2607.25244#S3.SS1.p1.2),[§7](https://arxiv.org/html/2607.25244#S7.SS0.SSS0.Px2.p1.1)\.
- D\. Zhang, S\. Yang, X\. Yuan, and P\. Zhang \(2021\)Interpretable deep learning for automatic diagnosis of 12\-lead electrocardiogram\.iScience24\(4\),pp\. 102373\.External Links:[Document](https://dx.doi.org/10.1016/j.isci.2021.102373)Cited by:[§2](https://arxiv.org/html/2607.25244#S2.SS0.SSS0.Px4.p1.1)\.

## Appendix AAdditional Results

### A\.1\.Hyperparameter selection

We selected the dictionary sizeFFand sparsitykkby a grid search overF∈\{4096,8192,16384,32768\}F\\\!\\in\\\!\\\{4096,8192,16384,32768\\\}andk∈\{32,64,128,256\}k\\\!\\in\\\!\\\{32,64,128,256\\\}at Layer\-6 \([FigureA4](https://arxiv.org/html/2607.25244#A1.F4)\), scoring each configuration on three axes: reconstruction explained variance \(EV, higher is better\), the fraction of dead atoms \(lower is better, since dead atoms waste dictionary capacity\), and the number of*phenotype\-specific*atoms—atoms that align selectively with a single clinical phenotype—as a proxy for interpretability yield \(higher is better\)\.

Three regularities decide the choice\. \(i\)EV rises withkkbut is essentially flat inFF: atk=128k\{=\}128, EV moves only0\.966→0\.9690\.966\\\!\\to\\\!0\.969asFFgrows8×8\\times, so a larger dictionary buys no reconstruction\. \(ii\)Dead atoms grow steeply with bothFFandkk: atk=128k\{=\}128the dead fraction jumps from21%21\\%\(F=8192F\{=\}8192\) to49%49\\%\(1638416384\) to70%70\\%\(3276832768\), i\.e\., most of a larger dictionary is wasted\. \(iii\)Interpretability yield peaks atk=128,F=8192k\{=\}128,\\,F\{=\}8192\(459459phenotype\-specific atoms, the global maximum of the grid\); enlargingFFdoes not add specific atoms \(458458at1638416384,440440at3276832768\) while denser codes \(k=256k\{=\}256\) sharply reduce them\. The chosen configuration \(k=128,F=8192k\{=\}128,\\,F\{=\}8192; orange star\) thus sits at the Pareto knee: EV within0\.010\.01of thek=256k\{=\}256ceiling at half the density, an acceptable21%21\\%dead fraction, and the maximum number of phenotype\-specific atoms\. This is consistent with the corpus\-size saturation reported in[App\.A\.10](https://arxiv.org/html/2607.25244#A1.SS10): neither more tokens nor a larger dictionary improves the decomposition at this backbone width\.

### A\.2\.SAE atoms vs\. neurons across depth

SAE atoms are far cleaner concept detectors than the raw CSFM embedding dimensions, and the gap widens with depth \([FigureA4](https://arxiv.org/html/2607.25244#A1.F4)\)\. Comparing the per\-feature best held\-out AUROC of SAE atoms against the dense embedding dimensions and a random\-noise control \([FigureA4](https://arxiv.org/html/2607.25244#A1.F4)\), atoms reach substantially higher peak association than dense dimensions at every layer, with the strongly\-associated tail expanding with depth, while the noise control stays at the0\.50\.5floor\.

### A\.3\.Downstream probes with bootstrap confidence intervals

[FigureA4](https://arxiv.org/html/2607.25244#A1.F4)reproduces the per\-layer probe comparison of[Figure2](https://arxiv.org/html/2607.25244#S2.F2)b with95%95\\%patient\-level bootstrap confidence bands around the SAE probe \(in place of the SAE\-over\-dense margin shading\)\. The CIs are narrow at every layer, so the SAE\-atom advantage over the dense embedding is well outside sampling noise, most clearly in the shallow layers\.

Table[A1](https://arxiv.org/html/2607.25244#A1.T1)breaks the aggregate best\-atom curve of[Figure2](https://arxiv.org/html/2607.25244#S2.F2)a down to the individual concept level\. For each concept and layer it reports the best\-atom AUROC of the SAE dictionary alongside the best\-unit AUROC of the raw CSFM dense embedding, each with a95%95\\%patient\-level bootstrap confidence interval\. The depth trend of[Figure2](https://arxiv.org/html/2607.25244#S2.F2)a holds concept by concept: the best single atom for most concepts is found at L5–L6, and the sparse atom matches or exceeds the best dense unit for almost every concept, with the margin widening in the deep layers for conduction, rhythm, infarct and interval concepts\.

The sparse\-probing decomposition of[Figure4](https://arxiv.org/html/2607.25244#S4.F4)b is not specific to PVC and paced rhythm\.[FigureA1](https://arxiv.org/html/2607.25244#A1.F1)shows the same decomposition for sinus bradycardia and right bundle branch block \(RBBB\): for RBBB \(panel b; test AUROC0\.9880\.988\) the highest\-weight atoms concentrate on \(incomplete\) RBBB and rSR′\(V1\) report statements, and their activation geometry localizes to the terminal QRS of the right precordial leads\.

![Refer to caption](https://arxiv.org/html/2607.25244v1/figure_probe_decomp_more.png)Figure A1\.Sparse\-probe decomposition generalizes across diagnoses\.For two further phenotype probes, the five highest\-weight atoms ranked by linear\-probe weight, each labeled by its most over\-represented \(auxiliary\) report statement:\(a\)sinus bradycardia and\(b\)right bundle branch block \(RBBB\) \(test AUROC in each title\)\. As for PVC and paced rhythm \([Figure4](https://arxiv.org/html/2607.25244#S4.F4)b\), each probe’s weight concentrates on a small stack of atoms whose enriched statements match the target diagnosis; the RBBB atoms are each report\-enriched for \(incomplete\) RBBB\. Report enrichment shares its source with the labels and is auxiliary, not independent proof\.![Refer to caption](https://arxiv.org/html/2607.25244v1/figure_gridsearch.png)

Figure A2\.SAE hyperparameter grid search at Layer\-6\.Each panel plots one selection criterion against dictionary sizeFF, with a line per sparsity levelkk\. Left to right: explained variance rises withkkbut is flat inFF; dead\-atom fraction grows steeply with both; and phenotype\-specific atom count, our interpretability proxy, peaks at the chosen configurationk=128,F=8192k\{=\}128,\\,F\{=\}8192, marked by the orange star\.Three line\-chart panels sharing an x\-axis of dictionary size F \(4k to 32k\) with four lines for k in 32,64,128,256\. Left: explained variance separates by k and is flat across F\. Middle: dead\-atom percentage increases with F and k\. Right: phenotype\-specific atom count peaks at k=128, F=8192, marked with a star\.

![Refer to caption](https://arxiv.org/html/2607.25244v1/figure_layers_ab.png)

Figure A3\.SAE atoms are cleaner concept detectors than neurons\.Best held\-out concept\-association AUROC per feature at each layer, for a green random\-noise control, blue CSFM dense embedding dimensions, and orange SAE atoms\.![Refer to caption](https://arxiv.org/html/2607.25244v1/figure5_four_ci.png)

Figure A4\.Downstream probe performance with bootstrap CIs, cf\.[Figure2](https://arxiv.org/html/2607.25244#S2.F2)b\. Per\-layer test AUROC of the solid SAE\-atom probe and the dotted CSFM dense embedding for the three task families; shaded bands are95%95\\%patient\-level bootstrap confidence intervals for the SAE probe\.Table A1\.Per\-concept best\-atom \(SAE\) vs\. best\-unit \(CSFM dense\) AUROC across the six CSFM layers, with 95% patient\-level bootstrap confidence intervals\.For each concept and layer we select, on the full test set, the single best sign\-folded SAE atom and the single best sign\-folded CSFM dense unit, then bootstrap that fixed unit’s AUROC by resampling patients \(B=1000B\{=\}1000\)\. The larger of the two point estimates in each layer is in bold\. Sparse atoms match or exceed the dense embedding for almost every concept, and the margin widens with depth for conduction, rhythm, infarct and interval concepts\. This is the concept\-level, dense\-comparison decomposition of the aggregate best\-atom curve in[Figure2](https://arxiv.org/html/2607.25244#S2.F2)a\.L1L2L3L4L5L6ConceptSAEdenseSAEdenseSAEdenseSAEdenseSAEdenseSAEdenseAtrial fibrillation0\.677\[\.66,\.69\]0\.724\[\.71,\.74\]0\.716\[\.70,\.73\]0\.724\[\.71,\.74\]0\.867\[\.86,\.88\]0\.821\[\.81,\.83\]0\.901\[\.89,\.91\]0\.864\[\.86,\.87\]0\.950\[\.94,\.96\]0\.891\[\.88,\.90\]0\.954\[\.95,\.96\]0\.910\[\.90,\.92\]Atrial flutter0\.700\[\.68,\.72\]0\.783\[\.77,\.80\]0\.730\[\.71,\.75\]0\.793\[\.78,\.81\]0\.823\[\.81,\.84\]0\.820\[\.81,\.84\]0\.844\[\.83,\.86\]0\.837\[\.82,\.85\]0\.872\[\.86,\.89\]0\.846\[\.83,\.86\]0\.874\[\.86,\.89\]0\.854\[\.84,\.87\]Sinus bradycardia0\.723\[\.71,\.73\]0\.843\[\.84,\.85\]0\.758\[\.75,\.77\]0\.869\[\.86,\.87\]0\.894\[\.89,\.90\]0\.883\[\.88,\.89\]0\.981\[\.98,\.98\]0\.923\[\.92,\.93\]0\.981\[\.98,\.98\]0\.957\[\.95,\.96\]0\.984\[\.98,\.99\]0\.957\[\.95,\.96\]PVC0\.665\[\.64,\.68\]0\.709\[\.69,\.73\]0\.706\[\.69,\.73\]0\.717\[\.70,\.73\]0\.767\[\.75,\.78\]0\.738\[\.72,\.76\]0\.853\[\.84,\.87\]0\.747\[\.73,\.76\]0\.905\[\.89,\.91\]0\.779\[\.76,\.79\]0\.904\[\.89,\.91\]0\.805\[\.79,\.82\]Paced rhythm0\.753\[\.72,\.78\]0\.822\[\.79,\.85\]0\.803\[\.77,\.83\]0\.845\[\.82,\.87\]0\.834\[\.80,\.86\]0\.859\[\.83,\.88\]0\.879\[\.85,\.90\]0\.872\[\.85,\.89\]0\.905\[\.88,\.93\]0\.870\[\.85,\.89\]0\.915\[\.89,\.93\]0\.856\[\.83,\.88\]LBBB0\.895\[\.88,\.91\]0\.887\[\.87,\.90\]0\.928\[\.92,\.94\]0\.941\[\.93,\.95\]0\.970\[\.96,\.98\]0\.934\[\.92,\.94\]0\.974\[\.97,\.98\]0\.919\[\.91,\.93\]0\.975\[\.97,\.98\]0\.884\[\.87,\.89\]0\.979\[\.97,\.99\]0\.891\[\.88,\.90\]RBBB0\.825\[\.81,\.84\]0\.725\[\.71,\.74\]0\.885\[\.88,\.89\]0\.774\[\.76,\.79\]0\.958\[\.95,\.96\]0\.818\[\.81,\.83\]0\.970\[\.96,\.97\]0\.836\[\.83,\.84\]0\.974\[\.97,\.98\]0\.872\[\.86,\.88\]0\.969\[\.96,\.97\]0\.797\[\.79,\.81\]LAFB0\.808\[\.80,\.82\]0\.755\[\.74,\.77\]0\.854\[\.84,\.87\]0\.770\[\.75,\.79\]0\.921\[\.91,\.93\]0\.809\[\.79,\.82\]0\.928\[\.92,\.93\]0\.807\[\.79,\.82\]0\.927\[\.92,\.93\]0\.783\[\.77,\.80\]0\.938\[\.93,\.94\]0\.770\[\.76,\.78\]Left\-axis dev\.0\.813\[\.81,\.82\]0\.758\[\.75,\.77\]0\.842\[\.83,\.85\]0\.769\[\.76,\.78\]0\.882\[\.87,\.89\]0\.770\[\.76,\.78\]0\.923\[\.92,\.93\]0\.795\[\.79,\.81\]0\.913\[\.91,\.92\]0\.762\[\.75,\.77\]0\.922\[\.92,\.93\]0\.730\[\.72,\.74\]LVH0\.656\[\.64,\.67\]0\.653\[\.64,\.67\]0\.671\[\.66,\.68\]0\.675\[\.66,\.69\]0\.682\[\.67,\.70\]0\.678\[\.67,\.69\]0\.681\[\.67,\.70\]0\.703\[\.69,\.72\]0\.667\[\.66,\.68\]0\.659\[\.65,\.67\]0\.684\[\.67,\.70\]0\.631\[\.62,\.64\]Low QRS voltage0\.621\[\.61,\.63\]0\.642\[\.63,\.65\]0\.654\[\.64,\.67\]0\.686\[\.67,\.70\]0\.673\[\.66,\.68\]0\.668\[\.66,\.68\]0\.678\[\.67,\.69\]0\.680\[\.67,\.69\]0\.704\[\.69,\.72\]0\.664\[\.65,\.68\]0\.712\[\.70,\.72\]0\.654\[\.64,\.67\]Inferior infarct0\.672\[\.66,\.68\]0\.642\[\.63,\.65\]0\.738\[\.73,\.75\]0\.644\[\.63,\.66\]0\.795\[\.79,\.80\]0\.645\[\.63,\.66\]0\.827\[\.82,\.83\]0\.679\[\.67,\.69\]0\.850\[\.84,\.86\]0\.648\[\.64,\.66\]0\.830\[\.82,\.84\]0\.642\[\.63,\.65\]Anterior infarct0\.756\[\.74,\.77\]0\.711\[\.70,\.72\]0\.737\[\.72,\.75\]0\.707\[\.69,\.72\]0\.830\[\.82,\.84\]0\.715\[\.70,\.73\]0\.831\[\.82,\.84\]0\.702\[\.69,\.71\]0\.828\[\.82,\.84\]0\.686\[\.67,\.70\]0\.836\[\.83,\.85\]0\.647\[\.63,\.66\]ST depression0\.718\[\.68,\.75\]0\.761\[\.73,\.79\]0\.761\[\.73,\.79\]0\.799\[\.77,\.83\]0\.816\[\.79,\.84\]0\.825\[\.80,\.85\]0\.822\[\.79,\.84\]0\.826\[\.80,\.85\]0\.810\[\.78,\.83\]0\.786\[\.76,\.81\]0\.822\[\.80,\.84\]0\.790\[\.77,\.81\]Prolonged PR0\.703\[\.69,\.72\]0\.668\[\.65,\.68\]0\.740\[\.73,\.75\]0\.679\[\.67,\.69\]0\.789\[\.78,\.80\]0\.704\[\.69,\.72\]0\.884\[\.87,\.89\]0\.797\[\.79,\.81\]0\.940\[\.93,\.95\]0\.797\[\.78,\.81\]0\.944\[\.94,\.95\]0\.857\[\.85,\.87\]Prolonged QT0\.638\[\.62,\.65\]0\.693\[\.68,\.71\]0\.710\[\.70,\.72\]0\.726\[\.71,\.74\]0\.717\[\.70,\.73\]0\.736\[\.72,\.75\]0\.785\[\.77,\.80\]0\.766\[\.75,\.78\]0\.851\[\.84,\.86\]0\.756\[\.74,\.77\]0\.850\[\.84,\.86\]0\.746\[\.73,\.76\]

Table A2\.Per\-concept linear\-probe AUROC \(SAE atoms vs\. CSFM dense embedding\) across the six CSFM layers, with 95% patient\-level bootstrap confidence intervals\.For each concept and layer we fit anℓ1/ℓ2\\ell\_\{1\}/\\ell\_\{2\}\-regularized logistic probe on the sparse atom vector \(max\-pooled\) and, separately, on the dense CSFM embedding \(mean\-pooled\), with a subject\-held\-out fit and a patient\-level bootstrap \(B=500B\{=\}500\) of the test AUROC\. The larger of the two point estimates in each layer is in bold\. The sparse atom probe matches or exceeds the dense probe for almost every concept and layer, with the largest gains in the shallow layers and for conduction, axis and infarct concepts; the two converge at the deep layers for the easiest rhythm concepts\. This is the concept\-level, per\-layer decomposition of the aggregate probe comparison in[Figure2](https://arxiv.org/html/2607.25244#S2.F2)b\.L1L2L3L4L5L6ConceptSAEdenseSAEdenseSAEdenseSAEdenseSAEdenseSAEdenseAtrial fibrillation0\.861\[\.85,\.87\]0\.819\[\.81,\.83\]0\.899\[\.89,\.91\]0\.874\[\.87,\.88\]0\.935\[\.93,\.94\]0\.934\[\.93,\.94\]0\.962\[\.96,\.97\]0\.960\[\.96,\.96\]0\.971\[\.97,\.97\]0\.970\[\.97,\.97\]0\.974\[\.97,\.98\]0\.973\[\.97,\.98\]Atrial flutter0\.854\[\.84,\.87\]0\.834\[\.82,\.85\]0\.871\[\.86,\.88\]0\.866\[\.85,\.88\]0\.897\[\.89,\.91\]0\.894\[\.88,\.90\]0\.906\[\.89,\.92\]0\.905\[\.89,\.92\]0\.914\[\.90,\.92\]0\.911\[\.90,\.92\]0\.916\[\.91,\.93\]0\.913\[\.90,\.92\]Sinus bradycardia0\.918\[\.91,\.92\]0\.887\[\.88,\.89\]0\.936\[\.93,\.94\]0\.933\[\.93,\.94\]0\.973\[\.97,\.98\]0\.970\[\.97,\.97\]0\.987\[\.98,\.99\]0\.986\[\.98,\.99\]0\.990\[\.99,\.99\]0\.990\[\.99,\.99\]0\.991\[\.99,\.99\]0\.990\[\.99,\.99\]PVC0\.765\[\.75,\.78\]0\.743\[\.73,\.76\]0\.792\[\.78,\.81\]0\.764\[\.75,\.78\]0\.880\[\.87,\.89\]0\.808\[\.79,\.82\]0\.925\[\.92,\.93\]0\.870\[\.86,\.88\]0\.942\[\.93,\.95\]0\.902\[\.89,\.91\]0\.946\[\.94,\.95\]0\.915\[\.91,\.92\]Paced rhythm0\.883\[\.86,\.91\]0\.875\[\.85,\.90\]0\.908\[\.89,\.93\]0\.902\[\.88,\.92\]0\.927\[\.91,\.94\]0\.922\[\.90,\.94\]0\.938\[\.92,\.95\]0\.938\[\.92,\.96\]0\.947\[\.93,\.96\]0\.936\[\.92,\.95\]0\.952\[\.94,\.97\]0\.940\[\.92,\.96\]LBBB0\.975\[\.97,\.98\]0\.962\[\.95,\.97\]0\.981\[\.97,\.99\]0\.975\[\.97,\.98\]0\.988\[\.98,\.99\]0\.982\[\.98,\.99\]0\.990\[\.99,\.99\]0\.986\[\.98,\.99\]0\.991\[\.99,\.99\]0\.986\[\.98,\.99\]0\.990\[\.99,\.99\]0\.984\[\.98,\.99\]RBBB0\.949\[\.94,\.95\]0\.879\[\.87,\.89\]0\.975\[\.97,\.98\]0\.949\[\.94,\.95\]0\.984\[\.98,\.99\]0\.968\[\.96,\.97\]0\.987\[\.98,\.99\]0\.982\[\.98,\.98\]0\.987\[\.99,\.99\]0\.982\[\.98,\.98\]0\.988\[\.99,\.99\]0\.980\[\.98,\.98\]LAFB0\.892\[\.88,\.90\]0\.849\[\.84,\.86\]0\.941\[\.94,\.95\]0\.909\[\.90,\.92\]0\.964\[\.96,\.97\]0\.931\[\.93,\.94\]0\.970\[\.97,\.97\]0\.943\[\.94,\.95\]0\.971\[\.97,\.97\]0\.943\[\.94,\.95\]0\.971\[\.97,\.97\]0\.942\[\.94,\.95\]Left\-axis dev\.0\.879\[\.87,\.89\]0\.839\[\.83,\.85\]0\.926\[\.92,\.93\]0\.888\[\.88,\.89\]0\.951\[\.95,\.95\]0\.910\[\.90,\.92\]0\.957\[\.95,\.96\]0\.923\[\.92,\.93\]0\.960\[\.96,\.96\]0\.923\[\.92,\.93\]0\.958\[\.96,\.96\]0\.923\[\.92,\.93\]LVH0\.746\[\.74,\.76\]0\.715\[\.70,\.73\]0\.790\[\.78,\.80\]0\.759\[\.75,\.77\]0\.839\[\.83,\.85\]0\.821\[\.81,\.83\]0\.865\[\.86,\.87\]0\.849\[\.84,\.86\]0\.872\[\.86,\.88\]0\.851\[\.84,\.86\]0\.875\[\.87,\.88\]0\.853\[\.84,\.86\]Low QRS voltage0\.721\[\.71,\.73\]0\.694\[\.68,\.71\]0\.768\[\.76,\.78\]0\.763\[\.75,\.77\]0\.832\[\.82,\.84\]0\.827\[\.82,\.83\]0\.863\[\.86,\.87\]0\.867\[\.86,\.87\]0\.872\[\.87,\.88\]0\.873\[\.87,\.88\]0\.874\[\.87,\.88\]0\.870\[\.86,\.88\]Inferior infarct0\.769\[\.76,\.78\]0\.676\[\.66,\.69\]0\.828\[\.82,\.84\]0\.751\[\.74,\.76\]0\.900\[\.89,\.91\]0\.839\[\.83,\.85\]0\.927\[\.92,\.93\]0\.890\[\.88,\.90\]0\.939\[\.93,\.94\]0\.896\[\.89,\.90\]0\.943\[\.94,\.95\]0\.896\[\.89,\.90\]Anterior infarct0\.834\[\.83,\.84\]0\.772\[\.76,\.78\]0\.859\[\.85,\.87\]0\.828\[\.82,\.84\]0\.890\[\.88,\.90\]0\.855\[\.85,\.86\]0\.911\[\.90,\.92\]0\.891\[\.88,\.90\]0\.919\[\.91,\.93\]0\.885\[\.88,\.89\]0\.919\[\.91,\.93\]0\.877\[\.87,\.89\]ST depression0\.848\[\.82,\.87\]0\.817\[\.79,\.84\]0\.866\[\.84,\.89\]0\.869\[\.85,\.89\]0\.893\[\.87,\.91\]0\.892\[\.87,\.91\]0\.916\[\.90,\.93\]0\.912\[\.90,\.93\]0\.914\[\.90,\.93\]0\.902\[\.88,\.92\]0\.926\[\.91,\.94\]0\.909\[\.89,\.92\]Prolonged PR0\.811\[\.80,\.82\]0\.747\[\.74,\.76\]0\.866\[\.86,\.87\]0\.828\[\.82,\.84\]0\.932\[\.93,\.94\]0\.909\[\.90,\.92\]0\.950\[\.94,\.96\]0\.947\[\.94,\.95\]0\.960\[\.95,\.97\]0\.956\[\.95,\.96\]0\.962\[\.96,\.97\]0\.957\[\.95,\.96\]Prolonged QT0\.789\[\.78,\.80\]0\.767\[\.75,\.78\]0\.832\[\.82,\.84\]0\.826\[\.82,\.84\]0\.897\[\.89,\.90\]0\.893\[\.89,\.90\]0\.924\[\.92,\.93\]0\.926\[\.92,\.93\]0\.947\[\.94,\.95\]0\.936\[\.93,\.94\]0\.950\[\.94,\.95\]0\.937\[\.93,\.94\]

### A\.4\.Rhythm\-level localization example

[FigureA8](https://arxiv.org/html/2607.25244#A1.F8)shows the rhythm\-level atom referenced in[Sec\.4\.2](https://arxiv.org/html/2607.25244#S4.SS2)\. A sinus\-bradycardia atom \(L6/3082\) activates on the*elongated diastolic baseline between beats*—the prolonged TP/RR interval—firing on every inter\-beat segment in lead II and staying silent through the QRS–T complexes\. Its “feature” is thus a multi\-beat timing property that no single time point causes, complementing the phenotype\- and morphology\-level atoms of[Figure2](https://arxiv.org/html/2607.25244#S2.F2)\.

A second rhythm example is atrial flutter: atom L6/418 fires on the regular sawtooth flutter waves in the baseline between QRS complexes \([FigureA8](https://arxiv.org/html/2607.25244#A1.F8)\)\.

A paced\-rhythm atom localizes a different waveform primitive: atom L6/5276 fires on the sharp pacing\-spike artifact preceding each QRS complex \([FigureA8](https://arxiv.org/html/2607.25244#A1.F8)\)\.

An interval atom localizes the conduction delay itself: atom L6/4213 fires on the prolonged PR segment between the P wave and the QRS \([FigureA8](https://arxiv.org/html/2607.25244#A1.F8)\)\.

A morphology atom localizes a sub\-beat waveform primitive: atom L6/1305 fires on the ST–T segment immediately after each QRS complex, across two example ECGs \([FigureA9](https://arxiv.org/html/2607.25244#A1.F9)\)\.

![Refer to caption](https://arxiv.org/html/2607.25244v1/figure2_rhythm.png)

Figure A5\.Rhythm\-level localization\.Sinus\-bradycardia atom L6/3082 fires, shown red, on the elongated diastolic baseline between beats in lead II across two 10 s example ECGs, remaining silent through the QRS–T complexes\.![Refer to caption](https://arxiv.org/html/2607.25244v1/figure_flutter.png)

Figure A6\.Rhythm\-level localization: atrial flutter\.Atom L6/418 fires, shown red, on the regular sawtooth flutter waves between QRS complexes in lead II across two 10 s example ECGs\.![Refer to caption](https://arxiv.org/html/2607.25244v1/figure_paced.png)

Figure A7\.Paced\-rhythm localization\.Atom L6/5276 fires, shown red, on the pacing\-spike artifact before each QRS complex, across two example ECGs \(leads V1 and V2\)\.![Refer to caption](https://arxiv.org/html/2607.25244v1/figure_pr.png)

Figure A8\.Prolonged\-PR localization\.Atom L6/4213 fires, shown red, on the extended PR segment between the P wave and QRS across two example ECGs \(lead V2\)\.![Refer to caption](https://arxiv.org/html/2607.25244v1/figure_af_crossdataset.png)Figure A9\.Morphology\-level localization: ST segment\.A morphology atom \(L6/1305\) fires, shown red, on the ST–T segment immediately after each QRS complex, across two MIMIC example ECGs \(lead V4\)\.
### A\.5\.Phenotype relationships: AUROC\-fingerprint variant

The phenotype\-relationship structure of[Figure3](https://arxiv.org/html/2607.25244#S4.F3)is robust to the similarity metric\.[FigureA13](https://arxiv.org/html/2607.25244#A1.F13)shows the same curated concepts under a signed\-AUROC fingerprint \(each concept’s vector of signed per\-atom AUROCs\), whose cosine recovers the identical conduction / wide\-QRS block; per\-pair values agree with the prototype cosine\.

### A\.6\.External validation on PTB\-XL

To test whether the dictionary generalizes beyond its training distribution, we apply the*frozen*, MIMIC\-trained per\-layer SAEs—without any retraining or re\-estimation of their input statistics—to PTB\-XL\(Wagneret al\.,[2020](https://arxiv.org/html/2607.25244#bib.bib44)\), an independent, cardiologist\-labeled 12\-lead cohort from a different country and acquisition system \(21,79921\{,\}799ECGs; patient\-level train/test split\)\. PTB\-XL signals are preprocessed identically to pre\-training and passed through CSFM and the SAEs to obtain per\-ECG atom activations; the atom\-activation scale matches the MIMIC distribution \(median max≈22\.8\\approx 22\.8vs\.2424\), confirming the frozen dictionary is in\-distribution\. We reproduce the best\-single\-feature analysis of[Figure2](https://arxiv.org/html/2607.25244#S2.F2)a against two independent label sources \([FigureA13](https://arxiv.org/html/2607.25244#A1.F13),[FigureA13](https://arxiv.org/html/2607.25244#A1.F13)\): the five cardiologist\- adjudicated diagnostic superclasses \(AUROC\) and4949automated 12SL/ECGDeli*measurement*concepts grouped into five families \(association\|ρSpearman\|\|\\rho\_\{\\mathrm\{Spearman\}\}\|\)\. In both, the best SAE atom exceeds the best dense dimension at*every*layer with the margin widening with depth—mean best\-atom AUROC0\.8250\.825vs\. dense0\.7200\.720over the diagnostic tasks, and atoms ahead of dense in all five measurement families—so the Layer\-6 monosemanticity advantage of[Figure2](https://arxiv.org/html/2607.25244#S2.F2)a is not an artifact of the training cohort\. Of the4949measurement concepts, only∼\\sim5 \(heart rate, PR, QRS duration, QT/QTc, axis\) have a counterpart in our MIMIC concept set; the remaining∼\\sim40 \(amplitudes, areas, regional ST–T, additional axes\) are recovered by the frozen dictionary despite never being evaluated during development\.

Beyond per\-concept detection, the atom\-space geometry also transfers: on PTB\-XL, the cosine similarity of concept fingerprints reproduces the conduction/wide\-QRS block of[Figure3](https://arxiv.org/html/2607.25244#S4.F3)\([FigureA13](https://arxiv.org/html/2607.25244#A1.F13)\), recovering the interval\-containment link wide\-QRS – long\-QTc \(0\.770\.77\) and the conduction–QRS\-widening relations without any relational supervision\.

![Refer to caption](https://arxiv.org/html/2607.25244v1/figure_relations_auroc.png)Figure A10\.Phenotype relationships under the AUROC\-fingerprint metric, cf\.[Figure3](https://arxiv.org/html/2607.25244#S4.F3)\. Same curated concept set; cosine of signed\-AUROC fingerprints\. The data\-driven block and star\-marked known pairs are unchanged\.
![Refer to caption](https://arxiv.org/html/2607.25244v1/figure_ptbxl_fig2a_6layer.png)Figure A11\.External validation on PTB\-XL: best\-atom vs\. best\-dense across depth, diagnostic tasks\.Best single\-feature test AUROC for three representative cardiologist\-labeled diagnoses—rhythm/AF, ischemia/MI, conduction/CD; solid SAE atoms vs\. dotted CSFM dense dimensions, shaded SAE\-over\-dense margin\. Frozen MIMIC\-trained SAEs applied to PTB\-XL\.
![Refer to caption](https://arxiv.org/html/2607.25244v1/figure_ptbxl_concept_cosine.png)Figure A12\.External validation on PTB\-XL: concept relationships in atom space\.Cosine similarity of PTB\-XL AUROC\-fingerprints for the conduction/wide\-QRS concepts; yellow stars mark known cardiology pairs\. The same physiological block recovered on MIMIC \([Figure3](https://arxiv.org/html/2607.25244#S4.F3)\) is reproduced on the external cohort\.
![Refer to caption](https://arxiv.org/html/2607.25244v1/figure_ptbxl_fig2a_concepts.png)Figure A13\.External validation on PTB\-XL: best\-atom vs\. best\-dense across depth, 49 measurement concepts\.Best single\-feature association\|ρSpearman\|\|\\rho\_\{\\mathrm\{Spearman\}\}\|per CSFM layer, averaged within each of the five concept families; solid SAE atoms vs\. dotted CSFM dense, shaded margin\. Atoms lead dense in every family at every layer\.

### A\.7\.Automated atom descriptions: worked examples and the LLM prompt

#### Atom inclusion\.

Atoms are auto\-interpreted only if their evidence can be constructed and fairly validated; using pre\-hoc activation/geometry statistics \(not the validation score\), we exclude atoms that are*dead*\(negligible maximum activation\),*rare*\(fewer than2020high\-activation ECGs, so a strong/medium/weak example set cannot be built\),*near\-constant*\(firing on\>90%\>\\\!90\\%of ECGs, leaving no negatives\),*weak*\(maximum activation below a floor\), or*lead\-diffuse*\(per\-lead activation entropy\>0\.85\>0\.85, i\.e\. no localizable lead\)\. The last criterion is motivated by a pilot: the atoms whose descriptions failed validation were almost all lead\-diffuse\. Of the81928192atoms,29912991\(∼46%\\sim\\\!46\\%of active atoms\) pass; we interpret a random sample of them\.

#### Describe\.

For each included atom we take its highest\-activating tokens, render the corresponding lead×\\timesbeat\-phase waveform crops with the activated region highlighted plus a 12\-lead context, and prompt the model with the images and an activation\-geometry summary \(lead and beat\-phase histograms; the same geometry gives each atom a fixed, label\-independent name,[App\.A\.11](https://arxiv.org/html/2607.25244#A1.SS11)\) for a structured, geometry\-first JSON description \(dominant lead, beat phase, waveform shape\)\. No machine\-report text is provided to the describer; an ablation adding the full report text of top\-activating ECGs did not change faithfulness \(median Spearmanρ\\rho0\.665→0\.6700\.665\\\!\\to\\\!0\.670\), ruling out report\-label leakage\.

#### Validate\.

We validate each description by activation prediction\. From the description*alone*\(no activation overlay, no activation values\), the model scores a held\-out montage of crops spanning the atom’s*high*,*median*, and*low*activation levels; we correlate predicted with measured activation across these graded buckets\. We report the Spearman correlationρ\\rhobecause the model’s scores are ordinal and activations are heavily skewed\. A dissimilar\-atom control \(scoring a different concept’s atom with this description\) gives a null near zero\.

This section makes the automated description pipeline of[Sec\.4\.5](https://arxiv.org/html/2607.25244#S4.SS5)\([Figure5](https://arxiv.org/html/2607.25244#S4.F5)\) concrete on three Layer\-6 atoms\. Each description is produced bygpt\-5\.4\-minifrom activation*geometry*alone: the lead\- and beat\-phase histograms of the atom’s top\-activating tokens, plus two evidence images \(eight local activation crops and a 12\-lead context with the activated patch highlighted\)\. Clinical report text is passed only as auxiliary context and the model is instructed to name atoms by geometry, not diagnosis\. The description is then validated by having the same model score held\-out crops from the description alone; we report the Spearman correlationρ\\rhobetween predicted and measured activation\.[FigureA14](https://arxiv.org/html/2607.25244#A1.F14)shows the three atoms with their returned descriptions and faithfulness, and[FigureA15](https://arxiv.org/html/2607.25244#A1.F15)reproduces the exact prompt verbatim\.

The verbatimmorphology\_descriptionfield returned for each atom:

- •L6/1173:“In lead aVR, the atom fires on a consistent terminal QRS/early ST feature: a small upward rebound or positive notch immediately after a predominantly negative QRS complex, sometimes extending into the earliest ST segment\.”
- •L6/1576:“A small pre\-QRS deflection in aVR during the P/PR interval, typically a subtle notch or bump on a relatively flat baseline just before the QRS onset; examples are fairly consistent and mostly beat\-locked\.”
- •L6/7800:“A small pre\-QRS deflection in aVR, usually a brief negative P/PR\-era notch or dip just before the sharp QRS complex; the pattern is consistently localized to aVR across examples\.”

![Refer to caption](https://arxiv.org/html/2607.25244v1/figure_atom_desc_examples.png)Figure A14\.Three example atoms described by the pipeline\.Each row is one atom \(left label: itsgpt\-5\.4\-mininame and held\-out faithfulness\); the two panels are its top\-activating ECGs \(distinct records\) in the atom’s dominant lead, with red marking where the atom fires\. All three localize to lead aVR at a specific beat phase: L6/7800, “aVR pre\-QRS notch” \(P–PR, Spearmanρ=0\.54\\rho\{=\}0\.54\); L6/1576, “aVR pre\-QRS notch” \(P–PR,ρ=0\.68\\rho\{=\}0\.68\); and L6/1173, “aVR terminal upstroke” \(QRS\-terminal,ρ=0\.63\\rho\{=\}0\.63\)\. Faithfulness is the Spearman correlation between the model’s held\-out predicted and measured activations\.System prompt\.

Youinterpretasparse\-autoencoder\(SAE\)atomofanECGmodel\.TheatomactivatesonspecificLOCAL

leadxtime\-patch\(beat\-phase\)waveformpatterns,shownbyred\-highlightedpatchesinlocalcropsanda

12\-leadcontext\.IdentifytheLOCALmorphologytheatomcaptures:whichlead\(s\),whichbeatphase

\(P/PR/QRS\-onset/QRS\-peak/QRS\-terminal/ST/T\),whatwaveformshape,andwhetherexamplesareconsistent\.

NameitbyactivationGEOMETRY,notbydiagnosis,unlessthewaveformclearlysupportsit\.

ReporttextisAUXILIARYonly\.OutputJSONonly\.

User message\(per atom; two images attached\)\.

AtomL6/<id\>\.

Activationgeometrysummary\(primary\):leadhistogramovertoptokens=\{\.\.\.\},beat\-phasehistogram=\{\.\.\.\},

modal\_lead=<lead\>,lead\_group=<group\>\.

Image\[1\]:8localcropsattop\-activatingpatches\(red=activatedpatch\)\.

Image\[2\]:12\-leadcontextofthetoptoken\(red=activatedpatch\)\.

AUXILIARYONLY\(donotleadwiththis\):reportenrichment=\{\.\.\.\},bestphenotypeassoc=<pheno\>\(AUROC<x\>\)\.

<JSONschema,below\>

Required JSON schema\(appended to the user message\)\.

ReturnONLYthisJSON:

\{

"short\_name":"≤\\leq6words,geometry\-based",

"primary\_geometry":\{

"lead\_pattern":"single\-leadVx\|lateral\|inferior\|anteroseptal\|global",

"beat\_phase":"P\|PR\|QRS\-onset\|QRS\-peak\|QRS\-terminal\|ST\|T\|non\-beat\-locked",

"temporal\_extent":"point\|wave\-segment\|broad",

"activation\_consistency":"beat\-locked\|non\-beat\-locked"

\},

"morphology\_description":"thelocalwaveformshapewheretheatomfires\(nodiagnosisunlesssupported\)",

"report\_enrichment":\["auxphrasesorempty"\],

"confidence":"high\|medium\|low",

"caveat":"reportenrichmentauxiliary;primaryinterpretation=lead\-timeactivationgeometry"

\}

Figure A15\.Verbatim prompt used by the automated atom\-description pipeline \([Sec\.4\.5](https://arxiv.org/html/2607.25244#S4.SS5)\): a fixed system prompt, a per\-atom user message \(with two evidence images attached\), and the required JSON output schema\.

### A\.8\.Concept inventory

[TableA4](https://arxiv.org/html/2607.25244#A1.T4)lists every clinical concept CADENCE is evaluated against, grouped by clinical family\. Diagnostic*phenotypes*are derived by keyword matching on the MIMIC\-IV\-ECG machine\-report text;*continuous measurements*are read from the machine\-measurement numeric fields;*derived interval concepts*are clinical thresholds on those measurements \(used to build the concept\-fingerprint heatmaps of[Sec\.4\.3](https://arxiv.org/html/2607.25244#S4.SS3)\);*demographics and outcome*come from the linked MIMIC\-IV records; and*morphology primitives*are not a fixed list but an emergent class of sub\-diagnostic atoms localized to a single lead and beat phase\. Prevalences are fractions of the150150k training cohort; because rare phenotypes were over\-sampled during cohort construction \([App\.A\.10](https://arxiv.org/html/2607.25244#A1.SS10)\), these are cohort—not population—rates\. Two report keywords \(t\_wave\_abnormality,nonspecific\_st\_t\) matched almost no records in this cohort and are excluded from downstream phenotype analyses\.

Table A3\.Phenotype and morphology features recovered by CADENCE, grouped by category\.CategoryFeaturePhenotype featuresRhythm & ectopysinus bradycardiaatrial fibrillationatrial flutterpremature ventricular complexpaced rhythmConduction & axisleft bundle branch blockright bundle branch blockleft anterior fascicular blockPR prolongationleft\-axis deviationChamber & voltageleft ventricular hypertrophylow QRS voltageIschemia & infarctinferior infarctanterior infarctST depressionRepolarizationQT prolongationMorphology primitives \(lead×\\timesbeat\-phase\)P / PR segmentatrial / pre\-QRS deflectionQRS onsetQ wave / initial deflectionQRS peakR\-wave peakQRS terminalterminal notch / R′ST segmentST\-segment deviationT waveT\-wave repolarization
Table A4\.Concept inventory used throughout CADENCE\. Prevalence is the fraction of the 150k training cohort positive for each phenotype \(over\-sampled for rare classes; not population rates\)\.ConceptDefinition / sourcePrev\. \(%\)*Rhythm & ectopy*Sinus rhythmreport keyword49\.0Atrial fibrillationreport keyword10\.9Atrial flutterreport keyword4\.5Sinus bradycardiareport keyword16\.7Sinus tachycardiareport keyword13\.5Premature ventricular \(PVC\)report keyword3\.8Premature atrial \(PAC\)report keyword2\.5*Conduction & axis*LBBBreport keyword4\.1RBBBreport keyword6\.8First\-degree AV blockreport keyword1\.9Left\-axis deviationreport keyword11\.7Right\-axis deviationreport keyword2\.4*Chamber & voltage*LVHreport keyword10\.5Low QRS voltagereport keyword14\.6*Ischemia & repolarization*Myocardial infarctionreport keyword29\.6ST depressionreport keyword1\.3ST elevationreport keyword8\.5Prolonged QTreport keyword9\.2T\-wave abnormality†report keyword<<0\.1Nonspecific ST\-T†report keyword<<0\.1*Continuous measurements*Heart rate \(HR\)measurement \(bpm\)—PR intervalmeasurement \(ms\)—QRS durationmeasurement \(ms\)—QT / QTcmeasurement \(ms\)—Electrical axismeasurement \(∘\)—*Derived interval concepts*Wide QRSQRS≥\\geq120 ms—Long QTcQTc≥\\geq460 ms—Long PRPR≥\\geq200 ms—BradycardiaHR<<60 bpm—TachycardiaHR\>\>100 bpm—*Demographics & outcome*Agelinked record—Mortalitylinked record—*Morphology primitives*Lead×\\timesbeat\-phase atomsemergent \(P, PR, QRS\-on/—\(sub\-diagnostic\)peak/term, ST, T\)
†near\-zero prevalence in this cohort; excluded from phenotype analyses\.

### A\.9\.Model and training details

[TableA5](https://arxiv.org/html/2607.25244#A1.T5)summarizes the frozen CSFM backbone we interpret, and[TableA6](https://arxiv.org/html/2607.25244#A1.T6)the sparse autoencoder we train on its Layer\-6 activations\. All models are implemented in PyTorch\(Paszkeet al\.,[2019](https://arxiv.org/html/2607.25244#bib.bib42)\), and downstream probes use scikit\-learn\(Pedregosaet al\.,[2011](https://arxiv.org/html/2607.25244#bib.bib43)\)\.

Table A5\.Frozen ECG foundation\-model backbone \(CSFM\)\. We use the released pretrained encoder as a fixed feature extractor; the pretraining decoders and task head are discarded\.PropertyValueModelCSFM \(Cardiac Sensing FM\),TinyvariantModality12\-lead ECG \(multi\-modal model: ECG/PPG/text\)Pretrainingself\-supervised masked reconstructionPretraining scale∼\\sim1\.7M individualsHidden widthdd768Transformer layers6Patch size25 samples \(100 ms\)Tokens per ECG12​leads×100​patches=120012~\\text\{leads\}\\times 100~\\text\{patches\}=1200Interpreted layer6 \(residual stream\)Preprocessingresample500→250500\\\!\\to\\\!250Hz; per\-channelclean \(neurokit2\); per\-leadzz\-normUse in this workfrozen \(no fine\-tuning\)Table A6\.Sparse autoencoder trained on frozen CSFM Layer\-6 tokens\.HyperparameterValueTypeBatchTopK SAEInput dimdd768Dictionary sizeFF8192Active codeskk\(batch\-avgL0L\_\{0\}\)128Decoderunit\-normalized columns \(per step\)Input normalizationcentered; scaled to𝔼​∥x∥2≈d\\mathbb\{E\}\\lVert x\\rVert^\{2\}\\\!\\approx\\\!dObjectivereconstruction MSE\+\+AuxKAuxK \(kauxk\_\{\\mathrm\{aux\}\}, coef, dead thresh\.\)256, 0\.03, 200 stepsOptimizerAdam\(Kingma and Ba,[2015](https://arxiv.org/html/2607.25244#bib.bib41)\), lr4×10−44\\times 10^\{\-4\}Batch size4096 tokensTraining steps30000Training tokens∼9×106\\sim\\\!9\\times 10^\{6\}\(9M\) Layer\-6 tokensExplained variance0\.968Dead atoms1710 \(∼21%\\sim\\\!21\\%\)FFselectiongrid search \([App\.A\.1](https://arxiv.org/html/2607.25244#A1.SS1)\)
### A\.10\.Training cohort construction

The SAE is trained on Layer\-6 tokens extracted from a phenotype\-stratified, subject\-disjoint sample of MIMIC\-IV\-ECG\(Gowet al\.,[2023](https://arxiv.org/html/2607.25244#bib.bib12); Johnsonet al\.,[2023](https://arxiv.org/html/2607.25244#bib.bib13); Goldbergeret al\.,[2000](https://arxiv.org/html/2607.25244#bib.bib14)\)\. The cohort is built as follows\.

1. \(1\)Candidate pool\.All800,036800\{,\}036records in the MIMIC\-IV\-ECGrecord\_list\.csv\.
2. \(2\)Weak phenotype labels\.For every record we scan themachine\_measurementsreport text and assign∼\\sim20 binary phenotype labels \(e\.g\., AF, LBBB, PVC\) by keyword matching\. These labels are used only to*balance sampling*, never as supervision for the SAE\.
3. \(3\)Subject\-disjoint split\.Subjects are partitioned by a hash of the subject id \(hash​\(subject\_id\)mod104<1500⇒\\mathrm\{hash\}\(\\text\{subject\\\_id\}\)\\bmod 10^\{4\}<1500\\Rightarroweval, else train\), so no patient appears in both the training and evaluation sets and there is no subject leakage\.
4. \(4\)Stratified sampling to a target size\.Within the training pool we sample toward a target of150,000150\{,\}000ECGs: rarest phenotypes are drawn first \(each capped at12,00012\{,\}000ECGs\) to over\-represent uncommon morphologies, at most22ECGs are taken per subject, and the remainder is filled at random\. The held\-out evaluation set \(∼\\sim15k ECGs, used for all downstream analyses\) is sampled identically from the disjoint eval pool\.

Each of the150,000150\{,\}000training ECGs contributes6060Layer\-6 tokens \(evenly sampled across the 12 leads\), yielding the∼\\sim9M\-token training corpus\. We chose150,000150\{,\}000ECGs rather than the full pool because the dictionary saturates well below this scale: retraining on a10×10\\timeslarger corpus \(9​M→98\.89\\text\{M\}\\\!\\to\\\!98\.8M tokens\) at the sameF=8192F\{=\}8192,k=128k\{=\}128left explained variance essentially unchanged \(0\.968→0\.9700\.968\\\!\\to\\\!0\.970\) while*increasing*the dead\-atom fraction \(21%→33%21\\%\\\!\\to\\\!33\\%\), indicating that 9M tokens already suffices for this dictionary capacity\.

### A\.11\.Geometry\-based atom naming

Each atom receives a*label\-independent*primary name of the form “\{lead group\} \{beat phase\} atom,” computed only from where the atom fires on the waveform, so naming never depends on a diagnostic label\. We profile an atom on its top\-8 activating test ECGs, giving a1212\-lead activation\-mass vector and an activation template aligned to neurokit\-detected R\-peaks\(Makowskiet al\.,[2021](https://arxiv.org/html/2607.25244#bib.bib15)\)\. The*lead group*is the single lead, anatomical group, or “global” pattern that carries the activation mass; the*beat phase*is read from the template’s peak offset relative to the R\-peak—P\-wave, QRS\-onset, QRS, QRS\-terminal/J, ST, or T\-wave—with wide beat\-locked templates named “broad\-QRS” and non\-beat\-locked atoms named “rhythm\.” Thus “global broad\-QRS” denotes a wide, beat\-locked depolarization across most leads, the signature of paced or bundle\-branch widening; “broad\-QRS” is a geometric proxy for, not a measurement of, QRS duration\. Separately, and only as an*auxiliary*annotation, we report the GE\-MUSE report statement most enriched among each atom’s top\-activating ECGs; this shares its source with the diagnostic labels and is used for description only, the geometric name being the atom’s identity\. The fixed thresholds are as follows\.

We profile each atom on its top\-8 activating test ECGs; each patch spans2525samples \(100100ms\)\.Lead group, from the per\-lead activation massmm: a single lead ifmmax/∑m\>0\.5m\_\{\\max\}/\\sum m\>0\.5; else an anatomical group \(inferior, lateral, anteroseptal, …\) if that group holds more than half the mass; else “global” if at least nine of the twelve leads are jointly active \(each exceeding0\.4​mmax0\.4\\,m\_\{\\max\}\); else “\{group\}\-predominant\.”Beat phase, relative to neurokit\-detected R\-peaks: an atom is “non\-beat\-locked rhythm” if more than half of its active patches fall outside a per\-beat window\[R−2,R\+6\]\[R\\\!\-\\\!2,R\\\!\+\\\!6\]\. Otherwise we average the activation over offsetsd∈\[−4,\+8\]d\\in\[\-4,\+8\]patches into an R\-aligned template, whose peak offsetd⋆d^\{\\star\}and width \(patches above half the template peak\) give the phase: width≥3\\geq 3patches \(≳300\\gtrsim\\\!300ms\) is “broad\-QRS”; else P\-wave \(d⋆≤−2d^\{\\star\}\\\!\\leq\\\!\-2\), QRS\-onset \(−1\-1\), QRS\-locked \(0\), QRS\-terminal/J \(\+1\+1\), ST\-segment \(\+2\+2\), or T\-wave \(later\)\.