Label-Free Role-Aware Segmentation of Classroom Recordings: Recovering the Teacher from Transferable Structure

arXiv cs.LG Papers

Summary

A label-free, on-device pipeline for teacher-student segmentation in classroom recordings that chains lightweight VAD, compact speaker embeddings, agglomerative clustering, and logistic regression to identify the teacher via session-relative structure, matching pyannote performance while running 4.8× faster under GDPR constraints.

arXiv:2611.23609v1 Announce Type: new Abstract: How much the teacher talks and when are among the most informative and costly-to-measure signals of teaching. Yet, classroom audio is among the most sensitive data an AI system can process: recordings capture minors and fall under GDPR, preventing raw audio from leaving the recording device. We present a teacher-student segmentation pipeline designed to meet exactly this constraint: label-free at inference, running on a recording device, and requiring no per-session calibration. The pipeline chains a lightweight voice activity detector, compactly pretrained speaker embeddings, agglomerative clustering, and a small logistic regression that assigns the teacher role based on session-relative structure rather than voice identity. We show that our structural cue transfers across classrooms, languages, and acoustic conditions. In leave-one-teacher-out cross-validation on 292 recordings from 63 teachers across English and German corpora, our pipeline achieves teacher F1 of 0.81 and 0.87, respectively, outperforming both enrollment-based baselines and a 30-second supervised enrollment ceiling. On a commodity smartphone, its performance matches pyannote, a strong neural diarization tool, while running 4.8x faster at comparable peak memory at a real-time factor of ~0.18. We also find that transcribing filtered audio only marginally increases teacher's word error rate from $0.26$ to 0.28. Residual failures concentrate where teacher and student voices are not acoustically separable. However, we show how to flag these sessions label-free at inference so that they can be routed to human review, enabling selective, privacy-preserving deployment.
Original Article
View Cached Full Text

Cached at: 11/19/26, 09:01 AM

# Label-Free Role-Aware Segmentation of Classroom Recordings:Recovering the Teacher from Transferable Structure
Source: [https://arxiv.org/html/2611.23609](https://arxiv.org/html/2611.23609)
###### Abstract

How much the teacher talks and when are among the most informative and costly\-to\-measure signals of teaching\. Yet, classroom audio is among the most sensitive data an AI system can process: recordings capture minors and fall under GDPR, preventing raw audio from leaving the recording device\. We present a teacher\-student segmentation pipeline designed to meet exactly this constraint: label\-free at inference, running on a recording device, and requiring no per\-session calibration\. The pipeline chains a lightweight voice activity detector, compactly pretrained speaker embeddings, agglomerative clustering, and a small logistic regression that assigns the teacher role based on session\-relative structure rather than voice identity\. We show that our structural cue transfers across classrooms, languages, and acoustic conditions\.

In leave\-one\-teacher\-out cross\-validation on 292 recordings from 63 teachers across English and German corpora, our pipeline achieves teacher F1 of 0\.81 and 0\.87, respectively, outperforming both enrollment\-based baselines and a 30\-second supervised enrollment ceiling\. On a commodity smartphone, its performance matches pyannote, a strong neural diarization tool, while running 4\.8×\\timesfaster at comparable peak memory at a real\-time factor of∼\\sim0\.18\. We also find that transcribing filtered audio only marginally increases teacher’s word error rate from0\.260\.26to0\.280\.28\. Residual failures concentrate where teacher and student voices are not acoustically separable\. However, we show how to flag these sessions label\-free at inference so that they can be routed to human review, enabling selective, privacy\-preserving deployment\.

1Hector Research Institute of Education Sciences and Psychology, University of Tübingen

2Chair of Human\-Centered Technologies for Learning, Technical University of Munich

3Tübingen AI Center, University of Tübingen

nitin96\.jain@gmail\.com, babette\.buehler@tum\.de, \{tim\.fuetterer, cagatay\.yildiz\}@uni\-tuebingen\.de

## 1Introduction

How much a teacher talks, and when, provides informative but costly\-to\-measure signals of teaching\([Tao and Chen 2024](https://arxiv.org/html/2611.23609#bib.bib36)\)\. Distinguishing teacher from student speech enables analyses of talk\-time balance, turn\-taking, and instructional dialogue, but still requires intensive human annotation\([Wang et al\. 2025](https://arxiv.org/html/2611.23609#bib.bib10);[Kelly et al\. 2018](https://arxiv.org/html/2611.23609#bib.bib7);[Donnelly et al\. 2017](https://arxiv.org/html/2611.23609#bib.bib6)\)\. Automating this distinction supports scalable feedback and downstream analyses of classroom discourse and student engagement\([Schlotterbeck et al\. 2021](https://arxiv.org/html/2611.23609#bib.bib8);[Hou et al\. 2025](https://arxiv.org/html/2611.23609#bib.bib12)\)\.

Speaker diarization identifies*who spoke when*\([Anguera et al\. 2012](https://arxiv.org/html/2611.23609#bib.bib15);[Park et al\. 2022](https://arxiv.org/html/2611.23609#bib.bib16)\), but classroom deployment adds two requirements\. First, a deployed system meets a new, unknown teacher in every session, and it must identify the teacher in unseen classrooms without enrollment audio or speaker\-specific fine\-tuning\. Second, recordings are among the most sensitive data because they involve minors and sensitive interactions\. In our intended downstream workflow, teacher speech must leave the device for further processing, making it necessary to identify and filter teacher segments locally before transmission\. This motivates local processing without transferring raw audio or retaining reference voiceprints\([Sümer et al\. 2020](https://arxiv.org/html/2611.23609#bib.bib11);[Granqvist et al\. 2020](https://arxiv.org/html/2611.23609#bib.bib28)\)\. Together, these constraints rule out the commonly used large, cloud\-based GPU neural diarization pipelines with enrollment and demand a label\-free, CPU\-based inference system\.

We present such a system\. Our pipeline combines lightweight voice activity detection\([Silero Team 2024](https://arxiv.org/html/2611.23609#bib.bib25)\), compact pretrained speaker embeddings\([Wang et al\. 2023](https://arxiv.org/html/2611.23609#bib.bib23)\), and agglomerative clustering with a small logistic regression that assigns teacher/student roles using session\-relative features\. The strongest cue we consider iscentrality: the cosine similarity between speech segments in a lesson and a lesson’s mean embedding\. When the teacher is a recurrent, acoustically coherent voice in a classroom, this mean provides an in\-session reference for recovering the teacher across fragmented segments\. The role classifier is trained on annotated recordings; inference uses only audio and the frozen models\.

We evaluate on 292 recordings from 63 teachers: 151 close\-microphone English lessons and 141 far\-field German lessons\. Leave\-one\-teacher\-out teacher F1 reaches0\.810\.81and0\.870\.87, respectively, exceeding pseudo\-enrollment and supervised enrollment baselines and remaining competitive with pyannote\([Plaquet and Bredin 2023](https://arxiv.org/html/2611.23609#bib.bib21);[Bredin 2023](https://arxiv.org/html/2611.23609#bib.bib22)\)\. Training on either language transfers to the other with no loss or target\-language supervision\. On Samsung Galaxy S24\+ smartphone, the pipeline runs at real\-time factor≈0\.18\\approx 0\.18,4\.8×4\.8\\timesfaster than the matched pyannote reference at comparable memory\. We additionally show minimal degradation in a downstream transcription task, audit ground\-truth transcript\-derived labels, and evaluate label\-free diagnostics for routing unreliable sessions to human review\.

#### Task and constraints\.

We assign every instant of a single\-channel recording one of three labels:silence,teacher, orstudent, grouping all students into a single role rather than distinguishing individuals\. Labels are used only to train and calibrate the classifier; inference requires neither enrollment audio nor per\-session tuning\. The overall system is to be deployed on smartphones\.

Figure 1:35 s segment of a classroom recording, ground truth transcript annotation \(who spoke\), and the output of the VAD stage of our pipeline, identifying silent moments\.

## 2Datasets and problems with their labels

We use two corpora in two acoustic regimes:

- •Virginia \(English,[Pianta and Burchinal \(2016\)](https://arxiv.org/html/2611.23609#bib.bib30)\)\.Classroom videos in which teachers wear a microphone\. 151 lessons have human transcripts \(24 distinct teachers\)\.
- •GTI \(German, Global Teaching InSights[OECD \(2020\)](https://arxiv.org/html/2611.23609#bib.bib1)\)Room\-microphone recordings, 141 transcribed lessons and tutoring sessions \(39 distinct teachers\)\. This dataset was initially referred to as the Teaching and Learning International Survey \(TALIS\) Video Study\.

Onset\-anchored labels\.Transcript timestamps mark turn onsets, not offsets\. For instance, a span labeledTeacher 38→\\rightarrow56ends only where the next speaker begins, and hides pauses, backchannels, and interjections that never received their own onset\. VAD confirms that only69%69\\%\(English\) and46%46\\%\(German\) of teacher\-labeled interval time is actually voiced\. Treating entire intervals as teacher speech therefore contaminates training labels and distorts evaluation\. At*training*we intersect transcript intervals with VAD \(Section[3\.1](https://arxiv.org/html/2611.23609#S3.SS1)\); at*evaluation*we trust only the first44s after each onset \(Section[3\.5](https://arxiv.org/html/2611.23609#S3.SS5)\)\. The strategy ranking remains stable across alternative window lengths \(Appendix[C](https://arxiv.org/html/2611.23609#A3)\)\.

Boundary timing noise\.Onset timestamps are recorded to whole\-second resolution, so the final∼\\sim11s before each speaker change is still credited to the outgoing speaker although the incoming one has usually begun\. Excluding that second from the ground truth raises teacher F1 from0\.850\.85to0\.870\.87\. Because the model is already correct on these frames and only the label is wrong, we do not revise our numbers upward and treat the reported F1 as a conservative floor, with roughly two points of residual error attributable to transition\-zone label noise rather than model error\.

Human audit\.We sampled5,0005\{,\}000clips across six groups: two where our pipeline and the transcript disagree on the speaker’s role, two where they agree on teacher or student, and two containing VAD\-designated silence\. A trained annotator, blind to both pipeline outputs and transcripts, labeled the clips\. Duration\-weighted estimates indicate substantial transcript error:63%63\\%of sampled duration labeled teacher by our pipeline but student by the transcript was judged to be teacher speech\. Corrections almost entirely improve teacher precision without degrading recall\. Appendix[B](https://arxiv.org/html/2611.23609#A2)details the sampling, annotation, and corrected metrics\.

## 3Method

Algorithm 1Assign teacher/student roles to speaker clusters1:A classroom audio recording

2:Each cluster labelledteacherorstudent

3:Step 1: Embed and cluster\.Run VAD to filter out the silent regions\. Then, split the active regions into 1\.25 s speech windows, embed each window with WeSpeakerenglish\_resnet34, and group the window embeddings into speaker clusters by agglomerative clustering\.

4:Step 2: Describe each cluster\.For every cluster, compute eight structural features from the audio \(no labels\)\.centralityis defined in Sec\.[3\.4](https://arxiv.org/html/2611.23609#S3.SS4); the rest in App\.[F](https://arxiv.org/html/2611.23609#A6)\.

5:Step 3: Score each cluster\.Pass these features through a logistic regression model trained to predict if a summary is student or teacher\. The model returns a*teacher probability*

p^c∈\[0,1\]\\hat\{p\}\_\{c\}\\in\[0,1\]for each cluster

cc\.

6:Step 4: Assign teacher role\.Label asteacherall clusters with

p^c≥τ\\hat\{p\}\_\{c\}\\geq\\tau\. Label all remaining clustersstudent\. We find that the optimal

τ=0\.45\\tau=0\.45

The inference pipeline is four stages: voice activity detection \(VAD\), embedding, clustering, and teacher selection\.

### 3\.1Voice activity detection and why we need it

Silero VAD\([Silero Team 2024](https://arxiv.org/html/2611.23609#bib.bib25)\)identifies speech regions, assigning thesilencelabel and restricting embedding extraction at*inference*\. During*training*, we intersect detected speech with transcript intervals and assign each retained sub\-span the interval’s role label\.

A human audit, stratified by frame energy in VAD\-designated silence, estimates speech recall at93\.8%93\.8\\%\. Missed speech is distributed proportional to teacher and student talk time, suggesting no role\-specific bias in speech retention\.

### 3\.2Segmentation and embedding

Each VAD\-speech region is sliced into 1\.25 s windows \(0\.5 s hop\), and each window is embedded with a generic pretrained speaker model \(WeSpeakerenglish\_resnet34, 256\-dimensional,[Wang et al\. \(2023\)](https://arxiv.org/html/2611.23609#bib.bib23)\)\. We deliberately use*one*embedder for both languages since speaker embeddings encode voice timbre rather than language, and we confirm below that the resulting features transfer\.

### 3\.3Clustering

Window embeddings are clustered with agglomerative clustering \(cosine distance, average linkage, and distance threshold0\.50\.5\)\. We cross\-validate the distance threshold jointly with the classifier in App\.[E](https://arxiv.org/html/2611.23609#A5)and find accuracy flat across0\.40\.4–0\.850\.85before the clusters collapse near0\.90\.9\. In practice, the teacher’s voice is*fragmented*across many clusters \(often hundreds per 45\-minute lesson\), because content, distance, and channel drift push the same voice past the cosine threshold\.

### 3\.4Per\-cluster structural features

For each cluster we compute a set of label\-free,*session\-relative*features that capture the structural role of a voice without any annotation\. The most important of these iscentrality, the cosine similarity between the cluster centroid and the duration\-weighted session\-global centroid\. Centrality of clusterccis defined as:

centrality​\(c\)=cos⁡\(μc,μsession\),μc=∑i∈cdi​ei∑i∈cdi,\\texttt\{centrality\}\(c\)=\\cos\\\!\\big\(\\mu\_\{c\},\\;\\mu\_\{\\text\{session\}\}\\big\),\\quad\\mu\_\{c\}=\\frac\{\\sum\_\{i\\in c\}d\_\{i\}\\,e\_\{i\}\}\{\\sum\_\{i\\in c\}d\_\{i\}\},whereeie\_\{i\}is the embedding of windowii,did\_\{i\}its duration,μc\\mu\_\{c\}the cluster centroid, andμsession\\mu\_\{\\text\{session\}\}the average over*all*windows in the session\. Becauseμsession\\mu\_\{\\text\{session\}\}is duration\-weighted, it is pulled toward whoever talks most, usually the teacher, so a cluster that resembles the session’s dominant voice scores high\.

We also compute complementary cues: talk\-time share, temporal coverage and spread, turn count, cluster size, embedding consistency, and relative energy\. These describe each cluster’s share of the lesson’s total speech duration, its temporal distribution across the lesson, and its acoustic consistency, capturing role\-related patterns without requiring speaker identity\. We select features by treating clusters as labeled teacher/student examples under leave\-one\-teacher\-out cross validation, optimizing the lower per\-cluster labeling accuracy across English and German\. After ranking individual features, we perform greedy forward selection, adding the feature that most improves the current set\. Centrality is the strongest single predictor in both languages, and additional cues provide little improvement\. App\.[F](https://arxiv.org/html/2611.23609#A6)details the feature definitions, selection procedure, and per\-feature results \(Figure[7](https://arxiv.org/html/2611.23609#A6.F7)\)\.

Table 1:Teacher \(Tch\) and student \(Stu\) precision/recall and teacher F1 under leave\-one\-teacher\-out cross\-validation \(mean±\\pmstd\)\. Our pipeline outperforms enrollment\-based approaches and has the lowest variance; against the external pyannote reference, it wins on German but is edged on English\.†pyannote \(pyannote speaker\-diarization\-3\.1, teacher chosen by our dominance rule\) is a deployable CPU reference \(∼\\sim8 M params\) but requires license\-gated diarization weights; it is scored once per corpus \(no folds\), with precision/recall from the aggregate teacher\-vs\-rest confusion\.
### 3\.5Evaluation metric

We compare pipeline predictions with transcript\-derived role labels on0\.10\.1s frames\. For each role, precision is the fraction of predicted frames that match the reference, and recall is the fraction of reference frames correctly identified\. Teacher precision thus measures the purity of extracted teacher speech, while teacher recall measures how much reference teacher speech is retained within the evaluated frames\. We also report F1, the harmonic mean of precision and recall\. Frame\-level evaluation weights speech by duration rather than by cluster count, and results are reported separately for English and German\. All experiments use*leave\-one\-teacher\-out cross\-validation*: in each fold, the classifier is trained on all recordings from the remaining teachers and evaluated on the held\-out teacher, whose voice and session statistics are excluded from training\.

### 3\.6Three alternative teacher\-selection strategies

We compare our pipeline against a hand\-designed rule, a supervised upper bound, and an SOTA diarization tool:\(i\) Pseudo\-enrollmentbuilds a teacher centroid from the dominant speaker in the opening minutes and labels clusters within a cosine\-distance threshold as teacher\. It can select the wrong voice when the teacher does not dominate the opening\.\(ii\) Enrollmentuses the same threshold rule but builds its centroid from the teacher’s first3030s of labeled speech, providing a supervised reference that requires per\-session annotation\.\(iii\) pyannote speaker\-diarization\-3\.1is a strong external system that first diarizes an entire lesson\. We pick the teacher with a dominance rule detailed in App\.[A](https://arxiv.org/html/2611.23609#A1)\.

## 4Findings

We address six deployment questions\.Does it work?Our pipeline outperforms enrollment approaches \(Sec\.[4\.1](https://arxiv.org/html/2611.23609#S4.SS1), Table[1](https://arxiv.org/html/2611.23609#S3.T1)\)\.Why?A session\-relative feature,centrality, carries the signal across languages \(Sec\.[4\.2](https://arxiv.org/html/2611.23609#S4.SS2)\); an embedder ablation supports discourse*structure*as the key cue \(App\.[D](https://arxiv.org/html/2611.23609#A4)\)\.Does it generalize and deploy?Cross\-lingual transfer requires no retraining \(Sec\.[4\.3](https://arxiv.org/html/2611.23609#S4.SS3)\), and the pipeline runs∼4\.8×\\sim 4\.8\\timesfaster than a matched pyannote reference at equal memory on a commodity Android smartphone \(Sec\.[4\.4](https://arxiv.org/html/2611.23609#S4.SS4)\)\.How does filtering affect transcription?Filtering reduces substitutions and insertions but deletes some teacher speech, increasing teacher word error rate by2%2\\%compared to unfiltered audio \(Sec\.[4\.5](https://arxiv.org/html/2611.23609#S4.SS5)\)\.Can failures be detected?Failures arise when teacher and student embeddings do not separate \(Sec\.[4\.6](https://arxiv.org/html/2611.23609#S4.SS6)\); and they are*detectable at inference*, so low\-confidence sessions can be routed to human review \(Sec\.[4\.7](https://arxiv.org/html/2611.23609#S4.SS7)\)\.What did not help?Sec\.[4\.8](https://arxiv.org/html/2611.23609#S4.SS8)reports three unsuccessful interventions\.

### 4\.1Our pipeline outperforms enrollment methods

Main findings\.Table[1](https://arxiv.org/html/2611.23609#S3.T1)compares four teacher\-identification strategies under leave\-one\-teacher\-out cross\-validation across 24 English and 39 German teachers\. Our Pipeline achieves teacher F1 \(TchF1\) of0\.810\.81and0\.870\.87, respectively, outperforming both enrollment approaches with the lowest variability across folds\. The gain comes primarily from*recall*: supervised enrollment recovers just over half of teacher speech in English and two\-thirds in German, whereas our pipeline reaches0\.870\.87and0\.910\.91\. Under reverberation, the teacher’s voice fragments across clusters that can fall outside a fixed cosine threshold; our structural classifier evaluates each cluster and recovers more of these fragments\.

Accounting for wrong labels\.The human audit described in Sec\.[2](https://arxiv.org/html/2611.23609#S2)reveals substantial transcript errors\. Using human judgments and weighting each sampling stratum by its corpus duration raises estimated teacher F1 from0\.840\.84to0\.940\.94\(App\.[B](https://arxiv.org/html/2611.23609#A2)\)\. The correction is much larger for precision than recall: nearly two\-thirds of the speech flagged as teacher false positives is actually teacher speech\. Errors also persist where the pipeline and transcript agree\. Teacher\-labeled agreement regions are largely reliable, but56\.5%56\.5\\%of the audited student\-labeled agreement regions contain teacher speech or non\-speech\. The student channel’s apparent weakness in Table[1](https://arxiv.org/html/2611.23609#S3.T1)therefore reflects substantial reference\-label noise\. Since the audit covers full transcript intervals, its correction is an upper bound for the onset\-restricted main evaluation\.

Pyannote comparison\.Pyannote reaches TchF1 of0\.840\.84in English and0\.820\.82in German, slightly outperforming our pipeline on English but trailing it on German\. Both systems use the same WeSpeaker ResNet34 embedder\. What distinguishes our pipeline is not model size, but that it recovers the teacher from a*generic*embedder and session\-relative structure, without gated diarization weights or the neural segmentation stage, pyannote runs ahead of clustering\.

Overall, our approach generalizes to unseen teachers and improves recall over supervised enrollment through structural cluster features\. App\.[D](https://arxiv.org/html/2611.23609#A4)tests whether this advantage persists with a stronger embedder\.

### 4\.2Why it works:centralityis the dominant, transferable cue

Figure[2](https://arxiv.org/html/2611.23609#S4.F2)visualizes cluster centroids coloured by their majority ground\-truth role\. In successful recordings, teacher clusters are compact and well separated from student clusters, placing the session\-global mean near the teacher clusters\. Consequently,centralityassigns high scores \(cosine similarity\) to multiple teacher fragments\. In difficult recordings, teacher and student clusters overlap, weakening this cue\.

Single\-feature ablations support this explanation:centralityalone achieves teacher F1 of0\.850\.85, and adding other features through greedy forward selection provides little improvement \(App\.[F](https://arxiv.org/html/2611.23609#A6), Figure[7](https://arxiv.org/html/2611.23609#A6.F7)\)\. Embedding using6×6\\timeslarger ResNet221 also improves F1 by less than0\.010\.01in either language \(App\.[D](https://arxiv.org/html/2611.23609#A4)\)\. These support using a compact embedder with session\-relative features for on\-device inference\.

Figure 2:t\-SNE embeddings of*cluster centroids*, coloured by majority ground\-truth role\.Two good cases\(left two\): the teacher clusters are tight, and the global mean falls inside the teacher mass, so a single global centroid recovers the teacher\.Two bad cases\(right two\): the teacher clusters are fragmented, and the global mean drifts off the teacher mass, so the same proxy fails\.
### 4\.3Cross\-lingual transfer is free

The main experiments train a single role classifier jointly on German and English recordings\. Training on either language alone and evaluating on a held\-out teacher \(same language\) and lessons in the other language yields the same English/German TchF1 of0\.81/0\.870\.81/0\.87\(Table[1](https://arxiv.org/html/2611.23609#S3.T1)\)\. Thus, the session\-relative cue captured bycentralitytransfers across these corpora despite different recording conditions and classroom practices, allowing deployment in the other language without modification or additional annotation\.

### 4\.4Deployability: runtime, footprint, and clustering robustness

The full pipeline \(VAD, embedding, clustering, and role classification\) runs locally on CPU, allowing audio and intermediate speaker representations to remain on the device\. Having established competitive accuracy against pyannote \(Sec\.[4\.1](https://arxiv.org/html/2611.23609#S4.SS1)\), we compare their runtime and memory requirements\.

We benchmark both pipelines end\-to\-end on a laptop CPU and a commodity Android smartphone using matched inference settings\. Fig\.[3](https://arxiv.org/html/2611.23609#S4.F3)shows the results, and App\.[J](https://arxiv.org/html/2611.23609#A10)provides the hardware and protocol\. On the laptop, our pipeline achieves a real\-time factor \(RTF\) of0\.0220\.022: approximately25×25\\timesfaster than pyannote and45×45\\timesfaster than real time\. On the phone, where the baseline is itself an optimised ONNX build\(\([Next\-gen Kaldi Team 2026](https://arxiv.org/html/2611.23609#bib.bib37)\)\), it is4\.8×4\.8\\timesfaster\. Both approaches use the same RSS peak resident memory \(872 MB on laptop vs 356 MB on phone \(Figure[3](https://arxiv.org/html/2611.23609#S4.F3)\)\)\. We also benchmark the parallelisability of our pipeline by increasing the number of threads \(Table[9](https://arxiv.org/html/2611.23609#A10.T9), Fig\.[3](https://arxiv.org/html/2611.23609#S4.F3)\)\. We find that while there are efficiency gains on the phone with RTF nearly halving on 8 threads, the efficiency on laptop CPU stays flat or decreases\.

Figure 3:Efficiency of the proposed pipeline versuspyannote\.Left:Real\-time factor at matched accuracy \(recording A, single thread\); error bars span the recorded min–max, the dashed line is real time, the boxed figure is the within\-environment speed\-up\.Right:Thread scaling \(recording B\): the phone nearly halves its RTF from one to eight threads as embedding parallelises, while the laptop is already fast enough that RTF is flat\. Lower is faster\.
### 4\.5Downstream transcription analysis

We assess how role filtering affects teacher transcription by running Whisper\-large\-v2\([Radford et al\. 2023](https://arxiv.org/html/2611.23609#bib.bib35)\)on all 292 recordings in fixed3030s chunks\. We compare unfiltered audio with audio retaining only spans the pipeline labels as teacher\. Both conditions are evaluated against the teacher’s transcript turns using the teacher\-word scoring protocol in App\.[K](https://arxiv.org/html/2611.23609#A11)\.

Filtering modestly increases average teacher word error rate \(WER\), from0\.260\.26to0\.280\.28across the two languages\. The effect differs by corpus: English WER rises from0\.1600\.160to0\.2130\.213, with most recordings worsening, while German performance remains essentially unchanged\.

Filtering reduces substitutions and insertions in both corpora, but also removes some genuine teacher speech\. The resulting deletions outweigh the recognition gains, particularly in English\. Frequently deleted words include fillers, backchannels, and discourse markers, e\.g\.,*okay*,*right*,*ja*, and*mhm*\(Figure[10](https://arxiv.org/html/2611.23609#A11.F10)\)\. Although they carry little content, their loss may still matter for analyses of classroom interaction\.

Table 2:Embedding\-level oracle vs\. our pipeline\. Oracle is a supervised logistic probe on the raw embeddings \(the ceiling extractable from the representation\), and GT sil\. is the cosine silhouette of the*true*teacher/student partition\. Overall, the two match; only the low\-accuracy tail leaves a recoverable signal, and it remains a low\-separability regime\.
### 4\.6When the pipeline fails: separability sets the ceiling

Per\-recording accuracy depends strongly on how clearly teacher and student speech separate in the embedding space\. We measure this without reference labels using the*silhouette*of the predicted role partition: for each embedding, the difference between its mean cosine distance to the other role and to its own role, normalized by the larger of these distances and averaged across embeddings\. Teacher F1 correlates strongly with this measure \(ρ=0\.76\\rho=0\.76\) and drops sharply below a silhouette of0\.100\.10\(Figure[4](https://arxiv.org/html/2611.23609#A1.F4)\), linking the poorest\-performing recordings to weak separation between the two roles\.

To examine how much signal remains recoverable, we train a supervised, within\-recording linear probe on the raw256256\-dimensional window embeddings\. Using ground\-truth role labels, this probe assesses how well a linear classifier can distinguish teacher from student speech directly from the embeddings \(Table[2](https://arxiv.org/html/2611.23609#S4.T2)\)\. Its overall teacher F1 matches the pipeline at0\.870\.87\. In the lowest\-performing group, however, it improves F1 from from0\.630\.63to0\.780\.78, indicating that our pipeline does not recover some linearly separable signal\. Yet, even there, it reaches only0\.780\.78, i\.e\., some recordings are hard because the true embeddings barely separate\.

### 4\.7Failures are detectable at inference: routing to human review

The difficult recordings in Sec\.[4\.6](https://arxiv.org/html/2611.23609#S4.SS6)can often be flagged without reference labels\. Two inference\-time diagnostics correlate strongly with teacher F1: the*classifier margin*, the mean distance of the teacher probability from0\.50\.5, and the*predicted teacher share*, the fraction of speech assigned to the teacher \(Spearmanρ=0\.82\\rho=0\.82and0\.730\.73; see App\.[G](https://arxiv.org/html/2611.23609#A7), Table[8](https://arxiv.org/html/2611.23609#A7.T8)\)\.

Classifier margin is particularly useful for review routing\. Flagging the lowest\-margin20%20\\%of recordings captures85%85\\%of those with teacher F1 below0\.700\.70and raises mean F1 among retained recordings from0\.870\.87to0\.920\.92\. Predicted teacher share shows a similar association: recordings with lower shares tend to have lower accuracy \(Figure[8](https://arxiv.org/html/2611.23609#A7.F8)\)\. These diagnostics offer a practical way to prioritize human review; App\.[G](https://arxiv.org/html/2611.23609#A7)compares their predictive strength\.

### 4\.8What does not help

Three interventions do not change the results significantly\. We defer the details of these negative results to appendices\.

Accumulating a teacher voiceprint across lessons\.Pooling predicted teacher clusters from earlier lessons provides little benefit: teacher F1 increases by only0\.010\.01in German and remains unchanged in English\. Similarity to the accumulated voiceprint closely tracks similarity to the session\-global mean, suggesting that it adds little information beyondcentrality\. Label\-free accumulation also matches supervised enrollment, supporting its use when a cross\-lesson voiceprint is available to avoid enrollment \(App\.[H](https://arxiv.org/html/2611.23609#A8), Figure[9](https://arxiv.org/html/2611.23609#A8.F9)\)\.

Masking mixed windows\.Roughly9%9\\%of English and4%4\\%of German speech are flagged by pyannote/segmentation\-3\.0 as overlapping speech\. Excluding these windows from the feature calculations changes teacher F1 by at most0\.0020\.002across separability strata\. This suggests that the aggregate features are insensitive to this masking intervention, so we retain the simpler pipeline without overlap detection \(App\.[I](https://arxiv.org/html/2611.23609#A9)\)\.

Varying the onset\-trust window\.As noted in Sec\.[2](https://arxiv.org/html/2611.23609#S2), transcripts mark turn onsets rather than true speech offsets\. So, we restrict evaluation to a short window after each onset to reduce incorrect attribution of later pauses, backchannels, and other\-speaker speech\. Re\-scoring the predictions with onset\-trust windows of\{2,4,8\}\\\{2,4,8\\\}s and full transcript intervals preserves the strategy ranking\. Student precision falls as the window widens, consistent with longer interval tails introducing incorrect role labels\. The main comparison is therefore robust to the window choice, while student precision remains sensitive to transcript span inflation \(App\.[C](https://arxiv.org/html/2611.23609#A3)\)\.

## 5Related work

Speaker diarization\.Speaker diarization determines*who spoke when*\([Anguera et al\. 2012](https://arxiv.org/html/2611.23609#bib.bib15);[Park et al\. 2022](https://arxiv.org/html/2611.23609#bib.bib16)\), commonly through speaker embeddings and clustering\([Snyder et al\. 2018](https://arxiv.org/html/2611.23609#bib.bib17);[Desplanques et al\. 2020](https://arxiv.org/html/2611.23609#bib.bib24);[Wang et al\. 2023](https://arxiv.org/html/2611.23609#bib.bib23);[Landini et al\. 2022](https://arxiv.org/html/2611.23609#bib.bib18)\)or end\-to\-end neural approaches\([Fujita et al\. 2019](https://arxiv.org/html/2611.23609#bib.bib19);[Horiguchi et al\. 2020](https://arxiv.org/html/2611.23609#bib.bib20);[Plaquet and Bredin 2023](https://arxiv.org/html/2611.23609#bib.bib21);[Park et al\. 2024](https://arxiv.org/html/2611.23609#bib.bib27)\)\. Related systems use enrollment audio to detect target speakers\([Medennikov et al\. 2020](https://arxiv.org/html/2611.23609#bib.bib14)\)or combine diarization with ASR\([Bain et al\. 2023](https://arxiv.org/html/2611.23609#bib.bib26)\)\. Our task adds role assignment to this setting: the pipeline must identify teacher speech without enrollment and operate within a smartphone CPU budget\.

Classroom and child\-adult audio\.Teacher\-talk analysis spans wearable adult/child tagging\([Xu et al\. 2009](https://arxiv.org/html/2611.23609#bib.bib3);[Ganek and Eriks\-Brophy 2018](https://arxiv.org/html/2611.23609#bib.bib2)\)and classroom\-discourse analytics that rely on teacher/student segmentation\([Owens et al\. 2017](https://arxiv.org/html/2611.23609#bib.bib4);[Wang et al\. 2014](https://arxiv.org/html/2611.23609#bib.bib5);[Donnelly et al\. 2017](https://arxiv.org/html/2611.23609#bib.bib6);[Kelly et al\. 2018](https://arxiv.org/html/2611.23609#bib.bib7);[Schlotterbeck et al\. 2021](https://arxiv.org/html/2611.23609#bib.bib8)\)\. Short turns, cross\-talk, and limited representation of children’s speech in pretrained systems make this setting challenging\([Kothalkar et al\. 2024](https://arxiv.org/html/2611.23609#bib.bib32)\)\. Related work explores simulated child\-adult conversations to reduce dependence on sensitive recordings\([Xu et al\. 2025](https://arxiv.org/html/2611.23609#bib.bib33)\), therapist\-role identification in psychotherapy\([Flemotomos et al\. 2020](https://arxiv.org/html/2611.23609#bib.bib13)\), and video\-assisted classroom speaker identification\([Chen et al\. 2019](https://arxiv.org/html/2611.23609#bib.bib9);[Chrzan et al\. 2026](https://arxiv.org/html/2611.23609#bib.bib34)\)\. Our pipeline uses audio alone and assigns roles through session\-relative features\.

Closest systems\.[Wang et al\. \(2025\)](https://arxiv.org/html/2611.23609#bib.bib10)study a cluster\-ing\-based pipeline for classroom group discussions using ECAPA\-TDNN embeddings, Whisper\-derived segmentation, and enrollment audio\. Our pipeline instead combines dedicated VAD with explicit teacher\-role classification and requires no enrollment\.[Khan et al\. \(2025\)](https://arxiv.org/html/2611.23609#bib.bib31)address classroom noise through a multi\-stage pipeline combining denoising, neural VAD, and ASR timestamps, resulting in a server\-side and GPU\-scale setting\. We focus on a compact inference path that runs faster than real time on a commodity smartphone CPU\. Speech\-privacy research also studies federated learning\([Granqvist et al\. 2020](https://arxiv.org/html/2611.23609#bib.bib28);[Woubie and Bäckström 2021](https://arxiv.org/html/2611.23609#bib.bib29)\)and anonymisation\([Sümer et al\. 2020](https://arxiv.org/html/2611.23609#bib.bib11)\)\. Our deployment approach complements these directions by keeping audio and speaker representations on the device during inference\.

## 6Discussion and Conclusion

Our results suggest that teacher\-role recovery can rely on session\-relative structure without speaker enrollment\. Centrality supplies most of the predictive signal, transfers between the two evaluated languages, and gains little from a larger embedder or accumulated teacher voiceprints\. This supports a compact, on\-device design for classroom role segmentation, although its value for educational feedback remains to be established in operational use\.

Reliability remains conditional on teacher/student acoustic separability\. Low classifier margins identify many difficult sessions, supporting selective human review rather than uniformly trusting every output\. Filtering also introduces a downstream trade\-off: average teacher WER rises from0\.260\.26to0\.280\.28, with greater degradation in English\. Deleted backchannels and discourse markers may matter for interaction analysis even when they contribute little lexical content\.

Our findings are restricted to two corpora in which language and microphone regime vary together\. The timestamp audit also shows that transcript\-derived labels are imperfect to measure accuracy\. Future work should prioritize precisely timed role annotations, prospective evaluation of thresholds, and testing in more diverse classrooms\. Overall, the pipeline provides a practical basis for local teacher/student segmentation, with explicit reliability checks and task\-specific validation remaining essential for deployment\.

## Ethical Statement

We use two existing research corpora, collected under their originating studies’ informed\-consent procedures and ethics approvals and accessed under their data\-use agreements\. These recordings involve minors and are not redistributed\. Our analysis distinguishes discourse roles without identifying individual students\. Segmentation runs locally, allowing raw recordings and intermediate speaker embeddings to remain on the device\. However, filtering does not anonymize speech: teacher segments remain identifiable, and classification errors can retain student speech\.

The pipeline is intended to support aggregate classroom\-discourse analysis for research and formative feedback\. Its outputs should not serve as standalone measures of teaching quality or support high\-stakes decisions about teachers or students\. Errors may distort talk\-time estimates and remove meaningful speech, while performance beyond the evaluated classrooms remains uncertain\. Deployments should limit data access and retention and provide human oversight\. Our inference\-time diagnostics can prioritize potentially unreliable sessions for review \(Sec\.[4\.7](https://arxiv.org/html/2611.23609#S4.SS7)\), but do not guarantee accurate role assignments or prevent privacy\-sensitive errors\.

#### Acknowledgements

Çağatay Yıldız is a member of the Machine Learning Cluster of Excellence, funded by the Deutsche Forschungsgemeinschaft \(DFG, German Research Foundation\) under Germany’s Excellence Strategy – EXC number 2064/1 – Project number 390727645\. This research utilized compute resources at the Tübingen Machine Learning Cloud, DFG FKZ INST 37/1057\-1 FUGG\.

## References

- Angueraet al\.\(2012\)X\. Anguera, S\. Bozonnet, N\. Evans, C\. Fredouille, G\. Friedland, and O\. VinyalsSpeaker diarization: a review of recent research\.IEEE Transactions on Audio, Speech, and Language Processing20\(2\),pp\. 356–370\.External Links:[Document](https://dx.doi.org/10.1109/TASL.2011.2125954)Cited by:[§1](https://arxiv.org/html/2611.23609#S1.p2.1),[§5](https://arxiv.org/html/2611.23609#S5.p1.1)\.
- Bainet al\.\(2023\)M\. Bain, J\. Huh, T\. Han, and A\. ZissermanWhisperX: time\-accurate speech transcription of long\-form audio\.InProc\. Interspeech,Cited by:[§5](https://arxiv.org/html/2611.23609#S5.p1.1)\.
- Bredin \(2023\)H\. BredinPyannote\.audio 2\.1 speaker diarization pipeline: principle, benchmark, and recipe\.InProc\. Interspeech,pp\. 1983–1987\.Cited by:[§1](https://arxiv.org/html/2611.23609#S1.p4.1)\.
- Chenet al\.\(2019\)J\. Chen, H\. Li, W\. Wang, W\. Ding, G\. Y\. Huang, and Z\. LiuA multimodal alerting system for online class quality assurance\.InArtificial Intelligence in Education \(AIED 2019\),Lecture Notes in Computer Science, Vol\.11626,pp\. 381–385\.Cited by:[§5](https://arxiv.org/html/2611.23609#S5.p2.1)\.
- Chrzanet al\.\(2026\)M\. L\. Chrzan, M\. Krishnaswamy, R\. Gibboni, K\. Wetstone, W\. Ai, and J\. LiuMultimodal speaker identification in classroom environments\.arXiv preprint arXiv:2606\.13712\.Cited by:[§5](https://arxiv.org/html/2611.23609#S5.p2.1)\.
- Desplanqueset al\.\(2020\)B\. Desplanques, J\. Thienpondt, and K\. DemuynckECAPA\-TDNN: emphasized channel attention, propagation and aggregation in TDNN based speaker verification\.InProc\. Interspeech,pp\. 3830–3834\.Cited by:[§5](https://arxiv.org/html/2611.23609#S5.p1.1)\.
- Donnellyet al\.\(2017\)P\. J\. Donnelly, N\. Blanchard, B\. Samei, A\. M\. Olney, X\. Sun, B\. Ward, S\. Kelly, M\. Nystrand, and S\. K\. D’MelloWords matter: automatic detection of teacher questions in live classroom discourse using linguistics, acoustics, and context\.InProc\. 7th International Learning Analytics & Knowledge Conference \(LAK\),pp\. 218–227\.External Links:[Document](https://dx.doi.org/10.1145/3027385.3027417)Cited by:[§1](https://arxiv.org/html/2611.23609#S1.p1.1),[§5](https://arxiv.org/html/2611.23609#S5.p2.1)\.
- Flemotomoset al\.\(2020\)N\. Flemotomos, P\. Georgiou, and S\. NarayananLinguistically aided speaker diarization using speaker role information\.InProc\. Odyssey: The Speaker and Language Recognition Workshop,Note:arXiv:1911\.07994Cited by:[§5](https://arxiv.org/html/2611.23609#S5.p2.1)\.
- Fujitaet al\.\(2019\)Y\. Fujita, N\. Kanda, S\. Horiguchi, K\. Nagamatsu, and S\. WatanabeEnd\-to\-end neural speaker diarization with permutation\-free objectives\.InProc\. Interspeech,pp\. 4300–4304\.Cited by:[§5](https://arxiv.org/html/2611.23609#S5.p1.1)\.
- Ganek and Eriks\-Brophy \(2018\)H\. Ganek and A\. Eriks\-BrophyLanguage environment analysis \(LENA\) system investigation of day long recordings in children: a literature review\.Journal of Communication Disorders\.Cited by:[§5](https://arxiv.org/html/2611.23609#S5.p2.1)\.
- Granqvistet al\.\(2020\)F\. Granqvist, M\. Seigel, R\. van Dalen, Á\. Cahill, S\. Shum, and M\. PaulikImproving on\-device speaker verification using federated learning with privacy\.InProc\. Interspeech,Note:arXiv:2008\.02651Cited by:[§1](https://arxiv.org/html/2611.23609#S1.p2.1),[§5](https://arxiv.org/html/2611.23609#S5.p3.1)\.
- Horiguchiet al\.\(2020\)S\. Horiguchi, Y\. Fujita, S\. Watanabe, Y\. Xue, and K\. NagamatsuEnd\-to\-end speaker diarization for an unknown number of speakers with encoder\-decoder based attractors\.InProc\. Interspeech,pp\. 269–273\.Cited by:[§5](https://arxiv.org/html/2611.23609#S5.p1.1)\.
- Houet al\.\(2025\)R\. Hou, B\. Bühler, T\. Fütterer, E\. Bozkir, P\. Gerjets, U\. Trautwein, and E\. KasneciMultimodal assessment of classroom discourse quality: a text\-centered attention\-based multi\-task learning approach\.arXiv preprint arXiv:2505\.07902\.Cited by:[§1](https://arxiv.org/html/2611.23609#S1.p1.1)\.
- Kellyet al\.\(2018\)S\. Kelly, A\. M\. Olney, P\. Donnelly, M\. Nystrand, and S\. K\. D’MelloAutomatically measuring question authenticity in real\-world classrooms\.Educational Researcher47\(7\),pp\. 451–464\.External Links:[Document](https://dx.doi.org/10.3102/0013189X18785613)Cited by:[§1](https://arxiv.org/html/2611.23609#S1.p1.1),[§5](https://arxiv.org/html/2611.23609#S5.p2.1)\.
- Khanet al\.\(2025\)A\. S\. Khan, T\. Ogunremi, A\. A\. Attia, and D\. DemszkyMulti\-stage speaker diarization for noisy classrooms\.arXiv preprint arXiv:2505\.10879\.Cited by:[§5](https://arxiv.org/html/2611.23609#S5.p3.1)\.
- Kothalkaret al\.\(2024\)P\. V\. Kothalkar, J\. H\. L\. Hansen, D\. Irvin, and J\. BuzhardtChild\-adult speech diarization in naturalistic conditions of preschool classrooms using room\-independent ResNet model and automatic speech recognition\-based re\-segmentation\.The Journal of the Acoustical Society of America155\(2\),pp\. 1198–1215\.Cited by:[§5](https://arxiv.org/html/2611.23609#S5.p2.1)\.
- Landiniet al\.\(2022\)F\. Landini, J\. Profant, M\. Diez, and L\. BurgetBayesian HMM clustering of x\-vector sequences \(VBx\) in speaker diarization: theory, implementation and analysis on standard tasks\.Computer Speech & Language71,pp\. 101254\.External Links:[Document](https://dx.doi.org/10.1016/j.csl.2021.101254)Cited by:[§5](https://arxiv.org/html/2611.23609#S5.p1.1)\.
- Medennikovet al\.\(2020\)I\. Medennikov, M\. Korenevsky, T\. Prisyach, Y\. Khokhlov, M\. Korenevskaya, I\. Sorokin, T\. Timofeeva, A\. Mitrofanov, A\. Andrusenko, I\. Podluzhny, A\. Laptev, and A\. RomanenkoTarget\-speaker voice activity detection: a novel approach for multi\-speaker diarization in a dinner party scenario\.InProc\. Interspeech,pp\. 274–278\.Cited by:[§5](https://arxiv.org/html/2611.23609#S5.p1.1)\.
- Next\-gen Kaldi Team \(2026\)Next\-gen Kaldi TeamSherpa\-onnx: speech\-to\-text, text\-to\-speech, and speaker diarization using next\-gen kaldi with onnxruntime\.Note:https://github\.com/k2\-fsa/sherpa\-onnxVersion 1\.13\.4Cited by:[Appendix J](https://arxiv.org/html/2611.23609#A10.SS0.SSS0.Px2.p1.1),[§4\.4](https://arxiv.org/html/2611.23609#S4.SS4.p2.1)\.
- OECD \(2020\)OECDGlobal teaching insights: a video study of teaching\.OECD Publishing,Paris\.External Links:[Document](https://dx.doi.org/10.1787/20d6f36b-en)Cited by:[2nd item](https://arxiv.org/html/2611.23609#S2.I1.i2.p1.1.1)\.
- Owenset al\.\(2017\)M\. T\. Owens, S\. B\. Seidel, M\. Wong, T\. E\. Bejines, S\. Lietz, J\. R\. Perez, S\. Sit, Z\. Subedar, G\. N\. Acker, S\. F\. Akana,et al\.Classroom sound can be used to classify teaching practices in college science courses\.Proceedings of the National Academy of Sciences114\(12\),pp\. 3085–3090\.External Links:[Document](https://dx.doi.org/10.1073/pnas.1618693114)Cited by:[§5](https://arxiv.org/html/2611.23609#S5.p2.1)\.
- Parket al\.\(2022\)T\. J\. Park, N\. Kanda, D\. Dimitriadis, K\. J\. Han, S\. Watanabe, and S\. NarayananA review of speaker diarization: recent advances with deep learning\.Computer Speech & Language72,pp\. 101317\.External Links:[Document](https://dx.doi.org/10.1016/j.csl.2021.101317)Cited by:[§1](https://arxiv.org/html/2611.23609#S1.p2.1),[§5](https://arxiv.org/html/2611.23609#S5.p1.1)\.
- Parket al\.\(2024\)T\. J\. Park, N\. R\. Koluguri, W\. Wang, J\. Balam, and B\. GinsburgSortformer: seamless integration of speaker diarization and ASR by bridging timestamps and tokens\.Note:arXiv preprint arXiv:2409\.06656Cited by:[§5](https://arxiv.org/html/2611.23609#S5.p1.1)\.
- Pianta and Burchinal \(2016\)R\. Pianta and M\. BurchinalNational center for research on early childhood education teacher professional development study \(2007\-2011\)\.Inter\-university Consortium for Political and Social Research \[distributor\]\.External Links:[Document](https://dx.doi.org/10.3886/ICPSR34848.v2),[Link](https://doi.org/10.3886/ICPSR34848.v2)Cited by:[1st item](https://arxiv.org/html/2611.23609#S2.I1.i1.p1.1.1)\.
- Plaquet and Bredin \(2023\)A\. Plaquet and H\. BredinPowerset multi\-class cross entropy loss for neural speaker diarization\.InProc\. Interspeech,pp\. 3222–3226\.External Links:[Document](https://dx.doi.org/10.21437/Interspeech.2023-205)Cited by:[§1](https://arxiv.org/html/2611.23609#S1.p4.1),[§5](https://arxiv.org/html/2611.23609#S5.p1.1)\.
- Radfordet al\.\(2023\)A\. Radford, J\. W\. Kim, T\. Xu, G\. Brockman, C\. McLeavey, and I\. SutskeverRobust speech recognition via large\-scale weak supervision\.InInternational conference on machine learning,pp\. 28492–28518\.Cited by:[§4\.5](https://arxiv.org/html/2611.23609#S4.SS5.p1.1)\.
- Schlotterbecket al\.\(2021\)D\. Schlotterbeck, P\. Uribe, A\. Jiménez, R\. Araya, J\. Van der Molen Moris, and D\. CaballeroTARTA: teacher activity recognizer from transcriptions and audio\.InArtificial Intelligence in Education \(AIED 2021\),Lecture Notes in Computer Science, Vol\.12748,pp\. 369–380\.External Links:[Document](https://dx.doi.org/10.1007/978-3-030-78292-4%5F30)Cited by:[§1](https://arxiv.org/html/2611.23609#S1.p1.1),[§5](https://arxiv.org/html/2611.23609#S5.p2.1)\.
- Silero Team \(2024\)Silero TeamSilero VAD: pre\-trained enterprise\-grade voice activity detector \(VAD\), number detector and language classifier\.Note:GitHub repository:https://github\.com/snakers4/silero\-vadCited by:[Appendix J](https://arxiv.org/html/2611.23609#A10.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2611.23609#S1.p3.1),[§3\.1](https://arxiv.org/html/2611.23609#S3.SS1.p1.1)\.
- Snyderet al\.\(2018\)D\. Snyder, D\. Garcia\-Romero, G\. Sell, D\. Povey, and S\. KhudanpurX\-vectors: robust DNN embeddings for speaker recognition\.InProc\. IEEE ICASSP,pp\. 5329–5333\.External Links:[Document](https://dx.doi.org/10.1109/ICASSP.2018.8461375)Cited by:[§5](https://arxiv.org/html/2611.23609#S5.p1.1)\.
- Sümeret al\.\(2020\)Ö\. Sümer, P\. Gerjets, U\. Trautwein, and E\. KasneciAutomated anonymisation of visual and audio data in classroom studies\.arXiv preprint arXiv:2001\.05080\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2001.05080)Cited by:[§1](https://arxiv.org/html/2611.23609#S1.p2.1),[§5](https://arxiv.org/html/2611.23609#S5.p3.1)\.
- Tao and Chen \(2024\)Y\. Tao and G\. ChenThe relationship between teacher talk and students’ academic achievement: a meta\-analysis\.Educational Research Review45,pp\. 100638\.External Links:ISSN 1747\-938X,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/j.edurev.2024.100638),[Link](https://www.sciencedirect.com/science/article/pii/S1747938X24000472)Cited by:[§1](https://arxiv.org/html/2611.23609#S1.p1.1)\.
- Wanget al\.\(2023\)H\. Wang, C\. Liang, S\. Wang, Z\. Chen, B\. Zhang, X\. Xiang, Y\. Deng, and Y\. QianWespeaker: a research and production oriented speaker embedding learning toolkit\.InProc\. IEEE ICASSP,Cited by:[Appendix J](https://arxiv.org/html/2611.23609#A10.SS0.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2611.23609#S1.p3.1),[§3\.2](https://arxiv.org/html/2611.23609#S3.SS2.p1.1),[§5](https://arxiv.org/html/2611.23609#S5.p1.1)\.
- Wanget al\.\(2025\)J\. Wang, S\. Dudy, X\. He, Z\. Wang, R\. Southwell, and J\. WhitehillOptimizing speaker diarization for the classroom: applications in timing student speech and distinguishing teachers from children\.Journal of Educational Data Mining17\(1\),pp\. 98–125\.External Links:[Link](https://jedm.educationaldatamining.org/index.php/JEDM/article/view/841)Cited by:[§1](https://arxiv.org/html/2611.23609#S1.p1.1),[§5](https://arxiv.org/html/2611.23609#S5.p3.1)\.
- Wanget al\.\(2014\)Z\. Wang, X\. Pan, K\. F\. Miller, and K\. S\. CortinaAutomatic classification of activities in classroom discourse\.Computers & Education78,pp\. 115–123\.External Links:[Document](https://dx.doi.org/10.1016/j.compedu.2014.05.010)Cited by:[§5](https://arxiv.org/html/2611.23609#S5.p2.1)\.
- Woubie and Bäckström \(2021\)A\. Woubie and T\. BäckströmFederated learning for privacy preserving on\-device speaker recognition\.In2021 ISCA Symposium on Security and Privacy in Speech Communication \(SPSC\),pp\. 1–5\.External Links:[Document](https://dx.doi.org/10.21437/SPSC.2021-1)Cited by:[§5](https://arxiv.org/html/2611.23609#S5.p3.1)\.
- Xuet al\.\(2025\)A\. Xu, T\. Feng, H\. Tager\-Flusberg, C\. Lord, and S\. NarayananData efficient child\-adult speaker diarization with simulated conversations\.InICASSP 2025 \- IEEE International Conference on Acoustics, Speech and Signal Processing,Cited by:[§5](https://arxiv.org/html/2611.23609#S5.p2.1)\.
- Xuet al\.\(2009\)D\. Xu, U\. Yapanel, and S\. GrayReliability of the LENA Language Environment Analysis system in young children’s natural home environment\.Technical reportTechnical ReportLTR\-05\-2,LENA Foundation\.Cited by:[§5](https://arxiv.org/html/2611.23609#S5.p2.1)\.

Table 3:Overview of appendix sections and supplementary analyses\.ItemDescriptionComplementsFigure[4](https://arxiv.org/html/2611.23609#A1.F4)Relationship between teacher/student embedding separability and per\-recording teacher F1Sec\.[4\.6](https://arxiv.org/html/2611.23609#S4.SS6)App\.[A](https://arxiv.org/html/2611.23609#A1)Details of how we map pyannote annotations to teacher/student labelsSecs\.[3\.6](https://arxiv.org/html/2611.23609#S3.SS6),[4\.1](https://arxiv.org/html/2611.23609#S4.SS1)App\.[B](https://arxiv.org/html/2611.23609#A2)Human audit of transcript labels: sampling, annotation, and duration\-weighted corrected metricsSecs\.[2](https://arxiv.org/html/2611.23609#S2),[4\.1](https://arxiv.org/html/2611.23609#S4.SS1)App\.[C](https://arxiv.org/html/2611.23609#A3)Sensitivity of teacher F1 and student precision to the onset\-trust windowSecs\.[2](https://arxiv.org/html/2611.23609#S2),[4\.8](https://arxiv.org/html/2611.23609#S4.SS8)App\.[D](https://arxiv.org/html/2611.23609#A4)ResNet34 versus ResNet221 embeddings for our pipeline and supervised enrollmentSec\.[4\.2](https://arxiv.org/html/2611.23609#S4.SS2)App\.[E](https://arxiv.org/html/2611.23609#A5)Accuracy and cluster\-count sensitivity to the clustering distance thresholdSec\.[3\.3](https://arxiv.org/html/2611.23609#S3.SS3)App\.[F](https://arxiv.org/html/2611.23609#A6)Structural feature definitions, feature selection, and single\-feature performanceSecs\.[3\.4](https://arxiv.org/html/2611.23609#S3.SS4),[4\.2](https://arxiv.org/html/2611.23609#S4.SS2)App\.[G](https://arxiv.org/html/2611.23609#A7)Label\-free session diagnostics and their association with teacher F1Sec\.[4\.7](https://arxiv.org/html/2611.23609#S4.SS7)App\.[H](https://arxiv.org/html/2611.23609#A8)Cross\-lesson teacher accumulation and its limited benefit beyond centralitySec\.[4\.8](https://arxiv.org/html/2611.23609#S4.SS8)App\.[I](https://arxiv.org/html/2611.23609#A9)Effect of excluding overlap\-detected windows from feature computationSec\.[4\.8](https://arxiv.org/html/2611.23609#S4.SS8)App\.[J](https://arxiv.org/html/2611.23609#A10)Runtime implementations, benchmark protocol, hardware, and thread scalingSec\.[4\.4](https://arxiv.org/html/2611.23609#S4.SS4)App\.[K](https://arxiv.org/html/2611.23609#A11)Teacher transcription protocol, corpus\-level error breakdowns, and frequently deleted wordsSec\.[4\.5](https://arxiv.org/html/2611.23609#S4.SS5)#### A note on data preprocessing\.

We start by noting the audio preprocessing\. All audio is decoded to 16 kHz monofloat32\. Video sources are converted with a fixedffmpegrecipe \(\-vn \-ac 1 \-ar 16000 \-c:a pcm\_s16le\)\. For the English close\-mic case, we additionally check for a separate loud channel that would give teacher activity directly; our recordings are mono, so the mixed\-down path is used throughout\.

## Appendix AAssigning roles topyannotespeakers\.

pyannotepartitions a lesson into anonymous speakers but does not identify their roles\. We therefore map its output to teacher and student labels using a label\-free*dominance rule*\. Each VAD window is assigned to the speaker with the greatest temporal overlap\. We then score each speaker usingD=talk\_time\_frac×coverage×t​u​r​n​\_​c​o​u​n​tD=\\texttt\{talk\\\_time\\\_frac\}\\times\\texttt\{coverage\}\\times turn\\\_count\. For English recordings, where teachers wear a close\-talk microphone, we additionally multiply this score by a shiftedenergy\_zterm\. We decided this rule after cross validating the individual factors and combining pairs/triplets\.

The speaker with the highest score is labeled teacher, and all remaining speakers are labeled student\. The product favors speakers who combine three properties: they account for a large share of speech, participate throughout the lesson, and take many turns\. Together, these cues help distinguish the teacher from a talkative student who may score highly on one component but is less likely to score highly on all three\. This rule is particularly suitable forpyannote, whose output aims to group each speaker’s speech under a single identity, making the selection of one highest\-scoring speaker a natural way to assign the teacher role\.

Figure 4:Per\-recording tchF1 against teacher/student embedding separability \(silhouette\)\. The relationship is tight \(Spearmanρ=0\.80\\rho=0\.80\) and tchF1 collapses once separability falls below≈0\.10\\approx 0\.10: the low\-F1 recordings are the ones whose teacher and student voices the embeddings fail to pull apart\.
## Appendix BA human\-labeled audit of the ground truth

Sections[2](https://arxiv.org/html/2611.23609#S2)and[4\.1](https://arxiv.org/html/2611.23609#S4.SS1)claim that a large share of the residual error in Table[1](https://arxiv.org/html/2611.23609#S3.T1)is label error\. This appendix gives the measurement behind that claim\.

#### Design\.

We sampled5,0005\{,\}000clips from202202of our recordings \(134134German,6868English\) and had a single trained annotator label each one by ear as teacher/adult \(t\), student/child \(s\), non\-speech \(n\), or unclear \(u\), with a separate binary flag for audible overlap\. The annotator saw neither the model output nor the transcript, and clip order was shuffled, so no stratum was identifiable during labeling\.4,9984\{,\}998clips were returned labeled\.

Sampling is stratified into six strata, chosen so that the estimator can correct the*agreement*regions rather than only the disagreements\. Two strata cover the frames where the pipeline and the transcript disagree \(FP: pipeline teacher, transcript student;FN: pipeline student, transcript teacher\); two are agreement controls \(ctrlT,ctrlS\), which measure the transcript’s error rate where nothing flags a problem; and two cover the regions the VAD discards as silence, split at the median frame energy \(silHigh,silLow\) so that quiet regions are measured rather than assumed non\-speech\. Sampling is budgeted by clip*count*, since annotation effort scales with the number of decisions rather than with audio duration\.

Table 4:The six sampling strata: total duration in the corpus, clips sampled, and the duration\-weighted distribution of human labels within each stratum\.ovis the fraction of*speech\-labeled*clip time carrying the overlap flag\. Rows sum to11up to the unusedulabel \(see the caveat below\)\.
#### Estimator\.

Each stratum is a uniform sample of a known total duration, so a duration\-weighted fraction within a stratum extrapolates to the corpus by scaling with that total\. WritingT⁡\(⋅\)T\(\\cdot\)for a stratum’s total seconds andfx​\(⋅\)f\_\{x\}\(\\cdot\)for the fraction of its sampled clip\-time labeledxx, the corrected teacher confusion is

TP\\displaystyle\\mathrm\{TP\}=T⁡\(ctrlT\)​ft​\(ctrlT\)\+T⁡\(FP\)​ft​\(FP\),\\displaystyle=T\(\\text\{ctrlT\}\)f\_\{t\}\(\\text\{ctrlT\}\)\+T\(\\text\{FP\}\)f\_\{t\}\(\\text\{FP\}\),FP\\displaystyle\\mathrm\{FP\}=T⁡\(ctrlT\)​fs​\(ctrlT\)\+T⁡\(FP\)​fs​\(FP\),\\displaystyle=T\(\\text\{ctrlT\}\)f\_\{s\}\(\\text\{ctrlT\}\)\+T\(\\text\{FP\}\)f\_\{s\}\(\\text\{FP\}\),FN\\displaystyle\\mathrm\{FN\}=T⁡\(ctrlS\)​ft​\(ctrlS\)\+T⁡\(FN\)​ft​\(FN\),\\displaystyle=T\(\\text\{ctrlS\}\)f\_\{t\}\(\\text\{ctrlS\}\)\+T\(\\text\{FN\}\)f\_\{t\}\(\\text\{FN\}\),and symmetrically for the student role\. VAD recall compares the retained speech total against the speech recovered from the two silence strata,T⁡\(silHigh\)​fsp​\(silHigh\)\+T⁡\(silLow\)​fsp​\(silLow\)T\(\\text\{silHigh\}\)f\_\{\\mathrm\{sp\}\}\(\\text\{silHigh\}\)\+T\(\\text\{silLow\}\)f\_\{\\mathrm\{sp\}\}\(\\text\{silLow\}\)withfsp=ft\+fsf\_\{\\mathrm\{sp\}\}=f\_\{t\}\+f\_\{s\}\. Confidence intervals are the2\.52\.5/97\.597\.5percentiles of10001000bootstrap resamples drawn within each stratum\.

Table 5:Human\-corrected metrics\. “transcript” scores the same recordings with the transcript taken as truth; “human” treats the annotator as truth\. Both are pooled across corpora and scored over full transcript intervals, so the transcript column is the like\-for\-like reference\.transcripthuman95%95\\%CIteacher precision0\.860\.860\.980\.98\[0\.97,0\.99\]\[0\.97,0\.99\]teacher recall0\.830\.830\.910\.91\[0\.90,0\.92\]\[0\.90,0\.92\]teacher F10\.840\.840\.94\\mathbf\{0\.94\}\[0\.94,0\.95\]\[0\.94,0\.95\]student F10\.600\.600\.710\.71—transcript error, ctrlT—0\.0350\.035\[0\.023,0\.048\]\[0\.023,0\.048\]transcript error, ctrlS—0\.5650\.565\[0\.513,0\.621\]\[0\.513,0\.621\]VAD recall—0\.9280\.928\[0\.910,0\.945\]\[0\.910,0\.945\]
#### Results\.

Table[5](https://arxiv.org/html/2611.23609#A2.T5)collects the estimates\. Correcting the labels moves teacher F1 from0\.840\.84to0\.940\.94, i\.e\. roughly two thirds of the counted teacher error is not model error, and the correction lands on precision rather than recall\. The two control strata explain why: where the transcript and the model agree onteacherit is right96\.5%96\.5\\%of the time, but where they agree onstudentthe label is wrong on56\.5%56\.5\\%of the audited time—only44%44\\%of it is a student speaking, the rest being teacher speech \(18%18\\%\) or non\-speech \(38%38\\%\)\. Thestudentlabel is thus closer to a residual class than to a positive one, which depresses every student\-side number in the main text\.

The silence strata bound what the VAD discards:92\.8%92\.8\\%of speech is retained, and the missed7%7\\%divides between the roles in proportion to their talk time \(72\.8%72\.8\\%of it teacher, against a73\.3%73\.3\\%teacher base rate\), so it does not bias the teacher/student comparison\.

#### Caveats\.

Two limitations apply to our inspection\. First, the labels come from a*single*annotator\. We report no inter\-annotator agreement; the direction of the result replicates a smaller pilot audit by one of the authors on disjoint clips, but the magnitudes should be read as one rater’s judgment\. Second, the pool was drawn against ground truth over full transcript intervals, whereas the main results trust only the firsth=4h=4s after each onset \(Section[3\.5](https://arxiv.org/html/2611.23609#S3.SS5)\)\. Since that window already removes part of the same label noise, the correction reported here is an upper bound on what applies to Table[1](https://arxiv.org/html/2611.23609#S3.T1)\.

## Appendix CSensitivity to the ground\-truth onset\-trust window

The evaluation metric \(Section[3\.5](https://arxiv.org/html/2611.23609#S3.SS5)\) trusts only the firsth=4h=4s of speech after each transcript onset as ground truth since the transcripts are onset\-anchored\. More concretely, an interval marks*when*a turn begins, not how long the speaker talks, so the interval tail silently contains pauses, backchannels, and overlapping speeches from the other party\. The44s window is therefore a hyperparameter, and a natural worry is that the reported results are an artifact of that particular choice\. To rule this out, we re\-score the*same*per\-fold predictions against ground truth computed ath∈\{2,4,8\}h\\in\\\{2,4,8\\\}s and at the full interval \(h→∞h\\to\\infty, i\.e\. every frame from one onset to the next is attributed to the named speaker\)\. Figure[5](https://arxiv.org/html/2611.23609#A3.F5)reports teacher F1 and student precision for all three strategies, leading to two clear patterns:

![Refer to caption](https://arxiv.org/html/2611.23609v1/figures/headsweep.png)Figure 5:Effect of the ground\-truth onset\-trust window on the evaluation\. Rows are languages, columns are teacher F1 and student precision; within each panel the bars are grouped by window \(22/44/88s and full interval\) and coloured by strategy \(error bars: std across leave\-one\-teacher\-out folds\)\.Teacher F1 \(left\) is flat\-to\-risingas the window widens, and the ranking of strategies is preserved at every window\.Student precision \(right\) falls monotonically\. The signature of inflated interval tails being folded into the dominant speaker’s ground truth\.Teacher F1 is essentially flat\.For our pipeline, it moves from0\.770\.77to0\.860\.86\(English\) and0\.840\.84to0\.870\.87\(German\) ashhgrows from22s to the full interval\. More importantly, the ranking of the three strategies is identical at every window: our pipeline wins, enrollment is the next best, and then pseudo\-enrollment\. The main\-text conclusions do not depend on the value ofhh\.

Student precision, by contrast, falls monotonically \.As the window widens, our pipeline drops from0\.630\.63to0\.560\.56\(English\) and0\.780\.78to0\.630\.63\(German\) betweenh=2h=2s and the full interval\. This is exactly the inflated\-tail effect the metric is designed to avoid\. Ashhgrows, more of each interval’s tail, which in reality contains silence and short turns by the other speaker, is credited to the named speaker in the ground truth\. Frames the pipeline*correctly*assigns to the student are then scored against a label that has been overwritten as teacher, so student precision is penalized for being right\. Full\-interval scoring, therefore, does not measure the system more realistically; it measures the system against noisier labels, under\-crediting student purity\. Theh=4h=4s window sits where teacher recovery has already saturated, but tail contamination is still small, and we adopt it for the main results on that basis\. We note that this robustness check uses the existing onset\-anchored transcripts; a frame\-accurate evaluation of sustained turns would require a small subset re\-labeled with true speech offsets, which we leave to future work\.

## Appendix DEmbedding\-model robustness: ResNet34 vs\. ResNet221

All results in the main text use a compact VoxCeleb ResNet34 speaker embedder, chosen for its small footprint\. A natural question is whether the method is bottlenecked by embedding capacity\. In particular, whether the supervised enrollment ceiling and the gap between it and our label\-free pipeline would shrink with a stronger model\. To probe this, we rebuild both caches with a6×6\\timeslarger embedder from the same family, VoxCeleb ResNet221, and re\-run the entire leave\-one\-teacher\-out evaluation\.

The main finding is thatour pipeline is essentially invariant to the embedder\(Table[6](https://arxiv.org/html/2611.23609#A4.T6)\): teacher F1 changes by less than0\.010\.01in either language, and every other cell moves within fold\-level noise\. By contrast,every cosine\-distance strategy improveswith the larger model\. The clearest gain is for supervised enrollment \(Table[7](https://arxiv.org/html/2611.23609#A4.T7)\), whose teacher recall rises by\+0\.05\+0\.05\(English\) and\+0\.04\+0\.04\(German\) and F1 by\+0\.03\+0\.03/\+0\.02\+0\.02; pseudo\-enrollment shifts in the same direction\. The better embedding flows into exactly the mechanisms that decide membership by voiceprint geometry, and not into the structural\-feature classifier\.

Table 6:Embedding\-model ablation for our pipeline: ResNet34 vs\. a6×6\\timeslarger ResNet221, frame\-level mean±\\pmstd across leave\-one\-teacher\-out folds \(h=4h=4s\)\. The classifier is invariant to the embedder\.Table 7:Same ablation for the supervised*enrollment*upper bound, which decides cluster membership by cosine distance to a confirmed\-teacher centroid\. Unlike our pipeline, it benefits from the stronger embedder \(especially in recall\) yet still trails our pipeline of Table[6](https://arxiv.org/html/2611.23609#A4.T6)by∼\\sim0\.100\.10joint F1\.Two conclusions follow these observations\. First, the larger embedder*raises*the supervised geometry ceiling, and the label\-free classifier still beats it by roughly0\.100\.10F1 and0\.220\.22recall\. Thus, we conclude that the recall gap reported in Section[4](https://arxiv.org/html/2611.23609#S4)is not an artefact of a weak embedder, but a property of voiceprint\-distance aggregation itself: a stronger model cannot close it\. Second, that the structural classifier is unmoved while clustering thresholds were left un\-recalibrated indicates its inputs are robust to the embedding change, consistent with the features being session\-relative rather than tied to any particular embedding geometry\. Both observations reinforce the central claim that the teacher is recoverable from discourse structure rather than acoustic detail\. In practice, they justify using the smaller ResNet34, which matches the larger model on the deployment path at a fraction of the cost\.

## Appendix ESensitivity to the clustering cut

The agglomerative distance threshold \(Section[3\.3](https://arxiv.org/html/2611.23609#S3.SS3)\) is the most important clustering parameter\. Setting it too low shatters each speaker into noise\-level micro\-clusters, and too high merges teacher and student into one cluster that no role rule can separate\. We sweep the cut over\{0\.4,0\.5,0\.6,0\.7,0\.75,0\.8,0\.85,0\.9\}\\\{0\.4,0\.5,0\.6,0\.7,0\.75,0\.8,0\.85,0\.9\\\}\. At each cut, we re\-cluster, regenerate the structural features and per\-cluster labels, and run the full leave\-one\-teacher\-out evaluation\. We score each language by the symmetric*macro role\-F1*\(the mean of teacher and student F1\) and take the best decision thresholdτ\\tauper cut\.

Figure 6:Clustering\-cut sensitivity\.Top:best macro role\-F1 \(mean of teacher and student F1, leave\-one\-teacher\-out\) per language as the cosine\-distance threshold varies\. Accuracy is flat to within∼\\sim0\.0060\.006across the entire0\.40\.4–0\.850\.85range for both languages\.Bottom:the total cluster count \(log scale\) falls from3\.5×1053\.5\{\\times\}10^\{5\}to5\.7×1035\.7\{\\times\}10^\{3\}over the same range\. At0\.90\.9the clusters collapse as teacher and student voices merge, and macro F1 drops sharply\. The deployed0\.50\.5\(dashed\) sits in the middle of the flat plateau, far from the cliff\.Figure[6](https://arxiv.org/html/2611.23609#A5.F6)shows the result\. The macro objective isflat to within∼\\sim0\.0060\.006across the whole0\.40\.4–0\.850\.85range\. English moves only between0\.6860\.686and0\.6920\.692, German between0\.780\.78and0\.810\.81despite the cluster count falling by more than two orders of magnitude \(from∼\\sim3\.5×1053\.5\{\\times\}10^\{5\}clusters at0\.40\.4to∼\\sim5\.7×1035\.7\{\\times\}10^\{3\}at0\.850\.85\)\. This is the same robustness shown by the embedder ablation from a different angle: the session\-relative structural features the classifier relies on survive substantial changes in clustering granularity\. Only at0\.90\.9, the recordings collapse to∼\\sim55clusters per recording, making the teacher and student no longer separable\. These observations confirm that the cut matters as a*safety floor*rather than an accuracy lever\. We therefore keep the calibrated0\.50\.5, which sits in the middle of the plateau and well clear of the collapse, and decline the≤0\.005\\leq 0\.005macro\-F1 that the coarsest safe cuts would buy at the cost of operating one step from breakdown\.

## Appendix FFeature set, selection, and per\-feature analysis

This appendix gives the full structural feature set that complementscentrality\(Section[3\.4](https://arxiv.org/html/2611.23609#S3.SS4)\), the selection procedure summarised in Section[3\.4](https://arxiv.org/html/2611.23609#S3.SS4), and the per\-feature results that establish centrality as the dominant, language\-transferable cue\.

### F\.1Complementary structural features

Beyondcentrality, each cluster is described by the following label\-free, session\-relative features\. We chose features that capture the teacher’s*behavioral*signature in a lesson \(speaking the most, throughout the session, from the acoustic center of the room\) rather than any voice\-identity or acoustic cue tied to a specific speaker\. Because every feature is session\-relative and label\-free, the same set transfers across rooms, speakers, and both languages, letting a single learned rule generalize without per\-recording tuning\.

- •talk\_time\_frac: share of total speech time held by the cluster\. Computed on VAD\-cleaned window durations \(not raw transcript intervals\) to avoid inflating teacher dominance\.
- •coverage: fraction of2020equal session\-time bins in which the cluster appears\. A teacher presents from bell to bell, scores 1; a student who speaks only in one group activity scores much lower\.
- •spread: temporal standard deviation of window midpoints, normalized by session length\. It complementscoveragesince high coverage with low spread indicates a bursty cluster whose bins happen to be spread, while high spread confirms sustained presence\.
- •turn\_count: number of contiguous speaking runs in the time\-ordered label sequence\. Teachers typically hold many short turns \(e\.g\., questions, transitions, corrections\); students hold fewer\.
- •size: raw window count\. It correlates with talk time, but is separately informative when window duration varies\.
- •consistency: mean cosine similarity of the cluster’s windows to their own centroidμc\\mu\_\{c\}\. A tight cluster \(one voice\) scores close to 1; a cluster polluted by mis\-assigned voices scores lower\.
- •energy\_z: each window’s log\-RMS energy standardized within the recording \(standard deviations above the session\-mean level\)\. It has a high score when teachers wear a microphone to record their lessons\.

#### Why pooling across rooms is legitimate\.

To choose features, we pool every cluster from every recording, rank single features, and run greedy forward selection under leave\-one\-teacher\-out cross\-validation so that no teacher leaks between train and test\. We optimize the*worse*of the two languages, ensuring that the chosen set works for both languages\. Pooling works even though the teacher is a different person each session because every feature is*session\-relative*\. In other words, a teacher cluster lands in roughly the same region of feature space in every recording, whoever the teacher is\. Pooling across hundreds of rooms, therefore, gives the classifier many labeled examples of that region, from which it learns a single decision boundary \(“this region corresponds to the teacher”\) that transfers frozen to a new room with no labels\.

### F\.2Per\-feature results

Figure 7:Single\-feature teacher F1\. Centrality dominates and transfers across languages\.To understand which acoustic and structural signals actually drive teacher identification, we evaluate each feature in isolation: we train the classifier on a single feature at a time, which is equivalent to reducing the teacher\-identification step to that one cue alone\. Figure[7](https://arxiv.org/html/2611.23609#A6.F7)shows the resulting teacher F1 per feature for both languages\. The key finding is that the same compact feature set generalizes across both languages despite differences in recording setup and classroom culture, which is a necessary condition for a single deployable system\.

centralitydominates across both languages \(F1=0\.85=0\.85\), confirming that the teacher’s structural position in the turn\-taking graph is the single most reliable cue: the teacher addresses every student, whereas students mostly address the teacher, making the teacher’s centrality score a clear outlier among the clusters\. Running greedy forward selection on top of this baseline adds almost no further gain, indicating that the remaining features are largely redundant once centrality is known\.

energy\_z, the z\-scored speaking energy relative to the session mean, is a useful second feature\. Its contribution for German is mildly surprising given the far\-field, reverberant recording condition; nonetheless, teachers in both corpora tend to speak with consistently higher energy than students, and that contrast survives acoustic degradation sufficiently to contribute marginal but consistent improvement\.

The dominance\-style features \(coverage,talk\_time\_frac, andturn\_count\) exhibit a characteristic high\-precision, low\-recall profile\. They reliably identify the teacher when that speaker produces one large, contiguous dominant segment, but fail when teacher speech is fragmented across many short turns or when a verbose student temporarily matches the teacher’s activity level\. Because these features assign a high score to at most one cluster, they crown the right cluster but miss the scattered remainder of teacher speech, suppressing recall\.

## Appendix GSession diagnostics: what predicts per\-recording teacher F1

Section[4\.6](https://arxiv.org/html/2611.23609#S4.SS6)argues that per\-recording accuracy is governed by teacher/student separability\. Here we substantiate that with a broader set of*label\-free*session descriptors\. These quantities are computed at inference from the embeddings, the clustering, and the classifier’s own outputs, using no ground\-truth roles\. For each of the291291recordings, we correlate every descriptor with the recording’s teacher F1 \(Spearmanρ\\rho\)\.

Table 8:Label\-free session diagnostics vs\. per\-recording teacher F1 \(Spearmanρ\\rho\)\. Sorted by\|ρ\|\|\\rho\|\. Descriptors above the rule track teacher F1; those below do not\.The strongest predictors, by a clear margin, are the classifier’s own confidence and the separability of the split it produces\. The mean*classifier margin*\(how far cluster scores sit from the decision boundary, 0\.5\) leads atρ=0\.82\\rho=0\.82, with the cosine*silhouette*of the predicted teacher/student partition close behind \(0\.800\.80\)\. Both are label\-free and both say the same thing: teacher F1 is high exactly when the two roles separate cleanly in the representation and the classifier is confident about which cluster is which, which is the mechanism Section[4\.6](https://arxiv.org/html/2611.23609#S4.SS6)identifies\. A second group of geometry measures follows: 1\-NN cluster\-label agreement \(0\.730\.73\), teacher\-cluster cohesion \(0\.700\.70\), the fragmentation of the teacher’s speech across clusters \(−0\.58\-0\.58\), and cluster\-belief entropy \(−0\.55\-0\.55\)\. Two dominance\-flavoured descriptors also predict teacher F1: the predicted teacher shareT^\\hat\{T\}\(0\.730\.73\) and the global\-mean→\\toteacher cosine \(0\.650\.65\)\. This is expected, since a teacher who holds more of the floor is more fully sampled and easier to recover\. These are useful, interpretable flags \(we useT^\\hat\{T\}for exactly this in Figure[8](https://arxiv.org/html/2611.23609#A7.F8)\), but they trail the separability and confidence measures rather than leading them\.

Several intuitive descriptors carry little signal\. Teacher/student centroid separation, despite being a separability measure, correlates only weakly \(ρ=0\.30\\rho=0\.30\), as the silhouette already captures the same geometry more faithfully\. Embedding*effective rank*\(a participation\-ratio measure of anisotropy\) was hypothesised to fall under reverberant channel collapse; it does not track teacher F1 \(ρ=0\.19\\rho=0\.19\)\. Centrality bimodality is essentially noise \(−0\.08\-0\.08, wrong sign\)\. Most notably, the*acoustic*signal\-to\-noise ratio proxy, the ratio of mean frame log\-energy in VAD\-speech vs\. VAD\-silence regions, is only weakly correlated with F1 \(ρ=0\.19\\rho=0\.19\), far below every geometry or confidence measure\. This bears directly on the cross\-language comparison in Section[4](https://arxiv.org/html/2611.23609#S4): the German corpus has the*lower*SNR proxy \(8\.98\.9vs\.12\.312\.3dB for English\) yet the*higher*F1, so the English↔\\leftrightarrowGerman difference is structural \(session composition and separability\), not a matter of recording quality\.

Figure 8:Mean per\-recording teacher F1 with recordings binned by the model’s predicted teacher shareT^\\hat\{T\}, which is a label\-free quantity available at inference\. Teacher F1 increases withT^\\hat\{T\}in both corpora, from about0\.800\.80in the lowest bin to0\.920\.92–0\.970\.97in the highest, so the low\-scoring recordings concentrate at lowT^\\hat\{T\}and can be flagged for human review without any labels\.
## Appendix HOnline cross\-lesson accumulation

Figure 9:Online cross\-lesson teacher enrollment\. Held\-out teacher F1 vs\. the number of prior lessonskkpooled into the teacher centroid\. The blue curve pools only the teacher’s prior lessons, selecting each lesson’s teacher speech without labels\. The green line is pseudo\-enrollment \(the label\-free floor\) and the orange line is the3030\-sec enrollment setting\. Accumulation climbs from the pseudo\-enrollment floor and saturates within two to three lessons, reaching the3030\-sec enrollment seed without ever using labels on the past lessons\.The enrollment strategies of the main text treat each recording in isolation\. But in the deployment we target, earlier recordings of the same teacher are available, which invites an*online*variant: accumulate a teacher embedding from past lessons and use it to diarise the next one\. We test this by grouping recordings by teacher and building the teacher centroid from the teacher’s*previous*lessons \(we sweep the number of prior lessonskkand average over3030random lesson orderings per teacher\)\. Figure[9](https://arxiv.org/html/2611.23609#A8.F9)plots held\-out teacher F1 againstkk\. It depicts two references: pseudo\-enrollment \(the fully label\-free floor of Table[1](https://arxiv.org/html/2611.23609#S3.T1)\) and the3030\-sec enrollment seed \(which needs a hand\-marked teacher clip\)\. The accumulated centroid itself uses*no labels*on any lesson, so the online curve is deployable end\-to\-end\. Error bars are±1\\pm 1SE across teachers\.

A single prior lesson already clears the label\-free floor: atk=1k=1both languages sit above pseudo\-enrollment, and accuracy then climbs monotonically as more history is pooled\. German saturates within two to three lessons, reaching0\.810\.81atk=3k=3and meeting the3030\-sec enrollment seed; English rises more gradually but overtakes the enrollment approach\. The practical implication is the following: given a handful of a teacher’s earlier recordings, label\-free cross\-lesson accumulation matches or exceeds the hand\-seeded within\-session enrollment\. Hence, the manual enrollment clip and the labels behind it can be dropped entirely in the online deployment\.

We now ask whether our approach benefits from incorporating a teacher’s previous lessons\. For this, we pool each teacher’s predicted\-teacher clusters across their earlier lessons into a single voiceprint and score every cluster by its cosine to it\. This lifts German teacher F1 only from0\.870\.87to0\.880\.88and student F1 from0\.610\.61to0\.630\.63\(no change in English\)\. Interestingly, a*single*prior lesson already yields this gain, with more history adding nothing\. We attribute this to redundancy: across all clusters, a cluster’s cosine to the pooledcross\-lesson teacher voiceprintand to thesession\-global mean embeddingtrack each other atρ=0\.97\\rho=0\.97\(still0\.890\.89in the rooms with students speaking longer than teachers\)\. In other words, a more accurate teacher voiceprint buys little: the teacher is recovered from discourse structure, not from the precision of a voiceprint\.

## Appendix IMasking mixed frames does not help

A natural suspect for the low\-separability of some of the recordings is the*mixed frames*: with1\.251\.25s windows at0\.50\.5s hop, any window may include a speaker change or genuine cross\-talk\. This would then produce a blended embedding that could drag cluster centroids off\-target and corrupt thecentralityfeature\. We tested this directly with a trained overlap detector \(pyannote segmentation\-3\.0\), which marks8\.7%8\.7\\%\(English\) /3\.9%3\.9\\%\(German\) of speech as overlapped, with the flagged windows over\-representing students by∼\\sim1\.7×1\.7\\times, consistent with cross\-talk over the teacher\. Excluding every flagged window from the two embedding averages that feedcentralityandconsistency\(leaving cluster assignment and all timing features untouched\) moves teacher F1 by at most0\.0020\.002in either direction, across masking thresholds0\.10\.1–0\.50\.5and across every low/mid/high separability stratum in both languages\. This means a clean negative result, with no benefit even on the mixed\-frame\-heavy low\-separability sessions that the intervention was meant to help\. We believe that a duration\-weighted mean over hundreds of clusters is barely perturbed by dropping44–9%9\\%of its members, so mixed\-frame contamination never propagates through the centroid pathway far enough to matter\. This reinforces Section[4\.6](https://arxiv.org/html/2611.23609#S4.SS6): the tail is bounded by the*separability*of the representation, not by frame\-level label noise, so cleaning that noise upstream buys nothing and we keep the simpler pipeline without an overlap\-detection stage\.

## Appendix JDeployability Test Implementation Details

Table 9:Thread scaling of the proposed pipeline \(recording B, CPU execution provider\): RTF \(median±\\pmstandard deviation overN=3N\{=\}3passes, lower is faster\) as the CPU thread budget varies\.#### Implementations\.

We benchmark the pipeline described in Sec\.[3](https://arxiv.org/html/2611.23609#S3)\. The PyTorch implementation used for the accuracy experiments Silero VAD\([Silero Team 2024](https://arxiv.org/html/2611.23609#bib.bib25)\), the WeSpeakerenglish\_resnet34256\-d ResNet34\-LM embedder\([Wang et al\. 2023](https://arxiv.org/html/2611.23609#bib.bib23)\), agglomerative clustering at cosine cut0\.50\.5, and the role classifier which uses eight\-feature session\-relative\. The Android implementation runs VAD and the exported embedder through ONNX Runtime, with clustering and role classification implemented natively\. Both process one1\.251\.25s speech window per embedding call, with a0\.50\.5s hop and batch size one\. The ONNX embedder returns an embedding of shape\[1,256\]\[1,256\]\. Its outputs have mean cosine similarity of0\.99930\.9993with the PyTorch outputs on held\-out speech\. We test the CPU, XNNPACK, and NNAPI execution providers on Android\.

#### Baselines\.

The laptop baseline runs pyannote throughpyannote\.audio3\.1 in PyTorch, with evaluation mode, inference mode, and batch size one\. The Android baseline usessherpa\-onnx\([Next\-gen Kaldi Team 2026](https://arxiv.org/html/2611.23609#bib.bib37)\)to run segmentation\-3\.0 and WeSpeaker through the same ONNX Runtime environment as our application\. We report speedups separately for the laptop and phone because their baseline implementations differ\.

#### Protocol\.

We measure real\-time factor \(RTF\), peak resident set size \(RSS\), and processing time by pipeline stage\. RTF is processing time divided by audio duration, so values below one indicate processing faster than real time\. Model loading is timed separately and excluded from RTF\. Each configuration receives one warm\-up pass, followed by five timed passes for recording A or three for recording B\. We report median RTF, with standard deviations in Table[9](https://arxiv.org/html/2611.23609#A10.T9)and minimum\-to\-maximum ranges in Fig\.[3](https://arxiv.org/html/2611.23609#S4.F3)

#### Hardware and data\.

We run the benchmarks on a 12\-core Apple M4\-Pro Macbook Pro laptop and a Samsung Galaxy S24\+\(SM\-S926B, 10\-cores5e9945SoC, Android 16\)\. Recordings A and B are two held\-out1010\-minute clips from the German GTI corpus, recorded with room microphones and processed as1616kHz mono audio\. The primary runtime comparison uses one inference thread\. We separately test thread counts of\{1,2,4,8\}\\\{1,2,4,8\\\}on both devices\. On the phone, we also compare execution providers at one thread, obtaining RTFs of0\.2280\.228for CPU,0\.2210\.221for XNNPACK, and0\.2630\.263for NNAPI\.

#### Per\-stage cost\.

Speaker embeding accounts for approximately96%96\\%of processing time on the phone and91%91\\%on the laptop\. VAD accounts for33\-9%9\\%, while clustering and role assignment together account for less than2%2\\%\. The pyannote baseline takes approximately507507s per pass on the phone\. We report its total runtime without a per\-stage breakdown\.

## Appendix KDownstream transcription analysis

#### Protocol\.

All292292recordings are transcribed with Whisper\-large\-v2 in fixed3030s chunks whose returned text is concatenated, under two conditions:*no filtering*\(the whole recording\) and*pipeline*\(only the spans the classifier labels teacher, at the deployed operating pointτ=0\.45\\tau\{=\}0\.45\)\. Both are scored against the teacher’s turns in the human transcript, and*only teacher words are counted*: student speech is neither credited when recognised nor charged when transcribed from unfiltered audio, so the comparison isolates recognition of the teacher’s own words rather than rewarding the removal of student speech\.

Table 10:Teacher word error rate with and without role filtering, pooled over reference words within each corpus\. Substitutions and insertions fall under filtering in both corpora; deletions rise and dominate, because an imperfect filter mutes some genuine teacher speech that the recogniser then never sees\. On German the two conditions are statistically indistinguishable \(macro\-averaged over recordings:0\.36350\.3635without filtering versus0\.36310\.3631with;6969recordings improve and7171worsen\)\.Figure 10:The teacher words most often lost under role filtering \(top2020per corpus, counts normalised to one4545\-minute lesson\)\. Each bar is split into two: the lower segment is celetions already present*without*filtering, i\.e\. imperfect transcription because of the recorded audio, and the upper segment the additional loss filtering causes\. The distinction matters, because Whisper already omits many German fillers from untouched audio \(*eh*is77%77\\%deleted with no filtering at all\)\. The lost vocabulary is dominated by fillers, backchannels and discourse markers in both languages\.

Similar Articles

Multimodal Speaker Identification in Classroom Environments

arXiv cs.CL

This paper evaluates a multimodal framework for speaker identification in K-12 classrooms by combining acoustic embeddings (ECAPA-TDNN) with LLM-derived semantic context from transcripts, improving accuracy from 39% to 50.3% overall and from 64.9% to 76.9% for longer utterances.

Redact or Keep? A Fully Local AI Cascade for Educational Dialogue De-Identification

arXiv cs.CL

This paper proposes a fully local AI cascade for de-identifying educational dialogue, combining a recall-first candidate proposer with a contextual Redact/Keep reviewer. The approach achieves high accuracy without sending data to external APIs, outperforming both smaller local models and commercial APIs on math tutoring transcripts.

Speaker-Disentangled Chunk-Wise Regression for Syllabic Tokenization

Hugging Face Daily Papers

This paper proposes a speaker-disentangled syllabic tokenizer that uses regression of perturbed student representations toward clean teacher targets, achieving state-of-the-art syllable boundary detection and a 7% relative improvement in speech language model understanding over SpiRit-LM.