Edge Phoneme Recognition for Children's Speech through Age-Aware Training

# Edge Phoneme Recognition for Children’s Speech through Age-Aware Training
Source: [https://arxiv.org/html/2608.10206](https://arxiv.org/html/2608.10206)
,Ryan ArboledaOccidental CollegeLos AngelesCAUSA[rarboleda@oxy\.edu](mailto:[email protected]),Sophie HaakOccidental CollegeLos AngelesCAUSA[haak@oxy\.edu](mailto:[email protected]),Sam HjelmesetOccidental CollegeLos AngelesCAUSA[hjelmeset@oxy\.edu](mailto:[email protected]),Andrew FranckOccidental CollegeLos AngelesCAUSA[franck@oxy\.edu](mailto:[email protected]),Bingrui YangOccidental CollegeLos AngelesCAUSA[byang@oxy\.edu](mailto:[email protected]),Jose Bustamante OrtizOccidental CollegeLos AngelesCAUSA[bustamanteor@oxy\.edu](mailto:[email protected]),Yuanrong ShenOccidental CollegeLos AngelesCAUSA[sheny@oxy\.edu](mailto:[email protected]%20)andJoel WalshOccidental CollegeLos AngelesCAUSA[jwalsh2@oxy\.edu](mailto:[email protected])

\(2026\)

###### Abstract\.

Detecting phonemes from children’s speech has historically been difficult due to the scarcity of training data, and unique characteristics of children’s speech\. During a phoneme detection competition, we found that training a lightweight model to predict the age of the learner, as well as the phoneme sequence, enabled a 94M\-parameter model to outperform WavLM Large models \(317M\) on the target DrivenData distribution, and fall within approximately 0\.04 CER of competition ensembles with 90 times the parameters\. This has enabled the creation of PhonemeTrainer, an application that can run on most modern cellular phones\. This will ultimately enable better Automated Speech Recognition \(ASR\) and pronunciation helper apps for children’s speech, with the privacy and compliance benefits that come with edge processing\.

children’s speech recognition, phoneme recognition, multi\-task learning, edge deployment, mobile speech processing, pronunciation assessment

††copyright:cc††journalyear:2026††conference:The 13th ACM Conference on Learning @ Scale; June 29–July 3, 2026; Seoul, South Korea††ccs:Computing methodologies Speech recognition††ccs:Computing methodologies Multi\-task learning††ccs:Applied computing E\-learning††ccs:Applied computing Interactive learning environments![Refer to caption](https://arxiv.org/html/2608.10206v1/x1.png)Figure 1\.Multitask CTC with auxiliary age classification\.Training diagram showing WavLM encoder branching to CTC and age heads with combined loss, and simplified inference diagram showing trained model with age output discarded\.## 1\.Introduction

Modern Automated Speech Recognition \(ASR\) and Speech to Phoneme pipelines are often built on Self\-Supervised Learning representations trained on large amounts of adult speech\(Li et al\.,[2024](https://arxiv.org/html/2608.10206#bib.bib10); Sinha et al\.,[2025](https://arxiv.org/html/2608.10206#bib.bib16)\)\. This results in representations that often fail at speech tasks that involve children’s speech\(Potamianos et al\.,[1997](https://arxiv.org/html/2608.10206#bib.bib13); Dubagunta et al\.,[2019](https://arxiv.org/html/2608.10206#bib.bib8); Wang et al\.,[2026](https://arxiv.org/html/2608.10206#bib.bib17); Block Medin et al\.,[2024](https://arxiv.org/html/2608.10206#bib.bib2)\)\.

To this end, the Gates Foundation launched the “On Top of Pasketti: Children’s Speech Recognition Challenge”, a large\-scale data science competition hosted on the DrivenData platform \(concluded April 2026\)\.\(DrivenData,[2026](https://arxiv.org/html/2608.10206#bib.bib7); Bull et al\.,[2016](https://arxiv.org/html/2608.10206#bib.bib3)\)\. The challenge utilized a curated corpus of child speech derived from the Arizona Child Acoustic Database\(Bunton and Story,[2016](https://arxiv.org/html/2608.10206#bib.bib4)\), TalkBank\(Rose and MacWhinney,[2014](https://arxiv.org/html/2608.10206#bib.bib14); MacWhinney,[2019](https://arxiv.org/html/2608.10206#bib.bib11)\), Jibo Kids\(Shankar et al\.,[2024](https://arxiv.org/html/2608.10206#bib.bib15)\), and the ReadNet project\(MIT Integrated Learning Initiative,[2024](https://arxiv.org/html/2608.10206#bib.bib12)\)\. The phonetic track of this challenge required participants to build models that predict International Phonetic Alphabet phoneme sequences from children’s speech utterances\. The data included metadata about the speaker’s age group buckets; ages 3–4, 5–7, 8–11, and 12\+\.

As with many data competitions like Kaggle, the top contest winners in “On Top of Pasketti” used ensemble models in order to achieve the lowest Character Error Rate \(CER\)\. The second place winner \(0\.2607 CER on the blind competition set\) reported using an ensemble of 13 models, with each model utilizing encoders in the range of 317 million \(WavLM Large\) to 1\.5 billion parameters \(Whisper Large\)\(Dieleman,[2026](https://arxiv.org/html/2608.10206#bib.bib6)\)\. The solution in this case provided negligible improvement in CER, with exponentially higher compute requirements \(NVIDIA A100 GPU with∼\\sim80 GB RAM\)\. In order to create solutions that scale in primary/secondary school settings where cost is constrained, privacy is mandatory, and internet access is spotty, developers must have access to near state\-of\-the\-art models that run on the edge\.

## 2\.Training and Results

Table 1\.Validation CER by Model Configuration\. All runs usedN=117,500N=117\{,\}500\. Per\-age CERs are on the DD validation subset\. \(Age 3–4: 37,377 utterances, 5–7: 24,285 utterances, 8–11: 58,557 utterances\)\.The Age\-aware training utilized a WavLM Base\+ \(94M parameters\)\(Chen et al\.,[2022](https://arxiv.org/html/2608.10206#bib.bib5)\)pretrained encoder and two decoder heads, one that predicts the age bucket, and one that uses Connectionist Temporal Classification\(Graves et al\.,[2013](https://arxiv.org/html/2608.10206#bib.bib9)\)to predict International Phonetic Alphabet symbols \(see Figure[1](https://arxiv.org/html/2608.10206#S0.F1)\)\. Predicting age was not part of the competition criteria, but there was some hope that adding this head would result in the model learning a regularization signal\. The age prediction was discarded for the purposes of the competition\.

While training, this model converged three times faster than any other model we trained\. All runs shared the same 117,500 utterance training set \(10,696 DrivenData \+ 106,804 TalkBank\), split by child ID to prevent data leakage\. As Table[1](https://arxiv.org/html/2608.10206#S2.T1)shows, the age\-aware model was not merely competitive with WavLM Large \(317M parameters\) configurations—it matched the Large RNN\-T on mixed CER \(both 0\.306\) and outperformed it on the target DrivenData distribution \(0\.306 vs\. 0\.343\), despite being 3\.4×\\timessmaller\. The WavLM Large \+ RNN\-T model did perform better on TalkBank \(TB\) data, no doubt due to the training data imbalance\. On the actual competition DrivenData \(DD\), the age\-aware model performed significantly better, neither under\- nor overfitting on either subset of the data\. This suggests that the age\-auxiliary objective encourages the encoder to learn age\-invariant phoneme representations rather than overfit on the more numerous \(10:1 ratio\) TalkBank data\.

## 3\.Discussion

Post\-training tests of the Age\-aware head showed that it was accurately predicting the age bucket of the speech samples with 72% accuracy, which essentially meant predicting the dominant class every time\. So while the head clearly sent a useful signal to the loss function, it was a subpar predictor\. We conjecture that the age gradient may encourage age\-invariant phoneme representations, preventing the encoder from overfitting to the acoustic characteristics of the majority age group\. Future research will apply mechanistic interpretability techniques to explore the mechanisms by which this age\-aware objective helps training and inference\.

The most important implication of this unique combination of size, accuracy, and quick convergence means that the model takes up a relatively small amount of device memory and can easily be trained and retrained as necessary to mitigate distribution drift\. This model can then serve as a stable backbone for a number of speech recognition and pronunciation feedback applications, without sending children’s speech data over the internet\. This has significant privacy implications in countries with strict regulations \(laws like COPPA, FERPA, GDPR\) governing the transmission of children’s speech and academic records\. A key technical limitation arises when trying this with larger, more accurate models such as the finetuned WavLM\-large \(317M parameters\) backbones\. Those models often exceed 1 GB, creating challenges for mobile deployment including longer initial download times, RAM requirements that exceed Android’s per\-app memory limits, and inference delays that make real\-time feedback impractical on mid\-range devices\.

## 4\.Demonstration

![Refer to caption](https://arxiv.org/html/2608.10206v1/figures/AndroidELPACUI.png)Figure 2\.A compact view of the Android ELPAC UI\.Screenshot of the PhonemeTrainer Android app in dark mode, scrolled to show a completed pronunciation report\. A “Pronunciation Match” card reports 90%, 9 of 10 phonemes matched, with the message “Excellent\! Your pronunciation is very accurate\.” Below it, an ELPAC level badge reads “Level 4 – Minimal errors”\. A “What you said” card shows the transcribed sentence “this is a test” with each word colour\-coded against a legend of green for good, amber for needs work, and red for incorrect; the word “a” is amber and the rest are green\. Per\-word score chips follow: this 88, is 92, a 93, test 87\. An “Areas to improve” note explains that “a” is mostly correct and suggests practising for clarity\. A “Pronunciation Score” card shows four ring gauges: 93 overall, 90 accuracy, 100 fluency, and 90 completeness\. A “Phoneme Timeline” section begins at the bottom edge\.This demonstration will show the phoneme transcription capabilities in real\-time on an Android phone, as part of an application that gives users feedback on their pronunciation of short English, Spanish, or Korean sentences\. During this demonstration participants will be asked to speak phrases into an Android device\. These phrases will then be analyzed via an application called PhonemeTrainer\. They will then be able to analyze the phonetic output of their speech, and analyze the accuracy to the reference phonetic sequence for that particular phrase \(see Figure[2](https://arxiv.org/html/2608.10206#S4.F2)\)\. The application performs this analysis of participant speech by routing the \.wav file to the finetuned WavLM Base\+ acoustic model, which outputs a symbolic phoneme sequence\. This spoken sequence is then mapped to ARPAbet phonemes and compared to the ground truth from the CMU Pronouncing Dictionary\. Users are then shown a visual representation of their spoken sequence, and where, if at all, their phonetic output diverges from the ground truth sequence\.

The demonstration video shows two audio samples, a three year old child and an adult each saying, “this is a test”, to illustrate how the tool responds to different speaker profiles and voice characteristics\. There will likely be no children in attendance\. To show how the application performs on children’s speech, we will pre\-load several examples of children speaking the phrases for demonstration\.

The participants will speak target sentences drawn from a language proficiency assessment used in many U\.S\. schools, selected to cover a representative range of phonemes\. To manage the model’s 360 MB size, the model weights are downloaded once from a remote host on first launch and saved to the device’s internal storage, so that all subsequent sessions run fully offline without re\-downloading\.

Future work will focus on using this speech\-to\-phoneme model as a complement to a language model that can take multilingual phrases and generate plausible phoneme sequences\. This will allow PhonemeTrainer to work on any sentences, not just those that have been pre\-loaded\.

## References

- \(1\)
- Block Medin et al\.\(2024\)Lucas Block Medin, Thomas Pellegrini, and Lucile Gelin\. 2024\.Self\-Supervised Models for Phoneme Recognition: Applications in Children’s Speech for Reading Learning\. In*Proceedings of Interspeech 2024*\. ISCA, Kos Island, Greece, 5168–5172\.[doi:10\.21437/Interspeech\.2024\-1095](https://doi.org/10.21437/Interspeech.2024-1095)
- Bull et al\.\(2016\)Peter Bull, Isaac Slavitt, and Greg Lipstein\. 2016\.Harnessing the power of the crowd to increase capacity for data science in the social sector\.*arXiv preprint arXiv:1606\.07781*\(2016\)\.[https://arxiv\.org/abs/1606\.07781](https://arxiv.org/abs/1606.07781)
- Bunton and Story \(2016\)Kate Bunton and Brad H\. Story\. 2016\.Arizona Child Acoustic Database Repository\.*Folia Phoniatrica et Logopaedica*68, 3 \(2016\), 107–111\.[doi:10\.1159/000452128](https://doi.org/10.1159/000452128)
- Chen et al\.\(2022\)Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, Jian Wu, Long Zhou, Shuo Ren, Yanmin Qian, Yao Qian, Jian Wu, Michael Zeng, Xiangzhan Yu, and Furu Wei\. 2022\.WavLM: Large\-Scale Self\-Supervised Pre\-Training for Full Stack Speech Processing\.*IEEE Journal of Selected Topics in Signal Processing*16, 6 \(2022\), 1505–1518\.[doi:10\.1109/JSTSP\.2022\.3188113](https://doi.org/10.1109/JSTSP.2022.3188113)
- Dieleman \(2026\)Willem Dieleman\. 2026\.Now that we are done, who wants to talk about what worked?DrivenData Community Forum\.[https://community\.drivendata\.org/t/now\-that\-we\-are\-done\-who\-wants\-to\-talk\-about\-what\-worked/11436/9](https://community.drivendata.org/t/now-that-we-are-done-who-wants-to-talk-about-what-worked/11436/9)Post \#9\.
- DrivenData \(2026\)DrivenData\. 2026\.On Top of Pasketti: Children’s Speech Recognition Challenge — Phonetic Track\.[https://www\.drivendata\.org/competitions/309/childrens\-phonetic\-asr/](https://www.drivendata.org/competitions/309/childrens-phonetic-asr/)\.Accessed: 2026\-04\-08\.
- Dubagunta et al\.\(2019\)S\. Pavankumar Dubagunta, Selen Hande Kabil, and Mathew Magimai\-Doss\. 2019\.Improving Children Speech Recognition Through Feature Learning from Raw Speech Signal\. In*Proceedings of the 2019 IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\)*\. IEEE, Brighton, UK, 5736–5740\.[doi:10\.1109/ICASSP\.2019\.8682826](https://doi.org/10.1109/ICASSP.2019.8682826)
- Graves et al\.\(2013\)Alex Graves, Abdel\-rahman Mohamed, and Geoffrey Hinton\. 2013\.Speech recognition with deep recurrent neural networks\. In*Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\)*\. IEEE, Vancouver, BC, Canada, 6645–6649\.[doi:10\.1109/ICASSP\.2013\.6638947](https://doi.org/10.1109/ICASSP.2013.6638947)
- Li et al\.\(2024\)Jialu Li, Mark Hasegawa\-Johnson, and Nancy L\. McElwain\. 2024\.Analysis of Self\-Supervised Speech Models on Children’s Speech and Infant Vocalizations\. In*Proceedings of the 2024 IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops \(ICASSPW\)*\. IEEE, Seoul, Republic of Korea, 550–554\.[doi:10\.1109/ICASSPW62465\.2024\.10626416](https://doi.org/10.1109/ICASSPW62465.2024.10626416)
- MacWhinney \(2019\)Brian MacWhinney\. 2019\.Understanding spoken language through TalkBank\.*Behavior Research Methods*51, 4 \(2019\), 1919–1927\.[doi:10\.3758/s13428\-018\-1174\-9](https://doi.org/10.3758/s13428-018-1174-9)
- MIT Integrated Learning Initiative \(2024\)MIT Integrated Learning Initiative\. 2024\.ReadNet: Preventing Reading Failure with Speech Recognition\-Powered Assessment\.[https://mitili\.mit\.edu/research/readnet\-preventing\-reading\-failure\-speech\-recognition\-powered\-assessment](https://mitili.mit.edu/research/readnet-preventing-reading-failure-speech-recognition-powered-assessment)\.Accessed: 2026\-04\-08\.
- Potamianos et al\.\(1997\)Alexandros Potamianos, Shrikanth Narayanan, and Sungbok Lee\. 1997\.Automatic speech recognition for children\. In*Proceedings of the 5th European Conference on Speech Communication and Technology \(Eurospeech 1997\)*\. ISCA, Rhodes, Greece, 2371–2374\.[doi:10\.21437/Eurospeech\.1997\-623](https://doi.org/10.21437/Eurospeech.1997-623)
- Rose and MacWhinney \(2014\)Yvan Rose and Brian MacWhinney\. 2014\.The PhonBank Project: Data and software\-assisted methods for the study of phonology and phonological development\.In*The Oxford Handbook of Corpus Phonology*, Jacques Durand, Ulrike Gut, and Gjert Kristoffersen \(Eds\.\)\. Oxford University Press, Oxford, UK, 380–401\.
- Shankar et al\.\(2024\)Natarajan Balaji Shankar, Amber Afshan, Alexander Johnson, Aurosweta Mahapatra, Alejandra Martin, Haolun Ni, Hae Won Park, Marlen Quintero Perez, Gary Yeung, Alison Bailey, Cynthia Breazeal, and Abeer Alwan\. 2024\.The JIBO Kids Corpus: A speech dataset of child\-robot interactions in a classroom environment\.*JASA Express Letters*4, 11 \(2024\), 115201\.[doi:10\.1121/10\.0034195](https://doi.org/10.1121/10.0034195)
- Sinha et al\.\(2025\)Abhijit Sinha, Hemant Kumar Kathania, and Mikko Kurimo\. 2025\.Beyond Traditional Speech Modifications: Utilizing Self Supervised Features for Enhanced Zero\-Shot Children ASR\. In*Proceedings of Interspeech 2025*\. ISCA, Rotterdam, The Netherlands, 1963–1967\.[doi:10\.21437/Interspeech\.2025\-1874](https://doi.org/10.21437/Interspeech.2025-1874)
- Wang et al\.\(2026\)Zilai Wang, Natarajan Balaji Shankar, Kaiyuan Zhang, Zihan Wang, and Abeer Alwan\. 2026\.Mind the Shift: Using Delta SSL Embeddings to Enhance Child ASR\.*arXiv preprint arXiv:2601\.20142*\(2026\)\.[https://arxiv\.org/abs/2601\.20142](https://arxiv.org/abs/2601.20142)Accepted to ICASSP 2026\.