JSPG: Dynamic Dictionary Filtering via Joint Semantic-Pinyin-Glyph Retrieval for Chinese Contextual ASR

# JSPG: Dynamic Dictionary Filtering via Joint Semantic-Pinyin-Glyph Retrieval for Chinese Contextual ASR
Source: [https://arxiv.org/html/2605.16896](https://arxiv.org/html/2605.16896)
Shilin Zhou, Zhenghua Li††thanks:Corresponding author School of Computer Science and Technology, Soochow University, Suzhou, China slzhou\.cs@outlook\.com; zhli13@suda\.edu\.cn

###### Abstract

Contextual Automatic Speech Recognition \(ASR\) faces challenges with large\-scale keyword dictionaries, as excessive irrelevant candidates introduce noise that degrades accuracy\. To address this, dynamic filtering typically uses a base ASR model to generate preliminary hypotheses, followed by semantic text retrievers to fetch a concise subset of relevant keywords\. However, this approach frequently fails in Chinese ASR\. Base models often produce homophonic or near\-homophonic errors that preserve the phonetic cues of the target keywords but severely distort their semantic meaning, rendering standard semantic retrievers ineffective\. To resolve this, we propose a filtering framework that jointly integrates Semantic, Pinyin, and Glyph features \(JSPG\)\. Pinyin effectively retrieves targets based on phonetic similarity, while glyph provides complementary structural cues to filter out numerous irrelevant homophones inherent in Chinese\. To bridge the gap between character\-level pinyin/glyph metrics and sequence\-level filtering, we introduce an extended Smith\-Waterman algorithm that computes similarity scores between the N\-best hypothesis sequences and keywords\. Experiments on the Aishell\-1 and RWCS\-NER datasets demonstrate that JSPG significantly outperforms single\-feature baselines\. Furthermore, downstream contextual ASR models guided by JSPG achieve substantial improvements in keyword recognition accuracy\.

JSPG: Dynamic Dictionary Filtering via Joint Semantic\-Pinyin\-Glyph Retrieval for Chinese Contextual ASR

Shilin Zhou, Zhenghua Li††thanks:Corresponding authorSchool of Computer Science and Technology,Soochow University, Suzhou, Chinaslzhou\.cs@outlook\.com; zhli13@suda\.edu\.cn

## 1Introduction

Automatic Speech Recognition \(ASR\) systems have achieved remarkable success in general transcription tasksRadford et al\. \([2022](https://arxiv.org/html/2605.16896#bib.bib18)\); Bai et al\. \([2024](https://arxiv.org/html/2605.16896#bib.bib2)\); Omnilingual et al\. \([2025](https://arxiv.org/html/2605.16896#bib.bib13)\)\. However, accurately recognizing contextual keywords, such as technical terminologies and rare entities, remains a challenge\. The keywords appear infrequently in general training corpora, which makes it difficult for models to transcribe them effectivelyPeters et al\. \([2018](https://arxiv.org/html/2605.16896#bib.bib15)\); Sudo et al\. \([2024b](https://arxiv.org/html/2605.16896#bib.bib22)\)\.

To address this, researchers have proposed the contextual ASR taskPundak et al\. \([2018](https://arxiv.org/html/2605.16896#bib.bib16)\); Alon et al\. \([2019](https://arxiv.org/html/2605.16896#bib.bib1)\); Zhou et al\. \([2024b](https://arxiv.org/html/2605.16896#bib.bib28)\), which utilizes a predefined dictionary of potential keywords to guide the recognition process\. Current approaches have achieved promising results when the keyword dictionary is small\. However, experiments show that as the dictionary size increases, the presence of irrelevant keywords leads to a significant decline in transcription accuracyZhou et al\. \([2024b](https://arxiv.org/html/2605.16896#bib.bib28)\); Sudo et al\. \([2024a](https://arxiv.org/html/2605.16896#bib.bib21)\); Zhou and Li \([2025](https://arxiv.org/html/2605.16896#bib.bib26)\)\.

Hypothesis:“买入弃权”\(BuyAbstention\)Base ASR OutputSemantically Confusing“买入期权”\(BuyOptions\)Input SpeechKeyword Dictionary\{…放弃, 期权, …\}SemanticRetrieverJSPGRetriever放弃 \(Give Up\)\[Semantic Match\]Wrong Target期权 \(Options\)\[Phonetic Match\]Correct Target

Figure 1:Illustration of the retrieval process\.Upper Path:The semantic retriever is misled by the ASR error “弃权 \(Abstention\)” and retrieves “放弃 \(Give Up\)”\.Lower Path:Our JSPG retriever utilizes the joint semantic\-pinyin\-glyph features to correctly retrieve the target “期权 \(Options\)”\.To alleviate the negative impact of large dictionaries, recent work has explored dynamically filtering the dictionary into a high\-relevance subset of keywords for each speech inputMathur et al\. \([2024](https://arxiv.org/html/2605.16896#bib.bib12)\); Li et al\. \([2024a](https://arxiv.org/html/2605.16896#bib.bib10)\); Chen et al\. \([2025](https://arxiv.org/html/2605.16896#bib.bib5)\); Lei et al\. \([2025](https://arxiv.org/html/2605.16896#bib.bib9)\)\. In a typical cascaded pipeline, a base ASR model first transcribes the speech input into preliminary N\-best hypotheses\. Then these methods usually either select the top\-ranked hypothesis as the query or concatenate all hypotheses into a single query sequence\. Subsequently, the query and the keywords in the dictionary are fed into a pre\-trained text embedding model, which converts them into embedding representations\. Finally, the system calculates the cosine similarity scores between them to select the top\-scoring keywords as the filtered subsetXiao et al\. \([2025](https://arxiv.org/html/2605.16896#bib.bib24)\); Dimitrios et al\. \([2025](https://arxiv.org/html/2605.16896#bib.bib7)\)\. Since text embeddings are designed to encode the contextual meaning of the input, we refer to these methods as semantic retrievers\.

However, relying solely on semantic features is insufficient for precise keyword filtering in Chinese contextual ASR\. Since ASR is acoustic\-based and Chinese has abundant homophones, ASR errors typically sound similar to the target keyword but carry entirely different meanings\. Semantic retrievers fetch keywords that are semantically related to the erroneous transcription but completely irrelevant to the actual target\. For instance, as illustrated in Figure[1](https://arxiv.org/html/2605.16896#S1.F1), the target “期权” \(Options\) is misrecognized as “弃权” \(Abstention\) due to homophonic confusion\. Consequently, the retriever incorrectly fetches the irrelevant keyword “放弃” \(Give Up\) based on its high semantic similarity with “弃权”\.

Since these errors preserve phonetic similarity to the target, we can leverage this cue for retrieval\. In Chinese, pinyin directly encodes pronunciation, making it a natural metric for phonetic matching\. However, many different Chinese characters share the exact same pronunciation\. Therefore, relying solely on pinyin introduces substantial noise, as it retrieves many irrelevant characters alongside the correct keyword\.

Fortunately, the glyph features of Chinese characters offer a complementary solution to this phonetic noise\. Since most characters are phono\-semantic compounds, ASR phonetic errors frequently share structural components with the target keywordsTan et al\. \([2005](https://arxiv.org/html/2605.16896#bib.bib23)\)\. Crucially, while many different characters share the exact same pronunciation, every individual character has a unique glyph structure\. This one\-to\-one property creates sharp discriminative signals, allowing the system to easily distinguish the target from unrelated homophones retrieved by pinyin\.

In this work, we propose JSPG, a Joint Semantic\-Pinyin\-Glyph retrieval approach for Chinese contextual ASR\. JSPG integrates two types of features: a globalsemanticscore derived from pre\-trained text embeddings to capture the broad contextMathur et al\. \([2024](https://arxiv.org/html/2605.16896#bib.bib12)\); Lei et al\. \([2025](https://arxiv.org/html/2605.16896#bib.bib9)\), and a unifiedphonetic\-glyphscore that fuses character\-level pinyin and glyph similaritiesQiao et al\. \([2025](https://arxiv.org/html/2605.16896#bib.bib17)\)to ensure precise local matching\. Since pinyin and glyph metrics inherently operate at the character level, we introduce an extended Smith\-Waterman algorithmSmith et al\. \([1981](https://arxiv.org/html/2605.16896#bib.bib19)\)that extends them to measure the sequence\-level similarity between a short keyword and a longer, noisy ASR hypothesis sequence\.

We evaluate JSPG on the standard Aishell\-1Bu et al\. \([2017](https://arxiv.org/html/2605.16896#bib.bib3)\); Chen et al\. \([2022](https://arxiv.org/html/2605.16896#bib.bib4)\)and the complex, real\-world RWCS\-NER datasetsZhou et al\. \([2024a](https://arxiv.org/html/2605.16896#bib.bib27)\)\. Experimental results demonstrate that JSPG significantly outperforms baselines relying solely on individual semantic, pinyin, or glyph features\. The ablation studies further reveal that each component contributes uniquely to the system, with their integration yielding the most robust retrieval performance\. Furthermore, when contextual ASR models are guided by the keywords filtered by JSPG, they achieve substantial improvements in keyword recognition accuracy\.

## 2Related Work

Standard Contextual ASR approaches face significant scalability bottlenecks when handling massive dictionaries, as the influx of irrelevant candidates introduces noise that degrades recognition accuracyZhou et al\. \([2024b](https://arxiv.org/html/2605.16896#bib.bib28)\); Sudo et al\. \([2024a](https://arxiv.org/html/2605.16896#bib.bib21)\)\. To address this, recent research adopts dynamic dictionary filtering, which retrieves a concise, high\-relevance keyword subset for each speech input before feeding it into the downstream contextual ASR model\. Existing implementations of this mechanism typically fall into two primary categories: text\-based cascaded retrieval and audio\-based end\-to\-end retrieval\.

### 2\.1Text\-based Cascaded Retrieval

The first category, text\-based cascaded retrieval, operates by first utilizing a lightweight base ASR model to transcribe audio into preliminary hypotheses or N\-best lists\. These hypotheses serve as queries to match against the keyword dictionary\. Existing methods predominantly rely on dense text embeddings to perform this matching\. For instance,Mathur et al\. \([2024](https://arxiv.org/html/2605.16896#bib.bib12)\)utilize BERTDevlin et al\. \([2019](https://arxiv.org/html/2605.16896#bib.bib6)\), whileXiao et al\. \([2025](https://arxiv.org/html/2605.16896#bib.bib24)\)employ MPNetSong et al\. \([2020](https://arxiv.org/html/2605.16896#bib.bib20)\)to encode both hypotheses and keywords, ranking candidates based on semantic similarity\. The advantage of this paradigm lies in its training\-free nature, allowing for the direct application of powerful text embedding models to capture rich semantic knowledge\.

While effective for capturing semantic similarity, these semantic\-only retrievers are susceptible to phonetic recognition errors, which are prevalent in Chinese\. Since base ASR models frequently produce homophonic or near\-homophonic errors, the transcribed text often diverges significantly in meaning despite sounding similar to the target\. Consequently, retrievers relying solely on semantic embeddings are misled by these textual errors, fetching keywords that are semantically related to the incorrect transcripts but irrelevant to the actual speech\. Such irrelevant candidates fail to provide accurate context, rendering them ineffective for guiding the downstream contextual ASR model\. This limitation underscores that relying solely on global semantics is insufficient, necessitating the incorporation of other features, such as phonetic and glyph features\.

![Refer to caption](https://arxiv.org/html/2605.16896v1/x1.png)Figure 2:Overview of the proposed JSPG filtering framework\. Given an input utterance, a base ASR model generates N\-best hypotheses, which are compared to each candidate keyword in the dictionary via semantic, pinyin, and glyph similarity\. The scores are combined to rank and retrieve the most relevant keywords for downstream contextual ASR\.
### 2\.2Audio\-based End\-to\-End Retrieval

In text\-based cascaded methods, errors in this preliminary transcription will propagate to the retrieval stage\. It may lead to the failure of retrieving the correct keywords\. To avoid the reliance on intermediate text, audio\-based end\-to\-end retrieval directly utilizes the input speech to retrieve keywords\. Specifically, these methods employ a speech encoder to generate embeddings for the audio and a text encoder to generate embeddings for the keywords\. They are trained using a contrastive loss to maximize the similarity score between the input speech and the relevant ground\-truth keywordsLi et al\. \([2024b](https://arxiv.org/html/2605.16896#bib.bib11)\); Gong et al\. \([2025](https://arxiv.org/html/2605.16896#bib.bib8)\)\. While this approach effectively bypasses error propagation, it typically necessitates extensive additional training on large\-scale paired datasets to align the audio and text representations, making it data\-hungry and computationally expensive\.

In contrast, our proposed JSPG framework maintains the cascaded paradigm to directly leverage the sophisticated semantic capabilities of LLMs without additional training\. Crucially, we mitigate the negative impact of transcription errors by integrating phonetic and glyph features\.

### 2\.3Glyph and Pinyin Features in Chinese

Chinese characters are unique in that they integrate semantics, phonetics, and structuresTan et al\. \([2005](https://arxiv.org/html/2605.16896#bib.bib23)\)\. Specifically, phonetic and structural attributes are commonly measured by pinyin and glyph features, respectively\. Previous works in Chinese Spelling Check \(CSC\) and Named Entity Recognition \(NER\) have demonstrated that integrating these features is essential for resolving ambiguity\.

Recently,Qiao et al\. \([2025](https://arxiv.org/html/2605.16896#bib.bib17)\)proposed an approach for quantifying the similarity between Chinese characters based on their phonetic and structural attributes\. However, it is designed forcharacter\-levelcomparison and cannot be directly applied to our task\. The core challenge lies in computing the similarity between the keyword and the hypothesis sequence\. Typically, there is a significant length discrepancy between the short keyword and the longer hypothesis, and the hypothesis sequence usually contains recognition errors\. To bridge this gap, our JSPG method adapts the character\-level similarity scores into a modified Smith\-Waterman algorithmSmith et al\. \([1981](https://arxiv.org/html/2605.16896#bib.bib19)\)\. This allows us to leverage phonetic and glyph cues to identify the optimal matching part within the hypothesis and return a sequence\-level similarity score, which is used to assist the retrieval process in JSPG\.

## 3Our JSPG Approach

#### Big Picture\.

Our goal is to determine a small number of keywords that are really related to the input speech from a keyword dictionary containing numerous keywords\. Only the selected few keywords are provided to the subsequent contextual ASR models\. Specifically, we follow the N\-best hypothesis paradigmMathur et al\. \([2024](https://arxiv.org/html/2605.16896#bib.bib12)\); Dimitrios et al\. \([2025](https://arxiv.org/html/2605.16896#bib.bib7)\), which consists of two steps\. First, we obtain N\-best transcriptions using a non\-contextual basic ASR model\. Second, we obtain theKKmost related keywords from the dictionary using the N transcriptions\. Previous works use implicit embedding\-based similarity to represent relatedness, using LLMs to encode each transcription and each keyword into dense vectors\. The main contribution of this work is incorporating explicit character similarity based on pronunciation and glyph\. We follow the recent work on Chinese spelling check ofQiao et al\. \([2025](https://arxiv.org/html/2605.16896#bib.bib17)\), and borrow the method for computing character similarity\. To compute the relatedness between a transcription and a keyword, we extend the standard Smith\-Waterman algorithmSmith et al\. \([1981](https://arxiv.org/html/2605.16896#bib.bib19)\)to incorporate character similarity\.

### 3\.1Problem Formulation

Given an input speechxxand a large dictionaryD=\{w1,w2,…,wM\}D=\\\{w\_\{1\},w\_\{2\},\\dots,w\_\{M\}\\\}, the goal of our JSPG is to filterDDinto a concise, high\-relevance subset of a predefined sizeKK\(K≪MK\\ll M\)\. Following previous practicesXiao et al\. \([2025](https://arxiv.org/html/2605.16896#bib.bib24)\), the process begins with a base ASR model that transcribes the speechxxinto a preliminary list of N\-best hypothesesQ=\{q1,q2,…,qN\}Q=\\\{q\_\{1\},q\_\{2\},\\dots,q\_\{N\}\\\}, where eachqjq\_\{j\}is a character sequence representing a transcription\. JSPG computes a joint scoreF​\(Q,w\)F\(Q,w\)betweenwwandQQ\. The top\-K keywords are selected as the subset\. Specifically,F​\(Q,w\)F\(Q,w\)is computed by integrating a semantic scoreFs​\(Q,w\)F^\{s\}\(Q,w\)and a phonetic\-glyph scoreFp​g​\(Q,w\)F^\{pg\}\(Q,w\)\. The phonetic\-glyph score fuses pinyin\-based scoreFp​\(Q,w\)F^\{p\}\(Q,w\)and glyph\-based scoreFg​\(Q,w\)F^\{g\}\(Q,w\)\. The final score combines semantic and phonetic\-glyph scores:

Fp​g​\(Q,wi\)=α⋅Fp​\(Q,wi\)\+\(1−α\)⋅Fg​\(Q,wi\)\.F^\{pg\}\(Q,w\_\{i\}\)=\\alpha\\cdot F^\{p\}\(Q,w\_\{i\}\)\+\(1\-\\alpha\)\\cdot F^\{g\}\(Q,w\_\{i\}\)\.\(1\)F​\(Q,wi\)=β⋅Fs​\(Q,wi\)\+\(1−β\)⋅Fp​g​\(Q,wi\)\.F\(Q,w\_\{i\}\)=\\beta\\cdot F^\{s\}\(Q,w\_\{i\}\)\+\(1\-\\beta\)\\cdot F^\{pg\}\(Q,w\_\{i\}\)\.\(2\)

### 3\.2Semantic\-based Sequence Similarity

To calculate the semantic similarity between the ASR hypotheses and the candidate keywords, we leverage dense text embeddings\. We begin by concatenating the N\-best hypotheses into a single sequenceQ′Q^\{\\prime\}, following the approach inDimitrios et al\. \([2025](https://arxiv.org/html/2605.16896#bib.bib7)\)\. Next, we employ the Qwen3\-Embedding modelZhang et al\. \([2025](https://arxiv.org/html/2605.16896#bib.bib25)\)to encode bothQ′Q^\{\\prime\}and the candidate keywordswiw\_\{i\}\. To maximize the model’s retrieval capability, we follow its recommended settings and add a task\-specific instruction beforeQ′Q^\{\\prime\}: “*Given a list of candidate transcriptions predicted by a speech recognition model as a query, retrieve keywords relevant to the query\. The candidate transcriptions are:q1q\_\{1\},q2q\_\{2\},…\\dots,qNq\_\{N\}\.*”\. The semantic scoreFs​\(Q,wi\)F^\{s\}\(Q,w\_\{i\}\)is then derived using cosine similarity:

Fs​\(Q,wi\)=Emb​\(Q′\)⋅Emb​\(wi\)‖Emb​\(Q′\)‖​‖Emb​\(wi\)‖F^\{s\}\(Q,w\_\{i\}\)=\\frac\{\\text\{Emb\}\(Q^\{\\prime\}\)\\cdot\\text\{Emb\}\(w\_\{i\}\)\}\{\\\|\\text\{Emb\}\(Q^\{\\prime\}\)\\\|\\,\\\|\\text\{Emb\}\(w\_\{i\}\)\\\|\}\(3\)
While text embeddings effectively capture the global context, ASR errors inQ′Q^\{\\prime\}often change the actual meaning of the transcription, causing pure semantic retrieval to fail\. This limitation necessitates the incorporation of pinyin and glyph features\.

### 3\.3Phonetic\-Glyph Character Similarity

ASR prediction errors typically disrupt the semantic context but preserve the phonetic cues of the target keyword\. To retrieve the target keywords using this cue, we compute character\-level similarity using two complementary features: pinyin and glyph\. First, pinyin directly encodes pronunciation in Chinese, so it can be used to measure phonetic similarity and recall the target keywords\. However, in Chinese, a single pronunciation often corresponds to multiple distinct characters\. Therefore, using pinyin to retrieve keywords inevitably fetches many irrelevant candidates\. Second, the glyph feature serves as another effective metric to retrieve the target keywords\. Because most Chinese characters are phono\-semantic compounds, characters with similar pronunciations frequently share similar structural components\. Furthermore, unlike the one\-to\-many mapping of pinyin, every character has a unique glyph structure\. The one\-to\-one property enables the glyph feature to effectively exclude irrelevant keywords that sound similar to the target\.

#### Pinyin Similarity\.

FollowingQiao et al\. \([2025](https://arxiv.org/html/2605.16896#bib.bib17)\), for any two charactersc1c\_\{1\}andc2c\_\{2\}, we extract their complete pinyin sequences, denoted aspy​\(c1\)\\texttt\{py\}\(c\_\{1\}\)andpy​\(c2\)\\texttt\{py\}\(c\_\{2\}\)\. The pinyin similarity scores​i​mp​\(c1,c2\)sim\_\{p\}\(c\_\{1\},c\_\{2\}\)is then calculated using the normalized Levenshtein Distance \(LD\):

s​i​mp​\(c1,c2\)=1−LD​\(py​\(c1\),py​\(c2\)\)\|py​\(c1\)\|\+\|py​\(c2\)\|,sim\_\{p\}\(c\_\{1\},c\_\{2\}\)=1\-\\frac\{\\texttt\{LD\}\(\\texttt\{py\}\(c\_\{1\}\),\\texttt\{py\}\(c\_\{2\}\)\)\}\{\|\\texttt\{py\}\(c\_\{1\}\)\|\+\|\\texttt\{py\}\(c\_\{2\}\)\|\},\(4\)where\|⋅\|\|\\cdot\|represents the length of the pinyin string\.

#### Glyph Similarity\.

To measure the glyph similarity between characters, we adopt the character\-level metric proposed byQiao et al\. \([2025](https://arxiv.org/html/2605.16896#bib.bib17)\)\. This metric comprehensively evaluates glyph similarity across four distinct dimensions\. For any two charactersc1c\_\{1\}andc2c\_\{2\}, the glyph similarity scores​i​mg​\(c1,c2\)sim\_\{g\}\(c\_\{1\},c\_\{2\}\)is defined as the average of four sub\-metrics:

1. 1\.Four\-corner Code \(s​i​mg1sim\_\{g\}^\{1\}\): Measures the similarity between numerical codes that represent the shapes of the character’s four corners\.
2. 2\.Structure\-aware Code \(s​i​mg2sim\_\{g\}^\{2\}\): Encodes radical decomposition and spatial layout \(e\.g\., left\-right vs\. top\-bottom structures\)\.
3. 3\.Stroke Sequence Edit Distance \(s​i​mg3sim\_\{g\}^\{3\}\): Computes the normalized LD between standard stroke\-order sequences to capture fine\-grained writing differences\.
4. 4\.Stroke Sequence LCS \(s​i​mg4sim\_\{g\}^\{4\}\): Computes the normalized Longest Common Subsequence \(LCS\) of stroke sequences to identify shared internal writing patterns\.

The final character\-level glyph similarity is the average of these four components:

s​i​mg​\(c1,c2\)=14​∑k=14s​i​mgk​\(c1,c2\)\.sim\_\{g\}\(c\_\{1\},c\_\{2\}\)=\\frac\{1\}\{4\}\\sum\_\{k=1\}^\{4\}sim\_\{g\}^\{k\}\(c\_\{1\},c\_\{2\}\)\.\(5\)
Currently, boths​i​mpsim\_\{p\}ands​i​mgsim\_\{g\}operate strictly at the character level\. However, a key challenge is extending them to quantify the sequence\-level relevance between a short keyword and a longer, noisy ASR hypothesis sequence\. To bridge this gap, we introduce an extended Smith\-Waterman algorithm to extend these character\-level metrics into sequence\-level scores, yieldingFp​\(Q,wi\)F^\{p\}\(Q,w\_\{i\}\)andFg​\(Q,wi\)F^\{g\}\(Q,w\_\{i\}\)\.

### 3\.4Sequence\-Level Similarity via an Extended Smith\-Waterman Algorithm

The similarities defined above operate at the single\-character level\. In other words, given a pair of characters,simg\(\.\)sim\_\{g\}\(\.\)\(orsimp\(\.\)sim\_\{p\}\(\.\)\) can give a similarity score between them\.

However, our retrieval module needs to assess how likely it is that a short keyword sequence𝒘=c1​c2​…​cs\\boldsymbol\{w\}=c\_\{1\}c\_\{2\}\.\.\.c\_\{s\}is related to a longer sequence𝒒=x1​x2​…​xn\\boldsymbol\{q\}=x\_\{1\}x\_\{2\}\.\.\.x\_\{n\}\. Specifically,𝒘\\boldsymbol\{w\}is considered related to𝒒\\boldsymbol\{q\}if𝒘\\boldsymbol\{w\}itself, or a modified version of it, is a substring of𝒒\\boldsymbol\{q\}\. To achieve this, we employ an extended Smith\-Waterman algorithmSmith et al\. \([1981](https://arxiv.org/html/2605.16896#bib.bib19)\), a dynamic programming approach based on edit distance\.

#### Edit Distance\.

As a preliminary, the edit distance algorithm uses dynamic programming to efficiently compute the distance between two sequences, such asa1​a2​…​asa\_\{1\}a\_\{2\}\.\.\.a\_\{s\}andb1​b2​…​btb\_\{1\}b\_\{2\}\.\.\.b\_\{t\}\. We useED​\(⋅\)\\texttt\{ED\}\(\\cdot\)to denote the edit distance\. Specifically,ED​\(a1​a2​…​as,b1​b2​…​bt\)\\texttt\{ED\}\(a\_\{1\}a\_\{2\}\.\.\.a\_\{s\},b\_\{1\}b\_\{2\}\.\.\.b\_\{t\}\)represents the total weight of the operations required to transforma1​a2​…​asa\_\{1\}a\_\{2\}\.\.\.a\_\{s\}intob1​b2​…​btb\_\{1\}b\_\{2\}\.\.\.b\_\{t\}\. There are four standard operations: match \(no change\), deletion, insertion, and substitution\. Their corresponding weights are typically set to 0, 1\.0, 1\.0, and 1\.0, respectively\.

#### Standard Smith\-Waterman\.

The standard Smith\-Waterman algorithm is built uponED\(\.\)\\texttt\{ED\}\(\.\)and a more complex dynamic programming algorithm\. Given𝒘=c1​…​cs\\boldsymbol\{w\}=c\_\{1\}\.\.\.c\_\{s\}and𝒒=x1​…​xn\\boldsymbol\{q\}=x\_\{1\}\.\.\.x\_\{n\}, the algorithm finds the optimal indicesu,vu,v\(1≤u≤v≤s1\\leq u\\leq v\\leq s\) for𝒘\\boldsymbol\{w\}andi,ji,j\(1≤i≤j≤n1\\leq i\\leq j\\leq n\) for𝒒\\boldsymbol\{q\}, which yield the lowest edit distance between the substringscu​…​cvc\_\{u\}\.\.\.c\_\{v\}andxi​…​xjx\_\{i\}\.\.\.x\_\{j\}\.SW\(w,q\) = min\_u,v,i,jED\(c\_u…c\_v, x\_i…x\_j\)

#### ExtendedSW\(\.\)\\texttt\{SW\}\(\.\)\.

To accommodate our situation, we make a two\-fold extension\. First, we further consider character similarity for the substitution operation\. Takings​i​mgsim\_\{g\}as an example, we change the weight of substitutingc1c\_\{1\}withc2c\_\{2\}from a fixed value of1\.01\.0to: 1 \- sim\_g\(c\_1, c\_2\) The more similarc1c\_\{1\}is toc2c\_\{2\}, the lower the weight\. Whenc1=c2c\_\{1\}=c\_\{2\}, which corresponds to the not\-change operation, the weight is zero\.

Second, we prohibit deletion operations at the beginning and end positions of𝒘\\boldsymbol\{w\}\(i\.e\., forc1c\_\{1\}andcsc\_\{s\}\)\. This constraint mathematically enforcesu=1u=1andv=sv=s, forcing the entirety of𝒘\\boldsymbol\{w\}to be aligned against a substring of𝒒\\boldsymbol\{q\}\.

Then, the relatedness likelihood is computed as:RL\(w,q\) = \|w\|\-SWext\(w,q\)\|w\|

Considering that the base ASR model typically generates an N\-best list of hypothesesQ=\{q1,q2,…,qN\}Q=\\\{q\_\{1\},q\_\{2\},\.\.\.,q\_\{N\}\\\}rather than a single transcription, we compute the relatedness likelihoodRL​\(w,qj\)\\texttt\{RL\}\(w,q\_\{j\}\)for each individual hypothesisqj∈Qq\_\{j\}\\in Q\. We take the maximum likelihood score across all N hypotheses\. Therefore, the final sequence\-level similarity score between the keywordwwand the N\-best hypothesesQQis formulated as: F^\*\(Q, w\) = max\_q\_j ∈QRL\(w, q\_j\) where∗\*represents eitherpporgg, depending on whether pinyin or glyph features are used when calculating the relatedness likelihoodRL\.

1351020501000\.10\.10\.20\.20\.30\.30\.40\.40\.50\.50\.60\.60\.70\.70\.80\.80\.90\.911FbF^\{b\}FsF^\{s\}FgF^\{g\}FpF^\{p\}FF

\(a\)Aishell\-1
1351020501000\.10\.10\.20\.20\.30\.30\.40\.40\.50\.50\.60\.60\.70\.70\.80\.80\.90\.911FbF^\{b\}FsF^\{s\}FgF^\{g\}FpF^\{p\}FF

\(b\)DC
1351020501000\.10\.10\.20\.20\.30\.30\.40\.40\.50\.50\.60\.60\.70\.70\.80\.80\.90\.911FbF^\{b\}FsF^\{s\}FgF^\{g\}FpF^\{p\}FF

\(c\)ICI

Figure 3:The performance of retrieval methods on Aishell, DC, and ICI datasets\. The x\-axis indicates the K value in top\-K, and the y\-axis shows the corresponding Recall@K\.

## 4Experiments

### 4\.1Experimental Setup

#### Datasets\.

To comprehensively evaluate the proposed approach across varying domains and speaking styles, we conduct experiments on two datasets\. Aishell\-1Bu et al\. \([2017](https://arxiv.org/html/2605.16896#bib.bib3)\); Chen et al\. \([2022](https://arxiv.org/html/2605.16896#bib.bib4)\)serves as our standard benchmark, containing 150 hours of Mandarin speech recorded in a clean environment\. To validate robustness in complex, real\-world scenarios, we utilize the RWCS\-NER datasetZhou et al\. \([2024a](https://arxiv.org/html/2605.16896#bib.bib27)\), which comprises two distinct domains: open\-domain daily conversation \(DC\) and intelligent cockpit instructions \(ICI\)\. It features spontaneous speech, background noise, and a high density of rare entities, making it a challenging task for contextual ASR\.

#### Pipeline Settings\.

We evaluate our approach using a three\-stage pipeline\. First, we employ WhisperRadford et al\. \([2022](https://arxiv.org/html/2605.16896#bib.bib18)\), fine\-tuned on the Aishell\-1 training set, as the base ASR model to generate the initial N\-best hypothesesQQ\. Second, for the retrieval stage, we compare our JSPG against an exact character\-matching baseline \(FbF^\{b\}\) that uses the same Smith\-Waterman framework, as well as single\-feature retrievers relying solely on semantic \(FsF^\{s\}, powered by Qwen3\-Embedding\-0\.6BZhang et al\. \([2025](https://arxiv.org/html/2605.16896#bib.bib25)\)\), glyph \(FgF^\{g\}\), or pinyin \(FpF^\{p\}\) scores\. Finally, to verify downstream effectiveness, we integrate the retrieved keywords into two distinct contextual ASR paradigms: end\-to\-end models \(CopyNEZhou et al\. \([2024b](https://arxiv.org/html/2605.16896#bib.bib28)\)and a multi\-grained modelZhou and Li \([2025](https://arxiv.org/html/2605.16896#bib.bib26)\)\) and a two\-stage LLM\-refinement paradigm utilizing GPT\-OSS\-120BOpenAI \([2025](https://arxiv.org/html/2605.16896#bib.bib14)\)to correct the N\-best hypotheses based on the retrieved prompts\.

#### Evaluation Metrics

We evaluate our system from two perspectives: the quality of the retrieved keywords and the final ASR recognition accuracy\. To assess retrieval quality, we adopt Recall@KK\(R@KK\) as our core metricChen et al\. \([2025](https://arxiv.org/html/2605.16896#bib.bib5)\)\. It measures the proportion of ground\-truth keywords captured within the top\-K retrieved list\. In our experiments, we report R@KKforK∈\{1,3,5,10,20,50,100\}K\\in\\\{1,3,5,10,20,50,100\\\}\.

To assess downstream recognition accuracy and verify whether the retrieved keywords effectively guide the ASR model, we report four metrics\. We measure the overall Character Error Rate \(CER\) on the full text, and the Unbiased\-CER \(U\-CER\) on non\-keyword text to ensure general transcription quality is preserved\. For keyword\-specific performance, we report Keyword\-CER \(K\-CER\) on the keyword segments and Keyword Recall \(Recall\), which represents the percentage of ground\-truth keywords correctly transcribed\.

### 4\.2Results of Retrieval

#### Overall Trend\.

Figure[3](https://arxiv.org/html/2605.16896#S3.F3)illustrates the recall performance asKKvaries\. A consistent trend is observed across all datasets: the recall of all retrieval methods increases monotonically withKKand eventually saturates\. This aligns with the theoretical expectation that a larger candidate pool covers more ground\-truth targets\. Crucially, our proposed JSPG algorithm \(FF\) consistently achieves the highest recall among all methods across allKKsettings\. This empirical evidence validates that fusing semantic, glyph, and pinyin features provides a more robust retrieval signal than any single modality or the character\-matching baseline\.

#### Comparison between Different Retrieval Approaches\.

The performance gap between methods is most pronounced at lowerKKvalues \(e\.g\.,K=1,3K=1,3\), which reflect the precision of the retriever\. In this setting, the joint algorithmFFdemonstrates a decisive advantage, significantly outperforming other methods\. Notably, on the relatively clean Aishell\-1 and DC datasets, the character\-matching baselineFbF^\{b\}performs comparably to the glyph\-based methodFgF^\{g\}\. This suggests that many keywords in these scenarios can be recalled via surface\-level or visual similarity\. Conversely, while the pinyin\-based retrieverFpF^\{p\}underperforms atK=1K=1\(especially on Aishell, dropping to 0\.04\), its recall rate grows rapidly asKKincreases, eventually matching or surpassingFbF^\{b\}andFgF^\{g\}\(e\.g\., reaching 0\.96 on DC atK=100K=100\)\. This indicates that pinyin features are highly effective at capturing phonetic confusion and homophonic errors, provided a slightly larger candidate pool is allowed to accommodate phonetic ambiguity\.

Table 1:Results of different contextual ASR models with JSPG retrieval\. The performance of base ASR is provided in the dataset headers for direct comparison\.Table 2:Ablation on feature weights\. It reveals their complementary roles: glyph heavily anchors top\-1 precision, pinyin expands the top\-100 recall ceiling, and semantics acts as a vital tie\-breaker upon a solid phonetic\-glyph foundation\.
#### Ablation on Phonetic\-Glyph Feature Weights\.

We conduct a two\-stage ablation study to investigate the complementarity among semantic, pinyin, and glyph features, as summarized in Table[2](https://arxiv.org/html/2605.16896#S4.T2)\.

Stage 1: Pinyin vs\. Glyph \(α\\alpha\)Varying the pinyin weightα\\alphareveals a strong symbiotic relationship\. Interestingly, R@1 remains highly stable \(∼\\sim18\.7%\) across all settings\. This stability demonstrates that top\-1 precision is heavily anchored by the glyph feature, whose one\-to\-one structural mapping successfully filters out exact homophone confusions\. However, for a larger candidate pool, R@100 rises substantially to peak at 99\.13% whenα=0\.7\\alpha=0\.7\. This proves that the pinyin feature is indispensable for capturing a broader range of structurally distinct but phonetically similar ASR errors\. We setα=0\.7\\alpha=0\.7to optimally balance broad phonetic tolerance and precise structural filtering\.

Stage 2: Semantic vs\. Phonetic\-Glyph \(β\\beta\)\.Fixingα=0\.7\\alpha=0\.7, we evaluate the semantic weightβ\\beta\(Table[2](https://arxiv.org/html/2605.16896#S4.T2)\) to demonstrate the necessity of feature fusion\. As shown in Figure[3](https://arxiv.org/html/2605.16896#S3.F3), the recall curve for pure semantic retrieval \(FsF^\{s\}\) flattens out early\. Because homophonic ASR errors severely distort sentence meaning, pure semantics assigns extremely low scores to the correct targets, pushing them completely out of the top\-100 pool\. Table[2](https://arxiv.org/html/2605.16896#S4.T2)confirms this limitation: over\-relying on semantics \(β=0\.9\\beta=0\.9\) causes the overall recall to drop to 89\.11%\. In contrast, fusing semantic and phonetic\-glyph features effectively overcomes this bottleneck\. At the optimal weight \(β=0\.4\\beta=0\.4\), the system achieves R@1 = 64\.52% while sustaining a high overall recall \(R@100 = 97\.03%\)\. This confirms that the semantic and phonetic\-glyph features are highly complementary, and their fusion leads to significantly better retrieval performance\.

### 4\.3Results of Contextual ASR

Having validated the retrieval performance, we now evaluate the impact of the proposed JSPG retriever on the downstream contextual ASR task\. We integrate JSPG into the pipeline and test it with three different recognition models: CopyNE \(Deep Biasing\), Multi\-grained \(LLM\-Fusion\), and LLM\-Refine \(Two\-stage correction\)\. Table[1](https://arxiv.org/html/2605.16896#S4.T1)details the performance of the retrieval\-augmented systems, with the zero\-shot performance of the fine\-tuned Whisper baseline provided directly in the dataset headers for direct comparison\.

#### Improvements over Baseline

Experimental results demonstrate that our framework significantly outperforms the Whisper baseline across all datasets\. On the standard Aishell\-1, the best\-performing configuration reduces the K\-CER from 10\.40% to 2\.70% and improves R from 80\.60% to 95\.53% \(Multi\-grained with Top\-K=10\)\. The gains are even more pronounced in the real\-world datasets\. On the noisy, task\-oriented ICI dataset, K\-CER drops dramatically from 30\.70% to 4\.85%, and R surges from 40\.80% to 93\.20% \(LLM\-Refine with Top\-K=10\)\. These results confirm that incorporating retrieved context via JSPG effectively mitigates the keyword recognition errors\.

#### Performance of Different Recognition Models

The effectiveness of the recognition models varies by domain\. The Multi\-grained model performs best on Aishell\-1 dataset\. However, for the highly complex ICI dataset, the LLM\-Refine method \(utilizing GPT\-OSS\-120B\) achieves the lowest K\-CER\. This indicates that while specialized end\-to\-end models are efficient for general tasks, large\-scale LLMs possess stronger capabilities for keyword correction in difficult scenarios\. Importantly, the U\-CER remains stable across all experiments, ensuring that our retrieval\-based biasing does not negatively impact the recognition of general non\-keyword text\.

## 5Conclusion

In this paper, we introduced JSPG, a novel filtering framework designed to address the challenge of utilizing large\-scale contextual dictionaries in ASR\. By synergizing semantic, glyph, and pinyin information, JSPG effectively bridges the gap between noisy hypotheses and target keywords, ensuring robust recall even in the presence of severe recognition errors\. We believe this work offers a practical solution for deploying contextual ASR in open\-domain scenarios and paves the way for future research in speech processing\.

## Limitations

While JSPG demonstrates promising results, we acknowledge two limitations in this work\.

First, our evaluation is currently limited to the Chinese language\. The specific implementation of our method relies on Chinese pinyin and character structures\. However, we believe that the core principle of leveraging phonetic and structural features is universal\. With appropriate modifications, we expect our approach can be adapted to other languages that face similar homophonic challenges, such as Japanese or Korean\.

Second, this work mainly focuses on the retrieval stage\. We aim to find the most relevant keywords to narrow down the search space\. We have not optimized how the downstream contextual ASR model utilizes these keywords\. Since the retrieved candidates are often phonetically or visually similar to the target, guiding the model to strictly distinguish the correct keyword from these confusing candidates is a critical step\. We plan to explore better ways to utilize the retrieved context in future work\.

## References

- Alon et al\. \(2019\)Uri Alon, Golan Pundak, and Tara N Sainath\. 2019\.Contextual speech recognition with difficult negative training examples\.In*ICASSP 2019\-2019 IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\)*, pages 6440–6444\. IEEE\.
- Bai et al\. \(2024\)Ye Bai, Jingping Chen, Jitong Chen, Wei Chen, Zhuo Chen, Chuang Ding, Linhao Dong, Qianqian Dong, Yujiao Du, Kepan Gao, and 1 others\. 2024\.Seed\-asr: Understanding diverse speech and contexts with llm\-based speech recognition\.*arXiv preprint arXiv:2407\.04675*\.
- Bu et al\. \(2017\)Hui Bu, Jiayu Du, Xingyu Na, Bengu Wu, and Hao Zheng\. 2017\.Aishell\-1: An open\-source mandarin speech corpus and a speech recognition baseline\.In*2017 20th Conference of the Oriental Chapter of the International Coordinating Committee on Speech Databases and Speech I/O Systems and Assessment \(O\-COCOSDA\)*, pages 1–5\.
- Chen et al\. \(2022\)Boli Chen, Guangwei Xu, Xiaobin Wang, Pengjun Xie, Meishan Zhang, and Fei Huang\. 2022\.Aishell\-ner: Named entity recognition from chinese speech\.In*2022 IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\)*, pages 8352–8356\.
- Chen et al\. \(2025\)Yifu Chen, Shengpeng Ji, Haoxiao Wang, Ziqing Wang, Siyu Chen, Jinzheng He, Jin Xu, and Zhou Zhao\. 2025\.[WavRAG: Audio\-integrated retrieval augmented generation for spoken dialogue models](https://doi.org/10.18653/v1/2025.acl-long.613)\.In*Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 12505–12523, Vienna, Austria\. Association for Computational Linguistics\.
- Devlin et al\. \(2019\)Jacob Devlin, Ming\-Wei Chang, Kenton Lee, and Kristina Toutanova\. 2019\.BERT: Pre\-training of deep bidirectional transformers for language understanding\.In*Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics \(NAACL\)*, pages 4171–4186, Minneapolis, Minnesota\. Association for Computational Linguistics\.
- Dimitrios et al\. \(2025\)Siskos Dimitrios, Stavros Papadopoulos, Pablo Peso Parada, Jisi Zhang, Karthikeyan Saravanan, and Anastasios Drosou\. 2025\.[Retrieval augmented generation based context discovery for ASR](https://doi.org/10.18653/v1/2025.findings-emnlp.768)\.In*Findings of the Association for Computational Linguistics: EMNLP 2025*, pages 14247–14254, Suzhou, China\. Association for Computational Linguistics\.
- Gong et al\. \(2025\)Xun Gong, Anqi Lv, Zhiming Wang, Huijia Zhu, and Yanmin Qian\. 2025\.Br\-asr: Efficient and scalable bias retrieval framework for contextual biasing asr in speech llm\.*arXiv preprint arXiv:2505\.19179*\.
- Lei et al\. \(2025\)Zhihong Lei, Xingyu Na, Mingbin Xu, Ernest Pusateri, Christophe Van Gysel, Yuanyuan Zhang, Shiyi Han, and Zhen Huang\. 2025\.Contextualization of asr with llm using phonetic retrieval\-based augmentation\.In*ICASSP 2025\-2025 IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\)*, pages 1–5\. IEEE\.
- Li et al\. \(2024a\)Shaojun Li, Hengchao Shang, Daimeng Wei, Jiaxin Guo, Zongyao Li, Xianghui He, Min Zhang, and Hao Yang\. 2024a\.La\-rag: Enhancing llm\-based asr accuracy with retrieval\-augmented generation\.*arXiv preprint arXiv:2409\.08597*\.
- Li et al\. \(2024b\)Shaojun Li, Hengchao Shang, Daimeng Wei, Jiaxin Guo, Zongyao Li, Xianghui He, Min Zhang, and Hao Yang\. 2024b\.La\-rag: Enhancing llm\-based asr accuracy with retrieval\-augmented generation\.*arXiv preprint arXiv:2409\.08597*\.
- Mathur et al\. \(2024\)Puneet Mathur, Zhe Liu, Ke Li, Yingyi Ma, Gil Karen, Zeeshan Ahmed, Dinesh Manocha, and Xuedong Zhang\. 2024\.[DOC\-RAG: ASR language model personalization with domain\-distributed co\-occurrence retrieval augmentation](https://aclanthology.org/2024.lrec-main.457/)\.In*Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation \(LREC\-COLING 2024\)*, pages 5132–5139, Torino, Italia\. ELRA and ICCL\.
- Omnilingual et al\. \(2025\)ASR Omnilingual, Gil Keren, Artyom Kozhevnikov, Yen Meng, Christophe Ropers, Matthew Setzler, Skyler Wang, Ife Adebara, Michael Auli, Can Balioglu, and 1 others\. 2025\.Omnilingual asr: Open\-source multilingual speech recognition for 1600\+ languages\.*arXiv preprint arXiv:2511\.09690*\.
- OpenAI \(2025\)OpenAI\. 2025\.[gpt\-oss\-120b and gpt\-oss\-20b model card](https://arxiv.org/abs/2508.10925)\.*Preprint*, arXiv:2508\.10925\.
- Peters et al\. \(2018\)Matthew E\. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer\. 2018\.Deep contextualized word representations\.In*Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics \(NAACL\)*, pages 2227–2237, New Orleans, Louisiana\. Association for Computational Linguistics\.
- Pundak et al\. \(2018\)Golan Pundak, Tara N Sainath, Rohit Prabhavalkar, Anjuli Kannan, and Ding Zhao\. 2018\.Deep context: end\-to\-end contextual speech recognition\.In*2018 IEEE spoken language technology workshop \(SLT\)*, pages 418–425\. IEEE\.
- Qiao et al\. \(2025\)Ziheng Qiao, Houquan Zhou, Yumeng Liu, Zhenghua Li, Min Zhang, Bo Zhang, Chen Li, Ji Zhang, and Fei Huang\. 2025\.[DISC: Plug\-and\-play decoding intervention with similarity of characters for Chinese spelling check](https://doi.org/10.18653/v1/2025.acl-long.1373)\.In*Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 28312–28324, Vienna, Austria\. Association for Computational Linguistics\.
- Radford et al\. \(2022\)Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever\. 2022\.Robust speech recognition via large\-scale weak supervision\.*arXiv preprint arXiv:2212\.04356*\.
- Smith et al\. \(1981\)Temple F Smith, Michael S Waterman, and 1 others\. 1981\.Identification of common molecular subsequences\.*Journal of molecular biology*, 147\(1\):195–197\.
- Song et al\. \(2020\)Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie\-Yan Liu\. 2020\.Mpnet: Masked and permuted pre\-training for language understanding\.*Advances in neural information processing systems*, 33:16857–16867\.
- Sudo et al\. \(2024a\)Yui Sudo, Yosuke Fukumoto, Muhammad Shakeel, Yifan Peng, and Shinji Watanabe\. 2024a\.Contextualized automatic speech recognition with dynamic vocabulary\.In*2024 IEEE Spoken Language Technology Workshop \(SLT\)*, pages 78–85\. IEEE\.
- Sudo et al\. \(2024b\)Yui Sudo, Muhammad Shakeel, Yosuke Fukumoto, Yifan Peng, and Shinji Watanabe\. 2024b\.Contextualized automatic speech recognition with attention\-based bias phrase boosted beam search\.In*ICASSP 2024\-2024 IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\)*, pages 10896–10900\. IEEE\.
- Tan et al\. \(2005\)Li Hai Tan, Angela R Laird, Karl Li, and Peter T Fox\. 2005\.Neuroanatomical correlates of phonological processing of chinese characters and alphabetic words: A meta\-analysis\.*Human brain mapping*, 25\(1\):83–91\.
- Xiao et al\. \(2025\)Cihan Xiao, Zejiang Hou, Daniel Garcia\-Romero, and Kyu J Han\. 2025\.Contextual asr with retrieval augmented large language model\.In*ICASSP 2025\-2025 IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\)*, pages 1–5\. IEEE\.
- Zhang et al\. \(2025\)Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, and 1 others\. 2025\.Qwen3 embedding: Advancing text embedding and reranking through foundation models\.*arXiv preprint arXiv:2506\.05176*\.
- Zhou and Li \(2025\)Shilin Zhou and Zhenghua Li\. 2025\.Improving contextual asr via multi\-grained fusion with large language models\.*arXiv preprint arXiv:2507\.12252*\.
- Zhou et al\. \(2024a\)Shilin Zhou, Zhenghua Li, Chen Gong, Lei Zhang, Yu Hong, and Min Zhang\. 2024a\.[Chinese spoken named entity recognition in real\-world scenarios: Dataset and approaches](https://doi.org/10.18653/v1/2024.findings-acl.111)\.In*Findings of the Association for Computational Linguistics: ACL 2024*, pages 1872–1884, Bangkok, Thailand\. Association for Computational Linguistics\.
- Zhou et al\. \(2024b\)Shilin Zhou, Zhenghua Li, Yu Hong, Min Zhang, Zhefeng Wang, and Baoxing Huai\. 2024b\.[CopyNE: Better contextual ASR by copying named entities](https://doi.org/10.18653/v1/2024.acl-long.147)\.In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 2675–2686, Bangkok, Thailand\. Association for Computational Linguistics\.

## Appendix AAppendix

### A\.1Algorithmic Walkthrough of Extended Smith\-Waterman

To demonstrate the process of our extended SW algorithm, we present a step\-by\-step walkthrough here\. This example computes the sequence\-level similarity score between the keywordw=w=“语音识别” \(speech recognition,\|w\|=4\|w\|=4\) and the ASR hypothesis “关于雨音的识别” \(about the recognition of rain sound\)\. Here the target character “语” is misrecognized as the homophone “雨” and an extra character “的” is inserted\.

#### Input: Substitution Cost Matrix

Table[3](https://arxiv.org/html/2605.16896#A1.T3)presents the substitution costs between the hypothesis characters and keyword characters\. As defined in Section[3\.4](https://arxiv.org/html/2605.16896#S3.SS4), each cost is computed as1−s​i​mp​\(ci,cj\)1\-sim\_\{p\}\(c\_\{i\},c\_\{j\}\), wheres​i​mpsim\_\{p\}is the pinyin similarity defined in Section[3\.3](https://arxiv.org/html/2605.16896#S3.SS3)\. A cost of 0 indicates identical pronunciation\. Notably, the homophonic pair \(“雨”, “语”\) shares identical pinyin \(yu3\), resulting in a substitution cost of 0\.

#### Process: Cost Accumulation Matrix \(DD\)

Using the substitution costs from Table[3](https://arxiv.org/html/2605.16896#A1.T3)and a gap cost of 1, we compute the cost matrixDD, whereDi,jD\_\{i,j\}represents the minimum alignment cost of the firstjjkeyword characters against hypothesis characters ending at positionii\. The initialization isDi,0=0D\_\{i,0\}=0\(free start in the hypothesis\) andD0,j=∞D\_\{0,j\}=\\inftyforj≥1j\\geq 1\. For general positions, the recurrence isDi,j=min⁡\(Di−1,j−1\+costi,j,Di−1,j\+1,Di,j−1\+1\)D\_\{i,j\}=\\min\(D\_\{i\-1,j\-1\}\+\\text\{cost\}\_\{i,j\},\\,D\_\{i\-1,j\}\+1,\\,D\_\{i,j\-1\}\+1\)\. As described in Section[3\.4](https://arxiv.org/html/2605.16896#S3.SS4), for the first and last keyword positions \(j=1j=1andj=\|w\|j=\|w\|\), the deletion moveDi,j−1\+1D\_\{i,j\-1\}\+1is excluded to ensure the entire keyword is matched\. The resulting matrix is shown in Table[4](https://arxiv.org/html/2605.16896#A1.T4)\. We highlight the calculation for the following critical nodes\.

Table 3:Substitution costs \(1−s​i​mp1\-sim\_\{p\}\) derived from pinyin similarity\. A cost of 0 indicates identical pronunciation\.Table 4:The cost matrixDD\. Lower values indicate better alignment\. The arrows visualize the optimal alignment path\.ForD3,1D\_\{3,1\}: Aligning雨3\\text\{雨\}\_\{3\}with语1\\text\{语\}\_\{1\}\. Sincej=1j=1, deletion of the first keyword character is prohibited\. With a substitution cost of0\(identical pinyin\), the diagonal move yields the minimum:

D3,1=min⁡\(D2,0\+cost3,1,D2,1\+1\)=min⁡\(0\+0,0\.17\+1\)=𝟎\.\\begin\{split\}D\_\{3,1\}&=\\min\(D\_\{2,0\}\+\\text\{cost\}\_\{3,1\},\\,D\_\{2,1\}\+1\)\\\\ &=\\min\(0\+0,\\,0\.17\+1\)=\\mathbf\{0\}\.\\end\{split\}
ForD4,2D\_\{4,2\}: Aligning音4\\text\{音\}\_\{4\}with音2\\text\{音\}\_\{2\}\. An exact match \(cost=0=0\) continues the diagonal path:

D4,2=min⁡\(D3,1\+cost4,2,D3,2\+1,D4,1\+1\)=min⁡\(0\+0,0\.60\+1,0\.43\+1\)=𝟎\.\\begin\{split\}D\_\{4,2\}&=\\min\(D\_\{3,1\}\+\\text\{cost\}\_\{4,2\},\\,D\_\{3,2\}\+1,\\,D\_\{4,1\}\+1\)\\\\ &=\\min\(0\+0,\\,0\.60\+1,\\,0\.43\+1\)=\\mathbf\{0\}\.\\end\{split\}
ForD5,2D\_\{5,2\}: The insertion character的5\\text\{的\}\_\{5\}has a high substitution cost with音2\\text\{音\}\_\{2\}\(cost=0\.67=0\.67\)\. The algorithm selects the vertical moveD4,2\+1=1D\_\{4,2\}\+1=1, which inserts的as a gap\. This effectively extends the alignment of “雨音”↔\\leftrightarrow“语音” to handle the extra character, rather than breaking the existing match:

D5,2=min⁡\(D4,1\+cost5,2,D4,2\+1,D5,1\+1\)=min⁡\(1\.10,1\.00,1\.60\)=1\.00\.\\begin\{split\}D\_\{5,2\}&=\\min\(D\_\{4,1\}\+\\text\{cost\}\_\{5,2\},\\,D\_\{4,2\}\+1,\\,D\_\{5,1\}\+1\)\\\\ &=\\min\(1\.10,\\,1\.00,\\,1\.60\)=\\mathbf\{1\.00\}\.\\end\{split\}
ForD6,3D\_\{6,3\}andD7,4D\_\{7,4\}: After bypassing the insertion, exact matches \(cost=0=0\) resume\. Sincej=4j=4at the last step, deletion is again prohibited:

D6,3\\displaystyle D\_\{6,3\}=min⁡\(D5,2\+cost6,3,D5,3\+1,D6,2\+1\)\\displaystyle=\\min\(D\_\{5,2\}\+\\text\{cost\}\_\{6,3\},\\,D\_\{5,3\}\+1,\\,D\_\{6,2\}\+1\)=min⁡\(1\.00,1\.67,2\.10\)=1\.00,\\displaystyle=\\min\(1\.00,\\,1\.67,\\,2\.10\)=\\mathbf\{1\.00\},D7,4\\displaystyle D\_\{7,4\}=min⁡\(D6,3\+cost7,4,D6,4\+1\)\\displaystyle=\\min\(D\_\{6,3\}\+\\text\{cost\}\_\{7,4\},\\,D\_\{6,4\}\+1\)=min⁡\(1\.00,2\.04\)=1\.00\.\\displaystyle=\\min\(1\.00,\\,2\.04\)=\\mathbf\{1\.00\}\.
Consequently, the keyword “语音识别” is successfully matched to the hypothesis part “雨音的识别”, with a total alignment cost ofSWext=D7,4=1\.0\\texttt\{SW\}\_\{\\text\{ext\}\}=D\_\{7,4\}=1\.0\. The relatedness likelihood is then computed asRL=\(\|w\|−SWext\)/\|w\|=\(4−1\)/4=0\.75\\texttt\{RL\}=\(\|w\|\-\\texttt\{SW\}\_\{\\text\{ext\}\}\)/\|w\|=\(4\-1\)/4=0\.75\.