Who Gets a Token, and What Does It Carry? Unequal Name Support and Concept Access in Large Language Models

Hugging Face Daily Papers Papers
large-language-models tokenization name-bias lexical-access fairness ai-ethics

Summary

The paper shows that unequal single-token support for names in large language models leads to biased concept accessibility across demographics, and introduces NameTrace to measure lexical comparability.

Names are personal identifiers, but they also carry social meaning and are widely used to evaluate how language models treat different people. Such evaluations typically assume that matched names are comparable model inputs. We show that this assumption often fails at the lexical interface: matched names are not necessarily matched inputs. Some names receive direct single-token access, while others are assembled from multiple subwords, creating unequal name-surface support. Across nearly half a million first names and 12 LLM-associated tokenizers, direct lexical access is highly selective, model dependent, and uneven across race- and gender-associated name metadata. We introduce NameTrace, a model-native, fine-grained, pre-behavioral framework for measuring whether unequal name-surface support remains a vocabulary property or becomes visible in task-relevant internal representations. NameTrace measures concept accessibility from the model's own probabilities over task-specific adjective axes with continuous task-aligned weights. On matched atomic and short-fragmented names within the same race/ethnicity--gender-associated strata, support predicts systematic differences in concept accessibility across fellowship, hiring, clinical assessment, and lending. These differences persist across all eight matched strata, extend across model families, and transfer to unseen names. Hidden-state interventions further show that the measured task directions have downstream leverage, shifting later constrained choices. Unequal lexical support is therefore demographically structured at the input and remains visible in task-relevant model computation. NameTrace makes lexical comparability measurable, supporting a broader principle: behavioral comparability begins with lexical comparability.
Original Article
View Cached Full Text

Cached at: 09/29/26, 08:12 PM

Paper page - Who Gets a Token, and What Does It Carry? Unequal Name Support and Concept Access in Large Language Models

Source: https://huggingface.co/papers/2609.34065

Abstract

Namesarepersonalidentifiers,buttheyalsocarrysocialmeaningandarewidelyusedtoevaluatehowlanguagemodelstreatdifferentpeople.Suchevaluationstypicallyassumethatmatchednamesarecomparablemodelinputs.Weshowthatthisassumptionoftenfailsatthelexicalinterface:matchednamesarenotnecessarilymatchedinputs.Somenamesreceivedirectsingle-tokenaccess,whileothersareassembledfrommultiplesubwords,creatingunequalname-surfacesupport.Acrossnearlyhalfamillionfirstnamesand12LLM-associatedtokenizers,directlexicalaccessishighlyselective,modeldependent,andunevenacrossrace-andgender-associatednamemetadata.WeintroduceNameTrace,amodel-native,fine-grained,pre-behavioralframeworkformeasuringwhetherunequalname-surfacesupportremainsavocabularypropertyorbecomesvisibleintask-relevantinternalrepresentations.NameTracemeasuresconceptaccessibilityfromthemodel’sownprobabilitiesovertask-specificadjectiveaxeswithcontinuoustask-alignedweights.Onmatchedatomicandshort-fragmentednameswithinthesamerace/ethnicity--gender-associatedstrata,supportpredictssystematicdifferencesinconceptaccessibilityacrossfellowship,hiring,clinicalassessment,andlending.Thesedifferencespersistacrossalleightmatchedstrata,extendacrossmodelfamilies,andtransfertounseennames.Hidden-stateinterventionsfurthershowthatthemeasuredtaskdirectionshavedownstreamleverage,shiftinglaterconstrainedchoices.Unequallexicalsupportisthereforedemographicallystructuredattheinputandremainsvisibleintask-relevantmodelcomputation.NameTracemakeslexicalcomparabilitymeasurable,supportingabroaderprinciple:behavioralcomparabilitybeginswithlexicalcomparability.

View arXiv pageView PDFProject pageGitHub0Add to collection

Get this paper in your agent:

hf papers read 2609\.34065

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.34065 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.34065 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.34065 in a Space README.md to link it from this page.

Collections including this paper1

Similar Articles

Apples to Apples? Towards Comparable Crosslingual Language Model Evaluation

arXiv cs.CL

This paper investigates the fairness of crosslingual evaluation methods for language models, showing that common normalized metrics can be biased due to tokenization and orthographic differences, and proposes using sentence-level negative log likelihood on semantically equivalent sequences for more consistent crosslingual comparisons.

Language Models are not Equally Robust to Non-Canonical Tokenization across Languages

arXiv cs.CL

This paper investigates whether language models remain robust to alternative (non-canonical) tokenizations across 27 languages, finding that invariance observed in English does not generalize and that languages with higher token fragmentation show greater sensitivity. The authors demonstrate that LoRA fine-tuning with multi-tokenization data can mitigate this sensitivity.

Byte-level models

Reddit r/LocalLLaMA

Discusses whether byte-level tokenizers outperform subword tokenizers for precise tasks like distinguishing similar names, counting characters, and case sensitivity, and asks for current recommendations.