A Fine-Tuned BERT Classifier for Personal-Letter Titles in Late-Ming and Early-Qing Collected Works

# A Fine-Tuned BERT Classifier for Personal-Letter Titles in Late-Ming and Early-Qing Collected Works
Source: [https://arxiv.org/html/2605.23103](https://arxiv.org/html/2605.23103)
\\setCJKmainfont

\[AutoFakeBold=true\]bsmi00lp\.ttf\\newCJKfontfamily\\cjkfbipaexm\.ttf

###### Abstract\.

I presentLepton \(Letter Prediction\), a fine\-tuned BERT classifier that predicts whether a title in a Classical Chinese*wenji*文集 table of contents is a personal letter \(書\) or a closely confusable preface \(序, particularly the farewell\-preface 送序\)\. Lepton fine\-tunesbert\-base\-chinese\([Devlin2019,](https://arxiv.org/html/2605.23103#bib.bib7)\)on 5,438 hand\-labeled wenji titles from thirty\-three late\-Ming and early\-Qing literati\. I’ve deployed the model111[https://huggingface\.co/cbdb/ClassicalChineseLetterClassification](https://huggingface.co/cbdb/ClassicalChineseLetterClassification)on Hugging Face and has been used at the China Biographical Database \(CBDB\) to identify approximately fifty\-five thousand letters across mid\-Ming through early\-Qing wenji, populating the Ming Letter Platform\.

Classical Chinese, BERT, text classification, digital humanities, China Biographical Database, NLP

## 1\.Introduction

The wenji is the canonical form in which a Chinese literatus’s lifetime output reaches the modern reader\. Its table of contents is a curated, genre\-mixed list assembled by an editor \(often the author’s student, son, or grandson\) who decided which texts merited preservation and under what generic heading\. For ten thousand wenji, that curated structure becomes the problem: a corpus no individual can read but that contains, in its title\-level metadata, exactly the genre signal that would make it tractable\.

The class of false positives that matters most is not poetry or memorials but prefaces \(*xu*序\), and the genre signal a regex would lean on turns out to be unreliable in both directions\. Only 9 % of letter titles in my labeled corpus end with the character 書; the rest end with\\cjkfb啓, with a recipient title, or with no terminal genre marker at all\. Letters are identifiable not by their final character but by their opening verb — 與, 答 \(and the orthographic variants 荅 and 畣\), 復, 報, 寄, 上, 奉 — and these verbs also appear at the head of preface titles\. A regex on titles ending in 書 misses 91 % of letters; a regex on the opening verb sweeps up prefaces and requires hand\-disambiguation\.

Lepton \(Letter Prediction\)is a fine\-tuned BERT classifier that predits whether a title denotes a personal letter \(書\) or a closely confusable preface \(序, particularly the farewell\-preface 送序\)\. It is finetuned on 3,206 letter titles and 2,232 preface titles from thirty\-three late\-Ming through early\-Qing literati\. The model is open\-source, deployed on Hugging Face, and has been used at the China Biographical Database \(CBDB\) to identify approximately fifty\-five thousand letters across a corpus of mid\-Ming through early\-Qing wenji, populating what the team calls the Ming Letter Platform\.

Section[2](https://arxiv.org/html/2605.23103#S2)places Lepton in three literatures: Classical Chinese pretrained language models, computational text classification, and the CBDB ecosystem\. Section[3](https://arxiv.org/html/2605.23103#S3)frames the central philological question, such as “what counts as a letter title?”, and reports the annotation decisions\. Section[4](https://arxiv.org/html/2605.23103#S4)describes the corpus and training pipeline and evaluates Lepton against regex and TF–IDF baselines on a held\-out test set\. Section[5](https://arxiv.org/html/2605.23103#S5)discusses what the model attends to, what it does not do, and where it is likely to fail beyond its training distribution\.

## 2\.Background and related work

Lepton sits at the intersection of two literatures: pretrained language models for Classical Chinese, and the digital humanities tradition of comparative evaluation on historical text\. By Lepton’s deployment \(November 2022\), three domain\-adapted Classical\-Chinese backbones were available — AnchiBERT\([Tian2020,](https://arxiv.org/html/2605.23103#bib.bib19)\), GuwenBERT\([Yan2021,](https://arxiv.org/html/2605.23103#bib.bib14)\), and SikuBERT / SikuRoBERTa\([Wang2022,](https://arxiv.org/html/2605.23103#bib.bib13)\); SikuGPT\([Chang2023,](https://arxiv.org/html/2605.23103#bib.bib4)\)arrived later\. Their published benchmarks target sentence\-level tasks \(word segmentation, punctuation, NER, POS\) rather than title\-level genre classification, so none offers a direct point of comparison\. I fine\-tunebert\-base\-chinese\([Devlin2019,](https://arxiv.org/html/2605.23103#bib.bib7)\)directly; §[4](https://arxiv.org/html/2605.23103#S4)gives the rationale\.

The closest digital humanities precedent on Classical Chinese is Broadwell, Chen and Shepard\([Broadwell2019,](https://arxiv.org/html/2605.23103#bib.bib3)\), who apply topic modeling and Jensen–Shannon divergence to the*Quan Tang shi*\. Beyond Chinese, work on historical\-text classification — Croatian census records\([Lauc2021,](https://arxiv.org/html/2605.23103#bib.bib10)\), Finnish OCR\([Kettunen2017,](https://arxiv.org/html/2605.23103#bib.bib9)\), ancient inscriptions\([Tagami2023,](https://arxiv.org/html/2605.23103#bib.bib18)\), eighteenth\-century Austrian newspapers\([Resch2023,](https://arxiv.org/html/2605.23103#bib.bib16)\)— establishes the comparative\-evaluation and annotation\-as\-construction expectations that Sections[4\.3](https://arxiv.org/html/2605.23103#S4.SS3)and[3](https://arxiv.org/html/2605.23103#S3)are written to meet\.

## 3\.What counts as a “letter title”?

In Classical Chinese collected works \(*wenji*文集\), letters circulate under several near\-synonymous genre labels:*shu*書 is the central case, with*qi*\\cjkfb啓,*jian*牋 / 箋,*tie*帖, and*die*牒 as adjacent boundary cases\. In the labeled corpus of 3,206 letter titles, the operative signal is at the*start*of the title rather than the end: only 9\.2 % end in 書, while a verb of transmission in the opening position characterises the class \(Table[1](https://arxiv.org/html/2605.23103#S3.T1)\)\.

Table 1\.Opening\-verb distribution in the 3,206 letter titles\.The first eight verbs cover 82 % of letter titles, and three of them — 答, 荅, 畣 — are orthographic variants of “reply” that a character\-level model handles automatically where a regex would have to enumerate each form\. The same character\-variation pattern runs through the preface side \(序 89 %, 敘 3\.4 %,\\cjkfb叙 1\.8 %, 引 2\.6 %\)\. A typical letter title reads\[verb\] \+ \[recipient\], with a terminal genre marker optional\.

Distinguishing letters from poetry, memorials, or inscriptions is straightforward; the hard case is the preface \(*xu*序\), particularly the occasion\-prefaces —*songxu*送序,*shouxu*壽序,*zengxu*贈序 — which like letters name a recipient and inhabit the same social\-network slot in CBDB’s relational schema, but encode a public tribute rather than a one\-to\-one address\. The opening syntax can be near\-identical; the discriminating signal is the verb \(答 vs\. nothing\) and the terminal marker \(書 vs\. 序\)\.

The labeled corpus contains 3,206 letter titles and 2,232 preface titles from thirty\-three mid\-Ming through early\-Qing literati, each row carrying CBDB metadata \(writer ID, recipient ID where known, relationship code, wenji, juan\)\. Lepton learns the binaryshu = 1, xu = 0; §[4](https://arxiv.org/html/2605.23103#S4)details corpus construction\.

### 3\.1\.What Lepton is and is not classifying

Lepton does*not*identify recipients, extract dates, distinguish private letters \(*sishu*私書\) from official correspondence \(*gongshu*公書\), or classify from bodies\. Three further limits:

- •Dynastic\.Trained on roughly 1550–1700; behaviour on Song or Yuan material is uncharacterised\.
- •Generic\.The negative class is the preface, not arbitrary non\-letter text; poetry titles and inscriptions are out\-of\-distribution\.
- •Architectural\.The binary choice is by design: building around the*one*false\-positive class that matters produces a sharper interpretable contrast than a flatter scheme\([Resch2023,](https://arxiv.org/html/2605.23103#bib.bib16)\)\.

## 4\.Methods and evaluation

### 4\.1\.The labeled corpus

For each of the thirty\-three authors, an annotator located the collection in*Lidai bieji ku*歷代別集庫,*Siku xilie shujuku*四庫系列數據庫, or*Zhongguo jiben guji ku*中國基本古籍庫 V7\.0, opened the table of contents \(目錄\), identified the juan containing letters and prefaces, and scraped or hand\-entered the titles\. Each row carries CBDB metadata: writer ID, recipient ID where named, relationship code \(e\.g\.致書Y,答Y書\), wenji text ID, and juan number\. The distribution across authors is uneven — Feng Mengzhen 馮夢禎 \(729\), Yuan Hongdao 袁宏道 \(528\), Qian Qianyi 錢謙益 \(487\), Zou Yuanbiao 鄒元標 \(470\), and Tu Long 屠隆 \(415\) account for 47 % of the data\.

### 4\.2\.Architecture and training

Lepton fine\-tunesbert\-base\-chinese\([Devlin2019,](https://arxiv.org/html/2605.23103#bib.bib7)\)\(∼\\sim102M parameters\) with aBertForSequenceClassificationhead over the\[CLS\]token; all parameters are updated\.

I chose a modern\-Chinese rather than a domain\-adapted Classical\-Chinese backbone for two reasons\. Empirically, the discriminating characters — the opening verbs and terminal markers of Table[1](https://arxiv.org/html/2605.23103#S3.T1)— are well\-represented in modern\-Chinese pretraining data, so I expected the checkpoint to transfer to a title\-level signal carried by shared characters rather than pre\-modern syntax\. At the time Lepton was trained \(2020\),bert\-base\-chinesewas the mature widely\-available Chinese BERT; domain\-adapted alternatives were not yet a low\-friction option\.

### 4\.3\.Test set, baselines, and metrics

I stratified the 5,438\-row labeled corpus into an 80 / 10 / 10 train / dev / test split \(seed=42, class balance preserved\)\. I evaluated:

1. \(1\)Majority class: predict “letter” for every title\.
2. \(2\)Regex \(ends with 書\): the naive historian’s baseline\.
3. \(3\)Regex \(starts with a verb of transmission\): titles whose first character is one of 與, 答, 荅, 畣, 柬, 復, 報, 又, 上, 寄, 奉, 再, 致\.
4. \(4\)Regex disjunction: starts\-with\-verb OR ends\-with\-書\.
5. \(5\)TF–IDF character n\-grams \+ logistic regression: with several n\-gram ranges,min\_df=2,class\_weight=’balanced’\.
6. \(6\)Lepton \(deployed\): the Hugging Face checkpoint \(fine\-tunedbert\-base\-chinese\)\.

I report accuracy, letter\-class precision / recall / F1, and macro\-F1\. I also report a confusion matrix because the class imbalance \(59 % letter\) makes accuracy a misleading single number\.

### 4\.4\.Results

Table 2\.System comparison on the held\-out test set\. The deployed Lepton row is evaluated on the matched subset for which deployment\-time predictions are available\.Three observations: First, the two regex baselines that rely on the terminal 書 marker fail decisively \(F1 = 0\.166 and 0\.150\)\. This confirms Section[3](https://arxiv.org/html/2605.23103#S3)’s observation: a\*書regex retrieves under one in ten letters\. Second, the opening\-verb regex reaches F1 = 0\.915 with near\-perfect precision \(0\.989\)\. This is the strong historian\-without\-ML baseline; any neural contribution has to clear this number, not majority class\. Addingverb\-start OR 書\-endraises F1 to 0\.920\. Third, the TF–IDF character n\-gram baselines \(ranges\(2,2\)\(2,2\)through\(2,4\)\(2,4\)\) cluster tightly at F1 = 0\.935–0\.937 with high recall \(≈0\.99\{\\approx\}\\,0\.99\) but lower precision \(≈0\.89\{\\approx\}\\,0\.89\), reflecting the same opening\-verb signal the regex picks up plus some additional discrimination from short character spans\.

The deployed Lepton checkpoint attainsF1 = 0\.984on the matched test subset — the strongest of any system reported here, with perfect precision on the letter class and a small recall gap that § 5\.3 unpacks\.

### 4\.5\.Error analysis

Lepton’s six errors on the matched test subset are all of one kind: very short, elliptical letter titles where the opening\-verb signal is absent because the title is the recipient’s name or a continuation marker:

- •*Mei yan*寐言 \(2 chars; Gu Xiancheng 顧憲成\);*Bo Xiu*伯修 \(2 chars, recipient’s*zi*; Yuan Hongdao 袁宏道\);*Yu Xi lu*諭西虜 \(3 chars; Sun Chengzong 孫承宗\);*Gu Sheng\-bo xiuzhuan*顧升伯修撰 \(Yuan Hongdao\);*Yu tie*諭帖 \(Sun Chengzong\)\.

Lepton, having learned what a canonical letter title “looks like” from the labeled distribution, defaults to preface on these out\-of\-template inputs\. The error mode is coherent that it falls along the axis the model was trained to discriminate, but it shows the limit of title\-surface classification: the six recipients above are letters because a Ming or Qing editor said so, not because their titles share the genre signal Section[3](https://arxiv.org/html/2605.23103#S3)names\.

## 5\.Discussion

The TF–IDF baselines’ top\-weighted features — 與, 答, 荅, 柬, 復, 報 on the letter side and 序, 敘, 引,\\cjkfb叙 on the preface side — are exactly the characters §[3](https://arxiv.org/html/2605.23103#S3)names, and Lepton’s perfect letter\-class precision on the matched subset suggests the deployed model has internalized the same signal robustly enough to admit no false positives\. The convergence between the linear baselines and the philological account meets Dobson’s\([Dobson2021,](https://arxiv.org/html/2605.23103#bib.bib8)\)call for ML to expose features rather than scores\.

In deployment, Lepton has identified roughly fifty\-five thousand letters across mid\-Ming through early\-Qing wenji, populating the Ming Letter Platform that now anchors social\-network research on the period\. Beyond this use case, the work contributes to the digital humanities community an open\-source classifier and labeled corpus for Classical Chinese epistolary\-genre identification, a worked example of modern\-Chinese pretraining transferring to a title\-level pre\-modern task without continued pretraining, and evidence that interpretable n\-gram baselines sit within striking distance of neural models on signal\-rich short\-text classification\.

###### Acknowledgements\.

I thank Katherine Enright, Hongsu Wang, Peter Bol for the support of this project\. This project is conducted under the China Biographical Database \(CBDB\) Group at Harvard University, whose data, tooling, and collective expertise made the work possible\.

## References

- \(1\)David M\. Berry\. 2023\.The Explainability Turn\.*Digital Humanities Quarterly*17, 2 \(2023\)\.
- \(2\)Peter K\. Bol\. 2007\.Creating a GIS for the History of China\.In*Placing History: How Maps, Spatial Data, and GIS Are Changing Historical Scholarship*, ESRI Press, 27–59\.
- \(3\)George Aaron Broadwell, Jack W\. Chen, and Jeffrey R\. Tharsen\. 2019\.Reading the*Quan Tang shi*: Literary History, Topic Modeling, Divergence Measures\.*Digital Humanities Quarterly*13, 4 \(2019\)\.
- \(4\)Liu Chang, Wang Dongbo, Zhao Zhixiao, Hu Haotian, Liu Jiangfeng, Lu Si, Si Shen, and Liu Liu\. 2023\.SikuGPT: A Generative Pre\-trained Model for Intelligent Information Processing of Ancient Texts\.arXiv:2304\.07778\.
- \(5\)Hilde De Weerdt, Brent Ho, and Wing Kong Hou\. 2016\.MARKUS: Text Analysis and Reading Platform\.[https://dh\.chinese\-empires\.eu/markus/](https://dh.chinese-empires.eu/markus/)\.
- \(6\)Hilde De Weerdt and Csaba Oláh Horváth \(eds\.\)\. 2023\.Special issue on East Asian digital humanities\.*International Journal of Digital Humanities*\.
- \(7\)Jacob Devlin, Ming\-Wei Chang, Kenton Lee, and Kristina Toutanova\. 2019\.BERT: Pre\-training of Deep Bidirectional Transformers for Language Understanding\.In*NAACL\-HLT*, 4171–4186\.
- \(8\)James E\. Dobson\. 2021\.Vector hermeneutics: On the interpretation of vector space models of text\.*Digital Scholarship in the Humanities*37, 1 \(2021\), 81–93\.
- \(9\)Kimmo Kettunen, Eetu Mäkelä, Teemu Ruokolainen, Juha Kuokkala, and Laura Löfberg\. 2017\.Old Content and Modern Tools — Searching Named Entities in a Finnish OCRed Historical Newspaper Collection 1771–1910\.*Digital Humanities Quarterly*11, 3 \(2017\)\.
- \(10\)Davor Lauc and Darko Vitek\. 2021\.Inferring Standard Name Form, Gender and Nobility from Historical Texts Using Stable Model Semantics\.*Digital Humanities Quarterly*15, 1 \(2021\)\.
- \(11\)Queenie Luo and Yung\-Sung Chuang\. 2024\.Cleansing Jewel: A Neural Spelling Correction Model Built on Google OCR\-ed Tibetan Manuscripts\.*ACM Transactions on Asian and Low\-Resource Language Information Processing*23, 5, Article 73 \(May 2024\), 11 pages\.[https://doi\.org/10\.1145/3654811](https://doi.org/10.1145/3654811)\.
- \(12\)Donald Sturgeon\. 2021\.Chinese Text Project: A Dynamic Digital Library of Premodern Chinese\.*Digital Scholarship in the Humanities*36, Suppl\. 1 \(2021\), i101–i112\.
- \(13\)Dongbo Wang, Chang Liu, Zhixiao Zhao, Si Shen, Liu Liu, Bin Li, Haotian Hu, Mengcheng Wu, Litao Lin, Xue Zhao, and Xiyu Wang\. 2022\.Construction and Application of Pre\-trained Models of Siku Quanshu in Orientation to Digital Humanities \(SikuBERT and SikuRoBERTa\)\.*Library Tribune*42, 6 \(2022\), 31–43\.
- \(14\)Ethan Yan\. 2021\.GuwenBERT: A Pre\-trained Language Model for Classical Chinese\.[https://github\.com/Ethan\-yt/guwenbert](https://github.com/Ethan-yt/guwenbert)\.
- \(15\)J\. Berenike Herrmann, Simone Rebora, Massimo Salgaro, and others\. 2023\.Tool criticism in computational literary studies\.*Digital Humanities Quarterly*\.
- \(16\)Claudia Resch, Daniela Fasching Rastinger, and Maria Kirchmair\. 2023\.Building an Iterative Annotation Schema for Eighteenth\- Century Austrian Newspapers\.*Digital Humanities Quarterly*\.
- \(17\)Carolyn Strange, Daniel McNamara, Josh Wodak, and Ian Wood\. 2014\.Mining for the Meanings of a Murder: The Impact of OCR Quality on the Use of Digitized Historical Newspapers\.*Digital Humanities Quarterly*8, 1 \(2014\)\.
- \(18\)Shinya Tagami and Michael L\. Satlow\. 2023\.Dating Ancient Inscriptions with Machine Learning\.*Digital Humanities Quarterly*\.
- \(19\)Huishuang Tian, Kexin Yang, Dayiheng Liu, and Jiancheng Lv\. 2020\.AnchiBERT: A Pre\-Trained Model for Ancient Chinese Language Understanding and Generation\.arXiv:2009\.11473\.