从转写到语义语料库分析:古代语言句子表征的无监督学习

arXiv cs.CL 论文

摘要

本文研究了将专用语言模型适配为面向特定语料库的古代语言句子编码器的无监督策略(TSDAE 和无监督 SimCSE),表明这些策略在拉丁语和古希腊语的圣经文本复用检测任务中优于基线方法,并将该处理流程作为 Paraphrasis 在线工具发布。

arXiv:2607.24542v2 Announce Type: replace Abstract: Automatic Text Recognition (ATR) now supplies digital humanities with large volumes of unstruc- tured, heterogeneous, and often noisy text in ancient languages. Downstream semantic analyses- text reuse identification, alignment, and semantic search-rely on sentence embeddings, yet ex- isting methods transfer poorly to ancient languages: generic multilingual encoders underperform, specialized language models yield anisotropic representation spaces, and labeled similarity data is scarce. We study two fully unsupervised strategies-TSDAE and unsupervised SimCSE (hence- forth CSE)-that adapt a specialized token-level language model into a corpus-specific sentence encoder using only raw sentences. We validate them on biblical reuse in patristic literature in Latin and Ancient Greek: 2,936 sentences carrying expert-verified reuses in works of Augustine, Jerome, and Athanasius, matched against the Vulgate, the Septuagint, and the Nestle--Aland New Testament. We decompose reuse identification into binary detection and correspondence retrieval, evaluate both tasks separately, and benchmark the adapted encoders against multilingual, specialized, distilled, and supervised fine-tuned baselines, on clean text and on text perturbed by simulated HTR errors and scribal abbreviations. On these test sets, the adapted encoders lead all baselines on both tasks, with complementary profiles: TSDAE leads detection given a large in-domain corpus, while CSE leads retrieval, reaches its optimum with as few as 4--8k raw in-domain sentences-a few tens of sec- onds of training on a laptop GPU-and transfers across works and authors. Under simulated noise, retraining CSE on the perturbed text itself outperforms a clean-trained encoder. Measurements of alignment, uniformity, and anisotropy relate the geometric effect of each strategy to the measured gains. The full pipeline-segmentation, fine-tuning, cross-corpus semantic search-is available to non-specialists through the online tool Paraphrasis.
查看原文
查看缓存全文

缓存时间: 2026/11/20 09:13

# From transcription to semantic corpus analysis: unsupervised learning of sentence representations for ancient languages
Source: [https://arxiv.org/abs/2607.24542](https://arxiv.org/abs/2607.24542)
[View PDF](https://arxiv.org/pdf/2607.24542)

> Abstract:Automatic Text Recognition \(ATR\) now supplies digital humanities with large volumes of unstruc\- tured, heterogeneous, and often noisy text in ancient languages\. Downstream semantic analyses\- text reuse identification, alignment, and semantic search\-rely on sentence embeddings, yet ex\- isting methods transfer poorly to ancient languages: generic multilingual encoders underperform, specialized language models yield anisotropic representation spaces, and labeled similarity data is scarce\. We study two fully unsupervised strategies\-TSDAE and unsupervised SimCSE \(hence\- forth CSE\)\-that adapt a specialized token\-level language model into a corpus\-specific sentence encoder using only raw sentences\. We validate them on biblical reuse in patristic literature in Latin and Ancient Greek: 2,936 sentences carrying expert\-verified reuses in works of Augustine, Jerome, and Athanasius, matched against the Vulgate, the Septuagint, and the Nestle\-\-Aland New Testament\. We decompose reuse identification into binary detection and correspondence retrieval, evaluate both tasks separately, and benchmark the adapted encoders against multilingual, specialized, distilled, and supervised fine\-tuned baselines, on clean text and on text perturbed by simulated HTR errors and scribal abbreviations\. On these test sets, the adapted encoders lead all baselines on both tasks, with complementary profiles: TSDAE leads detection given a large in\-domain corpus, while CSE leads retrieval, reaches its optimum with as few as 4\-\-8k raw in\-domain sentences\-a few tens of sec\- onds of training on a laptop GPU\-and transfers across works and authors\. Under simulated noise, retraining CSE on the perturbed text itself outperforms a clean\-trained encoder\. Measurements of alignment, uniformity, and anisotropy relate the geometric effect of each strategy to the measured gains\. The full pipeline\-segmentation, fine\-tuning, cross\-corpus semantic search\-is available to non\-specialists through the online tool Paraphrasis\.

## Submission history

From: Theotime de la Selle \[[view email](https://arxiv.org/show-email/0932ca51/2607.24542)\] \[via CCSD proxy\] **[\[v1\]](https://arxiv.org/abs/2607.24542v1)**Mon, 27 Jul 2026 15:20:13 UTC \(6,202 KB\) **\[v2\]**Thu, 19 Nov 2026 13:27:36 UTC \(6,763 KB\)

相似文章

利用形态学进行历史文字计量分析

Hugging Face Daily Papers

本文提出了一种基于Transformer的架构,结合原型学习,仅利用行级转录即可从历史文档中进行可扩展的古文字测量,并在仅有少量训练数据的160页手抄本上证明了其有效性。

利用少量标注学习拉丁文学中文本复用的词级解释

arXiv cs.CL

该论文提出了一种以极少标注学习拉丁文学中文本复用的词级解释的方法,结合合成训练样本对与主动学习;在1,490条引用中,仅使用200对已标注样本,联合指针网络就达到了使用全部数据训练时的性能。

通过无监督学习改进语言理解

OpenAI Blog

OpenAI 提出了一种两阶段方法来改进语言理解:首先在大规模无监督数据集上使用语言建模对 transformer 模型进行预训练,然后在较小的有监督数据集上针对特定任务进行微调。该方法在包括常识推理、语义相似度和阅读理解在内的多种任务上取得了最先进的成果,同时需要的超参数调优工作最少。

无语义的语法:教会大语言模型用未见过的语言编程

arXiv cs.CL

本文介绍了PyLang,一种在所有预训练语料库中都不存在的编程语言,并表明在其上微调的大语言模型可以学习语法但无法迁移算法推理,导致出现“实现忠实度差距”——模型理解算法但无法用不熟悉的语言表达它们。

学习古希腊字母形态的历时表征

arXiv cs.LG

本文介绍了三个数据集(Hell-Char、PaLit-Char、Med-Char),用于古希腊字母形态的历时表征学习,并提出了一种基于相似性加权的有监督对比损失函数,结合空缺驱动增强方法,以鲁棒地学习跨越数百年手写变化的字符嵌入。