Labeling Training Data for Entity Matching Using Large Language Models

arXiv cs.CL Papers
entity-matching knowledge-distillation large-language-models training-data-labeling benchmark

Summary

This paper investigates using LLMs as teacher models to label training data for entity matching, showing that student models trained on machine-labeled data perform on par with those trained on manually labeled benchmarks, with significant cost and speed advantages.

arXiv:2606.28823v1 Announce Type: new Abstract: Recent large language models (LLMs) achieve strong performance on entity matching without requiring task-specific training data. However, applying these models to large sets of candidate pairs remains slow and costly. In contrast, entity matchers using traditional machine learning methods or small language models (SLMs), such as RoBERTa, offer much faster inference but require task-specific training data. This paper investigates whether the need to provide task-specific training data can be avoided by using knowledge-distillation workflows, in which an LLM serves as a teacher model to label training pairs that are subsequently used to train a smaller student model. We investigate knowledge distillation for entity matching along the following dimensions: pair-selection strategy, teacher model, label post-processing method, and student model. We evaluate the workflows using the Abt-Buy, Walmart-Amazon, WDC Products, DBLP-ACM, and DBLP-Scholar benchmarks, and compare the performance of student models trained with machine-labeled data to the performance of the same models trained using the benchmark training sets. Our experiments show that student models trained using the machine-labeled sets perform approximately on par with models trained on the benchmark training sets, with the remaining differences in both directions staying below two F1 points. Using GPT-5.2 to label the training sets for all five benchmarks costs US\$28.31 to US\$40.88, whereas manually labeling the same training sets is estimated to require 470 hours of work. At inference time, Ditto is 41.5 to 534 times faster than directly using an LLM to perform the matching tasks. These results indicate that current LLMs, when combined with a suitable pair-selection method, can substantially reduce or even eliminate the manual effort required to label use case-specific training data for entity matching.
Original Article
View Cached Full Text

Cached at: 06/30/26, 05:28 AM 

# Labeling Training Data for Entity Matching Using Large Language Models
Source: [https://arxiv.org/abs/2606.28823](https://arxiv.org/abs/2606.28823)
[View PDF](https://arxiv.org/pdf/2606.28823)

> Abstract:Recent large language models \(LLMs\) achieve strong performance on entity matching without requiring task\-specific training data\. However, applying these models to large sets of candidate pairs remains slow and costly\. In contrast, entity matchers using traditional machine learning methods or small language models \(SLMs\), such as RoBERTa, offer much faster inference but require task\-specific training data\. This paper investigates whether the need to provide task\-specific training data can be avoided by using knowledge\-distillation workflows, in which an LLM serves as a teacher model to label training pairs that are subsequently used to train a smaller student model\. We investigate knowledge distillation for entity matching along the following dimensions: pair\-selection strategy, teacher model, label post\-processing method, and student model\. We evaluate the workflows using the Abt\-Buy, Walmart\-Amazon, WDC Products, DBLP\-ACM, and DBLP\-Scholar benchmarks, and compare the performance of student models trained with machine\-labeled data to the performance of the same models trained using the benchmark training sets\. Our experiments show that student models trained using the machine\-labeled sets perform approximately on par with models trained on the benchmark training sets, with the remaining differences in both directions staying below two F1 points\. Using GPT\-5\.2 to label the training sets for all five benchmarks costs US\\$28\.31 to US\\$40\.88, whereas manually labeling the same training sets is estimated to require 470 hours of work\. At inference time, Ditto is 41\.5 to 534 times faster than directly using an LLM to perform the matching tasks\. These results indicate that current LLMs, when combined with a suitable pair\-selection method, can substantially reduce or even eliminate the manual effort required to label use case\-specific training data for entity matching\.

## Submission history

From: Aaron Steiner \[[view email](https://arxiv.org/show-email/3d929592/2606.28823)\] **\[v1\]**Sat, 27 Jun 2026 09:15:09 UTC \(1,189 KB\)

Similar Articles

Observational Equivalence of LLM and Human Annotation

arXiv cs.CL

This paper demonstrates that large language models (LLMs) achieve annotation quality equivalent to human experts, suggesting the main challenge is minimizing ambiguity in coding rules rather than choosing between human and machine coders.