Physics-Distilled Neural Network enabled by Large Language Models for Manufacturing Process-Property Predictive Modeling

arXiv cs.LG Papers
knowledge-distillation large-language-models process-property-modeling manufacturing neural-network real-time-inference data-scarce

Summary

This paper proposes a novel framework that uses LLMs to extract analytical physics priors from scientific literature and distills them into a lightweight neural network for high-accuracy, real-time manufacturing process-property prediction, even with limited data.

arXiv:2606.11605v1 Announce Type: new Abstract: Predicting process-property relationships in manufacturing is often challenged by high experimental costs and the limited interpretability of complex 'black-box' models. This paper proposes a novel knowledge distillation framework designed to achieve high-accuracy predictions in data-scarce scenarios. The framework integrates analytical physics priors, which are systematically extracted from scientific literature via Large Language Models, into a privileged teacher model. We employ a Graph-Masked Attention layer to capture the complex physical dependencies among input variables showing strict setpoints or a combination of static and high-frequency temporal signatures. This privileged knowledge is distilled into a lightweight student predictor for inference. The feasibility and robustness of the framework are evaluated through a comprehensive experiment across five diverse manufacturing processes. To ensure statistical reliability, given the small dataset sizes, a repeated K-fold cross-validation technique is employed to quantify model stability and generalization. Results indicate that the proposed framework consistently achieves high predictive accuracy across all evaluated domains. Most importantly, the architecture demonstrates significant fault tolerance by maintaining robust predictive performance even in scenarios where LLM-derived analytical priors are suboptimal or incomplete. Furthermore, the student predictor achieves an inference frequency exceeding 6000 Hz, which facilitates real-time edge deployment on standard industrial hardware. This work provides a scalable solution for bridging the gap between theoretical physics and real-time industrial monitoring in data-limited environments.
Original Article
View Cached Full Text

Cached at: 06/11/26, 01:49 PM 

# Physics-Distilled Neural Network enabled by Large Language Models for Manufacturing Process-Property Predictive Modeling
Source: [https://arxiv.org/abs/2606.11605](https://arxiv.org/abs/2606.11605)
[View PDF](https://arxiv.org/pdf/2606.11605)

> Abstract:Predicting process\-property relationships in manufacturing is often challenged by high experimental costs and the limited interpretability of complex 'black\-box' models\. This paper proposes a novel knowledge distillation framework designed to achieve high\-accuracy predictions in data\-scarce scenarios\. The framework integrates analytical physics priors, which are systematically extracted from scientific literature via Large Language Models, into a privileged teacher model\. We employ a Graph\-Masked Attention layer to capture the complex physical dependencies among input variables showing strict setpoints or a combination of static and high\-frequency temporal signatures\. This privileged knowledge is distilled into a lightweight student predictor for inference\. The feasibility and robustness of the framework are evaluated through a comprehensive experiment across five diverse manufacturing processes\. To ensure statistical reliability, given the small dataset sizes, a repeated K\-fold cross\-validation technique is employed to quantify model stability and generalization\. Results indicate that the proposed framework consistently achieves high predictive accuracy across all evaluated domains\. Most importantly, the architecture demonstrates significant fault tolerance by maintaining robust predictive performance even in scenarios where LLM\-derived analytical priors are suboptimal or incomplete\. Furthermore, the student predictor achieves an inference frequency exceeding 6000 Hz, which facilitates real\-time edge deployment on standard industrial hardware\. This work provides a scalable solution for bridging the gap between theoretical physics and real\-time industrial monitoring in data\-limited environments\.

## Submission history

From: Hongyi Xu \[[view email](https://arxiv.org/show-email/69146324/2606.11605)\] **\[v1\]**Wed, 10 Jun 2026 03:05:42 UTC \(1,361 KB\)

Similar Articles

Natively Unlearnable Large Language Models

arXiv cs.LG

The paper proposes NULLs (Natively Unlearnable LLMs), a model class that isolates source-specific contributions in sparsely activated sinks while sharing backbone neurons, enabling clean unlearning of individual data sources without retraining and preserving general language capabilities.

Physics-Modeled Neural Networks

arXiv cs.LG

This paper introduces Dynamical Physics-Modeled Neural Networks (DynPMNNs), a continuous-time deep learning architecture where hidden layers are defined by ordinary differential equations. It presents a biologically inspired approach grounded in Reproducing Kernel Banach Spaces, demonstrating competitive performance on the California Housing dataset with fewer parameters than standard Neural ODEs.

Language Acquisition Device in Large Language Models

arXiv cs.CL

This paper proposes LAD-inspired pre-pretraining using a formal language called MP-Struct that encodes natural-language-like structures. It shows that this approach improves token efficiency and imparts human-like resistance to structurally implausible languages, challenging prior hypotheses about effective pre-pretraining languages.