STAR-XPC: Set Transformer for Estimating X-ray Transmission in Polymer Composites

# STAR-XPC: Set Transformer for Estimating X-ray Transmission in Polymer Composites
Source: [https://arxiv.org/html/2610.28709](https://arxiv.org/html/2610.28709)
Journal:Computer Physics CommunicationsCan ErerEmail:[salih\.erer@std\.bogazici\.edu\.tr](mailto:[email protected])Affiliation:Department of Computer Engineering, Boğaziçi University, Bebek, 34342, Istanbul, TurkiyeH\. Birkan YilmazEmail:[birkan\.yilmaz@bogazici\.edu\.tr](mailto:[email protected])Corresponding author:Corresponding authorAffiliation:Department of Computer Engineering, Boğaziçi University, Bebek, 34342, Istanbul, Turkiye

###### Abstract

Evaluating the energy transmission ratio of an X\-ray beam through a polymer composite is a challenging task in radiation shielding studies\. The reference solution with Monte Carlo transport simulation is highly accurate but computationally expensive: each of our reference simulations is capped at4×1094\\times 10^\{9\}photon histories and a20 00020\\,000s \(over55hour\) time limit, while closed\-form analytical methods fail to represent such complex scenarios\. We present an artificial neural network model that maps the unordered multiset of molecular constituents, their mass fractions, and four bulk physical descriptors directly to the transmission ratioI⁡\(t\)/I0I\(t\)/I\_\{0\}in 1\-3 milliseconds\. The model is a Set Transformer, which makes it permutation invariant by construction and accepts mixtures of different sizes up to 11 constituents within a fixed\-size padded set\. Each constituent is embedded with a chemistry\-pretrained transformer language model, ChemBERTa\. In our proposed method, training data are produced by an automated PENELOPE/penEasy pipeline using the radiation source at3030,8080,100100, and120kV120\\text\{\\,\}\\mathrm\{kV\}tube voltages over polyethylene\-based shielding composite slabs of thicknesses spanning0\.150\.15\-2\.52\.5mm\. After preprocessing, the dataset containsn=4697n=4697simulated configurations on which a leakage\-free, jointly stratified 10\-fold cross\-validation is performed, attaining mean squared errorMSE=\(1\.12±0\.56\)×10−4\\mathrm\{MSE\}=\(1\.12\\pm 0\.56\)\\times 10^\{\-4\}, normalized\-MSE percentageNMSE%=0\.0294%±0\.0147%\\mathrm\{NMSE\\%\}=0\.0294\\%\\pm 0\.0147\\%, andR2=0\.9988R^\{2\}=0\.9988\. The corresponding per\-sample percentage errors give anα\\alpha\-trimmed \(α=0\.10\\alpha=0\.10\) absolute percentage error \(APE\) of1\.66%1\.66\\%and a median APE of1\.15%1\.15\\%\. Our model’s inference is several orders of magnitude faster than the underlying Monte Carlo simulation \(sub\-millisecond per query vs\. several hours for the reference run\) with effective accuracy in predicting theI⁡\(t\)/I0I\(t\)/I\_\{0\}\.

###### Keywords:

Radiation attenuation , Polymer composites , Set Transformer , Monte Carlo , PENELOPE software , ChemBERTa

## 1Introduction

Calculating how a solid reduces the intensity of an X\-ray beam is one of the most crucial tasks for those who work in diagnostic imaging, image guided radiotherapy, industrial radiography and the engineering of radiation shields in medical or nuclear fields\. In practical applications, material designers rarely work with a single material, indeed they mostly work with composites, and polymer\-matrix composites loaded with high\-ZZfillers such as Bi2O3, WO3, or BaSO4are an active area of radiation\-shielding research in their own right\[[27](https://arxiv.org/html/2610.28709#bib.bib20),[19](https://arxiv.org/html/2610.28709#bib.bib21),[3](https://arxiv.org/html/2610.28709#bib.bib23),[4](https://arxiv.org/html/2610.28709#bib.bib22)\]\.

There are three characteristics that actually make this task complex and non\-trivial\. First, the composites consist of different constituents, which we call being*heterogeneous*: a thin slab contains molecules with interaction cross sections that differ by orders of magnitude across the kilovoltage range\. Secondly, the source emits many wavelengths at once, which we call being*polychromatic*: when we choose an energy level between 30\-120kV120\\text\{\\,\}\\mathrm\{kV\}the distribution belonging to that energy level varies a lot\. Lastly, there are various choices for possible designs, which we call being*combinatorial*: a mix of materials may consist of arbitrary materials and may have arbitrary thickness levels\. Due to the vast design space, engineers must test thousands of candidates in an optimization loop\. Hence, engineers need accurate and reliable methods to know how much energy passes through the given material based on its physical properties such as its shape and constituents\.

For heterogeneous mixtures and polychromatic energy spectrum, there is no closed\-form analytical solution\. Mass attenuation mixture rules evaluated with tabulated data, such as NIST’s X\-ray mass attenuation tables \(SRD 126, a\.k\.a\. XAAMDI\)\[[16](https://arxiv.org/html/2610.28709#bib.bib16)\]or cross\-section tools such as XCOM\[[8](https://arxiv.org/html/2610.28709#bib.bib29)\]and its derivatives WinXCom\[[13](https://arxiv.org/html/2610.28709#bib.bib30)\]and Phy\-X/PSD\[[31](https://arxiv.org/html/2610.28709#bib.bib31)\], are fast but inherit two restrictive assumptions: narrow\-beam geometry and a monoenergetic source\. In broad\-beam geometry, scattered photons reaching the detector must be accounted for with empirical buildup factors\[[14](https://arxiv.org/html/2610.28709#bib.bib32)\], which are tabulated for specific materials and energies and are not readily available for arbitrary multi\-component composites\. XAAMDI tabulates mass attenuation coefficients by mass\-weighting constituent elements at a single photon energy, which works only if the material is treated as one uniform blend\. Moreover, it gives one coefficient per energy, not a spectrum response, so using it for a 30\-120 kV polychromatic beam means manually querying and integrating many energy points, with no account for how the spectrum hardens as it penetrates the material\.

PENELOPE/penEasy\[[33](https://arxiv.org/html/2610.28709#bib.bib1),[34](https://arxiv.org/html/2610.28709#bib.bib2)\]uses Monte Carlo methods in order to simulate the interaction utilizing the physical laws\. The estimations of those codes are accurate, they are used as a reference to illustrate how radiation attenuates throughout the given material with respect to its physical properties\. The data they have and the algorithms used are extensively tested and concluded to be reliable\. However, their cost is prohibitive for use in an optimization loop\. Each reference run is configured with a nominal cap of4×1094\\times 10^\{9\}photon histories, a2%2\\,\\%relative\-uncertainty target, and a20 00020\\,000s time limit\. In practice, the time limit is the binding constraint for most configurations, so the number of histories actually completed, and the resulting precision, vary by configuration rather than uniformly reaching the nominal targets\. The statistical error decays only asσ∝M−1/2\\sigma\\propto M^\{\-1/2\}in the number of histories\. The main problem is obtaining a reliable estimation result requires a lot of computational power, hence it is not possible to use these codes to simulate all combinations in our design space\.

In short, current limitations mainly stem from the computational cost of running Monte Carlo simulations and the fact that we do not have an analytical solution for direct calculation\. The capabilities of classical machine learning \(ML\) methods can be considered as secondary limitation\.

The attenuation of a homogeneous mixture does not depend on the order of its constituents, so the target is a permutation\-invariant function of the constituent set\. Multi\-layer perceptrons over concatenated slots and sequence\-based models impose an artificial ordering and must learn this invariance from data, spending parameters and samples on repeating the same mapping for every ordering\. Tree\-based regressors share this problem once the constituents are flattened into fixed columns, and they treat each embedding dimension as an independent feature, which suits dense embeddings poorly\. A canonical ordering, e\.g\. by mass fraction, restores invariance but is arbitrary and changes abruptly when two fractions cross\. We therefore build the invariance into the architecture\.

This paper proposes a permutation\-invariant set transformer for estimating X\-ray transmission in polymer composites, namely STAR\-XPC\. Our proposed method treats a composite as an unordered set of tokens enriched with a vector of bulk physical descriptors, and predicts the energy transmission ratio\. The contributions of this work are:

1. 1\.
2. 2\.A chemistry\-aware embedding pipeline that summarizes each constituent with the attention\-mask\-weighted mean of the last four hidden layers of ChemBERTa\[[10](https://arxiv.org/html/2610.28709#bib.bib5)\]\(a 768\-dimensional vector\), followed byℓ2\\ell\_\{2\}normalization\. Aggregating the upper layers captures both syntactic and semantic features of the simplified molecular input line entry system \(SMILES\) string used in cheminformatics\.
3. 3\.A permutation\-invariant Set Transformer model based on Induced Set Attention Blocks \(ISAB\) and Pooling by Multihead Attention \(PMA\), which predicts the energy transmission ratio in 1\-3 milliseconds\. The STAR\-XPC model and inference codes are available at HuggingFace222[https://huggingface\.co/cmpe\-netlab41/STAR\-XPC](https://huggingface.co/cmpe-netlab41/STAR-XPC)\.

## 2Background Information

### 2\.1Radiation Attenuation Physics

For a narrow, monoenergetic photon beam of energyEEand intensityI0​\(E\)I\_\{0\}\(E\)traversing a homogeneous slab of thicknesstt, Beer Lambert law states that

I⁡\(t,E\)=I0​\(E\)​e−μ⁡\(E\)​t,I\(t,E\)=I\_\{0\}\(E\)\\,e^\{\-\\mu\(E\)\\,t\},\(1\)whereI⁡\(t,E\)I\(t,E\)is the transmitted intensity for the input thickness and energy,μ⁡\(E\)\\mu\(E\)is the linear attenuation coefficient for the input energy\[[5](https://arxiv.org/html/2610.28709#bib.bib17)\]\. For a multi\-component mixture, the standard mass\-additivity rule for the mass attenuation coefficient is

\(μ/ρ\)tot​\(E\)=∑i=1Nwi​\(μ/ρ\)i​\(E\),∑i=1Nwi=1,\\bigl\(\\mu/\\rho\\bigr\)\_\{\\mathrm\{tot\}\}\(E\)=\\sum\_\{i=1\}^\{N\}w\_\{i\}\\,\\bigl\(\\mu/\\rho\\bigr\)\_\{i\}\(E\),\\qquad\\sum\_\{i=1\}^\{N\}w\_\{i\}=1,\(2\)wherewiw\_\{i\}is the mass fraction of constituentii\. For a polychromatic source spectrumS⁡\(E\)S\\\!\(E\)incident on a slab of bulk densityρ\\rhoand thicknesstt, the integrated energy transmission ratio is

y⁡\(t\)≡I⁡\(t\)I0=∫0EmaxE​S​\(E\)​e−\(μ/ρ\)tot​\(E\)​ρ​t​𝑑E∫0EmaxE​S​\(E\)​𝑑Ey\(t\)\\;\\equiv\\;\\frac\{I\(t\)\}\{I\_\{0\}\}\\;=\\;\\frac\{\\displaystyle\\int\_\{0\}^\{E\_\{\\max\}\}E\\,S\\\!\(E\)\\,e^\{\-\(\\mu/\\rho\)\_\{\\mathrm\{tot\}\}\(E\)\\,\\rho\\,t\}\\,dE\}\{\\displaystyle\\int\_\{0\}^\{E\_\{\\max\}\}E\\,S\\\!\(E\)\\,dE\}\(3\)whereI⁡\(t\)I\(t\)is the transmitted intensity through a slab of thicknessttandI0I\_\{0\}is the incident intensity\. Equation \([3](https://arxiv.org/html/2610.28709#S2.E3)\) gives the narrow\-beam idealization, but our target is its Monte Carlo counterpart, which additionally captures scattered photons and characteristic X\-ray fluorescence reaching the scoring region\. is the target variable for our model, i\.e\. our model is trained to predict the energy transmission ratioy⁡\(t\)y\(t\)\.

In the kilovoltage range, X\-ray attenuation is governed by three interaction channels: photoelectric absorption \(τ∝Zn/Em\\tau\\propto Z^\{n\}/E^\{m\},n∈\[4,5\]n\\in\[4,5\]andm∈\[3,3\.5\]m\\in\[3,3\.5\]\), incoherent \(Compton\) scattering, and coherent scattering\[[5](https://arxiv.org/html/2610.28709#bib.bib17)\]\. At a fixed energy, the mass\-additivity rule \(Eq\. \([2](https://arxiv.org/html/2610.28709#S2.E2)\)\) is exact under the standard assumption that the atomic wavefunctions in the mixture are unaffected by the surrounding chemical or crystalline environment\. Combined with Beer\-Lambert’s law under narrow\-beam \(for a monoenergetic beam in narrow\-beam/scatter\-free\) geometry, this gives an exact analytic prediction of transmission\. However, two effects break this shortcut in practice: beam hardening reshapes the spectrumS⁡\(E\)S\\\!\(E\)along the depth coordinate as low\-energy photons are preferentially absorbed, and near an element’s K\-edge, the mass attenuation coefficient jumps sharply over a narrow energy window, so any closed\-form spectral integration must resolve this discontinuity precisely; moreover, the characteristic fluorescence photons emitted following K\-shell photoelectric absorption are transported and re\-absorbed within the surrounding matrix in a way no closed\-form correction captures\. These effects are jointly resolved via Monte Carlo simulation method\.

### 2\.2Monte Carlo Simulation Method

PENELOPE\[[33](https://arxiv.org/html/2610.28709#bib.bib1)\]\(PENetration and Energy LOss of Positrons and Electrons\) is a Fortran subroutine library that performs coupled photon\-electron\-positron transport in arbitrary materials and quadric geometries via a*mixed simulation*algorithm\. Hard interactions \(those with polar deflection or energy loss exceeding pre\-selected cutoff values\) are sampled individually, while soft interactions are condensed into a continuous\-slowing\-down approximation governed by the user\-controlled cutoff parameters\. PenEasy\[[34](https://arxiv.org/html/2610.28709#bib.bib2)\]is a structured general\-purpose main program that drives PENELOPE through a plain\-text input file and exposes a set of pre\-implemented source models, geometry handlers, tallies and variance reduction techniques\.

For a stationary, time\-independent source the photon angular fluxΦ⁡\(𝐫,E,𝛀\)\\Phi\(\\mathbf\{r\},E,\\boldsymbol\{\\Omega\}\)satisfies the linear Boltzmann transport equation

𝛀⋅∇Φ\+Σt​\(𝐫,E\)​Φ=∬Σs\(E′→E,𝛀′→𝛀\)×Φ⁡\(𝐫,E′,𝛀′\)​d​E′​d​𝛀′\+Q⁡\(𝐫,E,𝛀\),\\begin\{split\}\\boldsymbol\{\\Omega\}\\cdot\\nabla\\Phi\+\\Sigma\_\{t\}\(\\mathbf\{r\},E\)\\,\\Phi&=\\iint\\Sigma\_\{s\}\(E^\{\\prime\}\\\!\\to\\\!E,\\,\\boldsymbol\{\\Omega\}^\{\\prime\}\\\!\\to\\\!\\boldsymbol\{\\Omega\}\)\\\\ &\\qquad\\times\\Phi\(\\mathbf\{r\},E^\{\\prime\},\\boldsymbol\{\\Omega\}^\{\\prime\}\)\\,dE^\{\\prime\}\\,d\\boldsymbol\{\\Omega\}^\{\\prime\}\\\\ &\\qquad\+Q\(\\mathbf\{r\},E,\\boldsymbol\{\\Omega\}\),\\end\{split\}\(4\)whereΣt\\Sigma\_\{t\}is the macroscopic total cross\-section,Σs\\Sigma\_\{s\}the differential scattering kernel andQQthe volumetric source term\. PENELOPE samples particle histories from Eq\. \([4](https://arxiv.org/html/2610.28709#S2.E4)\) via the standard Markov\-process construction\.

Each history costs a fixed average time, soT∝M\{T\\\!\\propto\\\!M\}, while the statistical uncertainty of an estimator built fromMMindependent histories decays asσθ^∝M−1/2\\sigma\_\{\\hat\{\\theta\}\}\\propto M^\{\-1/2\}\. EliminatingMMyields,

T∝σθ^−2,T\\;\\propto\\;\\sigma\_\{\\hat\{\\theta\}\}^\{\-2\},\(5\)so halving the uncertainty quadruples the cost and, conversely, a10×10\\timesspeed\-up can only be bought at a3\.2×3\.2\\timeslarger uncertainty\. Our reference runs use up to4×1094\\times 10^\{9\}histories and take approximately∼\\sim5 CPU\-hours with almost all of them to reach a relative uncertainty of 2%\. No reduction ofMMbrings a converged transport run into the range required by an inner design\-optimisation loop, which is what motivates a learned ML model\.

### 2\.3Machine Learning for Physics Applications

ML modeling of costly simulators is a standard approach in computational physics\. Successful examples include deep operator networks that learn PDE solution operators directly from data\[[26](https://arxiv.org/html/2610.28709#bib.bib13),[23](https://arxiv.org/html/2610.28709#bib.bib14)\]\. The MC radiation transport literature is dominated by tabular regressors, with occasional deep\-learning exceptions in adjacent domains such as CT dose reconstruction\[[32](https://arxiv.org/html/2610.28709#bib.bib15)\]\. Within radiation shielding specifically, ANNs have been trained to predict mass or linear attenuation coefficients of glass systems\[[7](https://arxiv.org/html/2610.28709#bib.bib36)\]and of Bi2O3\-based and multi\-filler composites from Geant4\-generated data\[[2](https://arxiv.org/html/2610.28709#bib.bib37),[1](https://arxiv.org/html/2610.28709#bib.bib38)\]\. These models take a fixed\-length feature vector \(e\.g\. photon energy and density\) and predict a monoenergetic coefficient, whereas we predict the spectrum\-integrated transmission of a polychromatic tube beam directly from an unordered set of constituents\. None of these architectural preferences systematically encode the symmetry of a molecular mixture, the ordering of constituents in the input has no physical meaning\. The statistical error of an unbiased Monte Carlo estimator calculated using M histories decays at the standardM−1/2M^\{\-1/2\}rate, which makes high\-precision MC essentially costly and motivates the utilization of an ML model \(ML surrogate\)\.

An aggregate property of an*unordered multiset*of molecular constituents is what we are trying to estimate\. Deep learning on flattened tabular vectors does not fit this structure, since assigning constituents to fixed column positions makes the prediction depend on an arbitrary slot order, and recurrent networks would likewise introduce an ordering\. Padding a set to a fixed size is compatible with permutation invariance only if the padding tokens are interchangeable and the model is itself permutation invariant, as in our architecture \(Section[3\.3](https://arxiv.org/html/2610.28709#S3.SS3)\)\.

###### Definition 2\.1\(Permutation invariance\)\.

A functionffacting on a finite multiset𝒮=\{𝐱1,…,𝐱n\}⊂ℝd\\mathcal\{S\}=\\\{\\mathbf\{x\}\_\{1\},\\ldots,\\mathbf\{x\}\_\{n\}\\\}\\subset\\mathbb\{R\}^\{d\}is*permutation invariant*if, for every permutationπ∈𝔖n\\pi\\in\\mathfrak\{S\}\_\{n\},

f⁡\(\{𝐱1,…,𝐱n\}\)=f⁡\(\{𝐱π⁡\(1\),…,𝐱π⁡\(n\)\}\)\.f\\\!\\bigl\(\\\{\\mathbf\{x\}\_\{1\},\\ldots,\\mathbf\{x\}\_\{n\}\\\}\\bigr\)=f\\\!\\bigl\(\\\{\\mathbf\{x\}\_\{\\pi\(1\)\},\\ldots,\\mathbf\{x\}\_\{\\pi\(n\)\}\\\}\\bigr\)\.

While the Deep Sets\[[39](https://arxiv.org/html/2610.28709#bib.bib4)\]guarantees universality, its sum\-decomposition requires a latent dimension at least as large as the set cardinality, which can make the simple sum aggregator impractical for larger sets\[[37](https://arxiv.org/html/2610.28709#bib.bib8)\]\. The Set Transformer\[[22](https://arxiv.org/html/2610.28709#bib.bib3)\]replaces the sum aggregator with attention\.

We used Multi\-Head Attention \(MHA\)\[[36](https://arxiv.org/html/2610.28709#bib.bib6)\]and Induced Set Attention \(ISA\) blocks\[[22](https://arxiv.org/html/2610.28709#bib.bib3)\]in our ML surrogate\. Let𝐐∈ℝnq×d\\mathbf\{Q\}\\in\\mathbb\{R\}^\{n\_\{q\}\\times d\},𝐊,𝐕∈ℝnk×d\\mathbf\{K\},\\mathbf\{V\}\\in\\mathbb\{R\}^\{n\_\{k\}\\times d\}be query and key\-value matrices\. Lethhbe the number of heads anddh=d/hd\_\{h\}=d/h\. The scaled dot\-product attention is

Attn⁡\(𝐐,𝐊,𝐕\)=softmax⁡\(𝐐𝐊⊤dh\)​𝐕,\\mathrm\{Attn\}\(\\mathbf\{Q\},\\mathbf\{K\},\\mathbf\{V\}\)=\\mathrm\{softmax\}\\\!\\left\(\\frac\{\\mathbf\{Q\}\\mathbf\{K\}^\{\\top\}\}\{\\sqrt\{d\_\{h\}\}\}\\right\)\\mathbf\{V\},\(6\)extended to multi\-head form by per\-head linear projections𝐖Q\(j\),\\mathbf\{W\}\_\{Q\}^\{\(j\)\},𝐖K\(j\),𝐖V\(j\)∈ℝd×dh\\mathbf\{W\}\_\{K\}^\{\(j\)\},\\mathbf\{W\}\_\{V\}^\{\(j\)\}\\in\\mathbb\{R\}^\{d\\times d\_\{h\}\}and an output mix𝐖O∈ℝd×d\\mathbf\{W\}\_\{O\}\\in\\mathbb\{R\}^\{d\\times d\}:

MHA⁡\(𝐐,𝐊,𝐕\)=\[⨁j=1hAttn⁡\(𝐐𝐖Q\(j\),𝐊𝐖K\(j\),𝐕𝐖V\(j\)\)\]​𝐖O\.\\mathrm\{MHA\}\(\\mathbf\{Q\},\\mathbf\{K\},\\mathbf\{V\}\)=\\Bigl\[\\,\\bigoplus\_\{j=1\}^\{h\}\\mathrm\{Attn\}\\\!\\bigl\(\\mathbf\{Q\}\\mathbf\{W\}\_\{Q\}^\{\(j\)\},\\mathbf\{K\}\\mathbf\{W\}\_\{K\}^\{\(j\)\},\\mathbf\{V\}\\mathbf\{W\}\_\{V\}^\{\(j\)\}\\bigr\)\\Bigr\]\\mathbf\{W\}\_\{O\}\.\(7\)The Multi\-head Attention Block \(MAB\) wraps this with a residual connection, layer normalization \(LN\)\[[6](https://arxiv.org/html/2610.28709#bib.bib9)\], and a row\-wise feed\-forward networkrFFθ\\mathrm\{rFF\}\_\{\\theta\}:

𝐇\\displaystyle\\mathbf\{H\}=LN⁡\(𝐐\+MHA⁡\(𝐐,𝐊,𝐕\)\),\\displaystyle=\\mathrm\{LN\}\\\!\\bigl\(\\mathbf\{Q\}\+\\mathrm\{MHA\}\(\\mathbf\{Q\},\\mathbf\{K\},\\mathbf\{V\}\)\\bigr\),\(8\)MAB⁡\(𝐐,𝐊,𝐕\)\\displaystyle\\mathrm\{MAB\}\(\\mathbf\{Q\},\\mathbf\{K\},\\mathbf\{V\}\)=LN⁡\(𝐇\+rFFθ​\(𝐇\)\)\.\\displaystyle=\\mathrm\{LN\}\\\!\\bigl\(\\mathbf\{H\}\+\\mathrm\{rFF\}\_\{\\theta\}\(\\mathbf\{H\}\)\\bigr\)\.\(9\)
The Set Transformer recovers linear scaling by introducingm≪nm\\ll nlearnable*inducing points*𝐈∈ℝm×d\\mathbf\{I\}\\in\\mathbb\{R\}^\{m\\times d\}and the Induced Set Attention Block \(ISAB\):

𝐇\\displaystyle\\mathbf\{H\}=MAB⁡\(𝐈,𝒳,𝒳\)∈ℝm×d,\\displaystyle=\\mathrm\{MAB\}\(\\mathbf\{I\},\\mathcal\{X\},\\mathcal\{X\}\)\\in\\mathbb\{R\}^\{m\\times d\},\(10\)ISAB⁡\(𝒳\)\\displaystyle\\mathrm\{ISAB\}\(\\mathcal\{X\}\)=MAB⁡\(𝒳,𝐇,𝐇\)∈ℝn×d,\\displaystyle=\\mathrm\{MAB\}\(\\mathcal\{X\},\\mathbf\{H\},\\mathbf\{H\}\)\\in\\mathbb\{R\}^\{n\\times d\},\(11\)which runs inO⁡\(n​m\)O\(nm\)and is permutation*equivariant*in𝒳\\mathcal\{X\}\[[22](https://arxiv.org/html/2610.28709#bib.bib3)\]\(stacking ISAB blocks therefore preserves equivariance; permutation invariance is only obtained once the set is pooled to a fixed\-size summary, as described next\)\.

We also used Pooling by Multihead Attention \(PMA\) in our ML surrogate\. A permutation\-invariant fixed\-length summary of𝒳∈ℝn×d\\mathcal\{X\}\\in\\mathbb\{R\}^\{n\\times d\}is obtained by attendingkklearnable seed vectors𝐒∈ℝk×d\\mathbf\{S\}\\in\\mathbb\{R\}^\{k\\times d\}against𝒳\\mathcal\{X\}\. We omit the row\-wise feed\-forward pre\-processing of𝒳\\mathcal\{X\}used in the original formulation\[[22](https://arxiv.org/html/2610.28709#bib.bib3)\], giving the simplified form

PMAk​\(𝒳\)=MAB⁡\(𝐒,𝒳,𝒳\)∈ℝk×d\.\\mathrm\{PMA\}\_\{k\}\(\\mathcal\{X\}\)=\\mathrm\{MAB\}\(\\mathbf\{S\},\\mathcal\{X\},\\mathcal\{X\}\)\\in\\mathbb\{R\}^\{k\\times d\}\.\(12\)A subsequent self\-attention block \(SAB\) and a linear head produce the prediction\. Because𝐒\\mathbf\{S\}does not depend on the input, the compositionf=LinearHead∘SAB∘PMAk∘ISAB\(L\)∘⋯f=\\mathrm\{LinearHead\}\\circ\\mathrm\{SAB\}\\circ\\mathrm\{PMA\}\_\{k\}\\circ\\mathrm\{ISAB\}^\{\(L\)\}\\circ\\cdotsis permutation invariant by construction\.

## 3Methodology

For developing our proposed model, STAR\-XPC, the process included dataset generation, feature engineering, designing set transformer architecture, and cross\-validation protocol\.

### 3\.1Data Generation Pipeline

We used an automated PENELOPE/penEasy pipeline to generate the dataset\. Scripts generate directory tree for all simulations, the input file, the geometry file, and the material file for every composite, then launch and aggregate the simulation runs\. The incident photon spectra are four fixed, standardized beam qualities, each generated once with the XCOMP5R software\[[28](https://arxiv.org/html/2610.28709#bib.bib33)\]and shared by all composites: the ISO4037\-1narrow\-spectrum quality N\-30 at30kV30\\text\{\\,\}\\mathrm\{kV\}\[[18](https://arxiv.org/html/2610.28709#bib.bib35)\], and the IEC 61267 diagnostic qualities RQR 6, RQR 8 and RQR 9 at80kV80\\text\{\\,\}\\mathrm\{kV\},100kV100\\text\{\\,\}\\mathrm\{kV\}and120kV120\\text\{\\,\}\\mathrm\{kV\}, respectively\[[17](https://arxiv.org/html/2610.28709#bib.bib34)\]\.

The polymer used in all samples is polyethylene \(C2H\)n4\{\}\_\{4\}\)\_\{n\}with a density ofρpoly=0\.95\\rho\_\{\\mathrm\{poly\}\}=0\.95g/cm3\. It is represented as a single token \(labeledC2H4in the dataset and SMILESC=Cin the embedding pipeline\)\. Since the polymer is fixed in all samples, its embedding acts as a constant indicator vector\. Hence, the physical contribution of the polymer enters the model only through the bulk descriptors mentioned below and its mass fraction\. The shielding materials in this study consist of inorganic compounds\{Bi2​O3,BaSO4,BN,B4​C,WO3\}\\\{\\mathrm\{Bi\}\_\{2\}\\mathrm\{O\}\_\{3\},\\mathrm\{BaSO\}\_\{4\},\\mathrm\{BN\},\\mathrm\{B\}\_\{4\}\\mathrm\{C\},\\mathrm\{WO\}\_\{3\}\\\}and elemental shielders\{Sb,Sn,Bi,W,Cu,B,Ba\}\\\{\\mathrm\{Sb\},\\mathrm\{Sn\},\\mathrm\{Bi\},\\mathrm\{W\},\\mathrm\{Cu\},\\mathrm\{B\},\\mathrm\{Ba\}\\\}, for a total of twelve distinct shielding materials\. For each sample, the polymer mass fraction is drawn from the discrete uniform distribution over a fixed list and the remaining mass1−rpoly1\-r\_\{\\mathrm\{poly\}\}is split amongKK\(up to three\) materials via a flat Dirichlet, which ensures that fractions are always positive\.

rpoly∼𝒰⁡\{0\.20,0\.30,0\.40\},\\displaystyle r\_\{\\mathrm\{poly\}\}\\sim\\mathcal\{U\}\\\{0\.20,\\,0\.30,\\,0\.40\\\},\(r1,…,rK\)∼\(1−rpoly\)⋅Dir⁡\(𝟏K\),\\displaystyle\(r\_\{1\},\\ldots,r\_\{K\}\)\\,\\sim\\,\(1\-r\_\{\\mathrm\{poly\}\}\)\\cdot\\mathrm\{Dir\}\(\\mathbf\{1\}\_\{K\}\),\(13\)rpoly\+∑i=1Kri=1\.\\displaystyle r\_\{\\mathrm\{poly\}\}\+\\sum\_\{i=1\}^\{K\}r\_\{i\}=1\.The bulk mass density of the mixture is computed by the additivity of volumes rule

ρ=\(∑iriρi\)−1,\\rho\\;=\\;\\biggl\(\\sum\_\{i\}\\frac\{r\_\{i\}\}\{\\rho\_\{i\}\}\\biggr\)^\{\\\!\-1\},\(14\)and the mean excitation energy is delegated to PENELOPE’smaterialutility, which applies the Bragg additivity rule\[[33](https://arxiv.org/html/2610.28709#bib.bib1)\]

ln⁡Ie​x​c=∑ini​Zi​ln⁡Iie​x​c∑ini​Zi,\\ln I^\{exc\}\\;=\\;\\frac\{\\sum\_\{i\}n\_\{i\}\\,Z\_\{i\}\\,\\ln I\_\{i\}^\{exc\}\}\{\\sum\_\{i\}n\_\{i\}\\,Z\_\{i\}\},\(15\)withnin\_\{i\}the atomic number density \(atoms/cm3\) andZiZ\_\{i\}the atomic number of elementii, and tabulated elementalIie​x​cI\_\{i\}^\{exc\}from PENELOPE’s database\.

For each configuration the target variable, the energy transmission ratio,y=I⁡\(t\)/I0∈\(0,1\)y=I\(t\)/I\_\{0\}\\in\(0,1\)is computed by normalizing the transmitted energy against the source spectrum integral\. We discard configurations that fail the sanity checky=I⁡\(t\)/I0∈\(0,1\)y=I\(t\)/I\_\{0\}\\in\(0,1\), are missing bulk physical descriptors, or have no valid additive\-identity entry\. After this pre\-processing, the dataset containsn=4697n=4697valid samples\. The dataset spans y\-values of heavily attenuated and nearly transparent \(polymer dominated\) samples with a median of 0\.539\.

### 3\.2Input Feature Engineering and Representation

#### 3\.2\.1Molecular Embeddings

Each unique molecular constituent is mapped to a SMILES string\[[38](https://arxiv.org/html/2610.28709#bib.bib18)\]through a deterministic local lookup that handles both organic species \(e\.g\.,C2​H4↦𝙲=𝙲\\mathrm\{C\_\{2\}H\_\{4\}\}\\\!\\mapsto\\\!\\mathtt\{C=C\}\) and inorganic compounds \(e\.g\.,Bi2​O3↦𝙾=\[𝙱𝚒\]​𝙾​\[𝙱𝚒\]=𝙾\\mathrm\{Bi\_\{2\}O\_\{3\}\}\\\!\\mapsto\\\!\\mathtt\{O=\[Bi\]O\[Bi\]=O\}\)\. Pure elements are encoded by their canonical bracketed SMILES \(\[𝚆\]\\mathtt\{\[W\]\},\[𝙱𝚒\]\\mathtt\{\[Bi\]\}, etc\.\)\.

Each SMILES stringsis\_\{i\}is tokenized \(byte\-pair encoding, max lengthT=64T=64\) and fed through a ChemBERTa model \(seyonec/ChemBERTa\-zinc\-base\-v1\)\[[10](https://arxiv.org/html/2610.28709#bib.bib5),[35](https://arxiv.org/html/2610.28709#bib.bib19)\], withL=6L=6hidden layers of width768768\. Instead of collapsing the sequence onto a single\[CLS\]token, we average the four upper hidden layers \(ℓ=L−3,…,L\\ell=L\-3,\\ldots,L\), following the choice of layers used in the feature\-based ablation of\[[11](https://arxiv.org/html/2610.28709#bib.bib7)\]\(which pools them by concatenation or summation, we instead use the mean\), then take a mask\-weighted mean over tokens as follows:

𝐇¯i=14​∑ℓ=L−3L𝐇i\(ℓ\)∈ℝT×768,\\bar\{\\mathbf\{H\}\}\_\{i\}=\\frac\{1\}\{4\}\\sum\_\{\\ell=L\-3\}^\{L\}\\mathbf\{H\}^\{\(\\ell\)\}\_\{i\}\\in\\mathbb\{R\}^\{T\\times 768\},\(16\)𝐞~i=∑t=1Tmi,t𝐇¯i,t,:max⁡\(∑t=1Tmi,t,ϵ\)∈ℝ768,\\tilde\{\\mathbf\{e\}\}\_\{i\}=\\frac\{\\sum\_\{t=1\}^\{T\}m\_\{i,t\}\\,\\bar\{\\mathbf\{H\}\}\_\{i,t,:\}\}\{\\max\(\\sum\_\{t=1\}^\{T\}m\_\{i,t\},\\,\\epsilon\)\}\\in\\mathbb\{R\}^\{768\},\(17\)where𝐦i\\mathbf\{m\}\_\{i\}is the attention mask andϵ=10−9\\epsilon=10^\{\-9\}a numerical floor\. While mask\-weighted pooling treats all chemically significant tokens identically, regardless of where they are in the SMILES string, layer\-averaging collects a combination of syntactic and semantic information that is not available from any single layer\. Finally, we useℓ2\\ell\_\{2\}normalization, to remove norm\-induced scale heterogeneity between organic and inorganic species\.

#### 3\.2\.2Composition Features

A single composite is represented as a fixed\-size set ofN=11N=11component slots \(up to 10 material slots plus the polymer slot\)\. Each non\-empty slotiicarries the constituent’s molecular embedding𝐞i\\mathbf\{e\}\_\{i\}and its mass fractionrir\_\{i\}, renormalized so that∑iri=1\\sum\_\{i\}r\_\{i\}=1\. Empty slots receive a zero embedding andri=0r\_\{i\}=0, but, like every other slot, they also carry the global descriptor vector𝐠\\mathbf\{g\}\(Section[3\.2\.3](https://arxiv.org/html/2610.28709#S3.SS2.SSS3)\)\. Empty slots are not masked in attention: all padding tokens are identical, so they act as a repeated global\-descriptor token that the encoder and the PMA block can attend to\. Because the padding tokens are identical, the model remains exactly invariant to any permutation of the slots\. However, the number of padding tokens is part of the input, so the trained network is specific toN=11N=11\.

#### 3\.2\.3Global Physical Parameters

Four bulk descriptors are recorded per composite: the tube voltageVi​nV\_\{in\}, which indexes one of the four standard beam qualities \(one of\{30,80,100,120\}\\\{30,80,100,120\\\}kV\), the bulk mass densityρ\\rho\(g/cm3\), the slab thicknesstt\(mm\), and the Bragg\-additivity mean excitation energyIe​x​cI^\{exc\}\(eV\)\. We note thatIe​x​cI^\{exc\}is a charged\-particle stopping\-power parameter and is not directly related to the photon attenuation in the 30\-120 kV range and the secondary\-electron transport has a negligible effect on the transmitted energy for the slab thicknesses considered here\. We retain it purely as an engineered compositional summary\. By Eq\. \([15](https://arxiv.org/html/2610.28709#S3.E15)\),ln⁡Ie​x​c\\ln I^\{exc\}is anni​Zin\_\{i\}Z\_\{i\}\-weighted logarithmic mean over the constituent elements and therefore acts as a smooth scalar proxy for the high\-ZZcontent of the mixture\. Since it is a deterministic function of the constituents and their mass fractions, it carries no information beyond the token set\. In an ablation study, removingIe​x​cI^\{exc\}as an input, increased MAE in 9 of 10 folds\.

Each descriptor is standardized by Z\-normalization,

xnorm=x−μxσx,x\_\{\\mathrm\{norm\}\}=\\frac\{x\-\\mu\_\{x\}\}\{\\sigma\_\{x\}\},\(18\)whereμx,σx\\mu\_\{x\},\\sigma\_\{x\}are computed exclusively on the training partition of each cross\-validation fold \(Section[3\.4](https://arxiv.org/html/2610.28709#S3.SS4)\)\. The single\-run model used in Sections[4\.1](https://arxiv.org/html/2610.28709#S4.SS1),[4\.2](https://arxiv.org/html/2610.28709#S4.SS2)and[4\.4](https://arxiv.org/html/2610.28709#S4.SS4)is trained withμx,σx\\mu\_\{x\},\\sigma\_\{x\}computed on its 80% training split and is evaluated with the same statistics\. The standardized descriptor input vector is

𝐠=\[Vnorm,ρnorm,tnorm,Inorm\]∈ℝ4\.\\mathbf\{g\}\\;=\\;\[V\_\{\\mathrm\{norm\}\},\\,\\rho\_\{\\mathrm\{norm\}\},\\,t\_\{\\mathrm\{norm\}\},\\,I\_\{\\mathrm\{norm\}\}\]\\in\\mathbb\{R\}^\{4\}\.\(19\)IncludingVVas an explicit descriptor is what allows a single network to handle the four spectrally distinct beam settings without a per\-voltage head\. The vector𝐠\\mathbf\{g\}is concatenated to all component tokens, so that every element of the input set carries complete information about the global geometry, beam quality and bulk material state\. The final per\-component input is the concatenation of the768768\-dimensional molecular embedding, the scalar mass fraction, and the four global descriptors,

𝐱i=\[𝐞i;ri;𝐠\]∈ℝ773\.\\mathbf\{x\}\_\{i\}\\;=\\;\\bigl\[\\mathbf\{e\}\_\{i\};\\,r\_\{i\};\\,\\mathbf\{g\}\\bigr\]\\in\\mathbb\{R\}^\{773\}\.\(20\)
Figure 1:Set Transformer architecture of STAR\-XPC\. Each of theN=11N=11constituent tokens is the concatenation of its768768\-dimensional ChemBERTa embedding, its scalar mass fraction, and the four global descriptors𝐠\\mathbf\{g\}\. The token set is processed by two permutation\-equivariant ISAB encoder blocks, pooled by a PMA block intok=8k=8seed vectors, mixed by a self\-attention block \(SAB\), and mapped by a linear head to the scalar energy transmission ratioy^\\hat\{y\}\.

### 3\.3Set Transformer Architecture

#### 3\.3\.1Core Architecture

The network outputs a scalar predictiony^∈ℝ\\hat\{y\}\\in\\mathbb\{R\}which is regressed onto our target, the energy transmission ratioy∈\(0,1\)y\\in\(0,1\), after taking the padded set𝐗=\{𝐱1,…,𝐱11\}∈ℝ11×773\\mathbf\{X\}=\\\{\\mathbf\{x\}\_\{1\},\\ldots,\\mathbf\{x\}\_\{11\}\\\}\\in\\mathbb\{R\}^\{11\\times 773\}as an input\. The architecture \(Figure[1](https://arxiv.org/html/2610.28709#S3.F1)\) is a simple implementation of the Set Transformer of Lee et al\.\[[22](https://arxiv.org/html/2610.28709#bib.bib3)\]:

𝐙0\\displaystyle\\mathbf\{Z\}\_\{0\}=ReLU⁡\(𝐗​𝐖e\+𝐛e\),\\displaystyle=\\mathrm\{ReLU\}\\bigl\(\\mathbf\{X\}\\,\\mathbf\{W\}\_\{e\}\+\\mathbf\{b\}\_\{e\}\\bigr\),\(21\)𝐙ℓ\\displaystyle\\mathbf\{Z\}\_\{\\ell\}=ISABm\(ℓ\)\(𝐙ℓ−1\),ℓ=1,2,\\displaystyle=\\mathrm\{ISAB\}\_\{m\}^\{\(\\ell\)\}\\bigl\(\\mathbf\{Z\}\_\{\\ell\-1\}\\bigr\),\\quad\\ell=1,2,\(22\)𝐏\\displaystyle\\mathbf\{P\}=PMAk​\(𝐙2\),𝐐=SAB⁡\(𝐏\),\\displaystyle=\\mathrm\{PMA\}\_\{k\}\\bigl\(\\mathbf\{Z\}\_\{2\}\\bigr\),\\quad\\mathbf\{Q\}=\\mathrm\{SAB\}\\bigl\(\\mathbf\{P\}\\bigr\),\(23\)y^\\displaystyle\\hat\{y\}=𝐰o⊤​vec​\(𝐐\)\+bo\.\\displaystyle=\\mathbf\{w\}\_\{o\}^\{\\top\}\\mathrm\{vec\}\(\\mathbf\{Q\}\)\+b\_\{o\}\.\(24\)
Every MAB, and hence every ISAB, PMA and SAB block, contains a per\-element \(token\-wise\) feed\-forward sub\-network, which we instantiate as threeLinear\-ReLUlayers acting independently on each token \(therFF\\mathrm\{rFF\}block in the notation of Lee et al\.\[[22](https://arxiv.org/html/2610.28709#bib.bib3)\]\)\. The hyperparameters used in this work, summarized in Table[1](https://arxiv.org/html/2610.28709#S3.T1), were selected by testing alternative configurations\.

Table 1:Set Transformer hyperparameters are decided through extensive testing\. The resulting network has3,004,4173\{,\}004\{,\}417trainable parameters\.On a CPU \(no GPU acceleration\), a forward pass measures2\.62\.6ms per sample at batch size11and0\.990\.99ms per sample at batch size256256, both remain several orders of magnitude below the multi\-hour PENELOPE/penEasy reference runs regardless of hardware\.

#### 3\.3\.2Training Configuration

The model is trained by minimizing the mean squared error

ℒ⁡\(θ\)=1B​∑j=1B\(yj−y^j​\(θ\)\)2\\mathcal\{L\}\(\\theta\)\\;=\\;\\frac\{1\}\{B\}\\sum\_\{j=1\}^\{B\}\\bigl\(y\_\{j\}\-\\hat\{y\}\_\{j\}\(\\theta\)\\bigr\)^\{2\}\(25\)withB=256B=256, using the AdamW optimizer\[[25](https://arxiv.org/html/2610.28709#bib.bib10)\]at base learning rateα0=10−3\\alpha\_\{0\}=10^\{\-3\}and weight decayλ=10−5\\lambda=10^\{\-5\}\. We deliberately regress the raw targetyyunder a squared\-error loss rather than a logit\- or log\-transformed target\. The learning rate follows a cosine annealing schedule, without the warm restarts of\[[24](https://arxiv.org/html/2610.28709#bib.bib11)\], with periodTmax=200T\_\{\\max\}=200epochs,

αe=α02​\[1\+cos⁡\(π​eTmax\)\],Tmax=200,\\alpha\_\{e\}\\;=\\;\\frac\{\\alpha\_\{0\}\}\{2\}\\\!\\left\[1\+\\cos\\\!\\left\(\\frac\{\\pi e\}\{T\_\{\\max\}\}\\right\)\\right\],\\qquad T\_\{\\max\}=200,\(26\)so that the schedule completes a full half\-cosine over200200epochs\. Gradient norms are clipped to1\.01\.0to stabilize the early epochs\. Training runs for at most200200epochs with early stopping after5050consecutive epochs without validation\-loss improvement, so the schedule could in principle be cut short; in practice, every one of the ten cross\-validation folds \(Section[3\.4](https://arxiv.org/html/2610.28709#S3.SS4)\) and the single\-run training reported in this paper ran the full200200epochs without early stopping firing\.

### 3\.4Cross\-Validation Protocol

We estimate generalization with 10\-fold cross validation\[[21](https://arxiv.org/html/2610.28709#bib.bib24)\], jointly stratified on chemistry and target value\. To avoid leakage\[[20](https://arxiv.org/html/2610.28709#bib.bib39)\], all normalization statistics are fit on the training rows of each fold only \(see below\)\. The combined stratification method is indeed motivated by two factors: the chemistry of dataset not being uniform and the energy transmission ratio target spans several decades\. Some chemistries may be overrepresented by folds if we just define it onyy, while folds defined solely on chemistry may collapseyyinto a tight region\.

Hence, we decided to stratify based on the Cartesian product of \(i\) main shielding material and \(ii\) empirical\-quantile bin of the target\. Letmoli⋆=arg⁡maxj⁡ri​j\\mathrm\{mol\}\_\{i\}^\{\\star\}\\\!=\\\!\\arg\\max\_\{j\}r\_\{ij\}denote the highest\-mass\-fraction non\-polymer constituent in sampleii\. Given a desired number of energy binsBEB\_\{E\}, definebinE​\(y\)∈\{0,…,BE−1\}\\mathrm\{bin\}\_\{E\}\(y\)\\in\\\{0,\.\.\.,B\_\{E\}\\\!\-\\\!1\\\}via the empirical\-quantile boundariesF^−1​\(b/BE\)\\hat\{F\}^\{\-1\}\(b/B\_\{E\}\)forb=1,…,BE−1b=1,\.\.\.,B\_\{E\}\\\!\-\\\!1\. The joint stratum label issi=\(binE​\(yi\),moli⋆\)s\_\{i\}=\\bigl\(\\mathrm\{bin\}\_\{E\}\(y\_\{i\}\),\\;\\mathrm\{mol\}\_\{i\}^\{\\star\}\\bigr\)\.

BEB\_\{E\}is adaptively chosen as the largest value keeping at least 10 samples per stratum \(StratifiedKFold\[[29](https://arxiv.org/html/2610.28709#bib.bib25)\]\), ensuring every fold gets at least one sample from each stratum, with each fold using a 90/10 train\+validation/test split \(and 89/11 train /validation within that when early stopping is used\), while the separate single\-run training explained in Section[3](https://arxiv.org/html/2610.28709#S3)uses its own 80/20 train/validation random split\. Please note that, different ratio compositions correspond to different materials compositions and no material composition is seen twice in the dataset\. Therefore, each unique composition is tested with a single thickness, except for the generalization analysis at the end of the performance evaluation with held\-out compositions\.

The four global descriptors are applied unaltered to the validation and test partitions after being Z\-scored using statistics that are only calculated on the training rows of each fold\. This is crucial since it would boost the perceived accuracy without altering the underlying generalization if validation/test statistics were leaked\. We present root mean squared error \(RMSE\), mean squared error \(MSE\), mean absolute error \(MAE\), and the normalized\-MSE percentage \(NMSE%\)\.

NMSE%=100×MSE𝔼⁡\[ytrue2\]\.\\mathrm\{NMSE\\%\}\\;=\\;100\\times\\frac\{\\mathrm\{MSE\}\}\{\\mathbb\{E\}\[y\_\{\\mathrm\{true\}\}^\{2\}\]\}\.\(27\)

## 4Results and Analysis

### 4\.1Baseline Comparison

We argued that fixed\-length, order\-dependent regressors are a poor fit for this problem\. We compare the Set Transformer against three standard regressors given the exact same information, under the same 10\-fold protocol as Section[3\.4](https://arxiv.org/html/2610.28709#S3.SS4)\. Each baseline gets the same molecular embeddings, mass fractions and global descriptors as the Set Transformer \(STAR\-XPC\), just flattened into one fixed\-length vector \(the 11 slot embeddings, then the 11 mass fractions, then the 4 global descriptors,84638463numbers total\) instead of being read as a set\.

We tested Ridge regression\[[15](https://arxiv.org/html/2610.28709#bib.bib28)\]\(ℓ2\\ell\_\{2\}\-penalty\)α\\alphatuned on held\-out data \(please note that the plain linear regression cannot even be fit here, since there are more features than training rows\), Random Forest\[[9](https://arxiv.org/html/2610.28709#bib.bib26)\], and RBF\-kernel support vector regression \(SVR\)\[[12](https://arxiv.org/html/2610.28709#bib.bib27)\]\. All hyperparameters are selected withscikit\-learn\[[29](https://arxiv.org/html/2610.28709#bib.bib25)\]on the validation split of each fold\.

\(a\)Ridge Regression
\(b\)Random Forest
\(c\)SVR \(RBF\)
\(d\)STAR\-XPC

Figure 2:Predicted vs\. simulated energy transmission ratios for each of the three baselines and STAR\-XPC, aggregated over the out\-of\-fold predictions of the identical 10\-fold cross\-validation \(n=4697n=4697\)\.Figure[2](https://arxiv.org/html/2610.28709#S4.F2)shows the aggregate parity plots \(ground truth vs\. estimate\) for the three baselines and STAR\-XPC, and Table[2](https://arxiv.org/html/2610.28709#S4.T2)lists the corresponding NMSE% values\. Two things stand out\. First, Ridge regression is far worse than every other model \(R2=0\.84R^\{2\}=0\.84in\-distribution, versus≥0\.99\\geq 0\.99for the others\), confirming that the attenuation problem is nonlinear\. Second, in the standard 10\-fold comparison SVR comes close to STAR\-XPC \(NMSE%=0\.0312%\\mathrm\{NMSE\\%\}=0\.0312\\%vs\.0\.0278%0\.0278\\%on8080\-120kV120\\text\{\\,\}\\mathrm\{kV\}\)\. We attribute the competitiveness of SVR to the in\-distribution setting: the embedding cache contains only1313distinct molecule vectors, so an RBF kernel can effectively memorize “similar known composition⇒\\Rightarrowsimilar output”, and every fold’s test samples use the same fixed column order as training, so the flattened input is never penalized for depending on it\. The two experiments below test these two advantages\.

Table 2:NMSE% in\-distribution \(10\-fold CV, out\-of\-fold predictions restricted to8080\-120kV120\\text\{\\,\}\\mathrm\{kV\},n=3996n=3996\) versus the mean over the six held\-out unseen\-combination test sets, and the ratio between them\.In Table[2](https://arxiv.org/html/2610.28709#S4.T2), the two NMSE% columns come from different training regimes\. The CV column, like Figure[2](https://arxiv.org/html/2610.28709#S4.F2), aggregates the out\-of\-fold predictions of the ten cross\-validation models of each method, whereas the held\-out column is obtained from a single training run per method on the full training pool, evaluated once on the six held\-out compositions\. Since the held\-out compositions are restricted to the8080\-120kV120\\text\{\\,\}\\mathrm\{kV\}samples \(n=3996n=3996\), which is why the STAR\-XPC entry is lower than the all\-voltageNMSE%=0\.0294%\\mathrm\{NMSE\\%\}=0\.0294\\%reported in Section[4\.2](https://arxiv.org/html/2610.28709#S4.SS2)\.

The held\-out column of Table[2](https://arxiv.org/html/2610.28709#S4.T2)is a more informative comparison related to generalization\. There, STAR\-XPC reaches a mean NMSE% of0\.0496%0\.0496\\%against0\.0804%0\.0804\\%for SVR\. From CV to held\-out, the three flattened baselines degrade by a similar2\.42\.4\-2\.6×2\.6\\timesdespite in\-distribution errors spanning two orders of magnitude, whereas STAR\-XPC degrades by1\.8×1\.8\\times\. What separates the two models is invariance to input order\.

Another key point is that every fold’s test set preserves the fixed column order as training, risking that the flattened baselines memorize this input order rather than learning dependence\. A flattened vector has no notion that "slot 1" and "slot 2" are interchangeable and that ordering does not matter, while Set Transformer’s set input does, by construction\. To illustrate this even further, we took one sample per held\-out composition and generated 40 random reorderings of its 11 molecule slots and re\-ran all three trained models on every version\.

Figure 3:Predicted transmission ratio across 40 random slot reorderings of the same physical sample \(black tick: ground truth\), all three panels on a commonyy\-scale\. STAR\-XPC’s predictions collapse to a single point per case, SVR’s scatter across most of the target range, and the Random Forest’s, although far less sensitive than SVR’s, still fail to collapse to a single value per sample\.In Figure[3](https://arxiv.org/html/2610.28709#S4.F3), STAR\-XPC’s predictions do not move \(standard deviation≤1\.3×10−7\\leq 1\.3\\times 10^\{\-7\}, i\.e\. floating\-point noise\), while the standard deviation of predicted transmission ratios for SVR is between 0\.067 and 0\.098 for the identical physical sample, simply because the molecules were listed in a different order\. The Random Forest lies between these two extremes\. Its standard deviation stays within0\.00180\.0018\-0\.00780\.0078, since its axis\-aligned splits rest largely on the four global descriptors, which are per\-sample and therefore untouched by the reordering\. It is nevertheless not order\-invariant; over the 40 reorderings of a single sample its prediction still moves by up to0\.0340\.034, roughly9%9\\%of that sample’s transmission ratio\. In other words, the apparent superiority of the flattened baselines on in\-distribution test data may not reflect genuine physical understanding, it instead stems from the models memorizing the fixed order in which input columns happen to be arranged in the data\. STAR\-XPC, on the other hand, is unaffected by these random permutations, since it has no fixed input order to memorize in the first place\.

In short, a well\-tuned kernel method performs close to STAR\-XPC when interpolating compositions it has effectively already seen, but is outperformed on new combinations \(mean held\-out NMSE%0\.0804%0\.0804\\%vs\.0\.0496%0\.0496\\%\) and is not robust to input reordering\. STAR\-XPC is the only model here that is accurate, generalizes to new combinations, and is provably indifferent to how the input happens to be listed\.

### 4\.2Detailed Cross\-Validation Performance of STAR\-XPC

Figure[4](https://arxiv.org/html/2610.28709#S4.F4)shows the training and validation loss plots\. Training and validation loss both decrease rapidly in the first∼30\\sim\\\!30epochs and reach a plateau at the𝒪⁡\(10−4\)\\mathcal\{O\}\(10^\{\-4\}\)level\. Since the two curves closely follow each other throughout the training phase, we may deduce that we didn’t encounter overfitting\.

Figure 4:Training and validation MSE loss for the single\-run model\. The grey dashed line marks the best\-validation epoch and the red dot marks its loss value\.When tested on the whole pool of data\(n=4697\)\(n=4697\), STAR\-XPC exhibits high prediction accuracy over all 10 cross\-validation folds\. Aggregated across folds, we observeMSE¯=\(1\.12±0\.56\)×10−4\\overline\{\\mathrm\{MSE\}\}=\(1\.12\\pm 0\.56\)\\times 10^\{\-4\}andMAE¯=0\.00718±0\.00148\\overline\{\\mathrm\{MAE\}\}=0\.00718\\pm 0\.00148withR2=0\.9988R^\{2\}=0\.9988\(mean±\\pmsample standard deviation over folds\)\.

The aggregate metrics above are residual\-scale statistics that we used for the training\. For the performance evaluation, we complement them with a per\-sample relative measure, we additionally report the per\-sample absolute percentage error \(APE\),

APEi=100⋅\|y^i−yi\|max⁡\(\|yi\|,ϵ\),ϵ=10−9,\\mathrm\{APE\}\_\{i\}\\;=\\;100\\cdot\\frac\{\\lvert\\hat\{y\}\_\{i\}\-y\_\{i\}\\rvert\}\{\\max\\bigl\(\\lvert y\_\{i\}\\rvert,\\,\\epsilon\\bigr\)\},\\qquad\\epsilon=10^\{\-9\},\(28\)mean APE \(MAPE\), and anα\\alpha\-trimmed mean APE \(MAPEα\\mathrm\{MAPE\}\_\{\\alpha\}\)

MAPEα=1N⁡\(1−2​α\)​∑i∈ℐαAPEi,α=0\.10,\\mathrm\{MAPE\}\_\{\\alpha\}\\;=\\;\\frac\{1\}\{N\(1\-2\\alpha\)\}\\sum\_\{i\\in\\mathcal\{I\}\_\{\\alpha\}\}\\mathrm\{APE\}\_\{i\},\\qquad\\alpha=0\.10,\(29\)whereℐα\\mathcal\{I\}\_\{\\alpha\}is the index set of the central80%80\\%of theAPEi\\mathrm\{APE\}\_\{i\}values \(i\.e\. the lowest and highest10%10\\%are discarded\)\. In this case, trimming is quite needed due to the fact that samples with a transmission ratioyynear numerical zero \(i\.e\.,y^\\hat\{y\}even with a very small error\) produce a non\-representative, excessively high percentage error that dominates the MAPE without providing any insight about accuracy\.

Pooled over the out\-of\-fold predictions of all ten folds, we observeMAPE=203%\\mathrm\{MAPE\}=203\\%due to the near\-zero\-yysamples\. This figure is driven almost entirely by the129129near\-opaque samples withy<0\.01y<0\.01\(2\.7%2\.7\\%of the pool\)\. Please note that, their absolute errors are small, but the APE denominator vanishes asy→0y\\to 0\. The quantiles of the per\-sample APE distribution,\(P50,P75,P90,P95\)=\(1\.15%,2\.84%,6\.75%,12\.04%\)\(P\_\{50\},P\_\{75\},P\_\{90\},P\_\{95\}\)=\(1\.15\\%,\\,2\.84\\%,\\,6\.75\\%,\\,12\.04\\%\), make this concentration explicit: the median and the trimmed mean are of the same order, while the untrimmed mean is dominated by the extreme upper tail\. Discarding the lowest and highest10%10\\%of the APE values results inMAPE0\.10=1\.66%\\mathrm\{MAPE\}\_\{0\.10\}=1\.66\\%over all four tube voltages andMAPE0\.10=1\.32%\\mathrm\{MAPE\}\_\{0\.10\}=1\.32\\%over8080\-120kV120\\text\{\\,\}\\mathrm\{kV\}\(n=3996n=3996\)\. Therefore,MAPE0\.10\\mathrm\{MAPE\}\_\{0\.10\}is included as a more representative and practitioner\-friendly per\-sample relative\-error metric for non\-opaque configurations\.

### 4\.3Analysis of STAR\-XPC

For evaluating model robustness, we analyzed the effect of energy\-level and thickness\. Even if the30kV30\\text\{\\,\}\\mathrm\{kV\}tube voltage is not so common in practical applications, the30kV30\\text\{\\,\}\\mathrm\{kV\}is included in training in order to assist network to learn generalization across the entire kV range, and in the all\-voltage CV metrics\. Given that the model is cross\-validated, the overall cross\-validation figures of Section[4](https://arxiv.org/html/2610.28709#S4)above are reported over all four voltages\. Nonetheless, starting with this subsection, we report every stratified metric on the8080,100100, and120kV120\\text\{\\,\}\\mathrm\{kV\}data \(n=3996n=3996\)\.

#### 4\.3\.1Energy\-level performance

As expected, the beam spectrum has direct relation with the performance \(Table[3](https://arxiv.org/html/2610.28709#S4.T3)\)\. The representative relative error decreases monotonically with tube voltage, fromMAPE0\.10=1\.65%\\mathrm\{MAPE\}\_\{0\.10\}=1\.65\\%at80kV80\\text\{\\,\}\\mathrm\{kV\}to1\.25%1\.25\\%at100kV100\\text\{\\,\}\\mathrm\{kV\}and1\.13%1\.13\\%at120kV120\\text\{\\,\}\\mathrm\{kV\}, since harder spectra are attenuated less and the mean target rises accordingly \(y¯=0\.495\\overline\{y\}=0\.495,0\.5560\.556,0\.7030\.703\)\. The absolute and squared\-error metrics remain small in all three regimes \(MAE≤0\.0087\\leq 0\.0087,NMSE%≤0\.032%\\mathrm\{NMSE\\%\}\\leq 0\.032\\%\) but do not follow a monotonic trend, because MAE scales with the magnitude ofyywhile NMSE% is normalised by𝔼⁡\[y2\]\\mathbb\{E\}\[y^\{2\}\]on each subset\.

Table 3:Performance stratified by tube voltage \(8080\-120kV120\\text\{\\,\}\\mathrm\{kV\},n=3996n=3996\)
#### 4\.3\.2Thickness\-dependent accuracy

Our data\-generation pipeline sweeps single\-element shielding materials \(B, Ba, Bi, Cu, Sb, Sn, W\) only up to1\.51\.5mm, while molecular or compound fillers \(B4C, BN, BaSO4, Bi2O3, WO3\) are swept up to2\.52\.5mm, and the mix of the two classes is not constant across thickness bins\.

In Table[4](https://arxiv.org/html/2610.28709#S4.T4)and[5](https://arxiv.org/html/2610.28709#S4.T5), performance is stratified by thickness on single element and compounds cases\.MAPE0\.10\\mathrm\{MAPE\}\_\{0\.10\}and NMSE% increases monotonically as thickness increases, while MAE has no trend\. This is the expected first\-order behavior of the transmission ratioy=I⁡\(t\)/I0y=I\(t\)/I\_\{0\}\. Writingy⁡\(t\)=e−μ​ty\(t\)=e^\{\-\\mu t\}for an effective attenuation coefficientμ\\mu, an errorΔ​μ\\Delta\\mupropagates as

\|Δ​y\|y=t​\|Δ​μ\|,\|Δ​y\|=t​\|Δ​μ\|​e−μ​t,\\frac\{\|\\Delta y\|\}\{y\}\\;=\\;t\\,\|\\Delta\\mu\|,\\qquad\|\\Delta y\|\\;=\\;t\\,\|\\Delta\\mu\|\\,e^\{\-\\mu t\},\(30\)so the relative error scales linearly withttwhereas the absolute error stays bounded by the decaying exponential, explaining the flat MAE and the increasingMAPE0\.10\\mathrm\{MAPE\}\_\{0\.10\}reported here\.

#### 4\.3\.3Chemistry\-stratified performance

Table[6](https://arxiv.org/html/2610.28709#S4.T6)stratifies MAE, NMSE%, andMAPE0\.10\\mathrm\{MAPE\}\_\{0\.10\}by the main shielding material, and the table is sorted byMAPE0\.10\\mathrm\{MAPE\}\_\{0\.10\}\. STAR\-XPC estimates the compound constituents more accurately than the elemental ones\. The six worst rows byMAPE0\.10\\mathrm\{MAPE\}\_\{0\.10\}\(Sb, Sn, Cu, Bi, Ba, W\) are all single elements, and the lowest MAE \(0\.00460\.0046\-0\.00480\.0048\) is attained by BN, B4C and BaSO4\. The clearest evidence comes from the four elements that appear both in elemental and in compound form: in every such pair the compound has both the lower MAE and the lowerMAPE0\.10\\mathrm\{MAPE\}\_\{0\.10\}\(Bi2\.43%2\.43\\%vs\. Bi2O31\.19%1\.19\\%; W2\.58%2\.58\\%vs\. WO31\.23%1\.23\\%; Ba2\.44%2\.44\\%vs\. BaSO40\.73%0\.73\\%; B1\.24%1\.24\\%vs\. BN0\.42%0\.42\\%and B4C0\.47%0\.47\\%\)\. Sample count accounts for this only in part: across the twelve materials MAE decreases withnn\(Spearmanρ=−0\.70\\rho=\-0\.70, Pearsonr=−0\.64r=\-0\.64\), yet WO3\(n=251n=251\) is estimated more accurately than W \(n=289n=289\) despite having fewer samples\. One possible reason is that, at the same mass fraction, an elemental constituent puts more high\-ZZmass into the slab than its compound counterpart\. This leads to stronger attenuation with more spectral structure\. We leave a controlled test of this hypothesis to future work\.

Table 4:Performance stratified by thickness bins for samples whose main material is a single element \(B, Ba, Bi, Cu, Sb, Sn, W,8080\-120kV120\\text\{\\,\}\\mathrm\{kV\},n=1941n=1941\)\.Table 5:Performance stratified by thickness bins for samples whose main material is a compound \(B4C, BN, BaSO4, Bi2O3, WO3,8080\-120kV120\\text\{\\,\}\\mathrm\{kV\},n=2055n=2055\)\.Table 6:Per\-shielding\-material MAE, NMSE%, andα\\alpha\-trimmed MAPE \(8080\-120kV120\\text\{\\,\}\\mathrm\{kV\},n=3996n=3996\)\.\(a\)Bi\-W at80kV80\\text\{\\,\}\\mathrm\{kV\};
31/39/3031/39/30wt% Bi/W/\(C2H\)n4\{\}\_\{4\}\)\_\{n\}
\(b\)W\-Cu\-B at100kV100\\text\{\\,\}\\mathrm\{kV\};
48/9/3/4048/9/3/40wt% W/Cu/B/\(C2H\)n4\{\}\_\{4\}\)\_\{n\}
\(c\)Bi\-Sb at120kV120\\text\{\\,\}\\mathrm\{kV\};
60/20/2060/20/20wt% Bi/Sb/\(C2H\)n4\{\}\_\{4\}\)\_\{n\}
\(d\)Bi2O3\-BaSO4at80kV80\\text\{\\,\}\\mathrm\{kV\};
40/40/2040/40/20wt% Bi2O3/BaSO4/\(C2H\)n4\{\}\_\{4\}\)\_\{n\}
\(e\)BaSO4\-WO3at100kV100\\text\{\\,\}\\mathrm\{kV\};
35/35/3035/35/30wt% BaSO4/WO3/\(C2H\)n4\{\}\_\{4\}\)\_\{n\}
\(f\)Bi2O3at120kV120\\text\{\\,\}\\mathrm\{kV\};
80/2080/20wt% Bi2O3/\(C2H\)n4\{\}\_\{4\}\)\_\{n\}

Figure 5:Transmission ratio versus material thickness for held\-out data to analyze the generalization capability of STAR\-XPC\. Each sub\-caption gives the tube voltage and the mass fractions of the shielding material and of the polyethylene \(C2H\)n4\{\}\_\{4\}\)\_\{n\}matrix, which sum to100100wt%\.

### 4\.4Generalization Capability of STAR\-XPC

To assess generalization beyond the cross\-validation pool, we constructed six dedicated test sets corresponding to compositions not seen during training STAR\-XPC, each containing1010thickness samples spanning the full range\. In Figure[5](https://arxiv.org/html/2610.28709#S4.F5), thickness versus transmission ratios of simulated and estimated values are plotted\. Moreover, average NMSE% andMAPE0\.10\\mathrm\{MAPE\}\_\{0\.10\}\(%\) for each case are given in subfigures\. All six cases achieveNMSE%≤0\.129%\\mathrm\{NMSE\\%\}\\leq 0\.129\\%whileMAPE0\.10≤6\.86%\\mathrm\{MAPE\}\_\{0\.10\}\\leq 6\.86\\%\. The two largestMAPE0\.10\\mathrm\{MAPE\}\_\{0\.10\}occur at80kV80\\text\{\\,\}\\mathrm\{kV\}, Bi\-W and Bi2O3\-BaSO4, since transmission values are small and even a small error is large in terms of percentage error\. Overall, the proposed model generalizes compositions that are absent from the training dataset\.

Across the six hold\-out compositions the predicted curvey^​\(t\)\\hat\{y\}\(t\)coincides with the simulated transmission throughout the thicknesses of interest\. The two slightly harder cases, Bi2O3\-BaSO4and Bi\-W \(both at80kV80\\text\{\\,\}\\mathrm\{kV\}\), track the simulated curve in shape with only a small offset, while the four remaining cases are essentially indistinguishable from the simulation\.

The held\-out compositions above still draw every constituent from the twelve materials seen during training, only their*weight ratio combination*is new\. We wanted to test STAR\-XPC not only with the similar materials but also with unseen materials\. To probe the molecular embedding pipeline itself and the generalization to unseen materials, we ran additional PENELOPE/penEasy simulations for lead \(Pb\) and gadolinium \(Gd\), which were absent \(unseen\) in the training phase\. For these runs we considered a single shielding material with7070wt% in polyethylene \(the discreterpoly=0\.30r\_\{\\mathrm\{poly\}\}=0\.30\) at100kV100\\text\{\\,\}\\mathrm\{kV\}, swept over ten thicknesses from0\.150\.15to1\.51\.5mm\. All other inputs \(tube voltage, thickness range, mass\-fraction split and filler cardinality\) stay within the trained distribution, isolating the test to the molecular identity of Pb and Gd\. Their embeddings are computed with the identical ChemBERTa pipeline from their canonical bracketed SMILES \(\[Pb\],\[Gd\]\), then passed through the STAR\-XPC with no retraining or fine\-tuning\.

\(a\)Pb;70/3070/30wt% Pb/\(C2H\)n4\{\}\_\{4\}\)\_\{n\}
\(b\)Gd;70/3070/30wt% Gd/\(C2H\)n4\{\}\_\{4\}\)\_\{n\}

Figure 6:Transmission ratio versus thickness at100kV100\\text\{\\,\}\\mathrm\{kV\}for two shielding elements entirely absent from the training vocabulary\.Figure[6](https://arxiv.org/html/2610.28709#S4.F6)shows both thickness sweeps\. Despite zero exposure to Pb or Gd during training, STAR\-XPC tracks both curves closely:NMSE%=0\.192%\\mathrm\{NMSE\\%\}=0\.192\\%\(MAPE0\.10=5\.24%\\mathrm\{MAPE\}\_\{0\.10\}=5\.24\\%\) for Pb andNMSE%=0\.198%\\mathrm\{NMSE\\%\}=0\.198\\%\(MAPE0\.10=4\.03%\\mathrm\{MAPE\}\_\{0\.10\}=4\.03\\%\) for Gd\. This is a harder generalization task compared to previous held\-out compositions of similar materials\. The model has no prior exposure to either element’s embedding at all, whereas the held\-out compositions test still reuses embeddings seen during training\. On these two materials, Pb and Gd, this is an evidence against a pure lookup\-table interpretation of the embedding pipeline\.

## 5Discussion

### 5\.1Set Transformer Architecture Benefits

We chose set transformer architecture due to benefits like: permutation invariance, self\-attention capabilities, and linear scaling via inducing points\.

The bulk attenuation of a homogenized mixture is independent of the lexicographic order of its constituents\. The Set Transformer enforces this symmetry by construction and this is consistent with the observation that the model handles11\-,22\-, and33\-component mixtures with the same network\. The training data contain at most three shielding materials per sample\. The input format admits up to ten shielding constituents plus the polymer, so mixtures with more than three materials can be passed to the same network without modification; however, such mixtures would also contain fewer padding tokens than any training sample, and since the unmasked padding tokens carry the global descriptors and also receive attention, accuracy on higher\-order mixtures cannot be assumed and remains to be validated\. Likewise, the trained network is tied to the set sizeN=11N=11; changingNNchanges the number of padding tokens and therefore the prediction\. The ISAB mechanism keeps the attention cost linear in the set size, so a variant with masked padding and a dedicated global\-descriptor token could be scaled to larger mixtures, which we leave to future work\.

### 5\.2Limitations and Challenges

Because the polymer is identical across all samples \(polyethylene, \(C2H\)n4\{\}\_\{4\}\)\_\{n\}, encoded in the embedding pipeline by the SMILESC=C\), the polymer token’s embedding has zero discriminative power\. The bulk effect of the polymer enters exclusively through the four global descriptors and through its mass fraction\. A study that varies the polymer would require a richer embedding pipeline, e\.g\. a polymer\-aware encoder fine\-tuned on engineering polymers, and is beyond the scope of this paper\.

A related concern is the domain mismatch of the molecular embedding itself\. ChemBERTa\-zinc\-base\-v1\[[10](https://arxiv.org/html/2610.28709#bib.bib5)\]is pretrained on ZINC15\[[35](https://arxiv.org/html/2610.28709#bib.bib19)\], a corpus of drug\-like organic molecules\. Several of our shielding constituents \(e\.g\. elemental\[Bi\],\[W\], or inorganic oxides such as Bi2O3\) are far outside that pretraining distribution, and the resulting embeddings are not validated to carry chemically meaningful structure for these species\. With only1313distinct constituent molecules in the dataset \(Section[4\.1](https://arxiv.org/html/2610.28709#S4.SS1)\), the embedding pipeline could in principle function close to a per\-molecule lookup table rather than exploiting ChemBERTa’s generalization\. Section[4\.4](https://arxiv.org/html/2610.28709#S4.SS4)provides an evidence against a pure lookup\-table interpretation, since the model generalizes to two elements \(Pb, Gd\) never seen during training\.

## 6Conclusions and Future Directions

We have presented a permutation\-invariant Set Transformer model for X\-ray energy transmission in polyethylene/shielder composites, trained on a PENELOPE/penEasy\-generated dataset ofn=4697n=4697simulated configurations spanning four tube voltages and1212main shielding material chemistries\. Cross\-validated over all four voltages, the model achievesNMSE%=0\.0294%\\mathrm\{NMSE\\%\}=0\.0294\\%\(R2=0\.9988R^\{2\}=0\.9988\) withNMSE%≤0\.129%\\mathrm\{NMSE\\%\}\\leq 0\.129\\%on six held\-out compositions never seen during training andNMSE%≤0\.2%\\mathrm\{NMSE\\%\}\\leq 0\.2\\%on two unseen materials \(Pb and Gd\) to show its generalization capabilities\. We have open data and open codes on GitHub \([https://github\.com/cmpe\-netlab41/attenupy](https://github.com/cmpe-netlab41/attenupy)\) and HuggingFace \([https://huggingface\.co/cmpe\-netlab41/STAR\-XPC](https://huggingface.co/cmpe-netlab41/STAR-XPC)\), so that researchers focusing on estimating X\-Ray Transmission may use STAR\-XPC easily with example codes\.

Each architectural choice corresponds to a physical property of the problem\. Permutation invariance reflects the fact that the order of the constituents does not change the attenuation of the material\. The attention\-based ISAB encoder lets the constituents influence one another, capturing interactions that a simple linear mixture rule cannot\. With these architectural choices, the proposed model combines accuracy comparable to Monte Carlo simulations with an inference cost orders of magnitude lower \(sub\-millisecond per query on a single GPU vs\. hours per Monte Carlo run\), opening the door to interactive composite design for medical imaging, industrial radiography and shielding optimization\. In the proposed model, only the transmission ratio is predicted\. For the future directions, we plan to focus on other tally metrics that can also be added to the output head of ML\-surrogate\. We also plan to investigate physics\-informed regularizers\[[30](https://arxiv.org/html/2610.28709#bib.bib12)\]\.

## References

- \[1\]M\. R\. Alipoor, M\. Eshghi, and R\. Razavi\(2026\)Development of a model for design radiation shielding composite aprons using machine learning\.Scientific Reports16,pp\. 23827\.External Links:[Document](https://dx.doi.org/10.1038/s41598-026-50473-7)Cited by:[§2\.3](https://arxiv.org/html/2610.28709#S2.SS3.p1.1)\.
- \[2\]M\. R\. Alipoor, M\. Eshghi, and R\. Razavi\(2026\)Optimization radiation shielding for Bi2\{\}\_\{2\}O3\{\}\_\{3\}\-based composites with artificial neural networks\.Radiation and Environmental Biophysics65\(2\),pp\. 803–817\.External Links:[Document](https://dx.doi.org/10.1007/s00411-026-01212-z)Cited by:[§2\.3](https://arxiv.org/html/2610.28709#S2.SS3.p1.1)\.
- \[3\]N\. Aral, F\. Banu Nergis, and C\. Candan\(2016\)An alternative x\-ray shielding material based on coated textiles\.Textile Research Journal86\(8\),pp\. 803–811\.External Links:[Document](https://dx.doi.org/10.1177/0040517515590409)Cited by:[§1](https://arxiv.org/html/2610.28709#S1.p1.1)\.
- \[4\]N\. Aral, F\. B\. Nergis, and C\. Candan\(2017\)The x\-ray attenuation and the flexural properties of lead\-free coated fabrics\.Journal of Industrial Textiles47\(2\),pp\. 252–268\.External Links:[Document](https://dx.doi.org/10.1177/1528083716644287)Cited by:[§1](https://arxiv.org/html/2610.28709#S1.p1.1)\.
- \[5\]F\. H\. Attix\(1986\)Introduction to radiological physics and radiation dosimetry\.John Wiley & Sons,New York\.External Links:ISBN 9780471011460,[Document](https://dx.doi.org/10.1002/9783527617135)Cited by:[§2\.1](https://arxiv.org/html/2610.28709#S2.SS1.p1.2),[§2\.1](https://arxiv.org/html/2610.28709#S2.SS1.p2.1)\.
- \[6\]J\. L\. Ba, J\. R\. Kiros, and G\. E\. Hinton\(2016\)Layer normalization\.arXiv preprint arXiv:1607\.06450\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.1607.06450)Cited by:[§2\.3](https://arxiv.org/html/2610.28709#S2.SS3.p4.3)\.
- \[7\]A\. Benhadjira, M\. I\. Sayyed, O\. Bentouila, and K\. E\. Aiadi\(2024\)Artificial neural network approach for calculating mass attenuation coefficient of different glass systems\.Nuclear Engineering and Technology56\(1\),pp\. 100–105\.External Links:[Document](https://dx.doi.org/10.1016/j.net.2023.09.013)Cited by:[§2\.3](https://arxiv.org/html/2610.28709#S2.SS3.p1.1)\.
- \[8\]M\. J\. Berger and J\. H\. Hubbell\(1987\)XCOM: photon cross sections on a personal computer\.Technical reportTechnical ReportNBSIR 87\-3597,National Bureau of Standards,Gaithersburg, MD\.External Links:[Link](https://nvlpubs.nist.gov/nistpubs/Legacy/IR/nbsir87-3597.pdf)Cited by:[§1](https://arxiv.org/html/2610.28709#S1.p3.1)\.
- \[9\]L\. Breiman\(2001\)Random forests\.Machine Learning45\(1\),pp\. 5–32\.External Links:[Document](https://dx.doi.org/10.1023/A%3A1010933404324)Cited by:[§4\.1](https://arxiv.org/html/2610.28709#S4.SS1.p2.1)\.
- \[10\]S\. Chithrananda, G\. Grand, and B\. Ramsundar\(2020\)ChemBERTa: large\-scale self\-supervised pretraining for molecular property prediction\.arXiv preprint arXiv:2010\.09885\.Cited by:[item 2](https://arxiv.org/html/2610.28709#S1.I1.i2.p1.1),[§3\.2\.1](https://arxiv.org/html/2610.28709#S3.SS2.SSS1.p2.1),[§5\.2](https://arxiv.org/html/2610.28709#S5.SS2.p2.1)\.
- \[11\]J\. Devlin, M\. Chang, K\. Lee, and K\. Toutanova\(2019\)BERT: pre\-training of deep bidirectional transformers for language understanding\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(NAACL\-HLT\),pp\. 4171–4186\.External Links:[Document](https://dx.doi.org/10.18653/v1/N19-1423)Cited by:[§3\.2\.1](https://arxiv.org/html/2610.28709#S3.SS2.SSS1.p2.1)\.
- \[12\]H\. Drucker, C\. J\. C\. Burges, L\. Kaufman, A\. Smola, and V\. Vapnik\(1996\)Support vector regression machines\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.9,pp\. 155–161\.Cited by:[§4\.1](https://arxiv.org/html/2610.28709#S4.SS1.p2.1)\.
- \[13\]L\. Gerward, N\. Guilbert, K\. B\. Jensen, and H\. Levring\(2001\)X\-ray absorption in matter\. Reengineering XCOM\.Radiation Physics and Chemistry60\(1–2\),pp\. 23–24\.External Links:[Document](https://dx.doi.org/10.1016/S0969-806X%2800%2900324-8)Cited by:[§1](https://arxiv.org/html/2610.28709#S1.p3.1)\.
- \[14\]Y\. Harima\(1993\)An historical review and current status of buildup factor calculations and applications\.Radiation Physics and Chemistry41\(4–5\),pp\. 631–672\.External Links:[Document](https://dx.doi.org/10.1016/0969-806X%2893%2990317-N)Cited by:[§1](https://arxiv.org/html/2610.28709#S1.p3.1)\.
- \[15\]A\. E\. Hoerl and R\. W\. Kennard\(1970\)Ridge regression: biased estimation for nonorthogonal problems\.Technometrics12\(1\),pp\. 55–67\.External Links:[Document](https://dx.doi.org/10.1080/00401706.1970.10488634)Cited by:[§4\.1](https://arxiv.org/html/2610.28709#S4.SS1.p2.1)\.
- \[16\]J\. H\. Hubbell and S\. M\. Seltzer\(2004\)Tables of X\-ray mass attenuation coefficients and mass energy\-absorption coefficients 1 keV to 20 MeV for elementsZ=1\{Z\}=1to 92 and 48 additional substances of dosimetric interest\.Note:NIST Standard Reference Database 126[https://www\.nist\.gov/pml/x\-ray\-mass\-attenuation\-coefficients](https://www.nist.gov/pml/x-ray-mass-attenuation-coefficients)External Links:[Document](https://dx.doi.org/10.18434/T4D01F)Cited by:[§1](https://arxiv.org/html/2610.28709#S1.p3.1)\.
- \[17\]International Electrotechnical Commission\(2005\)IEC 61267:2005 – medical diagnostic X\-ray equipment – radiation conditions for use in the determination of characteristics\.Geneva\.Note:Edition 2\.0Cited by:[§3\.1](https://arxiv.org/html/2610.28709#S3.SS1.p1.1)\.
- \[18\]International Organization for Standardization\(2019\)ISO 4037\-1:2019 – Radiological protection – X and gamma reference radiation for calibrating dosemeters and doserate meters and for determining their response as a function of photon energy – Part 1: Radiation characteristics and production methods\.Geneva\.Note:Edition 2Cited by:[§3\.1](https://arxiv.org/html/2610.28709#S3.SS1.p1.1)\.
- \[19\]S\. Jayakumar, T\. Saravanan, and J\. Philip\(2023\)A review on polymer nanocomposites as lead\-free materials for diagnostic X\-ray shielding: recent advances, challenges and future perspectives\.Hybrid Advances4,pp\. 100100\.External Links:[Document](https://dx.doi.org/10.1016/j.hybadv.2023.100100)Cited by:[§1](https://arxiv.org/html/2610.28709#S1.p1.1)\.
- \[20\]S\. Kaufman, S\. Rosset, C\. Perlich, and O\. Stitelman\(2012\)Leakage in data mining: formulation, detection, and avoidance\.ACM Transactions on Knowledge Discovery from Data6\(4\),pp\. 15\.External Links:[Document](https://dx.doi.org/10.1145/2382577.2382579)Cited by:[§3\.4](https://arxiv.org/html/2610.28709#S3.SS4.p1.1)\.
- \[21\]R\. Kohavi\(1995\)A study of cross\-validation and bootstrap for accuracy estimation and model selection\.InProceedings of the 14th International Joint Conference on Artificial Intelligence \(IJCAI\),Vol\.2,pp\. 1137–1143\.Cited by:[§3\.4](https://arxiv.org/html/2610.28709#S3.SS4.p1.1)\.
- \[22\]J\. Lee, Y\. Lee, J\. Kim, A\. R\. Kosiorek, S\. Choi, and Y\. W\. Teh\(2019\)Set transformer: a framework for attention\-based permutation\-invariant neural networks\.InProceedings of the 36th International Conference on Machine Learning \(ICML\),pp\. 3744–3753\.Cited by:[§2\.3](https://arxiv.org/html/2610.28709#S2.SS3.p3.1),[§2\.3](https://arxiv.org/html/2610.28709#S2.SS3.p4.1),[§2\.3](https://arxiv.org/html/2610.28709#S2.SS3.p5.2),[§2\.3](https://arxiv.org/html/2610.28709#S2.SS3.p6.1),[§3\.3\.1](https://arxiv.org/html/2610.28709#S3.SS3.SSS1.p1.1),[§3\.3\.1](https://arxiv.org/html/2610.28709#S3.SS3.SSS1.p3.1)\.
- \[23\]Z\. Li, N\. Kovachki, K\. Azizzadenesheli, B\. Liu, K\. Bhattacharya, A\. Stuart, and A\. Anandkumar\(2021\)Fourier neural operator for parametric partial differential equations\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://openreview.net/forum?id=c8P9NQVtmnO)Cited by:[§2\.3](https://arxiv.org/html/2610.28709#S2.SS3.p1.1)\.
- \[24\]I\. Loshchilov and F\. Hutter\(2017\)SGDR: stochastic gradient descent with warm restarts\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://openreview.net/forum?id=Skq89Scxx)Cited by:[§3\.3\.2](https://arxiv.org/html/2610.28709#S3.SS3.SSS2.p1.2)\.
- \[25\]I\. Loshchilov and F\. Hutter\(2019\)Decoupled weight decay regularization\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://openreview.net/forum?id=Bkg6RiCqY7)Cited by:[§3\.3\.2](https://arxiv.org/html/2610.28709#S3.SS3.SSS2.p1.2)\.
- \[26\]L\. Lu, P\. Jin, G\. Pang, Z\. Zhang, and G\. E\. Karniadakis\(2021\)Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators\.Nature Machine Intelligence3\(3\),pp\. 218–229\.External Links:[Document](https://dx.doi.org/10.1038/s42256-021-00302-5)Cited by:[§2\.3](https://arxiv.org/html/2610.28709#S2.SS3.p1.1)\.
- \[27\]C\. V\. More, Z\. Alsayed, M\. S\. Badawi, A\. A\. Thabet, and P\. P\. Pawar\(2021\)Polymeric composite materials for radiation shielding: a review\.Environmental Chemistry Letters19\(3\),pp\. 2057–2090\.External Links:[Document](https://dx.doi.org/10.1007/s10311-021-01189-9)Cited by:[§1](https://arxiv.org/html/2610.28709#S1.p1.1)\.
- \[28\]R\. Nowotny and A\. Höfer\(1985\)Ein Programm für die Berechnung von diagnostischen Röntgenspektren\.RöFo – Fortschritte auf dem Gebiete der Röntgenstrahlen und der Nuklearmedizin142\(6\),pp\. 685–689\(german\)\.Note:In GermanExternal Links:[Document](https://dx.doi.org/10.1055/s-2008-1052738)Cited by:[§3\.1](https://arxiv.org/html/2610.28709#S3.SS1.p1.1)\.
- \[29\]F\. Pedregosa, G\. Varoquaux, A\. Gramfort, V\. Michel, B\. Thirion, O\. Grisel, M\. Blondel, P\. Prettenhofer, R\. Weiss, V\. Dubourg, J\. Vanderplas, A\. Passos, D\. Cournapeau, M\. Brucher, M\. Perrot, and E\. Duchesnay\(2011\)Scikit\-learn: machine learning in Python\.Journal of Machine Learning Research12,pp\. 2825–2830\.Cited by:[§3\.4](https://arxiv.org/html/2610.28709#S3.SS4.p3.1),[§4\.1](https://arxiv.org/html/2610.28709#S4.SS1.p2.1)\.
- \[30\]M\. Raissi, P\. Perdikaris, and G\. E\. Karniadakis\(2019\)Physics\-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations\.Journal of Computational Physics378,pp\. 686–707\.External Links:[Document](https://dx.doi.org/10.1016/j.jcp.2018.10.045)Cited by:[§6](https://arxiv.org/html/2610.28709#S6.p2.1)\.
- \[31\]E\. Şakar, Ö\. F\. Özpolat, B\. Alım, M\. I\. Sayyed, and M\. Kurudirek\(2020\)Phy\-X/PSD: development of a user friendly online software for calculation of parameters relevant to radiation shielding and dosimetry\.Radiation Physics and Chemistry166,pp\. 108496\.External Links:[Document](https://dx.doi.org/10.1016/j.radphyschem.2019.108496)Cited by:[§1](https://arxiv.org/html/2610.28709#S1.p3.1)\.
- \[32\]Y\. Salimi, A\. Akhavanallaf, Z\. Mansouri, I\. Shiri, and H\. Zaidi\(2023\)Real\-time, acquisition parameter\-free voxel\-wise patient\-specific Monte Carlo dose reconstruction in whole\-body CT scanning using deep neural networks\.European Radiology33\(12\),pp\. 9411–9424\.External Links:[Document](https://dx.doi.org/10.1007/s00330-023-09839-y)Cited by:[§2\.3](https://arxiv.org/html/2610.28709#S2.SS3.p1.1)\.
- \[33\]F\. Salvat\(2019\)PENELOPE\-2018: a code system for Monte Carlo simulation of electron and photon transport\.Workshop ProceedingsTechnical ReportNEA/MBDAV/R\(2019\)1,OECD Nuclear Energy Agency,Boulogne\-Billancourt, France\.Note:Workshop Proceedings, Barcelona, Spain, 28 January–1 FebruaryExternal Links:[Document](https://dx.doi.org/10.1787/32da5043-en)Cited by:[§1](https://arxiv.org/html/2610.28709#S1.p4.1),[§2\.2](https://arxiv.org/html/2610.28709#S2.SS2.p1.1),[§3\.1](https://arxiv.org/html/2610.28709#S3.SS1.p3.2)\.
- \[34\]J\. Sempau, A\. Badal, and L\. Brualla\(2011\)A PENELOPE\-based system for the automated Monte Carlo simulation of clinacs and voxelized geometries—application to far\-from\-axis fields\.Medical Physics38\(11\),pp\. 5887–5895\.External Links:[Document](https://dx.doi.org/10.1118/1.3643029)Cited by:[§1](https://arxiv.org/html/2610.28709#S1.p4.1),[§2\.2](https://arxiv.org/html/2610.28709#S2.SS2.p1.1)\.
- \[35\]T\. Sterling and J\. J\. Irwin\(2015\)ZINC 15–ligand discovery for everyone\.Journal of Chemical Information and Modeling55\(11\),pp\. 2324–2337\.External Links:[Document](https://dx.doi.org/10.1021/acs.jcim.5b00559)Cited by:[§3\.2\.1](https://arxiv.org/html/2610.28709#S3.SS2.SSS1.p2.1),[§5\.2](https://arxiv.org/html/2610.28709#S5.SS2.p2.1)\.
- \[36\]A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, L\. Kaiser, and I\. Polosukhin\(2017\)Attention is all you need\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.30,pp\. 5998––6008\.Cited by:[§2\.3](https://arxiv.org/html/2610.28709#S2.SS3.p4.1)\.
- \[37\]E\. Wagstaff, F\. B\. Fuchs, M\. Engelcke, I\. Posner, and M\. A\. Osborne\(2019\)On the limitations of representing functions on sets\.InProceedings of the 36th International Conference on Machine Learning \(ICML\),Proceedings of Machine Learning Research, Vol\.97,pp\. 6487–6494\.External Links:[Link](https://proceedings.mlr.press/v97/wagstaff19a.html)Cited by:[§2\.3](https://arxiv.org/html/2610.28709#S2.SS3.p3.1)\.
- \[38\]D\. Weininger\(1988\)SMILES, a chemical language and information system\. 1\. Introduction to methodology and encoding rules\.Journal of Chemical Information and Computer Sciences28\(1\),pp\. 31–36\.External Links:[Document](https://dx.doi.org/10.1021/ci00057a005)Cited by:[§3\.2\.1](https://arxiv.org/html/2610.28709#S3.SS2.SSS1.p1.1)\.
- \[39\]M\. Zaheer, S\. Kottur, S\. Ravanbakhsh, B\. Póczos, R\. Salakhutdinov, and A\. J\. Smola\(2017\)Deep sets\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Vol\.30,pp\. 3391––3401\.Cited by:[§2\.3](https://arxiv.org/html/2610.28709#S2.SS3.p3.1)\.