Numbers Already Carry Their Own Embeddings

# Numbers Already Carry Their Own Embeddings
Source: [https://arxiv.org/html/2606.14108](https://arxiv.org/html/2606.14108)
Suhyun Bae &Donghun Lee Department of Mathematics Korea University \{baeshstar, holy\}@korea\.ac\.kr

###### Abstract

We introduceAdelic operation\-preserved embeddings \(AOE\), a training\-free representation that captures both a number’s real value and its modular \(pp\-adic\) signatures\. This construction preserves additive and multiplicative structure by design, turning numerical input into embeddings that “speak in the language of mathematics\.” Unlike prior approaches that rely on task\-specific retraining, AOE is plug\-and\-play and drops seamlessly into existing architectures\. On algebraic combinatorics benchmarks, it delivers consistent gains—including the first\-ever perfect accuracy on the Weaving Pattern task—while suggesting a principled path forward for overcoming the long\-standing “number problem” in AI\.

## 1Introduction

LLMs have recently crossed remarkable milestones in mathematical reasoning—for instance, Gemini 2\.5 Pro attained gold medal–level performance at the International Mathematical Olympiad \(IMO\) 2025Huang and Yang \([2025](https://arxiv.org/html/2606.14108#bib.bib11)\)\. Yet a paradox has emerged: on certain core tasks, simpler baseline models like vanilla transformers or even MLPs can outperform state\-of\-the\-art systems such as Claude 3\.5 Sonnet and GPT\-4oChauet al\.\([2025](https://arxiv.org/html/2606.14108#bib.bib1)\)\. This surprising reversal highlights that scale alone has not solved one of the oldest bottlenecks in AI:numerical representation\.

Most LLMs still treat numbers as strings of characters\. From Byte Pair Encoding \(BPE\)Gage \([1994](https://arxiv.org/html/2606.14108#bib.bib12)\)to modern tokenization schemes, digits are fragmented into arbitrary subword units, stripping away the additive and multiplicative relationships that define numbers themselvesWallaceet al\.\([2019](https://arxiv.org/html/2606.14108#bib.bib13)\); Thawaniet al\.\([2021](https://arxiv.org/html/2606.14108#bib.bib15)\)\. The result is brittle reasoning, inconsistent generalization, and systematic errors in tasks that require arithmetic precision\. Prior efforts have explored continuous embeddings, digit\-wise positional encodings, or symbolic pretrainingGolkaret al\.\([2024](https://arxiv.org/html/2606.14108#bib.bib16)\); Levy and Geva \([2025](https://arxiv.org/html/2606.14108#bib.bib18)\)\. While valuable, these methods rely on the hope that modelsimplicitlyrediscover mathematical rules during training, rather than embedding them directly\.

We argue for a different path: numbers already carry their own embeddings\. Drawing on algebraic number theory, we introduce Adelic operation\-preserved embeddings \(AOE\), a training\-free representation that encodes each number by combining its real value with modular \(pp\-adic\) expansions\. This construction preserves both additive and multiplicative structures by design, providing a principled alternative to ad hoc tokenization\.

#### Contributions

Our contributions are as follows:

1. 1\.We propose AOE, a training\-free, algebraically grounded representation that directly respects numerical structure\.
2. 2\.We frame AOE as a step toward resolving the long\-standing “number problem” in AI, offering a plug\-and\-play remedy applicable across architectures\.
3. 3\.On algebraic combinatorics benchmarks, AOE\-equipped Transformers outperform baselines on every task, notably achieving the first\-ever perfect accuracy on the Weaving Pattern benchmark\.

## 2Method

Our goal is to design a numerical representation that preserves the algebraic structure already inherent in numbers\. Rather than fragmenting numbers into subword tokens, we construct embeddings directly in an*Adele space*, which combines a real component with multiplepp\-adic components\. Eachpp\-adic component lies inℤp\\mathbb\{Z\}\_\{p\}, a ring with well\-defined addition and multiplication inherited fromℤ\\mathbb\{Z\}\. As a result, this yields a product structure where arithmetic is preserved coordinate\-wise: for rationalsq1,q2q\_\{1\},q\_\{2\},

𝐀​\(q1\+q2\)=𝐀​\(q1\)⊕𝐀​\(q2\),𝐀​\(q1⋅q2\)=𝐀​\(q1\)⊗𝐀​\(q2\)\\mathbf\{A\}\(q\_\{1\}\+q\_\{2\}\)=\\mathbf\{A\}\(q\_\{1\}\)\\oplus\\mathbf\{A\}\(q\_\{2\}\),\\quad\\mathbf\{A\}\(q\_\{1\}\\cdot q\_\{2\}\)=\\mathbf\{A\}\(q\_\{1\}\)\\otimes\\mathbf\{A\}\(q\_\{2\}\)where⊕,⊗\\oplus,\\,\\otimesdenote coordinate\-wise operations on concatenated real andpp\-adic coordinates implemented viaNN\-digit truncations\.

#### Adele Space\.

The Adele Ring, a classical construction from algebraic number theoryLang \([1994](https://arxiv.org/html/2606.14108#bib.bib2)\), unifies the real line with allpp\-adic completions of the rationals\. Intuitively, each number carries a distinct “signature” across different primes, which AOE captures alongside its real value\. For example,7/5\\nicefrac\{\{7\}\}\{\{5\}\}is represented not only as1\.41\.4inℝ\\mathbb\{R\}but also throughpp\-adic expansions, such as…​11011\\dots 11011in the 2\-adics or…​01212\\dots 01212in the 3\-adics\. These complementary perspectives allow the embedding to encode both magnitude and modular structure\.

#### Embedding Construction\.

To build the embedding of a rational numberqq, we concatenate: \(i\) a real\-valued component, and \(ii\) a finite set ofpp\-adic expansions truncated toNNdigits of precision\. The resulting tensor has shape\(np\+1\)×N\(n\_\{p\}\+1\)\\times N, wherenpn\_\{p\}is the number of chosen primes\. Implementation details—such as digit lifting with Hensel’s Lemma and exactpp\-adic expansions—are provided in the[Appendix˜A](https://arxiv.org/html/2606.14108#A1)and[Appendix˜B](https://arxiv.org/html/2606.14108#A2)

#### Plug\-and\-play Integration\.

AOE is training\-free and architecture\-agnostic\. It can directly replace standard embedding layers in Transformers or other neural models without task\-specific tuning\. In practice, this means the input embedding lookup is replaced with pre\-computed Adelic representation tensors, which are flattened intodmodeld\_\{\\text\{model\}\}\-dimensional vectors\. We also add a lightweight 2D positional encoding to incorporate both the sequence position and the internal structure of each Adelic tensor\. The resulting pipeline integrates seamlessly with standard Transformer encoders, as illustrated in[Fig\.˜1](https://arxiv.org/html/2606.14108#S2.F1)\.

![Refer to caption](https://arxiv.org/html/2606.14108v1/x1.png)Figure 1:Diagram summarizing AOE\-equipped Transformer model workflow architecture for experiments\.

## 3Experiments and Results

### 3\.1Models

To isolate the effect of input representation, we implement two Transformer encoder models that are identical in architecture but differ in their embedding layers\. Both models use a 6\-layer Transformer encoder with 8 attention heads, hidden dimensiondmodel=128d\_\{\\text\{model\}\}=128, and a dropout rate of 0\.1\. The final hidden state corresponding to the \[CLS\] token is passed through a linear layer for classification\.

TheBaselinemodel employs a standardnn\.Embeddinglookup for integer tokens, followed by sinusoidal positional encodings\. Becausenn\.Embeddingis trainable, this baseline introduces additional learnable parameters\. By contrast, theAOE\-equippedmodel replaces the embedding lookup with pre\-computed Adelic representation tensors, which are training\-free and flattened intodmodeld\_\{\\text\{model\}\}\-dimensional vectors\. We also add a 2D positional encoding that jointly encodes the sequence index, the prime index, and the digit index of eachpp\-adic expansion, thereby preserving structural information\.

Crucially, this design ensures that any performance differences arise solely from the representation, not from model capacity: the baseline in fact has more learnable parameters, making AOE’s improvements all the more significant\.

### 3\.2Dataset

We evaluate our approach on theAlgebraic Combinatorics Dataset \(ACD\)Chauet al\.\([2025](https://arxiv.org/html/2606.14108#bib.bib1)\), a benchmark designed to support the conjecturing phase of mathematical research\. Unlike benchmarks such as`GSM8K`Cobbeet al\.\([2021](https://arxiv.org/html/2606.14108#bib.bib21)\)or other arithmetic word\-problem datasets—which primarily focus on middle or elementary school mathematics—ACD incorporates tasks linked to open problems in algebraic combinatorics\. It is therefore a uniquely demanding testbed: it not only exhibits scale and imbalance \(up to10710^\{7\}instances with heavy\-tailed distributions\) but also requires models to reason over algebraic structures well beyond rote calculation\.

From the nine tasks available in the repository, we focus on six classification problems spanning permutations, tableaux, lattice paths, quivers, and polynomial invariants\. This selection provides both breadth of combinatorial objects and diversity of input types \(strings, matrices, graphs\)\. Detailed task descriptions and mathematical context are deferred to Appendix[C](https://arxiv.org/html/2606.14108#A3)\.

### 3\.3Results

Both models are trained from scratch with the Adam optimizer and Cross\-Entropy loss\. To address class imbalance in the ACD tasks, we use a`WeightedRandomSampler`to balance training batches\. Additional hyperparameters, learning rate schedules, and per\-task settings are reported in[Appendix˜D](https://arxiv.org/html/2606.14108#A4)\.

Across all six tasks, AOE consistently outperforms the baseline\. This supports our central claim that*explicitly encoding algebraic structure enables stronger generalization than text\-based embeddings*\.

The improvements range from modest but consistent gains \(e\.g\., lattice path classification\) to dramatic jumps \(e\.g\., Schubert polynomials and quiver mutation classes\)\. Most strikingly, our model achieves the first\-ever perfect accuracy on the Weaving Patterns benchmark, a task that has resisted even large\-scale language models\.

These findings suggest that algebraically grounded representations can close long\-standing gaps in numerical reasoning\. Rather than relying on scale or implicit learning, embedding numbers in a mathematically faithful space offers a principled path to more reliable symbolic reasoning in neural models\.

[Table˜1](https://arxiv.org/html/2606.14108#S3.T1)and[Table˜3](https://arxiv.org/html/2606.14108#A5.T3)provide detailed accuracy and loss values\.

Table 1:Final test accuracy \(%\) comparison on the Algebraic Combinatorics benchmark\. Higher is better\.

## 4Discussion

Our current framework is restricted to rational numbers, since the Adele Ring is defined overℚ\\mathbb\{Q\}\. Extending the construction to irrationals, complex values, or transcendental constants requires new theoretical tools\.

Another limitation concerns the manual choice of primes and digit precision\. In this study, we fix these hyperparameters heuristically\. More principled or adaptive strategies may improve both efficiency and expressivity\.

Finally, while AOE is training\-free, its use introduces computational overhead: pre\-computations and 2D positional encodings make training roughly44–55times slower than standard embeddings\.

We view these challenges not as barriers but as opportunities for future work in broadening AOE toward more general and scalable settings\.

## 5Conclusion

This work begins from a simple observation: numbers already contain the structure they need to be represented\. By making this structure explicit through AOE, we show that neural models achieve levels of reliability on symbolic mathematical tasks that elude even the largest LLMs\.

The broader message is conceptual\. If numbers are more than tokens, then the future of AI reasoning hinges on embedding*mathematical structures of objects*, not merely the language that describes them\. AOE exemplifies how ideas from algebraic number theory operate as plug\-and\-play components in neural architectures, suggesting a path toward uniting symbolic rigor with neural scalability\.

## Acknowledgments and Disclosure of Funding

This work was supported by the National Research Foundation of Korea \(NRF\) grant funded by the Korea government \(MSIT\) \(RS\-2025\-24873052\)\.

## References

- \[1\]H\. Chau, H\. Jenne, D\. Brown, J\. He, M\. Raugas, S\. C\. Billey, and H\. Kvinge\(2025\)Machine learning meets algebraic combinatorics: a suite of datasets capturing research\-level conjecturing ability in pure mathematics\.InForty\-second International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=tlniJJFUW2)Cited by:[Appendix C](https://arxiv.org/html/2606.14108#A3.p1.1),[§1](https://arxiv.org/html/2606.14108#S1.p1.1),[§3\.2](https://arxiv.org/html/2606.14108#S3.SS2.p1.1)\.
- \[2\]K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. Schulman\(2021\)Training verifiers to solve math word problems\.External Links:2110\.14168,[Link](https://arxiv.org/abs/2110.14168)Cited by:[§3\.2](https://arxiv.org/html/2606.14108#S3.SS2.p1.1)\.
- \[3\]P\. Gage\(1994\-02\)A new algorithm for data compression\.C Users J\.12\(2\),pp\. 23–38\.External Links:ISSN 0898\-9788Cited by:[§1](https://arxiv.org/html/2606.14108#S1.p2.1)\.
- \[4\]S\. Golkar, M\. Pettee, M\. Eickenberg, A\. Bietti, M\. Cranmer, G\. Krawezik, F\. Lanusse, M\. McCabe, R\. Ohana, L\. Parker, B\. R\. Blancard, T\. Tesileanu, K\. Cho, and S\. Ho\(2024\)XVal: a continuous numerical tokenization for scientific language models\.External Links:2310\.02989,[Link](https://arxiv.org/abs/2310.02989)Cited by:[§1](https://arxiv.org/html/2606.14108#S1.p2.1)\.
- \[5\]K\. Hensel\(1908\)Theorie der algebraischen zahlen\.B\. G\. Teubner,Leipzig\.External Links:[Link](https://archive.org/details/in.ernet.dli.2015.493154)Cited by:[2nd item](https://arxiv.org/html/2606.14108#A1.I1.i2.I1.i2.p1.2)\.
- \[6\]Y\. Huang and L\. F\. Yang\(2025\)Gemini 2\.5 pro capable of winning gold at imo 2025\.External Links:2507\.15855,[Link](https://arxiv.org/abs/2507.15855)Cited by:[§1](https://arxiv.org/html/2606.14108#S1.p1.1)\.
- \[7\]S\. Lang\(1994\)Ideles and adeles\.InAlgebraic Number Theory,pp\. 137–154\.External Links:ISBN 978\-1\-4612\-0853\-2,[Document](https://dx.doi.org/10.1007/978-1-4612-0853-2%5F7),[Link](https://doi.org/10.1007/978-1-4612-0853-2_7)Cited by:[§2](https://arxiv.org/html/2606.14108#S2.SS0.SSS0.Px1.p1.7)\.
- \[8\]A\. A\. Levy and M\. Geva\(2025\)Language models encode numbers using digit representations in base 10\.External Links:2410\.11781,[Link](https://arxiv.org/abs/2410.11781)Cited by:[§1](https://arxiv.org/html/2606.14108#S1.p2.1)\.
- \[9\]A\. Ostrowski\(1916\)Über einige lösungen der funktionalgleichung φ\(x\)·φ\(y\)=φ\(xy\)\.Acta Mathematica41\(1\),pp\. 271–284\.External Links:[Document](https://dx.doi.org/10.1007/BF02422947),ISSN 0001\-5962Cited by:[§A\.1](https://arxiv.org/html/2606.14108#A1.SS1.p1.5)\.
- \[10\]A\. Thawani, J\. Pujara, P\. A\. Szekely, and F\. Ilievski\(2021\)Representing numbers in nlp: a survey and a vision\.External Links:2103\.13136,[Link](https://arxiv.org/abs/2103.13136)Cited by:[§1](https://arxiv.org/html/2606.14108#S1.p2.1)\.
- \[11\]E\. Wallace, Y\. Wang, S\. Li, S\. Singh, and M\. Gardner\(2019\-11\)Do NLP models know numbers? probing numeracy in embeddings\.InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing \(EMNLP\-IJCNLP\),K\. Inui, J\. Jiang, V\. Ng, and X\. Wan \(Eds\.\),Hong Kong, China,pp\. 5307–5315\.External Links:[Link](https://aclanthology.org/D19-1534/),[Document](https://dx.doi.org/10.18653/v1/D19-1534)Cited by:[§1](https://arxiv.org/html/2606.14108#S1.p2.1)\.

## Appendix AFormalization of Adelic Operation\-preserved Embedding

### A\.1Adele Ring

###### Definition 1\(Adele Ring\)\.

LetKKbe a global field, and𝔸K\\mathbb\{A\}\_\{K\}be an "Adele Ring" if

𝔸K=∏v\(Kv,𝒪v\)\\mathbb\{A\}\_\{K\}=\\prod\_\{v\}\(K\_\{v\},\\mathcal\{O\}\_\{v\}\)wherevvrepresents all possible valuations onKK\.

For the field of rationalsK=ℚK=\\mathbb\{Q\},Ostrowski’s theoremOstrowski \([1916](https://arxiv.org/html/2606.14108#bib.bib3)\)implies that all non\-trivial valuations correspond either to the realsℝ\\mathbb\{R\}or topp\-adic integersℤp\\mathbb\{Z\}\_\{p\}for some primepp\. Hence:

𝔸ℚ=ℝ×∏p\(ℚp,ℤp\)\.\\mathbb\{A\}\_\{\\mathbb\{Q\}\}=\\mathbb\{R\}\\times\\prod\_\{p\}\(\\mathbb\{Q\}\_\{p\},\\mathbb\{Z\}\_\{p\}\)\.
An elementa∈𝔸ℚa\\in\\mathbb\{A\}\_\{\\mathbb\{Q\}\}can be written as an infinite tuple:

a=\(a∞,a2,a3,a5,a7,…\)\.a=\(a\_\{\\infty\},a\_\{2\},a\_\{3\},a\_\{5\},a\_\{7\},\\dots\)\.

### A\.2Example Representation

For instance, the rational number75\\tfrac\{7\}\{5\}can be represented as:

𝐀ℚ​\(75\)=\(1\.4,…​110112,…​012123,1\.25,…​541307,…\)\\mathbf\{A\}\_\{\\mathbb\{Q\}\}\\left\(\\frac\{7\}\{5\}\\right\)=\(1\.4,~\\dots 11011\_\{2\},~\\dots 01212\_\{3\},1\.2\_\{5\},~\\dots 54130\_\{7\},~\\dots\)
where eachpp\-adic component is approximated with anNN\-digit precision\.

### A\.3Embedding Construction

Given a rationalq=nmq=\\frac\{n\}\{m\}, we approximate its Adele embedding𝐀​\(q\)\\mathbf\{A\}\(q\)as follows:

1. 1\.Real Component \(a∞a\_\{\\infty\}\)\. Representqqon the real line as a length\-NNvector: a∞↦\[0,0,…,q\]∈ℝNa\_\{\\infty\}\\mapsto\[0,0,\\dots,q\]\\in\\mathbb\{R\}^\{N\}
2. 2\.pp\-adic Components \(apa\_\{p\}\)\. For each primeppin a fixed set, compute theNN\-digitpp\-adic expansion ofqq: - •Solve the congruencem​x≡n\(modpN\)mx\\equiv n\\pmod\{p^\{N\}\}\. - •UseHensel’s LemmaHensel \([1908](https://arxiv.org/html/2606.14108#bib.bib4)\)to iteratively lift a solution moduloppto one modulopNp^\{N\}\. This method is described in detail in Appendix[B](https://arxiv.org/html/2606.14108#A2) - •Convertxxinto base\-pp, yielding digits\[cN−1,…,c0\]∈ℤpN\[c\_\{N\-1\},\\dots,c\_\{0\}\]\\in\\mathbb\{Z\}\_\{p\}^\{N\}\. Thus, ap↦\[cN−1,cN−2,…,c0\]a\_\{p\}\\mapsto\[c\_\{N\-1\},c\_\{N\-2\},\\dots,c\_\{0\}\]
3. 3\.Final Adelic Vector\. Concatenate the real component and all chosenpp\-adic vectors: 𝐀​\(q\)=concat​\(a∞,ap1,ap2,…,apnp\)\\mathbf\{A\}\(q\)=\\text\{concat\}\(a\_\{\\infty\},a\_\{p\_\{1\}\},a\_\{p\_\{2\}\},\\dots,a\_\{p\_\{n\_\{p\}\}\}\)The resulting tensor has shape\(np\+1\)×N\(n\_\{p\}\+1\)\\times N, wherenpn\_\{p\}is the number of selected primes andNNis the digit precision\.

## Appendix BHensel’s Lemma

In our method, we need to solve the congruence relationm​x≡n\(modpN\)mx\\equiv n\\pmod\{p^\{N\}\}to find thepp\-adic representation of a rational numbernm\\frac\{n\}\{m\}\. Hensel’s Lemma provides a powerful and efficient way to achieve this\.

At its core, Hensel’s Lemma is an analogue of Newton’s method for finding roots, but for modular arithmetic\. The core idea is "lifting": if you have an approximate solution to an equation modulopp, you can use that information to construct a more precise solution modulop2p^\{2\}, thenp3p^\{3\}, and so on, up to any desired powerpNp^\{N\}\. This iterative refinement process is exactly what we need to determine the digits of app\-adic number to a specific precision\.

More formally, one common version of Hensel’s Lemma is stated for polynomial roots\.

###### Theorem 1\(Hensel’s Lemma\)\.

Letf​\(x\)f\(x\)be a polynomial with integer coefficients, and letf′​\(x\)f^\{\\prime\}\(x\)be its derivative\. If an integerrris a root off​\(x\)f\(x\)modulo a primepp\(i\.e\.,f​\(r\)≡0\(modp\)f\(r\)\\equiv 0\\pmod\{p\}\), and if the derivative at that point is non\-zero modulopp\(i\.e\.,f′​\(r\)≢0\(modp\)f^\{\\prime\}\(r\)\\not\\equiv 0\\pmod\{p\}\), then for any integerk≥1k\\geq 1, there exists a unique integerssmodulopkp^\{k\}such thatf​\(s\)≡0\(modpk\)f\(s\)\\equiv 0\\pmod\{p^\{k\}\}ands≡r\(modp\)s\\equiv r\\pmod\{p\}\.

#### Application to Our Method\.

In our case, we are not solving a general polynomial equation, but a linear congruencem​x≡n\(modpN\)mx\\equiv n\\pmod\{p^\{N\}\}\. We can frame this as finding a root of the linear polynomialf​\(x\)=m​x−nf\(x\)=mx\-n\.

- •The polynomial isf​\(x\)=m​x−nf\(x\)=mx\-n\.
- •Its derivative isf′​\(x\)=mf^\{\\prime\}\(x\)=m\.

To apply Hensel’s Lemma, we first need a solutionx1x\_\{1\}modulopp\. This requires solvingm​x1−n≡0\(modp\)mx\_\{1\}\-n\\equiv 0\\pmod\{p\}\. A solution exists if and only ifm≢0\(modp\)m\\not\\equiv 0\\pmod\{p\}\(i\.e\.,ppdoes not dividemm\)\. The derivative condition,f′​\(x1\)=m≢0\(modp\)f^\{\\prime\}\(x\_\{1\}\)=m\\not\\equiv 0\\pmod\{p\}, is the very same condition\.

The iterative lifting process then works as follows: Suppose we have a solutionxkx\_\{k\}such thatm​xk≡n\(modpk\)mx\_\{k\}\\equiv n\\pmod\{p^\{k\}\}\. We seek a new solutionxk\+1x\_\{k\+1\}of the formxk\+1=xk\+t⋅pkx\_\{k\+1\}=x\_\{k\}\+t\\cdot p^\{k\}for some integert∈\{0,1,…,p−1\}t\\in\\\{0,1,\\dots,p\-1\\\}\. We substitute this into the congruence for the next level:

m​\(xk\+t⋅pk\)\\displaystyle m\(x\_\{k\}\+t\\cdot p^\{k\}\)≡n\(modpk\+1\)\\displaystyle\\equiv n\\pmod\{p^\{k\+1\}\}m​xk−n\+m​t⋅pk\\displaystyle mx\_\{k\}\-n\+mt\\cdot p^\{k\}≡0\(modpk\+1\)\\displaystyle\\equiv 0\\pmod\{p^\{k\+1\}\}
Sincem​xk−nmx\_\{k\}\-nis a multiple ofpkp^\{k\}, we can write it asc⋅pkc\\cdot p^\{k\}for some integercc\. Substituting this gives:

c⋅pk\+m​t⋅pk\\displaystyle c\\cdot p^\{k\}\+mt\\cdot p^\{k\}≡0\(modpk\+1\)\\displaystyle\\equiv 0\\pmod\{p^\{k\+1\}\}c\+m​t\\displaystyle c\+mt≡0\(modp\)\(after dividing by​pk\)\\displaystyle\\equiv 0\\pmod\{p\}\\quad\(\\text\{after dividing by \}p^\{k\}\)t\\displaystyle t≡−c⋅m−1\(modp\)\\displaystyle\\equiv\-c\\cdot m^\{\-1\}\\pmod\{p\}
wherem−1m^\{\-1\}is the modular multiplicative inverse ofmmmodulopp, andc=\(m​xk−n\)/pkc=\(mx\_\{k\}\-n\)/p^\{k\}\. This formula provides a direct way to compute the next "correction term"tt, thereby lifting the solution from precisionkktok\+1k\+1\. Our implementation programmatically applies this lifting fromk=1k=1up toNN\.

## Appendix CDataset Details

We conduct experiments on theAlgebraic Combinatorics Dataset \(ACD\) RepositoryChauet al\.\([2025](https://arxiv.org/html/2606.14108#bib.bib1)\), a suite of nine research\-level datasets designed to support the conjecturing phase of mathematical research in algebraic combinatorics\. Each dataset pairs raw discrete combinatorial objects with mathematically meaningful labels, providing tasks that go beyond closed\-form evaluation and emphasize pattern discovery for open mathematical problems\.

#### Task Coverage\.

The ACD Repository spans diverse problems in algebraic combinatorics:

- •Symmetric Group Characters: Regression of irreducible symmetric group charactersχμλ\\chi^\{\\lambda\}\_\{\\mu\}for partitionsλ,μ⊢n\\lambda,\\mu\\vdash n\(n∈\{18,20,22\}n\\in\\\{18,20,22\\\}\)\. This foundational result connects representation theory with combinatorial interpretations via Young diagrams\. Example \(n=18n=18\):Input x is a pair of partitions\. Output y is the integer character value\. ``` x = {’Irreducible rep...’: [5, 4, 3, ...], ’Conjugacy class’: [7, 7, 4]} y = 0 ```
- •mHeight Function: Classification of the mHeight statistic on permutations \(n∈\{8,9,10\}n\\in\\\{8,9,10\\\}\), a key tool in the recent proof of the Billey\-Postnikov conjecture about Kazhdan\-Lusztig polynomial coefficients\. Example \(n=8n=8\):Input ‘x‘ is a permutation\. Output ‘y‘ is its mHeight class\. ``` x = {’Permutation’: [2, 1, 5, 0, 7, 3, 6, 4]} y = 0 ```
- •Kazhdan\-Lusztig Polynomial Coefficients: Prediction of polynomial coefficientsPx,w​\(q\)P\_\{x,w\}\(q\)indexed by permutation pairs \(n∈\{5,6,7\}n\\in\\\{5,6,7\\\}\)\. These polynomials have deep connections to Schubert calculus and representation theory\. Example \(n=5n=5\):Input ‘x‘ is a pair of permutations\. Output ‘y‘ is a list of coefficients\. ``` x = {’Permutation 1’: [2, 1, 0, 3, 4], ’Permutation 2’: [2, 1, 4, 3, 0]} y = [1] ```
- •Robinson\-Schensted\-Knuth \(RSK\) Correspondence: Recovery of permutations from pairs of standard Young tableaux \(n∈\{8,9\}n\\in\\\{8,9\\\}\), testing whether models can learn this fundamental bijection in algebraic combinatorics\. Example \(n=8n=8\):Input ‘x‘ is a pair of standard Young tableaux\. Output ‘y‘ is the recovered permutation\. ``` x = {’SYT 1’: [[1, 3, 4, 6], [2, 8], ...], ’SYT 2’: [[1, 2, 4, 6], [3, 5], ...]} y = [2, 7, 3, 8, 5, 6, 4, 1] ```
- •Schubert Polynomial Structure Constants: Prediction of structure constantscα,βγc^\{\\gamma\}\_\{\\alpha,\\beta\}in the expansionSα​Sβ=∑γcα,βγ​SγS\_\{\\alpha\}S\_\{\\beta\}=\\sum\_\{\\gamma\}c^\{\\gamma\}\_\{\\alpha,\\beta\}S\_\{\\gamma\}\(n∈\{4,5,6\}n\\in\\\{4,5,6\\\}\)\. Finding combinatorial interpretations of these constants is a major open problem\. Example \(n=4n=4\):Input ‘x‘ is a triplet of indices\. Output ‘y‘ is the integer structure constant\. ``` x = {’Lower index 1’: [3, 1, 4, 2], ’Lower index 2’: [1, 4, 3, 2], ’Upper index’: [3, 7, 4, ...]} y = 0 ```
- •Grassmannian Cluster Algebras: Binary classification of whether semistandard Young tableaux of shape3×43\\times 4with entries from\{1,…,12\}\\\{1,\\ldots,12\\\}index cluster variables inGr​\(3,12\)\\text\{Gr\}\(3,12\)\. Example:Input ‘x‘ is a semistandard Young tableau\. Output ‘y‘ is a binary label\. ``` x = {’SSYT’: [[1, 1, 5, 7], [4, 5, 7, 9], ...]} y = 0 ```
- •Lattice Path Partial Orders: Classification of covering pairs in lattice paths from\(0,0\)\(0,0\)to\(n,n−1\)\(n,n\-1\)under Lagrange vs\. matching orderings \(n∈\{10,11,12,13\}n\\in\\\{10,11,12,13\\\}\), motivated by connections to number theory\. Example \(n=10n=10\):Input ‘x‘ is a pair of paths\. Output ‘y‘ is an integer label\. ``` x = {’Lattice path 1’: [1, 1, 0, ..., 0, 1, 0], ’Lattice path 2’: [1, 1, 0, ..., 1, 0, 0]} y = 0 ```
- •Quiver Mutation Classes: Classification of 11\-vertex quivers into seven mutation equivalence classes \(A11, D11, E11, BD11, BE11, DE11, BB11\), where determining mutation equivalence is an open algorithmic problem\. Example:Input ‘x‘ is a flattened adjacency matrix\. Output ‘y‘ is the mutation class name\. ``` x = {’Adjacency matrix’: [0, 0, 0, ..., 1, 0, ...]} y = ’A_11’ ```
- •Weaving Patterns: Binary classification ofn×\(n−1\)n\\times\(n\-1\)matrices with entries in\{1,…,n\}\\\{1,\\ldots,n\\\}as valid weaving patterns \(n∈\{6,7\}n\\in\\\{6,7\\\}\), related to reduced decompositions of the longest permutation\. Example \(n=6n=6\):Input ‘x‘ is a flattened matrix\. Output ‘y‘ is a binary label\. ``` x = {’Matrix’: [5, 4, 3, 2, 1, 6, 2, 3, ...]} y = 1 ```

#### Scale and Characteristics\.

The datasets exhibit substantial scale variation, ranging from thousands to over 10 million examples, with dataset sizes typically growing exponentially in the parameternn\. Class imbalance is prevalent across tasks—for instance, zero\-valued structure constants vastly outnumber non\-zero ones, and most quiver mutation classes are represented unequally\. The distributions often exhibit heavy tails \(e\.g\., symmetric group characters are concentrated around zero with long tails\), making standard accuracy metrics potentially misleading\.

#### Mathematical Significance\.

Unlike typical machine learning benchmarks, these datasets represent either:

1. 1\.Foundational resultswhere efficient algorithms exist \(e\.g\., RSK correspondence, symmetric group characters\) but learning them from examples tests a model’s ability to discover mathematical structure;
2. 2\.Open problemswhere no complete characterization is known \(e\.g\., Schubert polynomial structure constants, quiver mutation equivalence for general types\), making them genuine research\-level challenges\.

#### Tasks Used in Our Study\.

From this collection, we focus on six classification tasks that span the breadth of combinatorial objects and mathematical contexts: mHeight function, Grassmannian cluster algebras, lattice path partial orders, quiver mutation classes, weaving patterns, and Schubert polynomial structure constants\. These tasks were selected to provide diverse input representations \(permutations, tableaux, matrices, graphs\) and varying degrees of class imbalance\. Full dataset statistics, generation procedures, and train/test splits are provided with the ACD Repository\.

## Appendix DTraining Details

#### Optimizer and Objective\.

We use the Adam optimizer with defaultβ\\betavalues\(0\.9,0\.999\)\(0\.9,0\.999\)and weight decay set to0\. The training objective is Cross\-Entropy Loss for all classification tasks\.

#### Learning Rate Scheduler\.

We employ a cosine annealing learning rate scheduler`CosineAnnealingLR`\. For each task, the maximum number of epochs is set asTmaxT\_\{\\max\}, ensuring that the learning rate decays smoothly over the full training run\. Initial learning rates are task\-specific, as summarized in Table[2](https://arxiv.org/html/2606.14108#A4.T2)\.

#### Batching and Sampling\.

All experiments use mini\-batches of size 2048\. To address severe class imbalance, we adopt a`WeightedRandomSampler`, where the sampling weight for classiiis proportional to1ni\\frac\{1\}\{\\sqrt\{n\_\{i\}\}\}, withnin\_\{i\}denoting the number of samples in classii\. This strategy ensures minority classes are not neglected while avoiding over\-amplification of extremely rare labels\.

#### Training Epochs\.

Each task is trained with its own maximum epoch count \(see Table[2](https://arxiv.org/html/2606.14108#A4.T2)\)\. Validation is performed at the end of each epoch, and the checkpoint with the best validation accuracy is selected for reporting final results\.

#### Hardware\.

All models are trained on a single NVIDIA RTX 5090 GPU \(32GB\)\. Experiments are implemented in PyTorch \(version2\.6\.0a0\+df5bbc09d1\.nv24\.11\)\.

#### Hyperparameter Summary\.

Table[2](https://arxiv.org/html/2606.14108#A4.T2)lists the main hyperparameters and per\-task settings used in our experiments\.

Table 2:Per\-task training hyperparameters\. CosineAnnealingLR scheduler usesTmax=T\_\{\\max\}=maximum epoch for each configuration\. Batch size is fixed to 2048 for all tasks\. Learning rates were selected via Optuna\-based hyperparameter search\.

## Appendix ETest Loss Table

[Table˜3](https://arxiv.org/html/2606.14108#A5.T3)lists the test losses per\-task corresponding to the accuracy results reported in[Section˜3\.3](https://arxiv.org/html/2606.14108#S3.SS3)\. While accuracy provides a direct measure of task performance, the loss values offer an additional perspective on model calibration and confidence\. Consistent with the accuracy gains, AOE achieves lower or comparable losses across nearly all tasks, reinforcing that its improvements are not limited to classification correctness but extend to the stability of the underlying optimization\.

Table 3:Final test loss comparison on the Algebraic Combinatorics benchmark\. Lower is better\.