EML Trees are Universal Approximators [R]

Hey! The EML function made the rounds recently on the internet as a “cool trick” that allows for the representation of all elementary functions through composition. As a mathematical curiosity, we prove a universal approximation theorem for EML(-type) trees. Intuitively, one expects that if elementary functions can be presented by compositions of EMLs, then so too can polynomials, and polynomials are dense in other functional spaces (like continuous functions or certain Sobolev spaces), then one expects to be able to approximate (to desired accuracy) any function (in a reasonably general space) through an EML tree (with an upper bound on size and depth). One of the key steps in the proof (detailed in the appendix) is an explicit construction of EML(-type) representation of binary operations, polynomials, hyperbolic tangent, and approximate partitions of unity, and subsequently using them as “LEGO” blocks to get more complex functions. There are some technical difficulties that need to be dealt with in the proof, especially in what relates to the the ill-definedness of the natural logarithm for nonpositive inputs, which prompts us to do some “sign-based decompositions” in Theorem1.Step 5 and a suitable affine map in Corollary 1. Comments are welcome! Paper: https://arxiv.org/pdf/2606.23179 (Note: I use the term “EML(-type)” in the above description because, due to some theoretical and practical reasons detailed in the paper, we generalize the original EML function by adding some learnable parameters.)