[R] Deterministic attention-transformer with measured energy savings on H100 (0.63 J/token)

Reddit r/LocalLLaMA 07/13/26, 10:31 PM Papers

attention-transformer rust cuda energy-efficiency deterministic open-source benchmark

Summary

The article presents a custom Rust + CUDA attention-transformer engine achieving 0.63 J/token energy efficiency on H100 with bit-exact determinism, surpassing typical models, and plans open-source release after patent.

I’ve been working on a custom Rust + CUDA attention-transformer engine (GAE + ATE + WNSM + reversible training) aimed at determinism and real energy efficiency. Latest sustained numbers on H100 NVL (28-layer 7B-class stack, continuous batch): Throughput: ~403 tokens/second Energy: 0.63 J/token Power: ~254 W median board power The engine is bit-exact (AUDIT verified), supports both inference and training on the same stack, and runs on multiple hardware backends (H100, M2 Pro, consumer GPUs, WebGPU). Full benchmarks, methods, and evidence are on the site: https://luxiedge.com Open-source planned after patent issuance this month (dual license). Curious if anyone has thoughts on the energy numbers or similar work.

Original Article

Similar Articles

Hamiltonian-Inspired Attention Mechanism for Scalable RF Transmitter Fingerprinting

arXiv cs.AI

Proposes a Hamiltonian Transformer, a physics-informed attention mechanism that enforces norm-preserving value dynamics for RF transmitter fingerprinting, achieving 99.12% accuracy in same-day conditions and 61.64% with 150 transmitters, outperforming CNN and Transformer baselines.

Z1T (15 minute read)

TLDR AI

Extropic introduces Z1T, a sparse transformer-like model optimized for their probabilistic Z1 hardware to enhance energy efficiency in transformer inference through algorithmic-hardware co-design.

@h100envy: CMU PhD who built the kernels NVIDIA now ships in TensorRT-LLM explained fast attention in 68 minutes - better than $12…

X AI KOLs Timeline

A CMU PhD who developed the kernels now used by NVIDIA in TensorRT-LLM explains fast attention, covering fused CUDA kernels, FlashInfer, Triton, and paged-KV attention, enabling more tokens per second on the same GPU.

From Tokens to Watt-hours: Analytical Energy Estimation for LLM Inference on Modern GPUs

arXiv cs.LG

This paper presents an analytically structured, empirically calibrated methodology for estimating LLM inference energy on NVIDIA H100 GPUs without direct measurement, separating prefill and decoding phases and decomposing energy into compute, parameter-access, KV-cache write, and attention-read components.

@waterloo_intern: After reading up a bit on ML research post transformer era, I was upset that it seems to have converged on hyper-optimi…