post-transformer

#post-transformer

Pathway's BDH(post-transformer arch) matches GPT2 scaling from 10M to 1B params trained from scratch. runs on Normal GPUs

Reddit r/LocalLLaMA ↗ · 4h ago

Pathway's BDH, a post-transformer architecture, reportedly matches GPT-2 scaling from 10M to 1B parameters while training from scratch on standard GPUs.

0 favorites 0 likes

#post-transformer

Continual learning in mid-2026. A map of everyone trying to crack it: memory layers, "dreaming" agents, and the Post-Transformer models that learn inside the network

Reddit r/artificial ↗ · 2026-06-12

An overview of the current state and future outlook of continual learning in mid-2026, covering memory approaches including external memory, in-state memory, and weight updates, with analysis of various models like TTT, Titans, and Dragon Hatchling.

0 favorites 0 likes

#post-transformer

One of the authors of "Attention is All You Need" just argued we should move past it. Pathway’s Post-Transformer debate is worth watching

Reddit r/singularity ↗ · 2026-05-25

A co-author of the seminal 'Attention is All You Need' paper has argued that the field should move beyond transformers, and a debate hosted by Pathway explores this topic.

0 favorites 0 likes

post-transformer

Pathway's BDH(post-transformer arch) matches GPT2 scaling from 10M to 1B params trained from scratch. runs on Normal GPUs

Continual learning in mid-2026. A map of everyone trying to crack it: memory layers, "dreaming" agents, and the Post-Transformer models that learn inside the network

One of the authors of "Attention is All You Need" just argued we should move past it. Pathway’s Post-Transformer debate is worth watching

Submit Feedback