Tag
A developer shares lessons learned while building Apex-2, a small open LLM trained on ~80B tokens for about $2,000, covering GPU bottlenecks, DiLoCo-style distributed training with GH200 instances, and aggressive MinHash deduplication of FineWeb-Edu and DCLM.