Tag
A blogger shares Jia-Bin Huang's 9-episode, 230-minute video series "AI Training and Inference", which systematically covers the training and inference architecture of foundation models: from ZeRO/FSDP memory optimization, five-dimensional parallelism, LoRA variants, speculative decoding, on-policy distillation, GRPO reinforcement learning, to AdamW/Muon optimizers and LLM serving.
PhantaField introduces the PFG-1 'Sophon' monolithic-3D AI ASIC featuring 330 GB of on-die DRAM and pure digital compute-in-memory, eliminating HBM and delivering up to 4,200 TFLOPS FP8 for training and inference with significantly higher efficiency than current GPUs.
This article outlines the architectural building blocks for training and inferring foundation models on AWS, covering infrastructure, resource orchestration, ML software stacks, and observability.