Tag
The article questions whether the proliferation of AI agents is leading to increased complexity in management and maintenance, potentially outweighing their benefits.
Andrej Karpathy's Stanford lecture offers a concise explanation of AI engineering, focusing on building systems around models with context, memory, tools, and feedback loops.
InfraBench is a benchmark suite for evaluating AI agents on realistic infrastructure management tasks across the full stack, lifecycle, and risk levels. Experiments show that even the strongest agent configurations achieve only 40-88% effective scores and struggle with consistency and safety.
Discussion on Hacker News about whether computer systems are prepared for the first negative leap second, with comments debating the risks and alternatives like leap minutes.
A cheatsheet covering memory segmentation concepts, likely useful for students and developers.
A paper titled 'The Hitchhiker's Guide to Agentic AI: From Foundations to Systems' provides comprehensive textbook-like resources on the basics of LLMs and agentic AI.
A tweet sharing Brendan Gregg's curated list of recommended books covering systems performance, language performance, database performance, and other computer science domains.
Sharing a machine learning systems notes repo on GitHub, covering distributed computing, parallelization, quantization, and PyTorch internals related to LLM training and inference. Suitable for learners interested in ML systems.
Discusses how sandbox startup latency and scaling in RL training infrastructure can significantly impact training performance, referencing a detailed analysis by SemiAnalysis on matching trainer and generator throughput.
Developer built a multi-client terminal chat app in C from scratch using raw TCP sockets and select() for I/O multiplexing, as a step toward building EduOS, a privacy-first AI-native OS for African schools.
ObjectCache proposes using S3-compatible object storage for LLM KV cache reuse to reduce cost and increase capacity, with a co-designed storage protocol and transfer schedule that minimizes latency overhead. Experiments show it adds only 5.6% latency over local DRAM for 64K contexts.
A lecture on LLM deployment techniques covering AWQ, vLLM, FlashAttention, quantization, and activation smoothing for efficient serving.
This paper introduces DMI-Lib, a high-speed deep model inspector that enables efficient internal observability for LLM inference by decoupling monitoring from the inference hot path.
An open-source textbook on Machine Learning Systems from Harvard, covering principles and practices of engineering AI systems, with companion labs and kits.