Tag
This paper presents the first architectural characterization of agentic AI workflows, revealing fragmented, heterogeneous execution patterns that mismatch conventional server designs, and introduces a prototype server called Agora to improve CPU/GPU utilization and throughput.
Marker 2 is released, offering up to 5x faster and more accurate conversion of PDFs, images, and DOCX to markdown compared to mineru, docling, and liteparse.
KTransformers is a framework that optimizes inference and fine-tuning of large Mixture-of-Experts models by dynamically placing only active experts on the GPU while keeping the rest in CPU memory, enabling large models like DeepSeek-V3 to run on limited consumer GPU memory.
OpenInfer demonstrates "vertical disaggregation" that boosts Qwen 3.5 27B throughput by ~50% by co-executing quantized layers across a single node’s AMD EPYC CPU and Nvidia L40S GPU with a custom SLA-aware scheduler.