Tag
NVIDIA recsys-examples now offers an end-to-end HSTU generative recommender inference workflow on Dynamo-Triton, combining PyTorch AOTI compilation with FlexKV-backed KV caching to achieve up to 5.93x lower latency on an RTX PRO 6000 Blackwell GPU.
This article introduces how to use SQL on the Databricks platform to run the open-source decision model SemIf-OpenJev, enabling batch processing by calling the model without data leaving the platform.
This article argues that vLLM and similar serving frameworks are inefficient for running multiple small AI models on a single GPU due to design limitations. It introduces the SIE open-source inference engine as a solution for serving many models together to reduce costs.
This post shows how to serve Baidu's Unlimited-OCR model as a temporary, OpenAI-compatible endpoint on Hugging Face Jobs, enabling multi-page document parsing with features like table-to-HTML and equation-to-LaTeX extraction.