Tag
The user fixed crashes when running large AI models on an AMD GPU in headless Linux by disabling runtime power management, preventing model weights from being dumped into limited RAM and causing OOM errors.
A user expresses gratitude to open source developers, highlighting vllm's recent major releases that fixed OOM issues, allowing doubling of context window size to 240k on a 5090 with Qwen 27B.