SOTA ImageGen Locally NVIDIA Cosmos3(64B) INT4 quants CUDA/MLX
Summary
NVIDIA Cosmos3, a 64B parameter image generation model, is released with INT4 quantization for local deployment on CUDA and MLX, with code and weights available and performance demonstrated on Apple Silicon.
Similar Articles
MLX 16/8/4/2-bit quants of nvidia/llama-embed-nemotron-8b
The user converted Nvidia's Llama-Embed-Nemotron-8B model to MLX format with fp16, 8-bit, 4-bit, and 2-bit quantizations, enabling in-process embedding loading on Apple Silicon via mlx-embeddings.
nvidia/Cosmos3-Super
NVIDIA released Cosmos3, a collection of omnimodal world foundation models for Physical AI, capable of generating video, image, audio, and action commands from various inputs, with versions for different tasks like policy learning and image-to-video generation.
Qwen3.8-27B Hybrid IQ4_XS quantization for 16GB gang
This is a quantized version of the Qwen3.8-27B AI model using IQ4_XS quantization, optimized for 16GB RAM systems, with instructions for local deployment using various tools like llama.cpp and Ollama.
@swyx: roundup of links:
NVIDIA releases Cosmos 3 (Mixture-of-Transformers models up to 64B), Nemotron 3 Ultra (550B-A55B LLM), and previews RTX Spark personal superchip at Computex 2026, achieving SOTA on multiple open model leaderboards.
@basecampbernie: https://x.com/basecampbernie/status/2074262192304832535
This post details the author's setup and benchmarks for running NVFP4-quantized image and video generation models on a GIGABYTE AI TOP ATOM (DGX Spark) workstation, achieving impressive performance with models like FLUX.2, Qwen-Image, and LTX-2.3 for video with synchronized audio.