NVIDIA's 64B Cosmos3 Model Quantized to INT4 for Local MLX and CUDA Execution
Community developers have released INT4 quantized weights and open-source inference code for NVIDIA's 64-billion parameter Cosmos3 model. This work enables both text-to-image and image-to-video generation directly on consumer hardware via Apple MLX and NVIDIA CUDA. Running state-of-the-art 64B visual models locally previously required expensive enterprise-grade GPUs, but 4-bit quantization makes consumer execution feasible. It grants developers and creators local control, privacy, and cost-free experiment capabilities without relying on paid cloud APIs. The INT4 quantized model generates a single video clip in approximately 5 minutes on an Apple M4 Max Mac with 128 GB of unified memory. The project provides ready-to-use Hugging Face weights alongside a GitHub repository featuring performance benchmarks and visual quality comparisons against Grok.
## BACKGROUND
NVIDIA Cosmos is a platform of generative world foundation models built for physical AI, robotics, and advanced visual content generation. MLX is Apple's open-source array framework optimized for the unified memory architecture of Apple Silicon chips. INT4 quantization is a compression technique that converts 16-bit precision weights to 4-bit integers, significantly lowering the memory footprint required to run massive AI models.