https://koleslaw.ai/blog Notes from running a fine-tuned 30B model in production: quantization, spot GPUs, failure design, and what it all costs. en https://koleslaw.ai/blog/the-quantization-that-didnt-fit https://koleslaw.ai/blog/the-quantization-that-didnt-fit Wed, 29 Jul 2026 00:00:00 GMT Nemotron's hidden dimension is 2688, which is not divisible by 256. That one fact quietly inflated a 4-bit quant by a third and cost a weekend.