Unweight: how we compressed an LLM 22% without sacrificing quality

This report describes Unweight, a lossless compression system for LLM model weights that reduces H100 GPU memory bandwidth by compressing BF16 exponent bytes (via Huffman coding and palette transcoding) and fusing decompression into custom reconstructive matmul kernels; it explains four execution pipelines, autotuning per-projection/batch-size, performance trade-offs (≈15–22% model size reduction for MLP weights, ~3 GB VRAM savings on Llama-3.1-8B) and planned optimizations, and is focused on systems engineering rather than security incidents.