GN Beats Gzip and Brotli: How a Learning Sliding Window Outperforms Static Compressors
rust
dev.to
When we published our last article, GN was within 10% of gzip on LLM conversation data. We said the remaining gap was in the entropy backend. We were wrong about the solution — but right about the problem. This week GN beats gzip on every corpus we tested. And on all three corpora, it beats brotli. Here is what we learned. The ANS Dead End Our first instinct was to improve the entropy coder. Gzip uses Huffman coding. zstd uses ANS (Asymmetric Numeral Systems). We implemented byte-renorm ANS, bi