Inference on Yi's Personal Bloghttps://yiliu30.github.io/tags/inference/Recent content in Inference on Yi's Personal BlogHugo -- 0.146.0en-usSun, 26 Apr 2026 10:00:00 +0000DeepSeek V4 KV Cache Design: How 1M Tokens Fit in 10 GiBhttps://yiliu30.github.io/posts/ds-v4/Sun, 26 Apr 2026 10:00:00 +0000https://yiliu30.github.io/posts/ds-v4/<p>DeepSeek V4 supports 1M-token context, yet its KV cache for a 61-layer model fits in ~9.6 GiB (BF16) — a <strong>6.3× reduction</strong> over naive full attention. This post breaks down how three orthogonal techniques combine to make that possible.</p>