https://blog.ast.moe/en/tags/quantization/Recent posts: GorimuchuHugohttps://blog.ast.moe/en/blog/2026-09-24-2/Thu, 24 Sep 2026 22:07:00 +0900https://blog.ast.moe/en/blog/2026-09-24-2/<h2 id="overview">Overview</h2> <p>We compared image quality and file size when using a small image‑upscaling DNN with FP16 weights converted to FP6. In the search for optimal quantization settings, the PSNR improved on four validation images but decreased on another set of four images. Afterwards, we evaluated using 174 images divided into five groups by acquisition series, as well as 12 images that had not been used until the FP16 baseline was fixed. Even with roughly the same number of parameters, the degradation caused by uniform conversion to FP6 differed markedly between residual‑type and direct‑generation‑type models. The comparison of the 12 final FP6 images was added after viewing the FP16 results, so it does not constitute a new blind evaluation.</p>https://blog.ast.moe/en/blog/2026-09-24/Thu, 24 Sep 2026 13:43:00 +0900https://blog.ast.moe/en/blog/2026-09-24/<p>While researching quantization for LLMs, I found myself wondering in what situations FP8 would actually be useful. Further investigation revealed that FP6 and FP4 also exist. I became curious about how to store values in a 6‑bit format—an oddly sized width—and how much precision a floating‑point number with so few bits could actually represent.</p> <p>When you list the values representable by FP6, the spacing between adjacent numbers becomes clearly visible—a detail that is usually overlooked with the floating‑point formats we use daily. Even with the same 6 bits, the spacing changes depending on how the bits are allocated between the exponent and the mantissa. That raises the question: could we choose the representation to match the distribution of the weights?</p>