<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>MLX</title><link>https://blog.ast.moe/en/tags/mlx/</link><description>Recent posts: Gorimuchu</description><generator>Hugo</generator><item><title>Making Gemma 4 E2B Smaller: Vocabulary Trimming and Layer-Wise MLP Allocation</title><link>https://blog.ast.moe/en/blog/2026-11-18-3/</link><pubDate>Wed, 18 Nov 2026 19:36:38 +0900</pubDate><guid>https://blog.ast.moe/en/blog/2026-11-18-3/</guid><description>&lt;p&gt;&lt;a href="https://blog.ast.moe/en/blog/2026-11-18/"&gt;Previously, I removed whole MLP channels from Gemma 4 E2B and saved smaller matrices&lt;/a&gt;. At the nominal 10% pruning stage, total weight size fell about 3.5%. Generation became a little faster, but MLPs account for only about 34% of this model. Concentrating on them limits how much smaller the whole model can become.&lt;/p&gt;
&lt;p&gt;This time I tried two approaches. One trims vocabulary around English and Japanese to shrink the large embedding tables. The other keeps the total MLP pruning budget fixed while changing how much each layer loses. Neither updates the weights through training.&lt;/p&gt;</description></item><item><title>Reducing Gemma 4 E2B by Removing MLP Channels: Model Size and Inference Performance</title><link>https://blog.ast.moe/en/blog/2026-11-18/</link><pubDate>Wed, 18 Nov 2026 13:27:40 +0900</pubDate><guid>https://blog.ast.moe/en/blog/2026-11-18/</guid><description>&lt;p&gt;&lt;a href="https://blog.ast.moe/en/blog/2026-11-17/"&gt;In the previous experiment, I set Gemma 4 E2B weights to zero without retraining&lt;/a&gt;. Selecting small-magnitude weights separately in each matrix increased WikiText-2 perplexity by about 1.13 times when 20% of the target weights were removed. However, the arrays retained their shapes. Preserving some prediction performance did not mean that I had saved a smaller model or made it run faster.&lt;/p&gt;
&lt;p&gt;This time I removed entire intermediate MLP channels, making the matrices themselves smaller. No training updated the surviving weights. I investigated how to choose channels while preserving performance, then measured saved-model size, memory, and speed. I also compared the result with 8-bit quantization.&lt;/p&gt;</description></item></channel></rss>