<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Pruning</title><link>https://blog.ast.moe/en/tags/pruning/</link><description>Recent posts: Gorimuchu</description><generator>Hugo</generator><item><title>The Lottery Ticket Hypothesis and Pruning Gemma 4 E2B Without Retraining</title><link>https://blog.ast.moe/en/blog/2026-11-17/</link><pubDate>Tue, 17 Nov 2026 19:16:10 +0900</pubDate><guid>https://blog.ast.moe/en/blog/2026-11-17/</guid><description>&lt;p&gt;Reading &lt;a href="https://arxiv.org/abs/1803.03635v5"&gt;The Lottery Ticket Hypothesis&lt;/a&gt; made me wonder how much of a large neural network could be removed while leaving something useful. I tried pruning Gemma 4 E2B, which fits on my Mac.&lt;/p&gt;
&lt;p&gt;The short version: removing the smallest-magnitude weights separately in each target matrix increased perplexity by about 1.13 times at 20% pruning. Beyond that, degradation became steep: about 2.19 times at 30%, and 29.3 times at 40%. Random pruning reached about 13.4 times at just 10%. Which weights remain makes a substantial difference.&lt;/p&gt;</description></item></channel></rss>