Making Gemma 4 E2B Smaller: Vocabulary Trimming and Layer-Wise MLP Allocation
I tested English/Japanese vocabulary trimming and layer-wise MLP channel allocation without retraining Gemma 4 E2B. This article records BPB, throughput, and memory changes after a 44.6% parameter reduction at 64k vocabulary, and the effects of allocating pruning using measured layer sensitivity.