Author(s): Chew Loong Nian – AI ENGINEER

Originally published on Towards AI.

Claude Code Can Fine-Tune This 8.5 MB Model. Which of Its 19 Sizes Should You Ship?

Needle 3’s depth ladder has four cliffs. One extra transformer block costs 0.58 MB at depth 8 and 4.66 MB at depth 7. Here are all 19 rungs, priced against the published checkpoint.

After the lead, the article explains how Needle 3 is built as a “ladder” of deployable models by selecting a depth (2–20 layers) and re-slicing a single checkpoint, yet the per-depth byte size isn’t uniform: 14 steps cost an additional 0.58 MB while four specific steps jump by 4.66 MB. It traces those cliffs to the interaction between the layer-ordering logic (which only considers layer count) and the model’s fixed engram-site locations, then derives the exact file sizes by reading the repository’s quantization/packing scheme and using a script that estimates byte-exact archive sizes without downloading weights. The author validates the method against real packed files, itemizes what the “4.66 MB cliff” represents in terms of engram tables and parameters, shows how to choose depths just before cliffs to fit flash budgets, and concludes that the missing table of per-depth byte costs in the README hides a real deployment-cost structure.

Read the full blog for free on Medium.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI