Insights
Capacity planning
1 article tagged Capacity planning.
18 June 2026 · 4 min · QAI Labs engineering
Sizing GPUs for a 70B model you have to host yourself
The arithmetic that decides whether your cluster survives Monday morning (weights, KV cache, headroom) and the three assumptions that most often make it wrong.
Start with the constraint.
Most of these projects are shaped by what you cannot do rather than what you want. Data that cannot leave the estate, a model you cannot host with a third party, a decision somebody has to justify to a regulator. Tell us yours and we will say honestly whether we can work inside it.