LLM Inference Cost Estimator

How it works

The estimator computes the memory needed to host the model at the chosen precision, derives how many GPUs of the chosen SKU it takes to fit, and multiplies by the number of copies needed to serve the requested throughput. Cost = hourly rate × instance count × hours per month.

Assumptions you can override

• Throughput per GPU. Rough defaults per GPU type are baked in. Override in the advanced section if you have measured numbers.
• Utilization. Real systems rarely run at 100% — the default of 0.7 leaves headroom for spikes.
• KV-cache multiplier. A coarse 1.2 catch-all for activation and KV-cache memory on top of the weights.
• Replicas / redundancy. Multipliers for high-availability or N+M designs.

Edge cases and notes

• If a model exceeds a single GPU's VRAM, the calculator assumes tensor parallelism within the chosen instance. If the model also exceeds the instance's total VRAM, the feasibility banner says so.
• Mixture-of-experts models still need to keep all expert weights resident, even though only the active subset participates per token. The default params reflects total weights.
• Prices reflect on-demand list rates as of the catalog date. Spot, sustained-use, and committed-use discounts can lower the bill substantially.