API or self‑hosted?
Do the maths.
Compare what you pay for a proprietary LLM API with running an open-source model on your own GPUs — monthly cost, annual saving and the hardware you'd need.
In short
Self-hosting an open-source LLM is usually cheaper than a proprietary API once a workload is high-volume and predictable — typically from a few million requests a month for classification or extraction. The saving comes from matching the task to the smallest model that holds quality (often 7–8B parameters), quantizing it (FP8/INT4), and serving it with continuous batching (vLLM or SGLang) so GPUs stay busy. At low or very spiky volume, a cheaper API tier or a batch API is often the better choice.
Method
How the estimate works
- API cost = requests × (input tokens × input price + output tokens × output price).
- GPU time comes from sustained throughput for a quantized (FP8/INT4) model served with continuous batching on vLLM or SGLang. Input tokens are counted at 10% of an output token because prompt processing is far faster than generation.
- GPUs are planned at 60% average utilisation, leaving headroom for traffic peaks and latency targets.
- Real-time traffic keeps at least one serving replica running for the whole serving window; batch workloads pay only for the hours they run.
- GPU prices are typical on-demand cloud rates (L4 $0.80, A10G $1.00, L40S $1.80, A100 $2.50, H100 $3.00 per hour). Reserved or spot capacity is usually cheaper.
- Engineering and operations are a monthly figure you control. Don't set it to zero: someone has to run it.
These are conservative planning numbers, not a quote. Throughput on your own prompts can be higher or lower — that is what an inference audit measures before you commit.
Planning a move? Read our step-by-step playbook for migrating from OpenAI APIs to open-source LLMs.
FAQ
Questions,
answered.
Next step
Turn the estimate into a measured number.
In a fixed-scope audit we analyse your real usage, benchmark open models on your prompts and give you a cost model you can take to finance — then pilot the workloads that pay off.