Home
About
Services
Case Studies
Insights
Assessment
Contact
Get Started
Free tool

API or self‑hosted?
Do the maths.

Compare what you pay for a proprietary LLM API with running an open-source model on your own GPUs — monthly cost, annual saving and the hardware you'd need.

01 Your workload
1,000,000 ≈ 33,333 a day
02 Your current API
$
$

Tier prices are illustrative. Enter the rates on your current invoice for a real comparison.

03 Self-hosted setup
Classification, extraction, routing
$
Typical on-demand cloud price — edit to yours
$
Monitoring, updates, on-call
Traffic pattern
Real-time, round the clock

In short

Self-hosting an open-source LLM is usually cheaper than a proprietary API once a workload is high-volume and predictable — typically from a few million requests a month for classification or extraction. The saving comes from matching the task to the smallest model that holds quality (often 7–8B parameters), quantizing it (FP8/INT4), and serving it with continuous batching (vLLM or SGLang) so GPUs stay busy. At low or very spiky volume, a cheaper API tier or a batch API is often the better choice.

Method

How the estimate works

  • API cost = requests × (input tokens × input price + output tokens × output price).
  • GPU time comes from sustained throughput for a quantized (FP8/INT4) model served with continuous batching on vLLM or SGLang. Input tokens are counted at 10% of an output token because prompt processing is far faster than generation.
  • GPUs are planned at 60% average utilisation, leaving headroom for traffic peaks and latency targets.
  • Real-time traffic keeps at least one serving replica running for the whole serving window; batch workloads pay only for the hours they run.
  • GPU prices are typical on-demand cloud rates (L4 $0.80, A10G $1.00, L40S $1.80, A100 $2.50, H100 $3.00 per hour). Reserved or spot capacity is usually cheaper.
  • Engineering and operations are a monthly figure you control. Don't set it to zero: someone has to run it.

These are conservative planning numbers, not a quote. Throughput on your own prompts can be higher or lower — that is what an inference audit measures before you commit.

Planning a move? Read our step-by-step playbook for migrating from OpenAI APIs to open-source LLMs.

FAQ

Questions,
answered.

When the workload is high-volume and steady enough to keep GPUs busy. Classification and extraction at a few million requests per month usually cross the line first, because a quantized 7–8B model handles them well on a single low-cost GPU. Low-volume or bursty workloads usually stay cheaper on an API.

It is a planning estimate. Throughput figures are conservative averages for quantized models served with continuous batching; real numbers depend on prompt length, latency targets, the model and the quantization method. An inference audit replaces these assumptions with benchmarks on your own prompts.

Engineering to set up and maintain the serving stack, monitoring, evaluation against your current outputs, and on-call for production. The calculator includes these as a monthly operations figure you can change.

For well-defined tasks such as classification, extraction and templated generation, a smaller open model — sometimes fine-tuned — often matches it. Open-ended reasoning and long-form writing need careful testing. A pilot should compare quality on a sample of your real inputs before anything moves to production.

Next step

Turn the estimate into a measured number.

In a fixed-scope audit we analyse your real usage, benchmark open models on your prompts and give you a cost model you can take to finance — then pilot the workloads that pay off.

Talk to an inference engineer →Book a 30-min call ↗