A Self-Hosted LLM Costs Less Than You Think. Here Is the Bill.

A self hosted LLM has a reputation for being the expensive, masochistic option: buy GPUs, hire an ops team, end up behind on quality anyway. On a real production workload we priced line by line, it came in at 81% less than the equivalent frontier API over three years, and the hardware paid for itself in under four months.

What a self-hosted LLM actually costs

The numbers below come from a production deployment for a European insurance group: Qwen3-32B quantized to AWQ 4-bit, served with vLLM on two NVIDIA L40S 48 GB cards, handling around 42,000 inference requests a day at a peak concurrency of 34. The API comparison was calculated from actual measured request and token volume after launch, not from a forecast, which is the part most vendor calculators get wrong.

The 38,500 euro covers the GPU server, both L40S cards, storage, networking and three years of hardware support. The 1,030 euro a month covers power and cooling, infrastructure monitoring, backup and the hardware support allocation. Neither number includes engineering time to build the thing, which is the subject of the next section.

The three line items nobody puts in the spreadsheet

Most self-hosting business cases compare a GPU price against a token price and stop there. That comparison is not wrong so much as incomplete, and the missing pieces are where projects actually fail.

The serving stack. A model file is not a service. You need batching, quantization that does not wreck quality, an endpoint that your applications can actually call, health checks, and a controlled path for putting a new model version into production. vLLM behind an OpenAI-compatible endpoint solves most of this, and the compatibility matters more than it sounds: internal applications talk to it exactly as they would to a commercial API, so nothing downstream needs bespoke integration.

Somebody has to own it. When the API is down, that is the vendor's problem and their status page. When your node is down, it is yours, at whatever hour it happens. Budget for that person's attention, not just their salary line. This is the single most common reason a technically successful self-hosting project gets quietly abandoned a year later.

Evaluation. If you cannot measure whether the self-hosted model is good enough, you cannot make the decision at all, and you certainly cannot defend it to anyone. Building an evaluation set of real tasks, scored by people who know the domain, is unglamorous work that has to happen before the switch, not after. It is also the artifact that keeps paying: every model upgrade afterwards is a measurement instead of an argument.

Where the break-even actually sits

The arithmetic is simple once you have honest numbers. Monthly saving is API spend minus running cost. Divide the hardware capex by that, and you have your payback period. In this case: 10,900 minus 1,030 is 9,870 a month, and 38,500 divided by 9,870 is 3.9 months.

What moves that number is volume, and only volume. GPUs cost the same whether you send them ten requests a day or forty thousand, so the entire case rests on utilisation. A rough guide from the projects we have priced:

• Under 2,000 euro a month of API spend: self-hosting rarely pays on cost alone. Do it for privacy or control, not for the invoice
• 2,000 to 8,000 euro a month: it depends on how spiky your traffic is. Steady load is a good candidate, bursty load usually is not
• Above 8,000 euro a month of steady load: the payback is measured in months and the case largely makes itself

Batch work helps enormously here, because it fills the gaps between interactive requests. In this deployment the same endpoint serves an interactive copilot and a batch queue doing summarization, classification and fact extraction, which is what keeps utilisation high enough for the economics to work.

What you give up

Quality, a little. On a 600-task evaluation set covering real work, scored blind by the client's own domain experts, the self-hosted Qwen3-32B reached 93.4% accepted answers against 94.6% for the external frontier model. That is a gap of 1.2 percentage points, and it is a real gap: a 32B open-weight model is not stronger than a frontier model, and anyone telling you otherwise is selling something.

What you can gain is accuracy on your own material. Citation faithfulness went the other way, 97.1% self-hosted against 94.8% external, because the retrieval pipeline and reranking were tuned for those specific documents. That is not a model win, it is a systems win, and it is available to you precisely because you control the whole stack.

On speed, single-stream generation ran at about 72 tokens per second, and about 54 per stream under production load, with p95 time to first token at 1.3 seconds. Users see an answer begin within about a second, which is the number that actually governs whether people keep using the tool.

When self-hosting is the wrong answer

Do not self-host because it feels more serious. Stay on an API when your volume is low or unpredictable, when you need frontier reasoning on genuinely open-ended problems, when your team has no appetite for operating hardware, or when the fastest path to knowing whether the idea works at all is a pilot you can start this afternoon.

The honest version of the decision is that cost is usually the weakest reason to self-host. It is the easiest to quote and the least likely to be decisive. The strong reasons are control and permission: data that legally cannot leave your infrastructure, contractual obligations to customers, or a regulator who will ask where the documents went.

What this looked like in production

The deployment these numbers come from is written up in full, including the compliance work that made it possible: how a DACH insurance group moved its AI in-house after legal and security blocked the cloud pilot outright. That case covers the architecture, the access control and audit trail, and the DPIA, DORA assessment and penetration test that had to close before go-live.

If you want the mechanics rather than the economics, we have a practical walkthrough of setting up a private LLM on premise. And the six-week build timeline in that case study is a product of how we work, which is described in agentic coding as a process.

FAQ

How much does a self-hosted LLM cost?

For the deployment described here, 38,500 euro of hardware plus about 1,030 euro a month to run, against roughly 10,900 euro a month for a comparable external frontier API at the same measured volume. That is 75,580 euro against 392,400 euro over three years, a saving of about 81%, with the hardware paying for itself in 3.9 months.

Is a self-hosted LLM as good as GPT or Claude?

Not quite, and you should plan for that. On a 600-task evaluation set scored blind by domain experts, a self-hosted Qwen3-32B reached 93.4% accepted answers against 94.6% for an external frontier model. The gap was 1.2 percentage points. On citation faithfulness the self-hosted system was actually better, at 97.1% against 94.8%, because the retrieval pipeline was tuned for those specific documents.

What hardware do you need to self-host an LLM?

For a 32B model quantized to 4-bit serving around 42,000 requests a day at a peak of 34 concurrent users, two NVIDIA L40S 48 GB cards in a single node with 512 GB of RAM and mirrored NVMe was enough. Smaller models and lower concurrency need considerably less. The size of the model matters far less than the concurrency you have to hold.

When is self-hosting an LLM a bad idea?

When your volume is low, when your workload is spiky, when you need frontier-level reasoning on open-ended tasks, or when nobody on your team wants to own a GPU node at three in the morning. Below roughly 2,000 euro a month of API spend the economics rarely justify it on cost alone.

Published August 11, 2026. All costs, latency and quality figures come from a single production deployment delivered by Valletta Software and are quoted with the client's permission. The client is anonymized. Your numbers will differ with volume, region and hardware pricing.