Self-Hosted LLM vs API CostBreak-Even Analysis (2026)

Self-hosting an LLM becomes cheaper when its recurring operating cost falls below the API bill for the same accepted workload. Include idle capacity, incident cover and human review, then check whether the saving repays setup effort. First prove that both options meet the same quality, privacy and peak-latency requirements.

Decision worksheet reviewed: 19 October 2026. The vendor prices and numerical scenarios below retain their September 2026 reference date; they are examples, not current quotes or universal break-even thresholds. Recheck the linked vendor pages before budgeting.

When does self-hosting LLM inference become cheaper?

Start with the lowest-cost API option that passes your workload’s acceptance checks. Compare self-hosting only after an open model passes those same checks. A data-handling requirement may decide where inference must run even when self-hosting costs more; record that requirement separately from the savings calculation.

Write down two separate numbers:

monthly saving = comparable API operating cost - self-hosted operating cost
setup payback in months = one-off setup cost / positive monthly saving

Use the same currency and billing period. If the monthly saving is zero or negative, there is no cost payback on those assumptions. If the model fails the acceptance checks, a cheaper invoice does not qualify it. Use the existing break-even calculator to explore assumptions, then replace its estimates with measured workload and current quotes.

The independent local PII-redaction experiment shows why quality and remaining review work belong in the comparison; its small evaluation is not a production guarantee. If the workload reads company systems, bring this budget brief to a context-layer scoping conversation. The free call establishes fit; an assessment and implementation are separately commissioned.

Reasons to evaluate self-hosting

• Steady demand that can use the capacity you pay for, including busy periods and recovery headroom.
• A documented data boundary that the candidate managed endpoint cannot satisfy.
• Model weights or a deployment configuration that the candidate API does not offer.

None of these alone proves a saving. If self-hosting is unnecessary, use the production model-selection guide. For hosted usage estimates, see the LLM API cost comparison.

The cost model for APIs

The cost model for the Claude API is three numbers: input price, output price, and zero infrastructure.

For 2026:

Opus 4.7 and 4.8 are priced identically to Opus 5. Sonnet 5 introductory pricing of $2/$10 ended on 31 August 2026. Models from Claude 4.7 onward use a tokenizer that yields about 30 percent more tokens for the same text than Sonnet 4.6 and earlier, so compare cost per document, not per token.

That’s the whole pricing sheet. No idle cost, no reserved capacity, no GPU you pay for when nobody is using it. If you send zero requests this month you pay zero dollars. Scale is elastic in both directions.

On top of that you get prompt caching (90% discount on cached input), batch processing (50% discount for non-realtime jobs), and a global SDK that handles retries, rate limits, and streaming without any work on your end. The effective per-token rate for a well-tuned workload with prompt caching enabled is often 30-50% below the sticker price.

The failure modes are: per-token cost adds up at scale, your latency depends on the provider’s queue depth, and you’re subject to their rate limits (which you can raise on request but not eliminate).

The cost model for self-hosting

Self-hosting has four cost buckets, and most people only see the first one.

Hardware or rental. You either buy a GPU and rack it, or you rent one. On Hetzner, a GEX44 with an RTX 4000 Ada (20 GB VRAM) runs about 234 euros per month plus a 114 euro setup fee, excl. VAT, since the June 2026 price adjustment. That covers models up to roughly 14B parameters at 4-bit or 8B at full precision. For 27B to 70B-class models you need the GEX131 (RTX PRO 6000 Blackwell, 96 GB) at 1,197.30 euros per month excl. IPv4 plus 599 euros setup. A dedicated H100 on RunPod is $2.89/hr (PCIe) to $3.29/hr (SXM) on-demand, Lambda $3.29 to $4.29/hr for a single GPU. Running one H100 24/7 for a month is $2,100 to $3,100. A 96 GB RTX PRO 6000 on RunPod is $2.09/hr, an 80 GB A100 $1.39/hr.

For comparison on the cloud GPU side, I have a separate writeup on GPU cloud comparison for AI inference that covers spot pricing and reserved instances. And Hetzner vs AWS for AI workloads covers the bare-metal vs hyperscaler tradeoff for anything steady-state.

Idle time. A GPU at rest costs the same as a GPU at 100%. If your load is bursty (peak at 9am, nothing at 2am), you’re paying for capacity you’re not using. The Claude API charges you zero when you’re idle. This is the single biggest cost difference at low-to-medium scale.

Engineering time. Getting a production-grade inference setup running is 2-4 weeks of focused work for someone who’s done it before, longer if it’s their first time. That’s vLLM deployment, model download and validation, load testing, autoscaling, observability, and a deployment pipeline. Then it’s ongoing: model updates, serving stack upgrades, incident response. Budget 10-20% of one engineer’s time permanently if you want it to stay healthy.

The invisible stack. API queueing, retries, rate limit backoff, streaming, multi-region failover, cost monitoring, request logging, latency SLOs, security hardening of the inference endpoint. The Anthropic SDK does all of this. You get to rebuild it.

Break-even math at realistic volumes

Run the numbers on your own workload: Self-Hosted LLM vs API Break-Even Calculator — enter monthly tokens, batch share, prompt-cache hit rate, and GPU utilization to see the crossover point in seconds.

Here’s where the rubber meets the road. Four scenarios, all assuming Claude Sonnet 4.6 as the API baseline and a reasonable open model (Llama 3.3 70B or Qwen 2.5 72B) as the self-hosted alternative. Assume a 50/50 input/output split for simplicity.

Scenario: 1M tokens/day

Claude Sonnet 4.6: 500k input @ $3/M + 500k output @ $15/M = $1.50 + $7.50 = $9/day, or about $270/month.

A Hetzner GEX44 at about 234 euros per month costs roughly the same as the Sonnet 5 bill at this volume and about three times the Haiku 4.5 bill of $90 per month, before a single hour of engineering. The GPU would be idle most of the day. That’s a 3-5 year payback on the setup cost alone, before counting ongoing ops.

Verdict: API wins, not close.

Scenario: 10M tokens/day

Claude Sonnet 4.6: 5M input + 5M output = $15 + $75 = $90/day, roughly $2,700/month.

The smallest Hetzner server that holds a 27B to 70B-class open model is a GEX131 with 96 GB at about 1,199 euros per month ($1,397 in Hetzner’s USD price list), or a rented H100 at $2,100 to $3,100. Whether it sustains 10M tokens/day (about 115 tokens/second average, with peaks well above) depends on model size, quantization and context length; size it with a load test, not a brochure figure. Hardware: about $1,400 to $3,100 per month, plus 10 to 20 percent of a senior engineer. Engineering time: still 2-4 weeks upfront plus ongoing.

Break-even against Sonnet: yes, if quality is acceptable. Break-even against Haiku 4.5 ($1/$5, so $30/day, $900/mo) does not happen at this volume. Haiku handles many of the workloads people reach for self-hosting to cover.

Verdict: against Sonnet 5 it is a wash on a GEX131 at list price and the API wins on a rented H100; once you use the Batch API (50 percent off) or prompt caching (cached input at 10 percent of base), the API wins in both cases. Against Haiku 4.5 at $900 per month, the API wins clearly.

Scenario: 100M tokens/day

Claude Sonnet 5: $900/day, $27k/month. Haiku 4.5: $300/day, $9k/month. Opus 5: $1,500/day, $45k/month.

At 100M tokens/day (1,150 tokens/sec average), a single GEX44 is tapped out. You’re looking at 2-4 GPUs, likely H100-class if you want headroom, so $4.2k-9.6k/month on rented cloud GPUs or 2,400-3,600 euros for two to three Hetzner GEX131 (one GPU per server). Self-hosted wins decisively against Sonnet, and is competitive against Haiku when you factor in throughput advantages on bulk workloads.

This is where fine-tuning also enters the picture. If you can fine-tune a 7B-13B model on your specific task and serve it on a single A4000, unit economics get even better.

Verdict: self-hosted wins if an open model clears your quality bar.

Scenario: 1B tokens/day

At this scale, you’re running a small cluster. 4-8 H100s, dedicated ops, custom routing. Claude Sonnet would be $9k/day ($270k/month). Self-hosted with fine-tuned models is in the $30-80k/month range all-in including engineering overhead. The break-even is obvious, but it’s no longer a side project. You need a team.

Rough rule of thumb, at list prices and a 50/50 input/output split: break-even for self-hosting against Claude Sonnet 5 sits around 10-15M tokens/day on a Hetzner GEX131 and 10-20M on a rented H100, if a 27B to 70B-class open model meets your quality bar. Against Haiku 4.5 it is 25-45M tokens/day on a GEX131 and 35-60M on an H100. Against Opus 5 it is 5-10M on a GEX131 and 7-12M on an H100. If your API workload runs through the Batch API or hits prompt caching, roughly double every threshold, but you’re also accepting a quality drop that’s rarely worth it for the kind of workload that justifies Opus in the first place.

What you give up self-hosting

This is the list that kills most self-hosting plans when teams actually confront it.

Model quality at the top. Claude Opus 5 is the model to beat on reasoning, code and agentic tasks. Open-weight releases in 2026 (Qwen3.8-27B, DeepSeek V4) publish their own benchmarks against Claude Opus 4.6, not against Opus 5; whether they close the gap on your workload is something only your own eval set can answer. The gap on everyday tasks has narrowed a lot, but the gap on hard tasks (multi-step reasoning, complex tool use, long-form generation with strong coherence) is still real.

Structured output reliability. Compare the serving stack, not only the model. The Claude API offers native JSON outputs and strict tool schemas for supported constraints; a self-hosted stack needs its own constrained decoding and validation path. I show the JSON and tool patterns in the Claude API structured output guide, and the execution loop in the tool use guide.

Long-context recall. Open models with 128k context windows often fall off a cliff past 32k, especially for needle-in-haystack recall. Claude 4.6 and later models take up to 1M tokens of context at standard per-token pricing, with caching and batch discounts applying across the full window.

Vision and multimodal. Claude accepts images natively, handles PDFs, and generates structured output about visual content in the same API call. Self-hosted vision models exist but require a separate pipeline.

Prompt caching. Anthropic’s prompt caching gives you a 90% discount on cached input tokens and is a one-line code change. Self-hosted requires you to build your own caching layer, which is doable but non-trivial.

Ecosystem. SDK in every language, docs, community examples, debugging tools, third-party integrations. The hosted API ecosystem is years ahead of the open self-hosted tooling.

What you gain self-hosting

Data never leaves your network. This matters for regulated industries, classified workloads, and anything where your legal team has decided no external API is acceptable.

No rate limits. You’re capped by your own GPU, not by a provider’s rate limit policy. For workloads that need sustained high throughput, this is a real advantage.

Weights you own. Fine-tuned checkpoints you can redeploy anywhere. No vendor lock-in on the model itself.

Predictable flat-rate cost. Once the GPU is paid for, marginal cost per token is effectively zero. This makes budgeting trivial and removes the “we burned $30k in API calls this month” surprise.

Latency predictability. No API queue variance. Your p99 is whatever your GPU serves at, full stop.

Models nobody else hosts. Smaller specialized models, older versions frozen for a legacy pipeline, domain-specific tunes from Hugging Face with no hosted equivalent.

Which open models are realistic for self-hosting

Not every open model is production-ready. Here’s what I’d actually put into a serving stack in 2026.

Qwen3.8-27B. Current dense 27B under Apache 2.0, 262K context, BF16 weights of 55.6 GB fit a single 96 GB or 80 GB GPU; the FP8 checkpoint fits 48 GB.

Qwen3.6-27B. Previous generation (April 2026), same footprint, Apache 2.0.

DeepSeek V4 Flash. 284B total, 13B active, MIT license, 1M context; needs a multi-GPU node. DeepSeek V4 Pro. 1.6T total, 49B active; cluster only.

Llama 4 Scout and Maverick. Meta’s MoE generation under the Llama 4 Community License; have legal read the license terms before an EU deployment.

Llama 3.3 70B. Still widely tooled; 4-bit fits 40 GB.

Mistral. Older open-weight versions (Mixtral, Mistral Large before it went closed) are still solid for many workloads.

Phi-4. Microsoft’s smaller model, surprisingly strong on narrow tasks, runs on consumer GPUs (24GB VRAM is plenty). Good fit for classification, extraction, structured output where you don’t need frontier reasoning.

7B-13B models. For embedding, classification, simple extraction, and routing, a 7-13B model on a single 20GB GPU is often all you need. This is where Hetzner GEX44 economics start looking excellent.

If you’re deploying any of these on Kubernetes, I have a longer writeup on self-hosted LLM on Kubernetes covering vLLM deployment, autoscaling, and observability.

The serving stack options

The inference server matters almost as much as the model.

vLLM. My default. Fastest, production-ready, PagedAttention makes memory usage efficient for batched inference. Strong OpenAI-compatible API. Active development. If you’re serving at scale, this is the starting point.

Text Generation Inference (TGI). Hugging Face’s inference server. Solid, widely deployed, good integration with the HF model hub. Slightly behind vLLM on raw throughput in most benchmarks I’ve run but perfectly production-viable.

Ollama. Excellent for development and local experimentation. Not production-grade for multi-tenant serving. I use it on my dev box, I would not put it behind a production endpoint.

llama.cpp. CPU-friendly, runs on edge devices, great for Mac deployments. Slower than GPU-based stacks but useful when GPU isn’t an option.

SGLang. Newer, competitive with vLLM on throughput, particularly strong for structured output and constrained decoding. Worth watching.

For production, vLLM or TGI. Everything else is dev tooling.

The real cost categories people forget

The spreadsheet usually looks like “GPU cost + electricity = total”. It’s not.

GPU capex or ongoing rent. The obvious one.

Idle time. GPU at rest costs the same as GPU at 100%. If you can’t keep utilization above 60%, per-token economics look worse than the brochure suggests. The API provider amortizes idle across thousands of customers; you amortize it across your own traffic alone.

Engineering time, initial. 2-4 weeks of focused senior engineering to get a production serving stack up: deployment, autoscaling, monitoring, load testing, security.

Engineering time, ongoing. 10-20% of one senior engineer permanently: model updates, serving stack upgrades, incident response, cost optimization, quality regression tracking when you swap models.

Observability. Cost per request, latency percentiles, quality drift, token accounting. The API gives you most of this out of the box. You build it.

Model updates. When Anthropic ships the next Sonnet or Haiku, you get it with a model string change. When a new open model drops, you download, benchmark, re-validate on your eval suite, test serving performance, and redeploy. Budget a week per major model swap.

Security surface. Your inference endpoint is now an attack surface you own. Authentication, authorization, rate limiting, DDoS protection, input validation to prevent prompt injection from leaking to the model layer.

Redundancy. The API has multi-region failover built in. You rebuild it. A single-GPU deployment has a single failure domain, which is fine for batch but not for real-time.

If you add these honestly, self-hosting economics shift meaningfully. A setup that looks $2k/month cheaper on hardware often costs $4-6k/month more once you include engineering amortization.

The hybrid pattern

Here’s what most production teams I work with actually end up doing. Not pure self-hosted, not pure API. Both.

Self-host for:

• Bulk classification (millions of records, simple schema)
• Extraction from structured or semi-structured data
• Embeddings (a small model on a cheap GPU runs these for pennies)
• Simple summarization where quality bar is “good enough”
• Any workload where p95 volume is predictable and sustained

Claude API for:

• Reasoning, multi-step agents, complex tool use
• Multilingual content generation
• Long-context tasks (analyzing full contracts, long codebases)
• Vision and multimodal
• Anything customer-facing where quality variance matters
• Burst traffic that would push your self-hosted GPU past capacity

A simple routing layer decides per request. Classification input? Self-host. Customer support agent? Claude. Embedding a document batch? Self-host. Analyzing a 50-page contract? Claude.

This pattern captures the cost savings of self-hosting on the workloads that justify it, while keeping the quality and ops simplicity of API for the workloads that need it. Almost nobody ends up at 100% self-host. The teams that claim they did usually have a second hosted pipeline they forgot to mention.

Six questions before self-hosting

Walk through these honestly. Skip the ones you want to rationalize away at your own risk.

1. What’s my p95 token volume in the next 12 months? If it’s under 10M tokens/day against Sonnet 5, under 25M tokens/day against Haiku 4.5, the math almost certainly doesn’t work. Pay for the API.

2. Is my workload simple enough that an open 70B or smaller model performs well? Run an eval. Actually run it. “I think Llama will be fine” is not evidence. Take your top 200 prompts, run them through Sonnet and Llama, score the outputs. If Llama is within 10% on your metric, good. If it’s 30% worse, no self-hosting will fix that.

3. Do I need fine-tuning? If yes, self-host is the path. If no, the argument for self-host gets weaker.

4. Do I have GPU ops capability, or budget to hire it? If your team has never run a GPU in production, this is a hidden 3-6 month cost you’re not counting. If you can hire it, budget €90-130k/year for someone who’s actually done it.

5. Does my data HAVE to never leave my infra? Not “we’d prefer” but actually, legally, contractually required. Check first whether zero-retention endpoints from API providers satisfy your legal team, because increasingly they do.

6. Am I willing to re-benchmark every 6 months as open models improve and frontier models improve? Self-hosting is not a one-time decision. The gap between open and closed moves. Your decision has to move with it.

If you answered yes to 4 or more, self-hosting may be worth evaluating seriously. If you answered yes to fewer than 4, default to the API. That’s not a cop-out, it’s the right call for the volume and workload pattern most teams actually have.

Which should you choose?

For most teams in 2026, the answer is API. Claude Sonnet 5 for quality-sensitive work, Haiku 4.5 for cost-sensitive bulk, Opus 5 where the task justifies $5/$25, all with prompt caching to squeeze the effective rate down another 30-50%. The engineering time you save pays for a lot of tokens.

Self-host when volume is genuinely in the 15M+/day range on a workload where an open model meets your quality bar, or when you have hard data residency requirements no API can satisfy, or when you need fine-tuned weights you own. In those cases, start with vLLM on a Hetzner GEX131 or a single H100 rental, prove the unit economics with 30 days of real traffic, then scale.

The hybrid pattern is where most serious production setups land. Self-host the predictable bulk, use Claude for the high-value and bursty work. One router between them. That’s the architecture that actually ships.

Changelog

• 2026-10-19: added an equal-workload budget worksheet, separated setup payback from operating cost, and linked evaluation evidence to the context-layer assessment path. Price examples retain their September 2026 reference date.