aionprem.pl
TOOL · TCO

TCO calculator: on-prem vs cloud AI

Estimate the real cost of a private on-premise LLM vs a managed cloud subscription over several years. We count what competitors omit: power, cooling, people and real 40–65% GPU utilization — not the claimed 80–90%.

On-prem TCO · 3 years
PLN 779,233
Cloud TCO · 3 years
PLN 432,000
CAPEX PLN 180,000energy PLN 215,233people PLN 330,000support PLN 54,000

Cloud is cheaper by 45% over the horizon.

Break-even: none in horizon

// The biggest line item in on-prem TCO is usually NOT the GPU card — it's people and power. That's why we use real 40–65% utilization and the fully loaded FTE cost.

Want us to run these numbers for your real workload and architecture?

→ Book 30 min
Methodology & assumptions
  • 01Real production GPU utilization is usually 40–65%, not the claimed 80–90%. Higher figures are benchmark peaks, not a daily average.
  • 02Over a 3-year TCO, staff cost (MLOps/admin, FTE) often exceeds hardware cost. It's the most understated line item.
  • 03Cooling and power overhead (PUE ~1.3–1.5) typically add 25–40% on top of the GPU node's own energy draw.
  • 04The GPU sticker price is not the TCO — real 3-year TCO is often 1.8–3.5× the hardware quote.
  • 05The inference engine matters: vLLM delivers ~10–19× higher throughput than Ollama on the same GPU, lowering cost per query.
  • 06Cloud can look cheap at first, but token cost can spike with usage — so we compare the full horizon, not a single month.
How we calculate
  • // energy/yr = power kW × 24 × 365 × PUE × price per kWh (× 0.45 if powered down off-hours)
  • // people/yr = FTE fraction × annual cost of 1 FTE
  • // support/yr = CAPEX × HW support %
  • // on-prem TCO = CAPEX + years × (energy + people + support)
  • // cloud TCO = monthly cost × 12 × years
  • // break-even = first month where cumulative on-prem cost ≤ cumulative cloud cost
Frequently asked questions
Why 40–65% GPU utilization instead of 80–90%?

Because that's real, sustained utilization in production. The 80–90% figure reflects benchmark peaks, not a daily average. Overstating it artificially lowers on-prem TCO.

Why do staff (FTE) costs matter so much?

Over a 3-year horizon the fully loaded cost of an MLOps/admin team often exceeds hardware cost. It's the most frequently omitted and understated line in vendor calculations.

What does PUE account for?

PUE (~1.3–1.5) is the cooling and power overhead — it typically adds 25–40% on top of the GPU's own energy draw.

Is the GPU sticker price the TCO?

No. The quoted price is a fraction of the total — real 3-year TCO is often 1.8–3.5× the hardware quote once energy, people and support are added.

Does the inference engine affect cost?

Significantly. vLLM can deliver ~10–19× higher throughput than Ollama on the same GPU, directly lowering cost per query. Engine choice is one of the cheapest optimization levers.