AI architecture resilience: what you can audit on-prem, what the cloud hides

Reading time: approx. 9 minutes. Cluster: architecture. Author: Fryderyk.

Answer first

When a head of IT asks whether their AI is “resilient”, they usually mean “will it stay up”. That is the wrong question. Resilience is not the same as availability. A system that went down for a few hours can be accounted for after the fact, as long as you know exactly what broke and what it stood on. A system that never goes down but rests on dependencies you cannot name or audit is more fragile, you just do not know it yet. So the real dividing line between on-prem and public cloud does not run along “who has better uptime”, where hyperscalers usually win. It runs along auditability: what you can prove about your own system when an auditor, a regulator or the board asks. On-prem gives you visibility into layers a cloud model simply does not expose. This note sets out what you can audit yourself, what a cloud provider will not reveal, and why for an essential entity under NIS2 this is a requirement, not a matter of taste.

Table of contents

1. Why we confuse resilience with availability
2. The europe-west4 outage as evidence, but not the obvious kind
3. Three dependency layers you must be able to point to
4. What you can actually audit on-prem
5. What a hyperscaler will not reveal
6. Table: audit dimension, on-prem, public cloud
7. Auditability is a NIS2 requirement, not a preference
8. Where on-prem does not win (honestly)
9. FAQ
10. Disclosure and biases
11. What this note does not cover
12. Related notes

Why we confuse resilience with availability

Availability is the percentage of time a service responds. It is measurable, it goes into an SLA, and it is easy to argue about with a provider. Resilience is broader: the ability of a system to survive an event you did not anticipate, and to explain afterwards what actually happened. These two can pull in opposite directions. You can buy very high availability and still run an architecture that nobody in the organisation can draw end to end, because half the layers are sealed inside a provider.

For an AI system this gap is sharper than for an ordinary application. Inference sits on a longer chain of dependencies: accelerator hardware, drivers, the serving layer, the retrieval pipeline, data sources, model and system-prompt versions. A failure or a silent change in any of these layers can shift the system's behaviour without taking it off availability. If you cannot see those layers, your dashboard can glow green at the very moment the system has been answering worse for a week.

The europe-west4 outage as evidence, but not the obvious kind

On 15 and 16 July 2026 Google Cloud's europe-west4 region in the Netherlands suffered a near fifteen-hour outage. The cause was not a software bug but a physical event: an upstream electrical fault on the utility grid disrupted cooling, temperatures started to climb, and Google shut down servers, storage and switches to prevent heat damage. Among the services affected were Google Cloud VMware Engine, Bare Metal Solution and NetApp Volumes.

It is easy to draw the conclusion “the cloud fails, so go on-prem”. That is a lazy and dishonest reading, because your own server room can lose power and cooling too, sometimes worse than a hyperscaler. The interesting point is different. The services that went down are exactly the ones you buy in order to have dedicated, isolated hardware inside the cloud, a proxy for control. And yet the customer learned what was happening from status updates, not from their own telemetry, because they have no visibility into the power and cooling layer and never will. This is not an accusation against Google, every hyperscaler works this way for good reasons. It illustrates a single thesis: from an audit standpoint, the cloud's problem is less that something breaks and more that you cannot independently trace what your system stands on and why it behaved the way it did.

Three dependency layers you must be able to point to

Before comparing auditability, it helps to name what we are talking about. An AI system has three dependency layers an auditor will eventually ask about.

The physical and infrastructure layer. Where the hardware physically sits, who has access to it, how power, cooling and networking are arranged, who can open the rack. In a cloud model this layer is a black box with a certificate. On-prem it is a layer you see to the end, because it is yours.

The software and model layer. Driver versions, the system image, the serving layer, the model and its weights, the system prompt, the retrieval pipeline. The question here is whether you can reconstruct exactly which configuration served a given request on a given day, and whether that configuration changed without your knowledge.

The data and flow layer. Which documents entered the context, from where, which network boundaries the traffic crossed, where the logs landed and who had access to them. This is the layer that most often decides the outcome of a NIS2 or GDPR audit, because it touches data.

Auditability is the ability to show each of these three layers on demand, with evidence rather than a declaration.

What you can actually audit on-prem

On-prem is not magic, but it moves the boundary of what you can see substantially in your favour. Concretely, you can audit for yourself:

The physical path: you have the device inventory, the rack plan, access logs to the room and real control over who connects to the management network. When an auditor asks who could have touched the hardware, you answer from your own sources.

The full software stack: driver, image, serving-layer and model versions live with you and change only when you change them. You can freeze a version, reproduce the state from a month ago and prove that the system prompt on a specific day looked one way and not another.

The model event log: if observability is designed deliberately, you have a record of who asked, when, under which identity, what they asked, which documents entered the context and which model version answered. You keep that log in your own log-collection system, with your own retention policy and protection against overwriting. I have written separately about what to log in an on-prem LLM and how to tie it to an audit trail.

Network boundaries: you design and log segmentation, traffic rules and touch points yourself, so the zone diagram and the connection log come from a single source you control.

The common denominator is that on-prem the evidence comes from your systems. You do not have to trust someone else's report to answer the auditor's question.

What a hyperscaler will not reveal

The other side of the same coin. In a public cloud model there are things you cannot audit, not because the provider hides something in bad faith, but because the cloud's business model rests on those layers being shared and closed.

You will not see the physical layer: power, cooling, who had physical access to the machine your model ran on and when. You get a certificate and a compliance report, not your own log.

You will not see the full neighbourhood: on shared infrastructure you do not know what else was running on the same hardware and how it affected performance at a given moment. July's outage showed this from the physical side, but sharing also has a quieter, everyday version.

You will not see the subcontractor chain in depth: the cloud often rests on its own layers and sub-services whose full list the customer never receives. For a NIS2 Article 21 analysis this is a real problem, because you are mapping a supply chain you cannot see to the end. I covered this in the analysis of public cloud LLMs against NIS2.

You cannot freeze state on demand: updates to provider-managed layers happen on the provider's schedule, not yours. You may have API versioning, but you do not have full control over what changed underneath between Tuesday and Wednesday.

Table: audit dimension, on-prem, public cloud

Audit dimension	On-prem	Public cloud
Physical layer (power, access, hardware)	Your own log, full visibility	Provider certificate and report, no evidence of your own
Stack and model versions	Freezable, reproducible by you	Partly out of your control, provider's cadence
Model event log	In your own log system	Depends on what the provider exposes
Subcontractor chain	Known, because it is yours	Partly undisclosed
Network boundaries	Designed and logged by you	Configurable, but on someone else's fabric
Availability / uptime	Depends on your server room	Usually higher than a single company can reach

A note on the last row: this is the one dimension where the cloud usually wins, which is why on-prem should not be sold on the slogan “safer, because it does not go down”. It does go down, just differently. On-prem's advantage is auditability, not magical reliability.

Auditability is a NIS2 requirement, not a preference

NIS2, transposed in Poland through the national cybersecurity act, requires measures appropriate to the risk, including supply-chain security, incident handling and assessing the effectiveness of safeguards. No provision tells you outright to choose on-prem. But each of those three requirements comes down in practice to a question you must be able to answer with evidence: what your system is made of, who touches it and what happened inside it. That is a question about auditability.

The consequence is that the choice between on-prem and cloud stops being purely technical or cost-driven and becomes a question of whether you will be able to produce evidence in front of an auditor. If critical layers are sealed inside a provider, part of the answer will always be “we have the provider's certificate for that”, not “we have our own record”. For an essential entity, where management's personal liability is also in play, that difference can weigh more than a few percentage points of uptime.

Where on-prem does not win (honestly)

It would be dishonest to leave this without the other side. On-prem loses in several places, and it is worth naming them before someone does it for you in front of the board.

Raw availability is often lower: a single server room with a single power feed is more exposed to a physical event than a hyperscaler region with redundancy you cannot afford. Without engineered redundancy, on-prem is not automatically more resilient.

Cost and skills: the auditability I describe here does not follow from the mere fact that the hardware sits with you. It comes from observability, log policy and versioning discipline that have to be built and maintained. Without them, on-prem gives you only the illusion of control and a rack nobody looks inside. I have covered the choice of operating model, from DIY to managed, separately.

Scale and elasticity: if load fluctuates heavily or you need to experiment with many large models at once, the cloud's elasticity can be a real advantage, while a fixed on-prem GPU fleet can sit idle or choke.

So the conclusion is not “always on-prem”. It is: if your requirement is to prove control over the architecture, on-prem gives you layers the cloud will not expose by design, and then auditability, not uptime, is the right selection criterion.

FAQ

Is on-prem safer than public cloud?

Not in the sense of “goes down less often”. A hyperscaler usually has higher raw availability. On-prem wins on auditability: you can independently prove what the system is made of, who touches it and what happened inside it. For NIS2 that often matters more than uptime.

Does the Google Cloud outage prove you must flee the cloud?

No. Your own server room loses power and cooling too. The outage showed something else: even services bought for dedicated hardware give the customer no visibility into the physical layer. It is an argument about auditability, not about reliability.

What exactly can I audit on-prem that I cannot in the cloud?

On-prem: the physical layer, stack and model versions, an event log in your own system, network boundaries, a known supply chain. In the cloud those layers are partly closed and you get the provider's certificate instead of your own evidence.

Does simply placing hardware on-site give you auditability?

No. Auditability comes from observability, log-retention policy and versioning discipline. Without them, on-prem is a rack without a log. Hardware is a necessary condition, not a sufficient one.

// disclosure & biasesDisclosure and biases

I write from the perspective of someone working on an AI platform that runs outside the public cloud. That is a bias towards the on-prem category and worth knowing as you read. I have tried not to hide the places where the cloud wins, because a text that pretends on-prem has no drawbacks is useless to anyone actually making this decision. The on-prem AI market holds several categories of solution, from self-assembled open-source, through productized platforms, to managed models, and each has a different cost and risk profile. This text is not legal advice, nor an architecture design for any specific organisation.

What this note does not cover

I do not go into specific observability tools or logging stacks, because the choice depends on what you already run. I do not cover designing power and cooling redundancy in your own server room, though it is a real condition for on-prem to be resilient rather than merely auditable. I do not compute the TCO of either model here, because cost is a separate thread. I also leave aside hybrid models and cross-site federation, which deserve their own note. Legal details are worth confirming with your compliance team.

Related notes

• On-prem AI in European manufacturing 2026: a complete architecture guide
• Monitoring and observability for on-prem LLMs: what to log and how
• Network isolation for on-prem AI: the minimum NIS2 expects
• Public cloud LLMs and NIS2: a quick read of Article 21(1)(d)
• DIY, productized or managed: three on-prem AI models and who maintains them