Network isolation for on-prem AI: the minimum NIS2 expects

Reading time: approx. 9 min. Cluster: nis2. Author: Fryderyk.

The short answer

NIS2 does not hand you a network diagram for your AI system. What Article 21 of the Polish Cybersecurity Act (the NIS2 transposition) does require are measures proportionate to risk across network security, access control, and the development and maintenance of systems. In practice that means isolation. The smallest defensible layout for on-prem AI is three zones: an inference zone that holds the model and the accelerators, a data zone with the repositories that feed retrieval, and an access zone where users and applications connect. Traffic between zones travels only along defined, logged paths, and the inference zone has no open route to the internet. If your on-prem model sits on one flat network with everything else, that is not just technical debt. It is a missing risk-management measure, and an auditor will record it as one. This note lays out the minimum split and shows which Article 21 control each zone boundary closes.

Contents

1. Why isolation is a compliance question, not only a technical one
2. Three zones: the minimum model
3. Traffic rules between zones
4. Table: zone, what you allow, what you block
5. Mapping the zones to Article 21
6. Three mistakes an auditor will catch
7. FAQ
8. Disclosure and biases
9. What this note does not cover
10. Related notes

Why isolation is a compliance question, not only a technical one

Network segmentation is old, well understood security practice. What is new is that with on-prem AI it doubles as a compliance argument. The main reason a manufacturer runs a model in-house rather than in the public cloud is control over how sensitive operational data flows. That control is only credible if the network actually enforces it. A model running locally but wired into a flat network with an open path outward gives away most of the advantage it was placed there to protect.

Article 21(2) lists, among other things, network and information system security, access control, and security in system development and maintenance. None of these prescribes a specific topology, because the list is deliberately technology neutral. But an auditor asking about measures proportionate to risk expects a system that processes sensitive data to be separated from the rest, not sharing a segment with printers and workstations. Isolation is therefore where an architectural decision and a regulatory requirement meet in a single artifact: the firewall rules and the network diagram.

Three zones: the minimum model

The smallest split you can defend to an auditor and still maintain in practice rests on three zones.

Inference zone. This holds the model server, the accelerators, and the serving layer. It is the most protected segment. It has no free outbound access to the internet, and administrative access arrives only from a dedicated management subnet. Model and image updates enter through a controlled channel, not an open outbound connection.

Data zone. This is where repositories, vector indexes, and the sources that feed retrieval live. It accepts queries only from the inference zone and only on defined ports. It is never exposed directly to users. If you run RAG, this is where most of the data you must not let leave the organization sits, so the boundary of this zone is a compliance boundary. I covered the retrieval architecture itself in a separate note on on-prem RAG.

Access zone. This is where users, front-end applications, and integrations connect. It talks to the inference zone through one defined entry point, ideally a gateway that authenticates requests. The user sees neither the data zone nor the model internal ports.

Three zones is a floor, not a target. Larger deployments add a separate management segment, a logging segment, and a buffer zone for external integrations. But below three zones it is hard to speak of isolation that enforces anything.

Traffic rules between zones

The split alone achieves nothing if traffic between zones is unrestricted. The rule is simple: deny by default, allow explicitly. Every permitted connection has a known direction, a known port, and a reason recorded in the rule documentation.

Direction matters. The access zone initiates connections to the inference zone, not the other way around. The inference zone reaches into the data zone, but the data zone initiates nothing toward users. Outbound access from the inference zone to the internet is closed by default, and if the model needs an update it pulls it through a controlled, logged channel during a service window, not on a standing connection.

It is this set of rules, not the mere fact of hosting the model locally, that proves data does not leave the organization. The firewall rules and connection logs are the artifact you show an auditor. A network diagram without matching rules is a statement, not a control.

Table: zone, what you allow, what you block

Zone	You allow	You block
Inference	Requests from the access zone, management from the admin subnet	Internet egress, traffic straight from users
Data	Queries from the inference zone on defined ports	User access, outbound-initiated connections
Access	User and application traffic through a gateway	Direct sight of the data zone and model ports

Print this table and hold it against your real firewall configuration. Any line you cannot reproduce as a rule is a gap to document.

Mapping the zones to Article 21

Isolation is not a standalone NIS2 obligation. It is how you close several Article 21 areas at once. Network security is covered by the split and the traffic rules themselves. Access control is closed by the fact that a user reaches neither the model nor the data directly. Security in development and maintenance is supported by the controlled update channel, because changing the model does not mean opening the network. Connection logs between zones feed the effectiveness-assessment area, because they prove the rules actually work rather than merely existing on a diagram. I mapped all ten areas onto an AI system in a separate NIS2 cheat sheet.

Three mistakes an auditor will catch

First: a flat network with the model next to everything else. The most common and the easiest to catch, because one look at the diagram is enough. Running a model locally without segmentation creates an illusion of control that is not there.

Second: zones exist on the diagram but the traffic rules are too broad. An any-to-any allow inside the internal network is not isolation. The auditor compares the diagram with the configuration and the gap shows immediately.

Third: an open, standing outbound path from the inference zone to the internet under the banner of updates. This is the most dangerous mistake, because it creates exactly the exfiltration path that on-prem was meant to prevent. An update is a service window and a controlled channel, not a permanent link.

FAQ

Does NIS2 require a specific number of zones?

No. The Act specifies neither a topology nor a segment count. It requires measures proportionate to risk. Three zones is a practical minimum that is easy to defend, but the number follows from your risk analysis, not from the statute.

Does on-prem automatically satisfy the isolation requirement?

No. Hosting a model locally does not create isolation by itself. Without segmentation and traffic rules a local model can be as open as any other server on a flat network. Isolation comes from configuration, not location.

How does this compare to the public cloud?

In the public cloud the data-flow boundary also runs through the provider and its sub-processors, which shifts part of the analysis onto the supply chain. I set that out in a separate note on public cloud LLMs and NIS2. With on-prem the boundary is inside your own network, so the burden of proof rests on your firewall rules and logs.

Is segmentation enough for compliance?

No. Isolation covers part of the Article 21 areas, but it does not replace risk analysis, incident handling, the vendor agreement, or management oversight. It is one pillar, not the whole structure.

// disclosure & biasesDisclosure and biases

I write from the perspective of someone working on AI solutions deployed outside the public cloud, so I naturally emphasize data-flow control and on-prem architecture. I have tried to separate the regulatory requirement from the technical preference. This text is not legal advice and not a network design for any specific organization. The three-zone model is a working simplification, not the only correct pattern.

What this note does not cover

I do not go into specific firewall products, microsegmentation solutions, or vendor-specific configurations. I skip segmentation at the container and orchestration level, which deserves its own note. I also leave out the details of encrypting traffic within zones and of machine identity management, as those are separate areas. The focus here is the minimum network split and its link to Article 21.

Related notes

• AI on-prem in European manufacturing 2026: a complete architecture guide
• On-prem RAG: architecture, chunking, retrieval and what really drives quality
• Public cloud LLM and NIS2: a quick read of Article 21(1)(d)
• NIS2 plus AI: mapping Article 21 to concrete controls, one cheat sheet