Those are important, but they do not fully answer a practical architecture question:

Where should AI governance actually happen?

When an employee, application, copilot, agent or API workflow sends data to an AI model, the organization needs a way to inspect, control and record that access path.

This is where the distinction between the Control Plane and the Data Plane becomes useful.

The access path is where governance becomes real

In many enterprises, AI policies are written separately from the systems that actually send data to AI models.

That creates a gap.

A policy may say that sensitive customer data should not be sent to unapproved AI tools. But the organization still needs a technical way to answer questions such as:

• Who is sending the AI request?

• Which application or workflow is involved?

• What type of data is being sent?

• Is sensitive data present?

• Which model is being used?

• Is that model approved for this use case?

• Should the request be allowed, masked, blocked or routed elsewhere?

• Was the decision recorded?

• Can the activity be reviewed later?

These are not only governance questions.

They are system design questions.

A simple enterprise AI access pattern

A simplified AI access architecture can look like this:

Employees / Enterprise AI Apps
        |
        v
Enterprise AI Access Governance Layer
        |
        v
Approved AI Models

The governance layer does not need to replace every internal system.

Its role is to sit at the point where AI access can be inspected, controlled and recorded before sensitive data leaves enterprise control.

To make this practical, it is useful to separate the system into two concepts:

• Control Plane

• Data Plane

What the Control Plane does

The Control Plane is responsible for administration, configuration and governance visibility.

It can manage:

• tenant configuration;

• user and department settings;

• approved model routing rules;

• AI access policies;

• use case definitions;

• audit views;

• reporting views;

• security and governance administration workflows.

The Control Plane is where administrators define how AI access should be governed.

For example, a security or governance team may define that:

• certain departments can only use approved models;

• specific use cases require stricter data inspection;

• some sensitive-data types must be masked;

• some requests should be blocked;

• audit evidence must be retained for review.

The Control Plane is important, but it should not automatically mean that all sensitive prompt content or model outputs must be processed inside the same management interface.

That is where the Data Plane becomes important.

What the Data Plane does

The Data Plane is responsible for handling the actual AI request path.

It can process:

• prompts;

• model responses;

• sensitive-data detection;

• masking decisions;

• blocking decisions;

• route enforcement;

• request-level telemetry;

• policy execution results.

In enterprise environments, the Data Plane may need to be closer to the customer environment, especially when prompts or model outputs may contain regulated, confidential or customer-related data.

A customer-controlled Data Plane can help support stricter data boundary requirements.

This does not mean every enterprise needs the same deployment model.

But it gives organizations a clearer architecture option when they want governance controls without sending sensitive AI traffic through a fully external processing layer.

Why separation matters

Separating the Control Plane and the Data Plane helps address several practical concerns.

1. Sensitive-data boundary

Some organizations are comfortable with a SaaS management interface for policies and dashboards.

But they may not want sensitive prompts, model outputs or internal business data to leave their controlled environment.

A separate Data Plane can support that requirement.

2. Deployment flexibility

Different enterprises may have different deployment needs.

Some may prefer a SaaS model.

Some may require hybrid deployment.

Some may want a customer-controlled Data Plane for sensitive workloads.

A clear architecture should support discussion around these options without overpromising that one deployment model fits every organization.

3. Operational clarity

When the Control Plane and Data Plane are separated, it becomes easier to explain:

• where policies are configured;

• where prompts are inspected;

• where masking or blocking happens;

• where routing decisions are enforced;

• where audit views are managed;

• what data is visible to administrators.

This is useful during security review, compliance discussion and enterprise evaluation.

4. Audit evidence

Enterprise AI governance is not only about preventing bad requests.

It is also about producing useful evidence when a review happens.

Useful audit evidence may include:

• the user or application involved;

• the model selected;

• the policy applied;

• whether sensitive data was detected;

• whether content was masked, blocked or allowed;

• the timestamp of the decision;

• the request outcome;

• the review context.

A well-designed Control Plane can expose this evidence for review, while the Data Plane can generate the underlying decision and telemetry records.

Control Plane does not mean control of all data

One common misunderstanding is that a control system must directly collect all data in order to govern it.

For enterprise AI governance, that is not always desirable.

A better design question is:

What does the Control Plane need to know in order to manage policies, routing and audit views?

And what should remain inside the Data Plane, closer to the customer-controlled environment?

This distinction can help reduce unnecessary data exposure while still supporting governance visibility.

Data Plane does not mean no central governance

Another misunderstanding is that if the Data Plane is customer-controlled, governance becomes fragmented.

That does not have to be true.

A customer-controlled Data Plane can still follow centrally managed policies, route configuration and audit requirements.

The goal is not to remove central governance.

The goal is to avoid forcing sensitive AI traffic through a place where it does not need to be processed.

Applying this to AI access governance

For AI access governance, the Control Plane and Data Plane pattern can support capabilities such as:

• approved-model access control;

• policy-based routing;

• prompt inspection;

• sensitive-data detection;

• masking;

• blocking;

• audit evidence;

• usage visibility;

• token usage and estimated cost visibility.

This architecture is especially relevant when enterprises are evaluating generative AI adoption across departments, applications and workflows.

As AI usage expands, governance needs to move closer to the actual access path.

What we are building

I am working on SecureAI Gateway, an enterprise AI access governance platform developed by SecureAI Systems Limited, a Hong Kong registered company serving Hong Kong, Singapore and Southeast Asia.

SecureAI Gateway focuses on governing AI access before sensitive data leaves enterprise control.

The platform is designed around practical enterprise concerns such as:

• sensitive-data detection and masking;

• approved AI model access control;

• policy-based routing;

• audit evidence;

• AI usage visibility;

• token usage and cost visibility;

• hybrid deployment with customer-controlled Data Plane options.

The product is still early, but the architectural direction is clear:

Enterprise AI governance becomes more practical when the control point is connected to the actual AI access path.

Website:
https://secureaigateway.ai

Public resources:
https://github.com/XiaobinZhang6791/secureai-gateway-resources

I would be glad to exchange views with people working on enterprise AI governance, AI security, data protection, model risk and secure AI deployment architecture.