Cloudflare's internal AI stack is a lesson in control-plane design

TL;DR

Cloudflare's internal AI engineering stack is not mainly a story about choosing a better model. It is a control-plane story: Zero Trust authentication establishes identity, a Worker and AI Gateway govern routing and cost, Workers AI supplies local inference, the MCP Server Portal gives many tools one OAuth entry point, sandboxes contain execution, Backstage and AGENTS.md provide repository context, and the AI Code Reviewer turns those rules into an engineering feedback loop. Cloudflare reports 20 million requests through AI Gateway, 241 billion tokens processed, more than 3,683 internal users, and a security agent processing over 7 billion tokens per day. Netics' view is that the architecture is valuable because it makes access and evidence explicit, but the figures are operating signals rather than proof that every enterprise should copy the stack whole.

The important idea is the control plane

The source presents a stack built on the same products Cloudflare sells. That detail matters. Instead of treating internal AI as a collection of experiments outside the platform, the company connected identity, model access, tools, repository knowledge, and execution environments. This is a more demanding route than giving every team a model key and a prompt template, but it addresses the failure mode that appears after the first successful demo: nobody can explain who accessed which model, which tool was available, what context was used, or why a cost moved.

Our interpretation is narrower than a vendor case study. Cloudflare has reported its own usage and architecture; that does not establish a universal benchmark. It does show a useful design direction for a European company that needs an auditable internal AI layer rather than a parade of disconnected copilots.

Zero Trust authentication makes agents identifiable

The stack starts with Zero Trust authentication. A request from an engineering tool is not trusted merely because it comes from an internal network. Cloudflare describes a proxy Worker validating a Cloudflare Access JWT before rewriting headers and forwarding the request. That sequence puts an actual user identity at the boundary where model access begins.

This is the right place to be strict. An AI agent can be fast, but speed is not an identity primitive. The Access layer gives operators a way to associate activity with a person and a policy before the request reaches a provider. It also creates a place to revoke access without changing every local configuration. Netics would carry this pattern into any multi-tenant automation project: authenticate the principal at the edge, then pass only the claims required by the next control point.

AI Gateway is the economic and routing layer

AI Gateway is described as the common path for model traffic, not merely an observability dashboard. Cloudflare uses it for routing, cost visibility, provider choice, Bring Your Own Keys (BYOK), and zero data retention (ZDR) options. A single proxy Worker sends requests to AI Gateway, which gives the organization a stable address while providers and model choices can change behind it.

The economics make the decision concrete. Cloudflare reports 20 million requests routed through AI Gateway and 241 billion tokens processed. It also says a security agent processes over 7 billion tokens per day on Kimi; compared with an estimated $2.4M per year on a mid-tier proprietary model, Workers AI is reported as 77% cheaper for that workload. These are Cloudflare's reported figures, not an independent audit, so the honest lesson is measurement architecture: route centrally before you attempt to optimize cost.

Workers AI keeps selected inference close to the platform

Workers AI is the local inference component in the story. Cloudflare says it increasingly leverages Workers AI, including for the security agent workload. The attraction is not that local inference automatically wins. It is that a platform team can decide which tasks deserve a lower-cost or more controlled execution path while keeping the same gateway and identity conventions.

That separation is important. Routing policy should not live inside a prompt. A classifier, security scan, or summarization step can use Workers AI when its quality and latency are acceptable, while a more demanding task can use another provider through the same gateway. The application receives a policy decision instead of a hard-coded provider dependency.

One Worker URL is a better developer contract

Cloudflare describes a proxy Worker that accepts requests such as POST /anthropic/v1/messages, validates the Access token, rewrites headers, and forwards the call. The practical benefit is a single URL to configure. Developers do not need to learn every provider endpoint or embed a long-lived credential in each tool.

This is where platform design either helps or annoys engineers. The source notes that organization-wide OpenCode defaults are merged with local configuration, with local settings taking priority. Users can override the default model, add agents, and adjust project- or user-scoped permissions. That balance is more credible than a centrally managed configuration that forbids local experimentation. Governance should establish safe defaults, not make every exception a ticket.

The MCP Server Portal turns tool sprawl into one OAuth decision

The MCP Server Portal is the stack's tool-access layer. Cloudflare describes using portals so many MCP tools can be governed through a single OAuth connection. The appeal is obvious: an engineer or agent authorizes one controlled entry point rather than separately managing credentials for every server.

Single OAuth is not the same as unrestricted access. The portal still has to define which servers are available, which users can reach them, and what each tool can do. The source points to Cloudflare's broader enterprise MCP governance work, including MCP Server Portals. Netics' opinion is that a portal is useful only when its inventory remains explicit. A unified door does not remove the need for room-level permissions; it makes the door easier to monitor.

Sandboxes separate useful autonomy from risky execution

Cloudflare highlights the Sandbox SDK going generally available during Agents Week and points toward background agents. Sandboxes provide a place for an agent to run code or manipulate a working environment without placing that activity directly in a developer laptop or a production service.

The design implication is stronger than the product name. Model access and execution access should be different permissions. An agent may be allowed to inspect a repository or propose a change while its command execution happens in a contained environment. Long-running agents add a durability problem, and Cloudflare describes native support in the Agents SDK as a way to address sessions that previously required workarounds. Teams should still define expiration, network policy, artifact handling, and human approval around the sandbox.

Backstage and AGENTS.md provide repository context

A model cannot review a repository responsibly if it has no reliable map of that repository. Cloudflare describes generating AGENTS.md files at scale and using Backstage as part of the knowledge layer. AGENTS.md gives an agent local instructions and codebase context; Backstage provides a catalogue and organizational view that can help generate or maintain those instructions.

The operational warning is the important part: a stale AGENTS.md can be worse than no file. Cloudflare says its AI Code Reviewer can flag when repository changes suggest that AGENTS.md should be updated. This is a useful closed loop. Documentation is not treated as a one-time bootstrap artifact; it becomes an object whose freshness is reviewed alongside code.

The AI Code Reviewer makes enforcement visible

The AI Code Reviewer connects to AI Gateway for model access, pulls Engineering Codex rules from a central repository, reads the repository's AGENTS.md, and posts structured merge-request comments. That is a practical enforcement pattern because it puts the result back into the workflow engineers already use.

The source says a separate technical post covers routing between models, multi-agent orchestration, and cost optimization. We should not infer more than that. The defensible lesson is that review agents need three inputs: centrally maintained rules, repository-specific context, and a controlled model path. A comment is not proof that a change is safe, but it is a useful, inspectable intervention when it explains what failed and why.

What the monthly token figures do and do not prove

Cloudflare reports 241 billion tokens processed and more than 3,683 internal users, alongside the 20 million AI Gateway requests. Those figures demonstrate that the company is operating at meaningful internal scale. They do not prove that the same volume, provider mix, or savings profile exists for a smaller company.

The more transferable signal is the instrumentation around the numbers. A company can only discuss cost, BYOK, ZDR, routing, and model placement when requests pass through a layer that records those choices. Without that control point, monthly token totals become a retrospective invoice rather than an engineering input.

Netics' opinion: copy the boundaries, not the brand stack

Netics would not recommend copying Cloudflare's product list as a procurement checklist. Start with boundaries: one authenticated entry point, one policy-aware model gateway, a declared tool inventory, isolated execution, repository-specific context, and review feedback that returns to the merge workflow. Choose equivalent components according to data residency, latency, budget, and operational capacity.

For a French or European SME, ZDR and BYOK may matter more than a headline token count. A smaller internal gateway with careful logs can be more valuable than a large agent programme that cannot answer which user granted which tool. The architecture should earn complexity by reducing uncertainty, not by accumulating products.

A practical adoption sequence for engineering teams

Begin with Zero Trust authentication and a single proxy Worker or equivalent edge service. Add AI Gateway routing so provider selection, cost, BYOK, and ZDR are policy decisions. Introduce Workers AI or another controlled inference path for tasks whose quality can be tested. Then inventory MCP tools behind a portal and one OAuth flow.

After that foundation, add sandboxes for execution, generate repository guidance through Backstage or a similar catalogue, and make AGENTS.md freshness part of code review. Finally, deploy an AI Code Reviewer with structured comments and a human escalation path. This sequence prevents teams from starting with autonomous action before they can explain identity, context, and rollback.

A technical architecture review can be discussed on the Netics homepage or through a free 30-minute audit.

Sources

• Cloudflare, “The AI engineering stack we built internally — on the platform we ship,” 20 April 2026: https://blog.cloudflare.com/internal-ai-engineering-stack
• Cloudflare, MCP Server Portals documentation: https://developers.cloudflare.com/cloudflare-one/access-controls/ai-controls/mcp-portals/
• Related Netics analysis: AI agent permissions before platforms

Source: The AI engineering stack we built internally — on the platform we ship — blog.cloudflare.com, 20 April 2026