[EAAPL-MCP002] MCP Gateway

Category: MCP Sub-category: Integration Infrastructure Version: 1.0 Maturity: Emerging Tags: mcp-gateway, api-gateway, authentication, rate-limiting, routing, policy-enforcement, multi-server Regulatory Relevance: EU AI Act Art. 9/15, APRA CPS 234, ISO/IEC 42001 §8.4, NIST AI RMF (GOVERN 1.6, MANAGE 4.1), Privacy Act 1988 (Cth)

1. Executive Summary

The MCP Gateway Pattern introduces a dedicated intermediary layer between MCP clients (AI models and agents) and the fleet of MCP servers that expose enterprise capabilities. Rather than each client connecting directly to each server, all MCP traffic is routed through a centralised gateway that enforces authentication, authorisation, rate limiting, audit logging, and server discovery in a single, consistently governed enforcement point. The gateway presents a unified endpoint to clients while transparently proxying to the appropriate backend MCP servers based on capability routing rules.

For CIO/CTO audiences: the MCP Gateway is the API management platform for your AI integration fabric. Just as your organisation would never expose internal microservices directly to the internet without an API gateway enforcing auth and rate limits, you should not expose MCP servers directly to AI models without a corresponding enforcement layer. The gateway solves three compounding problems that emerge when MCP deployments scale beyond a handful of servers: authentication fragmentation (each server implementing auth independently with different standards), audit incompleteness (no single log of all AI-to-system interactions), and capability sprawl (clients must be reconfigured every time a server is added or retired). A well-designed gateway reduces these problems to solved infrastructure concerns, freeing application teams to focus on capability development.

2. Problem Statement

Business Problem

As MCP server deployments grow from a pilot (2–3 servers) to a production fleet (10–50+ servers), the cost of managing authentication, access policy, and audit logging per server grows linearly. Security teams cannot enforce consistent policy across independently-managed servers. Compliance teams cannot produce a unified audit trail of all AI model interactions with enterprise systems. Platform teams cannot manage server discovery and health centrally.

Technical Problem

Without a gateway, each MCP server must independently implement authentication token validation, rate limiting, and audit logging — leading to inconsistent implementations and gaps. Clients must maintain individual connection configurations for every server, and any server migration or endpoint change requires client reconfiguration. There is no single point at which to enforce cross-cutting policies (maximum tool argument payload size, blocked tool categories by caller identity, global emergency circuit breaker).

Symptoms of Absence

• Different MCP servers implement authentication using different mechanisms with no consistency
• A complete audit log of all AI tool invocations across the enterprise requires aggregating logs from dozens of independent servers
• When a new MCP server is deployed, every client that needs it must be manually reconfigured with its endpoint
• A single misbehaving AI agent consuming excessive resources cannot be throttled without server-by-server intervention
• Security teams cannot block a specific tool category across all servers without contacting each server owner

Cost of Inaction

• Security: Policy gaps between independently-managed servers create exploitable inconsistencies; no central kill-switch for rogue agent behaviour
• Compliance: Fragmented audit logs cannot satisfy APRA CPS 234 requirements for complete, centralised records of AI system actions
• Operational: Server endpoint changes break clients; no centralised health monitoring for the MCP fleet
• Engineering: Each new MCP server duplicates auth and logging boilerplate, consuming 20–30% of development time per server

3. Context

When to Apply

• The organisation operates 3 or more MCP servers and anticipates continued growth
• Regulatory requirements demand a unified, centralised audit log of all AI-to-system interactions
• Multiple AI clients (different models, different teams, different trust tiers) need differentiated access to the same set of servers
• Server endpoint management (discovery, migration, versioning) needs to be decoupled from client configuration
• A centralised emergency circuit breaker is required to halt all AI tool access during an incident

When NOT to Apply

• Single MCP server deployment where the overhead of a gateway exceeds the benefit
• All clients and servers are co-located in the same process or container and a gateway adds unnecessary network hop latency
• The organisation's existing API gateway (Kong, AWS API Gateway, Azure APIM) can be extended to handle MCP-specific routing and protocol proxying natively

Prerequisites

• At least one production MCP server (EAAPL-MCP001) already deployed and governed
• OAuth 2.1 authorisation server or equivalent identity provider for client authentication
• Service mesh or container orchestration platform for gateway deployment and availability
• Centralised log management infrastructure capable of receiving structured gateway audit events

Industry Applicability

4. Architecture Overview

The MCP Gateway sits at the network boundary between MCP clients and the MCP server fleet. It operates as a transparent protocol-aware proxy: it terminates the client's MCP connection, enforces all cross-cutting policies, and establishes a new connection to the appropriate backend server on the client's behalf. The gateway is stateful in the sense that it maintains the mapping between client sessions and backend server connections for the duration of each MCP session lifecycle.

Routing is capability-driven: the gateway maintains a server registry that maps tool names, resource URI prefixes, and prompt template identifiers to the backend server responsible for each. When a client sends a tools/call request, the gateway looks up the target server, verifies the client has access to that tool, and proxies the request. Capability discovery (tools/list, resources/list, prompts/list) is aggregated across all servers the client is permitted to access — the client sees a unified capability surface, not individual server boundaries.

Policy enforcement at the gateway covers four layers: authentication (validate the client's OAuth 2.1 bearer token or mTLS certificate before any MCP message is processed), authorisation (check the authenticated identity against the access policy for the requested tool or resource), rate limiting (enforce per-client and per-server token bucket limits to prevent resource exhaustion), and content policy (enforce maximum payload sizes and block prohibited tool invocation patterns based on organisational policy).

Observability is a first-class gateway concern. Every proxied request generates a structured trace event carrying the client identity, target server, capability name, latency, and result status. These events flow to the centralised audit log and to the metrics pipeline. The gateway health endpoint exposes aggregate server fleet health so that a single monitoring check can detect any degraded backend server. An administrative API allows operators to update routing rules, adjust rate limits, and activate the emergency circuit breaker without gateway redeployment.

Wire Protocol Detail

The MCP Gateway operates as a protocol-aware transparent proxy. It must handle the full JSON-RPC 2.0 message lifecycle on both the client-facing and server-facing legs independently. The following wire exchanges illustrate the gateway's dual-handshake behaviour.

Inbound client initialisation — the gateway completes the handshake with the client using an aggregated capability set:

// Client → Gateway: initialize
{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{"tools":{},"roots":{"listChanged":true}},"clientInfo":{"name":"agent-a","version":"2.1"}}}

// Gateway → Client: aggregated initialize response (capabilities merged from permitted backend servers)
{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":"2024-11-05","capabilities":{"tools":{"listChanged":true},"resources":{"subscribe":true,"listChanged":true}},"serverInfo":{"name":"mcp-gateway","version":"1.0"}}}

The gateway does NOT forward the client's initialize to any backend server at this point. It issues a separate initialize to each permitted backend server during gateway startup (or lazily on first route to that server), caches the capability responses, and synthesises the aggregated response.

Proxied tool invocation — the gateway rewrites the request to the target server, injecting the authenticated identity and correlation ID:

// Client → Gateway: tools/call
{"jsonrpc":"2.0","id":7,"method":"tools/call","params":{"name":"query_customers","arguments":{"filter":"region=AU","limit":50}}}

// Gateway → Backend Server: proxied tools/call (id remapped; _meta injected)
{"jsonrpc":"2.0","id":1042,"method":"tools/call","params":{"name":"query_customers","arguments":{"filter":"region=AU","limit":50},"_meta":{"callerIdentity":"agent-a@domain.com","correlationId":"550e8400-e29b-41d4-a716-446655440000","gatewayRequestId":"gw-20240614-001042"}}}

// Backend Server → Gateway: response
{"jsonrpc":"2.0","id":1042,"result":{"content":[{"type":"text","text":"[{\"id\":1,\"name\":\"Acme Corp\"}]"}],"isError":false}}

// Gateway → Client: response (gateway request id stripped; client's original id restored)
{"jsonrpc":"2.0","id":7,"result":{"content":[{"type":"text","text":"[{\"id\":1,\"name\":\"Acme Corp\"}]"}],"isError":false}}

Rate limit and authorisation error responses — the gateway returns structured JSON-RPC errors before proxying:

// Authorisation failure (client lacks scope for requested tool)
{"jsonrpc":"2.0","id":7,"error":{"code":-32000,"message":"Insufficient scope","data":{"required":"mcp:data:customers:read","granted":["mcp:data:orders:read"],"tool":"query_customers"}}}

// Rate limit exceeded
{"jsonrpc":"2.0","id":7,"error":{"code":-32000,"message":"Rate limit exceeded","data":{"retryAfterSeconds":30,"limit":"100 req/min","identity":"agent-a@domain.com"}}}

The gateway MUST preserve the client's original id in all responses, even though it remaps IDs internally when fanning out to multiple backend servers. ID collision between concurrent requests is managed via a gateway-internal request map (client-id → gateway-id → pending response slot).

5. Architecture Diagram

6. Components

7. Implementation Steps

Step 1: Deploy the Server Registry

Before building the gateway, deploy and populate the server registry. For each MCP server, register its endpoint, transport type, capability list, access tier requirements, and health check URL. The registry is the gateway's source of truth for routing — it must be available for the gateway to start. Implement a registration API that MCP server operators use to register capabilities; do not hand-edit the registry. Include a health polling job that probes registered servers on a 30-second interval and marks degraded servers as unavailable in the routing table.

Step 2: Implement the Protocol Proxy Core

Build the gateway's protocol proxy layer. For each inbound client connection, perform the MCP initialisation handshake with the client to establish the session. Do not forward the client's initialize request directly to a backend server at this point — instead, complete the handshake at the gateway, then establish a separate initialize handshake with each backend server the client is permitted to access. Aggregate the backend servers' capability responses into a unified capabilities declaration returned to the client. This aggregation is the mechanism by which the client sees a single unified MCP server, not a fleet.

Step 3: Layer Authentication and Authorisation

Implement the auth middleware as the first handler in the request pipeline — before any routing or capability lookup. Validate the client's bearer token via the authorisation server's introspection endpoint (preferred for revocation support) or local JWT signature verification (lower latency, higher throughput). Attach the validated identity claims to the request context. Implement the authorisation check in the routing layer: before proxying a tools/call request, verify the authenticated identity is permitted to invoke the named tool on the target server. Return a JSON-RPC -32000 application error with a descriptive message for authorisation failures.

Step 4: Implement Audit Logging and Rate Limiting

Implement the audit logger as middleware that fires synchronously after the backend server returns a response but before the gateway forwards it to the client. The audit record must include: client identity, session ID, tool or resource name, target server ID, request timestamp, response timestamp, result status (success/error), and error code if applicable. Arguments should be logged in sanitised form (PII fields redacted per a configured field mask). Implement rate limiting using a Redis-backed token bucket: separate buckets per client identity and per target server, with configurable refill rates. Return 429 with a Retry-After header — never silently drop requests.

AU Compliance Note — APRA CPS 234 Attachment C & APS 330: The MCP Gateway is the single most important artefact for satisfying APRA CPS 234 Attachment C's requirement to test "the completeness and accuracy of records of access to information assets." As the unified chokepoint for all AI tool invocations, the gateway audit log is the authoritative record. APRA examiners assessing CPS 234 compliance will expect to see: (1) a centralised, tamper-evident log of every AI-to-system interaction with authenticated identity; (2) evidence that no interaction can bypass the log path (architecture diagram + penetration test); (3) log retention of at least 7 years per APS 330. The gateway's sequence-numbered audit log (with alerting on any gap) is the mechanism by which "completeness" can be demonstrated — a log with gaps cannot satisfy Attachment C. Store audit records in WORM storage (S3 Object Lock in Compliance mode, or Azure Immutable Blob Storage) and retain signed log manifests separately so that a storage compromise cannot silently delete records.

8. Security Considerations

OWASP LLM Top 10 Mapping

Additional Controls

• Implement a gateway-level emergency circuit breaker (administrative API endpoint) that immediately halts all proxying; activate during incidents without requiring server-by-server intervention
• Enforce mutual TLS between the gateway and all backend MCP servers; plain HTTP between gateway and backend is prohibited in production
• Rotate gateway-to-server client credentials quarterly; store in secrets manager, not in gateway configuration files
• Log all gateway administrative API calls (routing rule changes, rate limit adjustments, circuit breaker activations) to a separate, highly privileged audit log
• Apply gateway egress filtering: the gateway process should only be able to reach registered backend server endpoints

9. Governance Artefacts

• MCP Server Fleet Register — live registry of all MCP servers with endpoint, capability set, access tier, owner, and health status
• Client Identity to Capability Allowlist — per-client-identity mapping of permitted tools and resources; reviewed quarterly
• Gateway Audit Log — immutable record of every proxied request, retained per APRA CPS 234 requirements (minimum 7 years for financial services)
• Rate Limit Configuration Record — documented per-client and per-server rate limits with justification and approval record
• Circuit Breaker Activation Log — record of every emergency circuit breaker activation with trigger, duration, and approver

10. SLOs

11. Cost Model

12. Trade-off Analysis

13. Failure Modes

14. Regulatory Mapping

15. Reference Implementations

AWS

Deploy the gateway as an ECS Fargate service (2 replicas across AZs) behind an Application Load Balancer. Use ElastiCache Redis for rate limiting state. Register backend MCP servers in AWS Cloud Map for service discovery. Route audit logs via Kinesis Data Firehose to S3 with server-side encryption and Object Lock for WORM compliance. Use AWS WAF on the ALB to enforce payload size limits and IP-based rate limiting as a first line of defence before gateway application logic.

Azure

Deploy as an Azure Container Apps environment with a minimum of 2 replicas. Use Azure Cache for Redis for rate limiting. Integrate with Azure API Management as the public-facing TLS termination layer, delegating MCP-specific routing to the container app backend. Use Azure Monitor diagnostic settings to route gateway access logs to a Log Analytics Workspace with the immutable log retention policy enabled. Azure Managed Identity handles gateway-to-backend-server authentication, eliminating stored credentials.

On-Premises / Self-Hosted

Package the gateway as a Helm chart deployed to Kubernetes. Use an Istio service mesh for mTLS between the gateway and backend MCP server pods, eliminating the need for application-level certificate management. Deploy Redis Cluster (3 nodes) for HA rate limiting state. Register backend servers in Consul with health checks. Forward audit logs via Fluentd DaemonSet to Elasticsearch with a dedicated index and ILM policy enforcing append-only writes and 7-year retention.

16. Related Patterns

• EAAPL-MCP001: MCP Server Design — the foundational server pattern that MCP Gateway routes to and governs
• EAAPL-MCP003: Multi-Server Orchestration — using the gateway as the entry point for orchestrated multi-server workflows
• EAAPL-MCP004: MCP Authentication & Authorisation — detailed identity and access management patterns enforced at the gateway
• EAAPL-AGT003: Agent Tool Registry — enterprise tool catalogue that feeds the gateway's server registry
• EAAPL-SEC001: API Security Gateway — parent pattern for enterprise API governance that MCP Gateway specialises

17. Maturity Assessment

18. Revision History