[EAAPL-PLT001] Enterprise AI Platform

Category: Platform Engineering Sub-category: Foundation Platform Version: 1.4 Maturity: Mature Tags: platform-engineering, internal-developer-platform, golden-path, shared-services, model-serving, developer-experience Regulatory Relevance: APRA CPS230, CPS234, EU AI Act (Article 9 Risk Management), ISO 42001, NIST AI RMF (GOVERN 1.1)

1. Executive Summary

The Enterprise AI Platform pattern establishes a shared, governed infrastructure layer that enables product teams to consume AI capabilities safely and efficiently without each team solving foundational concerns independently. Rather than allowing every business unit to procure models, build integrations, and manage compliance in isolation—creating exponential risk surface and duplicated cost—this pattern centralises platform concerns while preserving product team autonomy.

The platform delivers measurable outcomes: 60–80% reduction in time-to-first-AI-feature for new teams, consolidated cost visibility with per-team chargeback, a single control plane for policy enforcement (data classification, model access tiers, rate limits), and an audit trail satisfying regulatory obligations across all AI usage. The platform team operates as an internal product team serving engineering consumers, not a gatekeeping function. Adoption is driven through golden paths—opinionated, well-documented routes to common AI use cases—that make the right thing the easy thing. This pattern is the prerequisite upon which all other EAAPL platform patterns depend.

2. Problem Statement

Business Problem

Enterprises face uncoordinated AI adoption: each team independently evaluates models, negotiates vendor contracts, builds bespoke integrations, and manages compliance obligations. This creates duplicated investment, inconsistent risk posture, and no executive visibility into total AI spend or exposure. AI incidents (data leakage, hallucination in customer-facing output, cost overruns) are discovered reactively with no systematic controls.

Technical Problem

Without a shared platform, teams build thin wrappers around foundation model APIs, each implementing authentication, logging, error handling, and cost tracking differently. There is no consistent mechanism for prompt versioning, model failover, semantic caching, or response auditing. Security review is performed ad hoc. Infrastructure drift compounds over time.

Symptoms

• Multiple AWS/Azure/GCP AI accounts with no consolidated billing or spend alerts
• Product engineers spending >30% of AI feature development time on infrastructure concerns
• Security team performing point-in-time reviews rather than continuous enforcement
• No audit trail mapping AI outputs to the model version and prompt that produced them
• Data residency violations discovered post-deployment as teams use public endpoints without restriction
• Duplicate vendor contracts for the same model provider across business units

Cost of Inaction

• Regulatory non-compliance penalties (APRA operational risk, EU AI Act fines up to 3% global turnover)
• AI security incidents with no forensic trail, increasing breach disclosure obligations
• Cost inefficiency of 30–50% above market rate due to absence of volume commitments and caching
• 6–12 month delays in AI capability delivery as teams rebuild foundational patterns from scratch

3. Context

When to Apply

• Organisation has ≥3 product teams independently consuming or planning to consume AI services
• Enterprise has data classification requirements that must be enforced before prompts leave the perimeter
• AI spend is untracked or exceeds $50K/year across business units without consolidated visibility
• Regulatory obligations (APRA, EU AI Act, privacy legislation) require audit trails for AI-assisted decisions
• Platform or infrastructure team exists with mandate to provide shared engineering services

When NOT to Apply

• Single-product startup with one team: overhead of platform exceeds benefit; use a managed API gateway directly
• Proof-of-concept or time-boxed experiment: build direct integrations, migrate to platform post-validation
• Fully air-gapped deployment with no shared infrastructure capability: consider a simplified on-premises variant

Prerequisites

• Identity provider (IdP) capable of issuing service account credentials (OIDC/OAuth2)
• Centralised secrets management (HashiCorp Vault, AWS Secrets Manager, Azure Key Vault)
• Observability stack (metrics, logs, traces) available for platform instrumentation
• Executive sponsorship and cross-BU agreement on platform adoption mandate (voluntary adoption rarely scales past early adopters)
• Cloud landing zone or on-premises infrastructure with network segmentation capability

Industry Applicability

4. Architecture Overview

The Enterprise AI Platform is structured as five horizontal layers stacked atop shared cross-cutting services. Each layer has a clear ownership boundary and a defined interface contract. The deliberate separation of concerns between layers is what allows the platform to evolve (e.g., swapping model providers, adding new compute tiers) without disrupting product teams.

Layer 1 — Infrastructure and Compute provides the physical and virtual compute substrate: GPU/accelerator clusters for self-hosted model serving, cloud provider AI endpoints (Amazon Bedrock, Azure OpenAI, Google Vertex AI), and VPC/network controls enforcing data residency. This layer is owned by the Platform Infrastructure team and changes infrequently. The critical design decision here is whether to use a shared GPU pool, dedicated per-tenant compute, or a hybrid—this choice has profound cost and isolation implications addressed in the Trade-Off Analysis.

Layer 2 — Model Serving and Registry abstracts individual model deployment concerns. It hosts the Model Registry (model metadata, capability cards, approved versions, deprecation notices), the Serving Layer (OpenAI-compatible inference endpoints whether models are self-hosted via vLLM/TorchServe or proxied from cloud providers), and the Model Lifecycle Manager. The OpenAI-compatible API surface is a deliberate choice: it maximises ecosystem compatibility and allows product teams to switch underlying models with zero code change.

Layer 3 — AI API Gateway is the primary integration point for all platform consumers. It enforces authentication (API keys/OIDC JWT), authorisation (RBAC/ABAC on model and capability access), rate limiting per consumer/team, cost allocation tagging, prompt/response logging for audit, semantic caching, and circuit breaking. Every request transits this layer—there are no side-door paths to models. This is the enforcement perimeter for all security and governance controls.

Layer 4 — Developer Services includes the capabilities that accelerate product team velocity: the Prompt Registry (versioned prompts with promotion workflows), the Evaluation Framework (automated benchmarking against golden datasets), the Experimentation Service (A/B routing for model comparison), and the RAG Orchestration Service. These are optional services product teams can adopt; the gateway is mandatory.

Layer 5 — Developer Portal is the human-facing surface: API catalogue, self-service onboarding, per-team dashboards, AI playgrounds, policy transparency, and documentation. This layer drives adoption and reduces platform team support burden. The portal is built as a product—it has a roadmap, user research input, and a feedback loop with consuming teams.

Cross-cutting Shared Services underpin all layers: Identity and Access Management, Secrets Management, Observability (metrics/logs/traces), Policy Engine (OPA or equivalent), Cost Management and Chargeback, and Data Classification Service. These services are not AI-specific—they extend the existing enterprise platform—but they must be explicitly wired into the AI platform's control plane.

The Platform Team vs. Product Team operating model is critical. The platform team owns Layers 1–3 and the shared services. Product teams own their applications, their prompts, and their AI feature logic. Layer 4 services are joint-owned with a platform team as service provider and product teams as co-designers. The golden path concept operationalises this: the platform team publishes opinionated starter templates, SDKs, and runbooks that encode best practice so product teams can onboard a new AI capability in hours rather than weeks.

5. Architecture Diagram

6. Components

Layer 1 — Infrastructure and Compute

Layer 2 — Model Serving and Registry

Layer 3 — AI API Gateway

Layer 4 — Developer Services

Layer 5 — Developer Portal

7. Data Flow

Primary Flow — Product Team AI Request

Error Flow

8. Security Considerations

Authentication and Authorisation

• All consumers authenticate via short-lived OIDC JWT tokens or rotatable API keys stored in Secrets Manager; long-lived static credentials are prohibited
• RBAC model: model-viewer, model-invoker, prompt-editor, platform-admin; ABAC extends this with data classification attributes
• Service-to-service communication within the platform uses mTLS with certificates managed by the service mesh (Istio/Linkerd)

Secrets Management

• All model provider API keys (OpenAI, Anthropic, AWS Bedrock IAM roles) are stored in HashiCorp Vault or cloud-native secrets manager; zero hardcoded credentials
• Secrets rotation is automated; gateway refreshes credentials on a schedule without downtime
• Audit log of every secret access event

Data Classification and Encryption

• All prompts and responses classified at ingress by the Data Classification Service; classification label persists through the audit trail
• Data at rest: AES-256 encryption for audit log store, vector cache, and model registry
• Data in transit: TLS 1.3 minimum for all internal and external communication
• PII in prompts: masked or tokenised before sending to third-party cloud endpoints if data residency policy requires

Auditability

• Cryptographic hash of every prompt and response stored in the audit log; enables non-repudiation
• Audit log is append-only and stored in a separate security account with no delete permissions for platform operators
• Audit events emitted to SIEM (Splunk/Sentinel/Chronicle) in real time

OWASP LLM Top 10 Controls

9. Governance Considerations

Responsible AI Framework

• Every model onboarded to the registry must have a completed Model Risk Card covering intended use, limitations, bias evaluation results, and regulatory classification
• High-risk AI use cases (as defined by EU AI Act Annex III or organisational risk policy) require additional approval and enhanced monitoring
• Data used for model fine-tuning must go through the Data Ethics Review process

Model Risk Management

• Models are classified by risk tier: Low (content summarisation), Medium (customer-facing recommendations), High (automated decisions affecting individuals)
• High-tier models require a signed-off Model Risk Assessment before production promotion
• Ongoing model monitoring for performance drift, bias drift, and output quality degradation

Human Approval Gates

• Changes to platform-wide policies (rate limits, model access tiers, data classification rules) require approval from the AI Platform Governance Board
• High-risk model promotions to production require Platform Owner + Chief Risk Officer sign-off
• Agentic use cases that can initiate real-world actions (send emails, execute transactions) require explicit human-in-the-loop gate design

Policy and Traceability

10. Operational Considerations

Monitoring

SLOs

Logging

• Structured JSON logs emitted by all platform components; correlated by x-request-id and x-team-id headers
• Log levels: INFO for all gateway transactions, WARN for policy near-misses, ERROR for circuit openings and auth failures
• Security-sensitive events (policy violations, auth failures) shipped to SIEM within 60 seconds
• Log retention: 90 days hot (searchable), 7 years cold (compliance archive)

Incident Response

Disaster Recovery

11. Cost Considerations

Cost Drivers

Scaling Risks

• Token cost scales super-linearly with context window abuse: no-context-limit requests from one team can dominate platform spend
• Uncontrolled model tier usage: teams defaulting to most expensive model for every use case without routing intelligence
• Cache cold-start: new deployment or cache eviction causes temporary cost spike as cache warms

Optimisations

• Semantic caching: 20–40% token reduction on repetitive workloads (FAQ, summarisation)
• Model tier routing: route simple tasks to cheaper models (GPT-4o-mini, Claude Haiku); reserve frontier for complex reasoning
• Prompt compression: strip whitespace, compress system prompts via shared library; 10–15% token reduction
• Batch API for non-interactive: use provider batch APIs at 50% discount for overnight processing
• Reserved capacity: negotiate reserved throughput with cloud AI providers for predictable workloads

Indicative Cost Range

12. Trade-Off Analysis

Compute Architecture Options

Tenant Isolation Options

Architectural Tensions

13. Failure Modes

Cascading Failure Scenarios

• Semantic cache failure → cold-start cost spike: Cache failure causes all requests to hit model directly; combined with a traffic spike this can exhaust token budgets across multiple teams simultaneously and trigger cloud provider rate limits. Mitigation: circuit breaker on cache with graceful bypass; pre-emptive capacity buffer in token budgets.
• Policy engine outage → open or closed failure: If OPA becomes unavailable, the gateway must fail open (allow all, risk policy bypass) or fail closed (deny all, block all AI features). This is a critical design choice; most enterprises should fail closed with a break-glass procedure.
• Identity provider outage → complete gateway authentication failure: If the IdP issuing JWTs is unavailable, all JWT-authenticated requests fail. Mitigation: API key fallback path for critical production consumers; IdP HA configuration.

14. Regulatory Considerations

APRA CPS 230 (Operational Risk)

• The platform must be classified as a Critical or Important Business Service if AI features are material to regulated activities; this triggers BCP/DR obligations including RTO/RPO targets above
• Third-party model providers (OpenAI, Anthropic) must be assessed under CPS 230 third-party risk management obligations; contracts must include sub-contracting visibility, audit rights, and incident notification requirements
• Operational incidents affecting AI services must be reportable to APRA if material

APRA CPS 234 (Information Security)

• The audit log is an information asset requiring classification, protection, and retention per CPS 234
• All platform components handling sensitive data must be within the CPS 234 information security capability boundary
• Penetration testing of the AI API Gateway is required at least annually and after significant changes

Privacy Act 1988 (Australia) / GDPR (EU)

• Personal information in prompts and responses must be handled in accordance with the Privacy Act; prompt logging of PII-containing interactions requires a Privacy Impact Assessment
• Data minimisation principle applies: prompts should not contain more PII than necessary for the AI task
• Data residency controls must enforce storage of Australian personal information within Australia if required by APP 8 considerations

EU AI Act

• Article 9 requires risk management systems for high-risk AI applications; the Model Risk Card and platform governance artefacts satisfy this requirement
• Article 13 transparency obligations require AI-generated content to be identifiable as such in consumer-facing applications
• Article 17 quality management system requirements are met by the prompt version control, evaluation framework, and change governance processes

ISO 42001 (AI Management System)

• The platform governance artefacts (Model Risk Cards, audit logs, governance board minutes) constitute the AI management system records required by ISO 42001 Clause 7
• Continual improvement processes (evaluation framework, post-incident review) satisfy Clause 10

NIST AI RMF

• GOVERN 1.1: AI risk tolerance defined via model risk tiers and data classification policies
• MAP 2.1: AI risk context mapped through Model Risk Cards and use case classification
• MEASURE 2.3: Metrics for AI risk tracked through observability stack and governance dashboards
• MANAGE 3.1: Response plans for AI incidents documented in platform runbook

15. Reference Implementations

AWS

Azure

GCP

On-Premises

16. Related Patterns

17. Maturity Assessment

Overall Maturity: Mature This pattern is in production at multiple large enterprises across financial services, healthcare, and technology verticals. Reference implementations are available for all major cloud providers. Tooling ecosystem (Kong, LiteLLM, Backstage, vLLM) is stable and production-proven.

Scoring Matrix

18. Revision History