[EAAPL-SEC008] Secrets Management for AI

Category: Security / Credential Management Sub-category: API Key and Token Lifecycle Version: 2.0 Maturity: Mature Tags: secrets-management api-keys vault dynamic-secrets rotation audit credential-hygiene Regulatory Relevance: APRA CPS234 §22, ISO 27001 A.9.4, NIST CSF PR.AC-1, EU AI Act Art. 9, SOC 2 CC6.1

1. Executive Summary

Secrets Management for AI addresses one of the most prevalent and consequential security failures in enterprise AI deployments: the mishandling of model API keys, service tokens, and credentials used by AI systems. Leaked or improperly managed API keys for commercial LLM providers (OpenAI, Anthropic, Azure OpenAI) give attackers the ability to generate costs at the organisation's expense, access data sent in API calls, and exfiltrate model outputs — with no attribution back to the attacker.

The business risk is material and immediate. A single leaked sk- OpenAI API key has been used to generate tens of thousands of dollars in API charges within hours of exposure. AI API keys embedded in mobile applications, JavaScript bundles, GitHub repositories, or CI/CD pipeline logs represent standing vulnerabilities that can be exploited at any time, from anywhere, with no prior access to the organisation's infrastructure.

This pattern establishes the complete lifecycle for AI credentials: vault storage and retrieval (no secrets in code or environment variables), dynamic secret generation where supported, automated rotation, granular access control, and comprehensive audit logging of every secret access event. For VITE_-prefixed or client-side secret exposure in particular, this pattern defines the architectural controls that prevent secrets from reaching user-accessible environments — a critical failure mode in frontend AI applications.

2. Problem Statement

Business Problem

AI model API keys are high-value credentials: they grant direct access to model inference, with billing charged to the owner. They are also widely mishandled — treated as configuration values rather than secrets, hard-coded in source code, embedded in build artifacts, and shared across teams without access controls. Every insecure placement is a potential financial and security exposure.

Beyond cost, AI API keys may be used to submit requests that the organisation would not authorise — generating harmful content, extracting information, or probing model capabilities in ways that create legal and reputational risk, with the charges appearing on the organisation's bill.

Technical Problem

Common failure modes in AI credential management:

• Hard-coded in source code: OPENAI_API_KEY = "sk-..." in application code, committed to Git history.
• Environment variables in containers: Accessible to any process in the container; visible in orchestrator logs; leaked in crash dumps.
• VITE_-prefixed secrets in frontend builds: Build tools like Vite bundle VITE_* environment variables into client-side JavaScript, making them accessible to every user who loads the page.
• Shared keys across environments: Same key used in dev, staging, and production — a leaked dev key gives production access.
• No rotation: API keys never rotated; a key compromised 6 months ago may still be valid.
• No audit: No record of which systems use which keys; key compromise cannot be scoped.
• Broad-permission keys: Using an OpenAI "all models" key for an application that only needs GPT-3.5-turbo.

Symptoms

• AI API keys in Git history (discoverable via git log -S "sk-").
• VITE_OPENAI_API_KEY in production JavaScript bundles.
• Unexplained spikes in model API spend (possible key misuse).
• No centralised record of which applications hold which API keys.
• Application secrets in CI/CD pipeline logs.
• Keys with no expiry date and no rotation history.

Cost of Inaction

3. Context

When to Apply

• Any application, service, or pipeline that holds credentials for AI model providers.
• CI/CD pipelines that need model API access for testing or evaluation.
• Frontend or mobile applications that call AI APIs — the frontend must NEVER hold model provider credentials directly.
• Multi-environment deployments (dev/staging/prod) needing credential isolation.
• Teams managing more than one AI model provider integration.

When NOT to Apply

• Single-developer local development with credentials that never leave the developer's machine and are isolated to their personal API account.

Prerequisites

Industry Applicability

4. Architecture Overview

The secrets management architecture for AI systems is built on a single foundational principle: model provider credentials must never be present outside the vault and the AI Gateway's runtime memory. Every other location — source code, environment variables, build artifacts, logs, client bundles — is prohibited.

Vault as Single Source of Truth

All AI credentials are stored in a centralised vault. The vault is the only place where credentials exist at rest. Applications do not store credentials — they request them at runtime. This separation is enforced through:

• Architecture policy: no secret values in application configuration files or source code.
• Automated scanning: CI/CD pipeline executes a secret-grep step that fails the build if any known secret pattern (OpenAI sk-, Anthropic sk-ant-, sbp_ for Supabase) appears in source files, build artifacts, or environment variable names with VITE_ prefix.
• Vault access control: applications can only read the specific secrets they need, verified by workload identity.

Dynamic Secrets Where Supported

Some AI providers support dynamic or short-lived credential generation:

• AWS Bedrock: Uses IAM roles via assume_role; credentials are time-limited by STS.
• Azure OpenAI: Uses Azure Managed Identity; credentials retrieved from Azure AD, never static keys.
• Anthropic, OpenAI: Do not currently support dynamic credentials — static API keys must be managed and rotated manually.

For providers requiring static keys, vault rotation is the control: vault stores the key, tracks its age, and triggers rotation workflows. Rotation involves: generating a new key via the provider's API, updating vault, confirming applications are using the new key, and revoking the old key.

Key Scoping and Least Privilege

Every AI application is issued credentials scoped to its specific requirements:

• A customer service bot that only uses gpt-4o-mini is issued a key with rate limits and, where supported, model access restrictions.
• A batch processing pipeline with no human interaction is issued a separate key from the real-time serving key — isolating blast radius if either is compromised.
• Separate keys for dev, staging, and production environments. Dev key has the smallest spending limit; production key has the strictest access controls.

Frontend Architecture — The Critical Rule

Frontend applications (React, Vue, Angular) and mobile applications must NEVER hold model provider API keys. The technical reason: any value in a JavaScript bundle or mobile binary is publicly accessible — to every user, to security researchers, and to attackers. The architecture must be:

Browser/Mobile → Application Backend → AI Gateway → Model Provider

The application backend holds the session context, authenticates the user, enforces application-level access controls, and proxies requests to the AI Gateway (which holds provider credentials). The browser never sees a model provider API key.

Build systems that use VITE_, REACT_APP_, EXPO_PUBLIC_, or similar prefixes bundle the prefixed variable into client-side code. No secret should ever have these prefixes. CI/CD must fail on any VITE_OPENAI_, REACT_APP_ANTHROPIC_, or similar pattern.

Audit Logging

Every secret read from vault generates an immutable audit record: which application, which secret path, which accessor (human or workload identity), timestamp, and IP/network source. This enables:

• Scoping of a credential compromise (which systems accessed the key in the last 90 days).
• Detection of anomalous access patterns (secret accessed at 3am from an unusual IP).
• Compliance evidence (APRA CPS234 requires audit trails for access to critical information assets).

5. Architecture Diagram

6. Components

7. Data Flow

Primary Flow

Error Flow

8. Security Considerations

Authentication & Authorisation

• Applications authenticate to Vault using workload identity (Kubernetes SA, AWS IAM role, Azure Managed Identity) — no human-visible credentials required to access vault.
• Human operators access vault via SSO + MFA; break-glass access with dual approval and automatic alerting.
• Vault policies grant access to specific secret paths only — an application cannot read secrets outside its namespace.

Secrets Management (Meta)

• The vault itself requires protection: vault unseal keys are split using Shamir's secret sharing (e.g., 3-of-5) and distributed to security officers.
• Vault root token is generated once, used to configure vault, and then deleted — normal operations use lower-privileged tokens only.

Data Classification

• Secret paths in vault are classified: ai/providers/openai/prod-keys is RESTRICTED. Audit access to RESTRICTED paths generates alerts for any access outside business hours.

Encryption

• All vault secrets encrypted at rest with AES-256-GCM; encryption keys managed by vault's built-in seal (or externally via AWS KMS, Azure Key Vault HSM).
• All communication to vault over TLS 1.3.
• Vault audit log: write-ahead log; each entry signed to detect tampering.

OWASP LLM Top 10 Coverage

9. Governance Considerations

Governance Artefacts

10. Operational Considerations

SLOs

Incident Management

Key Compromise Response (15-minute target):

1. Receive alert (automated detection or developer report).
2. Immediately revoke key via provider API.
3. Generate and vault new key.
4. Verify all consumers have picked up new key (vault lease renewal cycle or forced restart).
5. Scope the incident: review vault audit log for all accesses using the compromised key path; review model provider's usage logs for anomalous queries.
6. File incident report; update runbook.

11. Cost Considerations

Cost Drivers

Indicative Cost Range

12. Trade-Off Analysis

Option Comparison

Architectural Tensions

13. Failure Modes

14. Regulatory Considerations

15. Reference Implementations

AWS

Azure

On-Premises

16. Related Patterns

17. Maturity Assessment

Overall Maturity: Mature

18. Revision History