[EAAPL-WRK008] Mixture of Agents

Category: Agentic Workflows Sub-category: Ensemble Architecture Version: 1.0 Maturity: Emerging Tags: mixture-of-agents, ensemble, consensus, multi-model, voting, aggregation, diverse-sampling Regulatory Relevance: ISO 42001 §8.4, EU AI Act (Art. 15), NIST AI RMF (MEASURE 2.5)

1. Executive Summary

The Mixture of Agents (MoA) Pattern defines an architecture in which multiple independent agents — potentially using different LLM providers, models, or sampling configurations — produce independent outputs for the same task, and a dedicated aggregator synthesises a final result from these diverse outputs. Unlike Fan-Out/Fan-In (EAAPL-WRK003), which fans out for coverage or throughput, MoA fans out for quality improvement through diversity: different models make different errors, and a well-calibrated aggregation can produce a result that consistently outperforms any individual model. Published benchmarks demonstrate 5–20% quality improvement over single-model approaches on complex reasoning and analysis tasks.

For CIO/CTO audiences: this is the AI equivalent of getting multiple independent expert opinions before making an important decision. A law firm, a medical panel, and a financial advisory board all use this approach — not because any individual expert is wrong, but because independent experts catch different errors and the consensus or synthesis is more reliable than any individual view. The cost is proportional to the number of agents (3 agents costs 3× as much as 1), so MoA is reserved for tasks where the quality improvement justifies the cost: high-stakes decisions, regulated analyses, outputs that will not be reviewed by a human expert.

2. Problem Statement

Business Problem

High-stakes enterprise decisions — legal risk assessments, clinical summaries, regulatory compliance determinations, executive briefings — require output reliability that a single LLM invocation cannot guarantee. The error rate of any individual model, even the most capable available, is non-zero and the failure mode is invisible: the model produces a confident, plausible-sounding output that contains material errors.

Technical Problem

Single-model outputs exhibit correlated errors: if the model makes a mistake in a particular domain or reasoning pattern, it makes the same mistake consistently. There is no mechanism within a single inference call to detect and correct errors that are within the model's systematic blind spots. Reflection (EAAPL-AGT006) addresses some errors but shares the same model's limitations in the critique phase.

Symptoms of Absence

• High-stakes outputs require expensive human expert review because single-model reliability is insufficient
• No diversity in output generation; all inference calls to the same model are correlated
• Quality ceiling is the individual model's capability, with no ensemble improvement possible
• Single-model failure modes are invisible until outputs are reviewed

Cost of Inaction

• Quality Risk: Single-model errors in high-stakes outputs create compliance and liability exposure
• Human Review Bottleneck: Expert review of every output is the only quality gate, creating a throughput bottleneck
• Opportunity: Peers using ensemble approaches achieve demonstrably better quality without proportionally higher cost for high-value tasks

3. Context

When to Apply

• Task output quality has material financial, legal, or safety consequences
• The task has an articulable quality benchmark against which improvement can be measured
• The additional cost (N× model inference) is justified by quality improvement and risk reduction
• Multiple capable models are available (different providers or model families)
• Aggregation strategy is clearly defined and produces consistently better results than individual outputs

When NOT to Apply

• High-volume, low-stakes tasks where N× cost is not justified
• Tasks requiring real-time responses with hard latency constraints
• Tasks where all available models have the same systematic blind spots (diversity is the source of benefit)
• Tasks that are too subjective to define a quality improvement metric

Prerequisites

• Access to multiple independent LLM models/providers
• Defined aggregation strategy (voting, synthesis, best-of-N)
• Quality benchmark for measuring MoA improvement over single-model baseline
• Fan-Out infrastructure (EAAPL-WRK003) for parallel worker execution

Industry Applicability

4. Architecture Overview

The MoA architecture has three layers: parallel proposers, optional discussion/critique, and an aggregator.

Proposer Layer Multiple proposer agents independently process the same task input and produce independent output proposals. The diversity between proposers is the source of quality improvement. Diversity can be achieved through: (a) different LLM providers (GPT-4o + Claude 3.5 Sonnet + Gemini 1.5 Pro), (b) different model sizes from the same provider, (c) same model with different sampling temperatures, (d) same model with different system prompts (each with a different expert persona), or (e) same model with different context window positions (to avoid position bias).

Optional Discussion Phase In higher-quality configurations, proposers are shown each other's outputs and produce a revised proposal that takes account of what others observed. This multi-round "discussion" mimics the expert panel model and can improve consensus quality. It adds latency and cost (each proposer makes an additional inference call per discussion round).

Aggregation Layer The Aggregator receives all proposer outputs and produces a final synthesised result. Aggregation strategies:

• Majority Voting: For categorical decisions (classify this document as "high/medium/low risk"), take the majority vote. Simple, deterministic, explainable.
• Weighted Voting: Weight votes by each proposer's historical accuracy on this task type. Requires tracked historical accuracy.
• Union Aggregation: For coverage-oriented tasks (find all risk flags), take the union of all proposer findings. Each finding attributed to the proposer(s) that identified it.
• LLM Synthesis: An aggregator LLM receives all proposer outputs and synthesises a unified final output. Highest quality, adds cost and latency. The aggregator is instructed to identify agreement, reconcile disagreements, and explain reasoning.
• Best-of-N Selection: Score all proposers' outputs on a quality rubric and return the highest-scoring output. Preserves full fidelity of the best output; does not synthesise.

Confidence and Agreement Reporting The aggregation result includes metadata: the number of proposers that agreed (for voting strategies), the agreement rate (for structured outputs), and individual proposer outputs (for audit and transparency). Low agreement (e.g., < 2/3 proposers agree) is a signal that the task is genuinely ambiguous or that the proposers have conflicting knowledge, and this signal should be reported to the caller and, for high-stakes tasks, escalated to human review.

5. Architecture Diagram

6. Components

7. Data Flow

Error Flow

8. Security Considerations

Cross-Provider Data Sharing

• In multi-provider MoA, task data is sent to multiple LLM provider APIs
• Mitigation: Review data classification against each provider's data processing agreement; use single-provider MoA (different models or sampling configurations) for sensitive data that cannot be sent to multiple providers

OWASP LLM Top 10

9. Governance Considerations

Multi-Provider Data Governance

• For each provider used in the proposer pool, verify data processing agreements, data residency requirements, and model training opt-out options
• For APRA-regulated data, confirm each provider is an approved supply chain entity

Governance Artefacts

10. Operational Considerations

SLOs

11. Cost Considerations

Optimisation Strategy

• Reserve MoA for tasks where single-model error rate is unacceptably high (> 5% material error rate)
• Use same-provider multi-temperature configuration as cheapest diversity strategy
• Route routine tasks to single-model; route high-stakes tasks to MoA via Router/Dispatcher (EAAPL-WRK004)

12. Trade-Off Analysis

13. Failure Modes

14. Regulatory Considerations

EU AI Act

• Art. 15 (Accuracy and Robustness): MoA directly implements the accuracy and robustness requirement for high-risk AI systems; the quality benchmark demonstrates measurable improvement.

ISO 42001

• §8.4: Multi-provider proposer pool requires supply chain AI governance; each provider must be assessed under the organisation's AI supplier framework.

NIST AI RMF

• MEASURE 2.5: The MoA quality benchmark (measured improvement over single-model baseline) directly implements the AI performance measurement requirement.

15. Reference Implementations

AWS

Azure

On-Premises

16. Related Patterns

17. Maturity Assessment

Overall Maturity: Emerging

18. Revision History