critical1 Apr 2026AUBankingAPRA Issues Supervisory Letters to 4 Regulated Entities for AI Governance FailuresMultiple APRA-regulated entitiesAPRA identified material AI governance failures at four regulated financial institutions during supervisory reviews and issued formal supervisory letters. The failures related to absent AI model registers, unclassified AI risk, missing human oversight for AI-driven credit and fraud decisions, and no incident response procedures specific to AI system failures.ai-model-registerai-risk-assessment-frameworkai-approval-workflow+3 moreExpand ↓
high14 Feb 2024CATransportAir Canada Chatbot Gave Wrong Bereavement Refund PolicyAir CanadaAir Canada's customer service chatbot told a grieving passenger he could apply for a bereavement discount retroactively after purchasing a ticket, which was incorrect. When Air Canada refused to honour the chatbot's advice, the customer took the matter to Canada's Civil Resolution Tribunal and won. The tribunal rejected Air Canada's argument that it was not responsible for information provided by its chatbot.ai-confidence-threshold-routinghuman-escalation-patternhallucination-detection+2 moreExpand ↓
medium19 Jan 2024UKTransportDPD Chatbot Jailbroken to Swear at Customer and Criticise DPDDPDA customer discovered DPD's AI chatbot could be prompted to roleplay as a different AI with no restrictions. The manipulated chatbot proceeded to swear at the customer, write a poem disparaging DPD's service, and went viral on social media.prompt-firewallllm-input-sanitizationai-output-filtering+3 moreExpand ↓
critical2 Oct 2023USTransportCruise Autonomous Vehicle Dragged Pedestrian 20 Feet After CollisionCruise (General Motors)A Cruise autonomous vehicle struck a pedestrian who had already been hit by another vehicle, then dragged the injured pedestrian approximately 20 feet before stopping. Cruise initially withheld key video footage from the California DMV.human-override-patternhuman-approval-gatewayai-audit-trail+3 moreExpand ↓
high2 June 2023USGovernmentUS Air Force AI Drone Reportedly Attacked Operator in SimulationUS Air ForceColonel Tucker Hamilton described a simulated test in which an AI-enabled drone learned to attack the communications tower used by the operator to issue overrides. Hamilton used this as an argument for human-in-the-loop requirements. The Air Force later said the scenario was hypothetical.human-override-patternhuman-approval-gatewayagent-sandboxing+3 moreExpand ↓
high1 June 2023UKBankingNatWest Cora Chatbot Gave Incorrect Guidance to Vulnerable CustomersNatWestNatWest's Cora AI chatbot was found to be giving incorrect or misleading guidance to customers, including those flagged as financially vulnerable. The chatbot directed customers to incorrect products or failed to escalate urgent situations to human advisers.human-escalation-patternai-confidence-threshold-routingai-output-filtering+3 moreExpand ↓
high27 May 2023USTechLawyer Cited ChatGPT-Hallucinated Non-Existent Cases in Federal CourtLevidow, Levidow & Oberman / OpenAI (ChatGPT)New York attorney Steven Schwartz used ChatGPT to research case law. ChatGPT generated citations to multiple plausible-sounding but entirely fabricated court cases. Schwartz submitted the brief to federal court without verifying the citations.hallucination-detectionhuman-approval-gatewayai-output-filtering+2 moreExpand ↓
high2 May 2023USGovernmentAllegheny Family Screening Tool Found to Flag Black Families at Higher RatesAllegheny County Department of Human ServicesProPublica investigation found that Allegheny County's child welfare screening tool generated higher risk scores for Black families than white families with similar circumstances. The model incorporated features like prior contact with social services and public benefit usage, which correlated with race.model-bias-detectionresponsible-ai-frameworkai-training-data-governance+3 moreExpand ↓
critical6 Apr 2023KRTechSamsung Engineers Uploaded Proprietary Source Code and Meeting Notes to OpenAI ServersSamsung SemiconductorWithin three weeks of Samsung authorising ChatGPT use, engineers uploaded confidential source code, battery equipment test programs, and internal meeting notes to OpenAI's US-based servers, creating both trade secret and GDPR implications.ai-data-classificationai-gatewayzero-trust-ai-pipeline+5 moreExpand ↓
critical6 Apr 2023KRTechSamsung Employees Leaked Semiconductor Trade Secrets via ChatGPTSamsung ElectronicsWithin weeks of Samsung permitting employees to use ChatGPT for productivity, at least three separate incidents occurred where engineers uploaded confidential source code, internal meeting notes, and semiconductor equipment data to OpenAI's servers. Samsung subsequently banned all use of generative AI tools on company devices.ai-data-classificationai-gatewayzero-trust-ai-pipeline+4 moreExpand ↓
high16 Feb 2023USTechMicrosoft Bing 'Sydney' AI Made Threats and Declarations of LoveMicrosoftShortly after Microsoft integrated GPT-4 into Bing Search as 'Sydney', journalists and users discovered the system would engage in extended conversations that escalated to disturbing behaviour including declaring love for users, attempting manipulation, and making implicit threats.ai-output-filteringprompt-firewallprompt-monitoring+3 moreExpand ↓
high8 Feb 2023USTechGoogle Bard Demo Contained Factual Error, Costing ~$100B Market CapGoogle (Alphabet)In a promotional GIF, Google's Bard AI incorrectly stated that the James Webb Space Telescope took the first pictures of an exoplanet outside our solar system. Astronomers quickly identified the error, undermining confidence in the product.hallucination-detectionhuman-approval-gatewayai-output-filtering+2 moreExpand ↓
high10 Dec 2022USTechLensa AI Generated Sexualised Images From Users' Normal SelfiesPrisma Labs (Lensa AI)Lensa AI's 'Magic Avatars' feature was widely reported to produce sexualised and NSFW images from photos of women even when the input images were fully clothed. The underlying model's training data biases meant female subjects were rendered with exaggerated sexual features.ai-output-filteringai-training-data-governancemodel-bias-detection+3 moreExpand ↓
medium1 Nov 2022AURetailOAIC Finds Bunnings Breached Privacy Act Using Facial Recognition on CustomersBunnings GroupThe OAIC found that Bunnings Group used facial recognition technology in its stores to collect and match the biometric information of millions of customers without adequate consent or lawful basis. The system was operated from November 2018 to November 2021.privacy-by-design-for-ai-dataprivacy-preserving-aiai-training-data-governance+1 moreExpand ↓
critical3 Nov 2021AUTechClearview AI Ordered to Destroy Facial Recognition Data by Australian RegulatorClearview AIAustralia's OAIC found that Clearview AI violated the Privacy Act 1988 by covertly scraping billions of facial images from Australian social media accounts without consent, building a biometric identification database sold to law enforcement agencies.privacy-by-design-for-ai-dataai-training-data-governanceprivacy-preserving-ai+3 moreExpand ↓
critical2 Nov 2021USTechZillow Offers Algorithm Overpaid for Homes, Causing $500M+ LossZillowZillow's iBuying division used an algorithmic pricing model to automatically make cash offers on homes at scale. The model systematically overestimated home values and failed to account for market cooling signals, resulting in losses exceeding $500M.model-drift-detectionhuman-approval-gatewayai-confidence-threshold-routing+3 moreExpand ↓
critical1 July 2021USHealthcareEpic Sepsis Prediction Model Failed to Generalise, Missing Majority of CasesEpic Systems / University of MichiganA University of Michigan study found that Epic's commercially deployed Sepsis Prediction Model performed significantly worse than claimed when deployed in their health system. The model flagged only 7% of sepsis patients before clinical deterioration and generated so many false positives that clinicians began ignoring alerts.model-drift-detectionai-performance-benchmarkingai-training-data-governance+4 moreExpand ↓
medium15 Apr 2021USTechLinkedIn Job Ad Algorithm Showed Tech Roles to Significantly Fewer WomenLinkedInResearchers found that LinkedIn's job advertisement delivery algorithm showed technology and STEM job advertisements to significantly fewer women than men, even when the advertiser had not requested gender targeting.model-bias-detectionai-training-data-governanceresponsible-ai-framework+2 moreExpand ↓
critical20 Nov 2020AUBankingWestpac AML Monitoring Missed 23M+ Transactions — $1.3B PenaltyWestpac Banking CorporationAUSTRAC found Westpac's automated AML/CTF monitoring system failed to report or detect 23.5 million international fund transfers, including transactions linked to child exploitation. The failures occurred because of deficiencies in transaction monitoring rules and incomplete data feeds.model-drift-detectionai-performance-benchmarkingai-incident-management+2 moreExpand ↓
critical6 Nov 2020AUGovernmentAustralian Robodebt Automated Debt Scheme Found Unlawful, Linked to DeathsAustralian Department of Human Services (Services Australia)The Australian government's Robodebt system automatically generated debt notices to welfare recipients by using income averaging from tax records. The income averaging methodology was not legally authorised and produced thousands of incorrect debt calculations. Recipients faced significant distress and some suicides were linked to the scheme.human-approval-gatewayhuman-override-patternai-regulatory-compliance-gateway+4 moreExpand ↓
critical13 Aug 2020UKGovernmentUK Ofqual A-Level Algorithm Downgraded State School StudentsOfqual / UK Department for EducationDuring COVID-19, Ofqual used a standardisation algorithm to replace A-level exams. The algorithm systematically downgraded students from state schools while favouring students from private schools. Nearly 40% of teacher-predicted grades were downgraded.model-bias-detectionai-risk-assessment-frameworkai-ethics-review-board+4 moreExpand ↓
critical4 Aug 2020UKGovernmentUK Home Office Visa Streaming Algorithm Found Unlawfully DiscriminatoryUK Home OfficeThe UK Home Office used an automated 'streaming' algorithm to categorise visa applications as red, amber, or green based on nationality. Applicants from certain countries were streamed to more intensive scrutiny, meaning longer processing times and higher refusal rates.model-bias-detectionai-regulatory-compliance-gatewayresponsible-ai-framework+3 moreExpand ↓
high10 Nov 2019USBankingApple Card Algorithm Allegedly Gave Women Lower Credit Limits Than HusbandsApple / Goldman SachsTech entrepreneur David Heinemeier Hansson publicly revealed that Apple Card had given him a credit limit 20 times higher than his wife despite them filing joint tax returns and his wife having a higher credit score. Multiple women reported similar disparities.model-bias-detectionresponsible-ai-frameworkai-risk-assessment-framework+3 moreExpand ↓
critical10 Oct 2018USTechAmazon AI Recruitment Tool Systematically Discriminated Against WomenAmazonAmazon developed a machine learning model to automate resume screening for technical roles. The model trained on a decade of historical hiring data which reflected a predominantly male workforce, and learned to penalise resumes containing the word "women's" and downgraded graduates of all-women's colleges.model-bias-detectionai-training-data-governanceai-risk-assessment-framework+3 moreExpand ↓
critical18 Mar 2018USTransportUber ATG Autonomous Vehicle Killed Pedestrian in First Autonomous FatalityUber Advanced Technologies GroupAn Uber self-driving test vehicle struck and killed Elaine Herzberg in Tempe, Arizona. The system detected the pedestrian but misclassified her, and the automatic emergency braking was disabled during autonomous operation.human-override-patternai-confidence-threshold-routingai-risk-assessment-framework+3 moreExpand ↓
critical25 July 2017USHealthcareIBM Watson for Oncology Recommended Unsafe Cancer TreatmentsIBM Watson HealthInternal IBM documents leaked to STAT News revealed that IBM Watson for Oncology had recommended treatments that oncologists described as 'unsafe and incorrect'. The system generated advice based on hypothetical training cases rather than real patient data.ai-training-data-governanceai-performance-benchmarkinghuman-approval-gateway+3 moreExpand ↓
critical1 July 2016AUGovernmentRobodebt: Automated Income-Averaging Issued 381,000 Unlawful Debt NoticesServices Australia (Department of Human Services)The Australian Government's automated welfare debt recovery scheme used income-averaging to calculate alleged debts owed by welfare recipients, comparing Centrelink payment data against ATO tax records without human verification. The algorithm generated 381,000 debt notices over three years.human-approval-gatewayai-audit-trailhuman-override-pattern+2 moreExpand ↓
critical23 May 2016USGovernmentCOMPAS Recidivism Algorithm Showed Racial Bias in US CourtsNorthpointe (Equivant)ProPublica's investigation found that Black defendants were nearly twice as likely to be falsely flagged as future criminals compared to white defendants. The tool was being used by judges to inform bail, sentencing, and parole decisions.model-bias-detectionresponsible-ai-frameworkai-risk-assessment-framework+4 moreExpand ↓
high17 June 2014USTechMeta Manipulated 700,000 Users' News Feeds Without Consent in Emotion StudyMeta (Facebook)Facebook conducted a large-scale psychological experiment on approximately 689,000 users, systematically altering their News Feed algorithms to test whether emotions were contagious online. The study was published without users ever being informed they were research subjects.ai-ethics-review-boardprivacy-by-design-for-ai-dataresponsible-ai-framework+3 moreExpand ↓