AI Medical Diagnosis Failures: Algorithmic Errors in Healthcare Decision-Making

Multiple studies documented significant failures in AI medical diagnosis systems, including racial bias in algorithms that determined patient care priority, diagnostic errors in radiology AI tools approved by the FDA, and algorithmic failures that denied patients necessary treatments. A widely cited study revealed that an algorithm used by major US health systems to allocate care systematically discriminated against Black patients by using healthcare spending as a proxy for health needs, failing to account for systemic disparities in healthcare access. The algorithm affected an estimated 200 million patients annually. Separately, FDA-cleared AI diagnostic tools for radiology showed significant performance drops when deployed on patient populations different from their training data, missing cancers and other critical diagnoses in demographic groups underrepresented in training datasets. The failures demonstrated that AI medical tools can embed and amplify existing healthcare disparities while presenting algorithmic outputs as objective clinical evidence.

The ramifications of this incident extend beyond the immediate data exposure. Privacy regulators in multiple jurisdictions have opened investigations, and affected individuals are organizing collective action to demand accountability and meaningful remediation. The case highlights systemic weaknesses in how organizations handle personal data and the gap between corporate privacy promises and operational reality.

For impacted individuals, immediate action is critical. Filing a data subject access request forces the company to disclose exactly what data they hold about you, providing the foundation for deletion requests, regulatory complaints, and potential legal action. Below, we outline the specific data types at risk and the concrete steps you can take to protect yourself.