The Future of Care or the Automation of Inequality? Part 2
- 5 days ago
- 7 min read
Written by Sajdah Wendy Muhammad, Business Advisor
Wendy is a multi-million-dollar business and real estate developer, global thought leader, crisis manager, emotional intelligence coach, and award-winning urban historic preservationist. An international entrepreneur, she has pioneered innovative healthcare business models and founded the Mind of an Entrepreneur® brand to empower marginalized communities through wealth-building, business ownership, and sustainable community development.
The same technologies capable of expanding healthcare access may also amplify disparities if implemented without intentional governance. Artificial intelligence systems do not emerge independently from healthcare systems. They learn from them. This distinction matters because healthcare data is not merely clinical information.
Healthcare data reflects decades of policy decisions, insurance structures, access patterns, provider behaviors, reimbursement models, geographic inequities, and historical discrimination. If historical healthcare systems produced unequal experiences, artificial intelligence systems trained on those experiences may inherit similar distortions.
The risk, therefore, is not simply technological failure. The risk is automated inequality.
Historical data does not necessarily equal objective data
Artificial intelligence systems depend upon historical information to identify patterns. However, historical healthcare data should not automatically be interpreted as objective truth. Healthcare utilization patterns themselves have historically been unequal.
Numerous structural factors influence who appears within healthcare datasets, including insurance status, provider availability, transportation access, geographic location, specialist availability, healthcare affordability, institutional distrust, research participation rates, and socioeconomic conditions.
This creates an important policy concern. Artificial intelligence systems do not simply learn from disease. They learn from who received care. Communities with reduced healthcare utilization may therefore become underrepresented within datasets used to train future systems.
The data availability problem: When underrepresentation becomes infrastructure
Training datasets require large volumes of information. However, large datasets do not necessarily produce representative datasets. Lower insurance coverage rates, reduced specialty access, lower diagnostic testing frequency, and reduced participation within research studies may create smaller data footprints for historically marginalized populations.
This produces an important question, "If certain populations receive fewer diagnostic tests, fewer specialist referrals, fewer imaging studies, fewer clinical encounters, and fewer opportunities for participation in research, how accurately can future systems learn to identify disease within those populations?"
Healthcare utilization itself becomes part of the training data problem. This concern becomes particularly important if artificial intelligence significantly increases healthcare capacity. If future AI systems are used to evaluate larger populations than those represented within original training datasets, policymakers must ask, "Will systems trained on smaller and potentially less representative populations perform equally well when expanded to millions of additional users?" Scaling healthcare capacity without scaling representative data may create new risks.
Algorithmic bias is often structural rather than intentional
Numerous studies have demonstrated algorithmic bias within healthcare applications. One widely cited example involved healthcare risk algorithms using healthcare spending as a proxy for medical need. Because Black patients historically generated lower healthcare expenditures, not necessarily because they were healthier, but because they often received fewer services, the algorithm systematically underestimated medical need.
The consequence was that Black patients with equivalent disease burdens received fewer referrals for advanced care.
The issue was that historical spending patterns reflected unequal access rather than equal need. Artificial intelligence systems optimize for patterns. They do not automatically distinguish between correlation and causation, access and need, utilization and disease burden, or spending and illness severity.
Without governance, algorithms may unintentionally transform historical disparities into future decision-making systems.
Diagnostic accuracy concerns
Several emerging concerns suggest that diagnostic performance may vary when healthcare technologies are applied across increasingly diverse populations. Diagnostic accuracy challenges extend beyond simple questions of pigmentation or physical appearance. Healthcare outcomes are influenced by a complex interaction between biological, environmental, social, geographic, and behavioral factors. Artificial intelligence systems attempting to predict disease, identify risk, or recommend interventions must therefore interpret far more than isolated clinical variables.
Examples frequently discussed include dermatology systems trained disproportionately on lighter skin tones, pulse oximeter performance differences associated with skin pigmentation, imaging datasets lacking sufficient demographic diversity, predictive risk models demonstrating uneven performance across populations, and facial recognition systems demonstrating different accuracy rates across demographic groups.
Human populations may experience meaningful differences in environmental exposures. Communities may experience differing exposure patterns related to pollution exposure, housing conditions, occupational risks, food access, environmental toxins, and neighborhood infrastructure.
Environmental conditions influence disease development and progression. Diagnostic systems failing to adequately account for environmental context may produce less accurate predictions.
Disease prevalence and population-level risk differences
Different populations may demonstrate different prevalence rates for particular diseases or conditions. Examples include hypertension prevalence differences, cardiovascular risk distributions, maternal health complications, diabetes prevalence patterns, and kidney disease incidence.
Diagnostic systems trained primarily on populations with different baseline risk patterns may experience performance variation when deployed more broadly.
Genetic and biological variation
Healthcare systems increasingly recognize that disease expression, medication response, biological variation, environmental exposures, inherited risk factors, and population-level health patterns may influence clinical outcomes. Human populations are not identical in terms of disease prevalence, environmental exposures, physiologic variation, ancestry-related risk factors, healthcare utilization patterns, or social conditions that influence health outcomes.
Although race itself may function imperfectly as a clinical variable, healthcare systems have long recognized that population-level differences sometimes influence disease risk, treatment response, and health outcomes. The challenge for artificial intelligence is not simply recognizing difference. The challenge is determining which differences are clinically meaningful, which are social or environmental, which are biologically relevant, and how these factors interact within increasingly diverse populations. Artificial intelligence systems trained without adequate representation or contextual understanding may therefore miss clinically meaningful variation.
Rather, healthcare systems must recognize that populations may possess different combinations of ancestry-related risk factors, medication response patterns, inherited conditions, biomarker distributions, disease progression patterns, anatomical variation, physiologic differences, and environmental exposure patterns.
Artificial intelligence systems trained without adequate representation may miss clinically meaningful variation.
However, representation alone does not fully solve the problem. An equally important challenge involves the accuracy of the clinical standards themselves. Artificial intelligence systems frequently learn from existing clinical guidelines, measurement thresholds, diagnostic criteria, imaging interpretations, and historical physician decision-making patterns.
This creates an important policy concern, "What happens when existing standards contain limitations?"
Consider aneurysm detection. Certain aneurysm measurement thresholds have historically relied upon standardized diameter thresholds. However, vessel size frequently differs across individuals based on sex, body size, anatomy, and other physiologic factors. A vessel measurement that may not trigger concern in one patient population could represent substantially elevated risk in another. If artificial intelligence systems simply learn existing thresholds without contextual interpretation, systems may reproduce those inaccuracies at scale. This challenge extends beyond aneurysms.
Healthcare increasingly recognizes that numerous measurements may vary based on factors such as body size, anatomy, age, sex, genetic variation, environmental exposure, baseline physiology, and comorbidity burden.
Artificial intelligence systems therefore face a more complex task than simple pattern recognition. They must determine what constitutes normal, normal for whom, under what conditions, and at what level of risk.
This distinction matters because life-threatening conditions often emerge precisely at the margins where standardized thresholds fail. If artificial intelligence systems learn incomplete measurement frameworks, errors may become amplified as systems scale. The future challenge is therefore not simply building larger datasets. The challenge is ensuring that artificial intelligence systems learn from accurate, representative, and context-sensitive frameworks capable of recognizing clinically meaningful variation. Healthcare AI should not merely automate historical standards. It should improve them.
Social determinants and behavioral context
Disease risk is not determined exclusively by biology. Healthcare outcomes are strongly influenced by income, education, transportation access, healthcare utilization patterns, nutrition, chronic stress exposure, and neighborhood conditions.
Artificial intelligence systems trained primarily using clinical variables while ignoring broader context may oversimplify risk prediction.
Why small errors become large problems
Diagnostic performance matters because healthcare outcomes depend heavily on early detection. Small differences in accuracy rates may produce substantial consequences when applied across millions of patients.
A diagnostic model performing only slightly worse within specific populations may generate delayed treatment initiation, increased disease severity at diagnosis, higher hospitalization rates, increased mortality, increased healthcare costs, increased disability rates, and reduced workforce participation.
The scaling problem
Healthcare leaders frequently promote artificial intelligence because it may dramatically increase capacity. This creates an important policy question. If AI systems allow clinicians to evaluate larger populations, will systems trained on smaller and potentially less representative populations perform equally well when expanded?
A system serving one hundred thousand patients incorrectly creates problems. A system serving tens of millions incorrectly creates infrastructure risk. The central question, therefore, is not simply whether artificial intelligence works.
The more important questions are, "For whom does it work? Under what conditions? How does performance change at scale? And what are the consequences when those systems fail?"
Healthcare AI should not simply increase access. It must increase accurate access.
Artificial intelligence may scale representation problems
Healthcare leaders frequently emphasize that artificial intelligence may expand capacity. This may be true. However, increased capacity alone does not guarantee equitable outcomes. A biased system capable of serving ten times more people may simply distribute bias more efficiently. Artificial intelligence does not automatically eliminate representation problems. It may magnify them.
This creates perhaps the central policy challenge of healthcare AI, healthcare systems must ensure that increased scale does not produce increased inequity.
The objective should not merely be larger systems. The objective should be more accurate, more equitable systems capable of improving outcomes across increasingly diverse populations.
So the risks are clear, but the story does not end with diagnosis and data. AI's reach extends directly into the workforce itself, and into a much bigger question about money and power, who actually owns the algorithms making these decisions, who profits from the data patients generate every day, and whether Black healthcare professionals and entrepreneurs will be partners in this transformation or simply subjects of it. Part 3 closes the brief by examining the impact on healthcare workers, the ownership economy behind healthcare AI, and the concrete policy recommendations needed to make sure this technology builds equity rather than erasing it.
Read more from Sajdah Wendy Muhammad
Sajdah Wendy Muhammad, Business Advisor
Wendy Muhammad is a multi-million-dollar business developer, Author of the best-selling book, The Art and Science of Business, an Award-Winning Urban Historic Preservationist and Real Estate Developer, with more than $500 million in projects across healthcare, real estate, infrastructure, and community development. Muhammad is a leading voice in empowering entrepreneurs and building generational wealth. Her Mind of an Entrepreneur brand includes podcasts, workshops, and books that blend strategy, spirituality, and economic empowerment.
.jpg)
