AI in the NDIS and Beyond, What Happens When Systems Don’t Understand People
- Apr 14
- 4 min read
Sarah McLoughlin is the creator of Strategic Self-Advocacy™, founder of EduLinked and EduPsyched, and developer of Microsoft-supported digital tools that turn burnout into strategy across disability, education, and mental health systems.
The introduction of AI in systems like the NDIS promises efficiency, but without being designed to understand real human needs, it can amplify confusion and create exclusion. This article discusses the ethical considerations and design requirements to ensure AI truly serves individuals, not just systems.
Start here
Big idea: When AI is introduced into complex systems without being designed for real people, it amplifies confusion and removes control over meaning.
A familiar moment
Imagine you're trying to understand what support you can access. The language is dense, and the process is unclear. You turn to AI for assistance, and it provides an answer that sounds certain. Yet, despite its apparent confidence, you're still unsure.
What is happening?
The system itself is complex, and AI simplifies it. However, AI doesn’t have your context. It cannot understand your unique situation, your history, or your specific communication needs. As a result, while the answer is structured, it doesn’t fully fit or address your individual circumstances.
Why this matters in the NDIS
The NDIS operates within a framework of layered rules, multiple decision points, and high stakes. People engaging with the NDIS are often under pressure, managing multiple needs, and navigating unclear information. Introducing AI into this environment doesn’t reduce this complexity; instead, it changes how individuals experience it, often making it harder to navigate.
Where AI breaks down
In these systems, AI falls short in several ways. It cannot fully interpret the lived context of the individual, nor can it assess consequences in the same way a human can. Instead, it relies on patterns and data, which may not accurately reflect individual realities. This gap between what is said and what is needed can lead to misinterpretation and misunderstanding.
What can go wrong?
Without proper design, AI can misinterpret needs, offer incomplete guidance, increase confusion, and shift responsibility onto the user. This is not a failure of AI’s intelligence but rather a failure of design and understanding of the user’s real needs. AI systems must be better equipped to handle the nuances of human context.
The risk of confident answers
AI outputs often appear certain, which can create a false sense of trust, even when the answers provided are incomplete or don’t apply. This can have serious consequences in systems like the NDIS, where individuals’ access to support, wellbeing, and long-term outcomes may be directly impacted by these seemingly authoritative yet incomplete answers.
What ethical AI requires
For AI to function effectively in systems like the NDIS, several key requirements must be met:
Accessibility
AI must provide information in plain language and in easy-to-read formats. It should offer multiple formats to ensure people understand the information being presented. Clear communication is crucial.
Context awareness
AI systems must reflect on the unique situations of individuals and clearly state their limits. The system should acknowledge what it doesn’t know and be transparent about it.
Transparency
AI must explain how answers are formulated and make its limitations visible to users. This transparency is essential so that people can question and understand the process.
Human oversight
AI should support decisions but not replace human judgment. Human review and shared responsibility are necessary to ensure that decisions made by AI are appropriate and aligned with individual needs.
Traceability
It’s important that AI systems track how information changes over time and preserve the original meaning. This allows people to see the process and understand how decisions are made.
Connecting to system design
Ethical AI is not a feature that can be added at the end of a system’s development; it needs to be built into the system from the start. Current research is focused on developing frameworks that track authorship, preserve meaning, support accessibility, and ensure that consent is visible throughout the process.
The real question
Rather than asking whether AI provides an answer, the real question is: Does the system understand the person using it? AI must be designed to serve individuals, not just process data.
Final thought
AI will inevitably shape how people interact with systems like the NDIS, but the key issue at the heart of this transformation is control over meaning. If systems are not designed with real human complexity in mind, AI will not simplify them; instead, it will intensify the challenges people face. If a system cannot adapt to the person using it, it is not innovation—it is exclusion scaled.
Read more from Sarah Ailish McLoughlin
Sarah Ailish McLoughlin, Neurodivergent and Disabled Founder
Sarah Ailish McLoughlin is the neurodivergent founder behind EduLinked and EduPsyched, and the creator of the Strategic Self-Advocacy™ framework. Her work transforms lived experience into trauma-informed, policy-smart tools that restore clarity and agency. Through digital apps, therapeutic messaging, and emotionally literate reform training, she helps carers, educators, and system-changemakers navigate complexity without self-erasure. Her Microsoft-backed NDIS Navigator app and emotional literacy campaigns are reshaping advocacy, access, and wellbeing across Australia.
