Building the Senses for the Machine – Interview with Jelle Pieter van der Tas, Founder and CEO of AIVAS

Jelle Pieter van der Tas is the founder and CEO of AIVAS, a Singapore-based AI infrastructure company building the interaction layer that enables machines to see, hear, and understand humans in real environments. Noisy food courts, busy airports, multilingual hospitals. Having moved from the Netherlands to pursue his vision in APAC, he is on a mission to solve what he calls AI’s three fundamental blind spots: environmental failure, emotional blindness, and identity fracture.

In this interview, Jelle Pieter shares why today’s AI falls apart the moment it leaves the demo room, how AIVAS’s AI Interaction Model (AIM) gives any AI the social skills it’s missing, and what it takes to build technology that truly adapts to humans rather than forcing humans to adapt to it.

Jelle Pieter van der Tas, Founder, AIVAS, Conversational AI Solutions

Can you share the story behind how discovering generative AI inspired you to leave your corporate career and start AIVAS?

I always had this problem where I’d come up with a new business idea every week, but never had the resources to actually build it. When vibe coding was born, I suddenly had the ability to make the visions I had a reality without needing a team of devs. So I started using my spare time to build and didn’t stop till I had something I felt proud showing. I started asking for feedback and it came to a point where I thought to myself, why wait? I had made a promise to myself that year: “I would live a year without fear.” So I made the decision to quit my job and move to the other side of the world. No safety net, no promises, just me, my laptop and a good amount of discipline.

I started out in Vietnam, worked in the middle of the rice fields of Hoi An and had a great time. Then, through a mutual acquaintance, I was invited to meet a potential investor in Singapore. Without any guarantee or prior conversations, I just packed my stuff and flew over for a meeting. That meeting changed my life. They liked what I’d built and decided to invest in me. Together, we founded AIVAS.

What sets AIVAS’s AI Interaction Model (AIM) apart from other AI systems, especially in environments that are unpredictable or multilingual?

Currently, all the big players are focusing on making AI smarter. We are trying to make it more human by giving it the senses and social skills it lacks. The problem I had was that whenever I wanted to show someone a conversational AI solution that I’d built, I had to ask everyone to be quiet because it doesn’t understand who is talking and when. It just listens if it hears someone talk, and as soon as there is some silence, it thinks it needs to respond. This isn’t human at all and is exactly the problem we are solving.

Our AI can not only hear but also see you and understand who is talking and when. We do this by giving it eyes, ears and interaction intelligence. These three things combined allow our model to perform in any unpredictable environment. It can be loud and messy. The AI understands who is trying to speak with it by mimicking human behaviour. Is the person looking at me? Are their lips moving? Do I hear the person speak? Our AI knows when it’s being addressed, when it needs to respond, and when it needs to shut up. It can do this in any language, for any application and any environment.

If there is a human trying to interact with AI using their voice, there is a need for AIVAS’s AIM model. We are the layer that sits in front of LLMs, which makes us model agnostic. As LLMs get better, we get better.

How do you envision the future of AI evolving in physical spaces like hospitals or airports, where traditional systems struggle?

AI adds a lot of value in busy airports, hospitals, schools and other unpredictable, loud environments. But these are exactly the environments where the current stateless, emotionally blind, noise-sensitive AI fails.

The next five years will look like this: AI moves from being a new way to Google stuff to helpful, humanlike friends. You will build relationships with AI that come eerily close to the ones you have with humans. Imagine you’re walking around a big shopping mall looking for a good restaurant. You see a robot and walk up to it. It sees you approach, understands you want to ask it something and greets you: “Hey there, can I help you?” You say: “Yeah, I’m looking for a good restaurant.” The robot replies: “Anything you’re craving?” You respond: “I could go for some good pizza.” The robot says, “Mamma Mia has some great pizza, want me to walk you there?” You agree, and together with your newly made friend, you walk over to the restaurant. But it doesn’t stop there. While walking, it asks your name and where you’re from. You have a little chat, and after arriving at the restaurant, you say goodbye.

Now, the following week, you happen to be in a different mall to check out Apple’s latest MacBook. You see another robot, and to your surprise, when you walk up to it, it greets you by name. It even asks how the pizza was. Confused, you ask how it knows. The robot explains that the AIM model it runs on can remember and recognise faces. No matter where you are or what product you interact with, as long as it runs AIM, it remembers you.

AI is often critiqued for being too rigid in understanding human emotions. How does AIVAS integrate emotional awareness to create more authentic interactions?

We train our model on hundreds of thousands of interactions. The current models focus on the voice sounding human, but not on actually being human. Because we train our model on how we talk and not on how we sound, it becomes much more natural and empathetic. Just like us, our AIM model uses all vocal and visual cues to understand not just what you are saying but how you are saying it.

This makes it able to understand if you are being sarcastic or serious, whether you’re saying something in a sad way or an angry one. Our AI has learned all the small nuances that we humans have evolved to understand. This allows it to be much more emotionally aware than the current models.

In what ways does AIVAS address the challenges of working in diverse, multilingual environments across APAC, especially with languages like Singlish and Hinglish?

APAC is the hardest place in the world to build voice AI, and that’s exactly why we built here. In a single food court in Singapore, a customer might order in Singlish, switch to Mandarin for the aunty behind the counter, and check with their kid in English. All in one sentence.

Most global AI is trained on clean, mostly Western data. It breaks the moment you introduce real Singlish, Hinglish, or any of the hundreds of dialects that actually live across this region. Our approach has two parts. First, because we are model agnostic, we can support 100+ languages, including the ones that rarely get first-class treatment. Second, and this is the part that compounds: every kiosk we deploy feeds real-world, noisy, multilingual data back into the model. Every frustrated customer, every aunty, every rushed order makes AIM better.

By the time a Western competitor realises APAC matters, we’ll have years of training data from environments they never even attempted.

As someone who’s moved from the Netherlands to Singapore to pursue your entrepreneurial vision, how has that international experience shaped AIVAS’s approach to AI solutions?

Being an outsider makes you see things differently. This can work for you and against you. But in my experience, even though there are small nuances everywhere in habits and customs, humans tend to be the same everywhere. This is also baked into AIVAS. “What makes us human is not defined by language, it’s how we interact with one another.”

This is why we focus on interactions instead of language. We believe that the way we communicate is much more than the words we use. That’s why we focus on making our AI both emotionally and culturally aware. An AI that communicates in a Western way might feel very out of place when you put it in an Asian food court. Our AI adapts.

You’ve mentioned that privacy is a right, not a luxury. How does AIVAS ensure privacy while still delivering seamless, human-like AI interactions?

In a world where privacy can no longer be guaranteed and companies are hard to trust, AIVAS wants to be transparent. We don’t store audio or video. We translate everything into numbers and train our models on a combination of metrics that is completely anonymous. But in our vision, we do want AI to recognise you for your own convenience. This, of course, needs your consent as a user. So as long as you haven’t agreed to us creating a personalised profile, we won’t store any data.

What personal growth habits do you rely on to stay grounded, and how have they influenced your leadership at AIVAS?

My most important habit is hitting the gym. Even though it’s hard to stay consistent when your life is anything but consistent, it’s the thing that will always keep me grounded, gets rid of my cortisol and keeps me healthy. It also motivates me to keep my diet in check, and whenever I do let go for a bit, I don’t have to feel guilty.

Besides the gym, continuous learning and staying curious are two of the most important things to improve yourself each day. I tend to take long walks and listen to audiobooks, read and watch a lot of YouTube. This keeps the mind sharp and open to new ideas.

Finally, journaling before bed is an amazing habit, and having a clear set of values and rules you live by keeps you stable and sound whenever you have to make decisions or deal with difficult situations. You have to program your computer, a.k.a. your brain, so that it can follow a script and you don’t have to overthink everything. That allows for living a life with a lot less stress.

Looking ahead, what’s the one takeaway you’d like to share with entrepreneurs or tech leaders about building AI solutions that truly make a difference in people’s lives?

The most important thing is asking yourself why. Why are you doing what you’re doing? If you had to do it for the coming 20 years, would you still do it? Then there are all the other W questions. Who are you solving it for? What does it solve? When do they want it? Where do they want it? Then, when you have an answer to why, who, what, when, and where you’re building, and you’ve decided you still want to do it, the most important thing is to focus.

I think one of the most common mistakes of entrepreneurs is not focusing on the right thing. But most importantly, sometimes you get a client who wants something you could offer, but it’s not what you’re trying to build. Too many founders pivot because they want to make money and leave their vision behind. “Sometimes you must do the hard thing and say no.” Keep your focus on the original vision if you believe it’s the right one.

For me, this has worked. For others, the pivot is what saved them. But then you’ve got to ask yourself: was the original idea even sound? I think as an entrepreneur, you have to have the natural gift of knowing when and how to change strategy and when to stay the course. But if you don’t believe your vision, no one else will.

Follow me on Instagram and LinkedIn for more info!

Read more from Jelle Pieter van der Tas