Q: How do you see the future of AI user experiences evolving beyond the current chat interface paradigm? What innovative applications of AI do you believe could redefine how users interact with technology?

It’s true that right now many people see AI experiences primarily through chatbots, but there are a lot of exciting developments beyond that. From my perspective as a developer, I expect we will see a continued improvement of the copilot experience. We’ll have super intellisense that will autocomplete and suggest code as we type. Although it’s still evolving and sometimes makes incorrect suggestions, its potential to improve as it gains more context about your entire project is significant.

At Artium, we believe in the value of pair programming, where two developers work together on the same codebase. An AI partner could be invaluable when no one else is available to pair with you. Imagine talking to an AI as you would with a human partner over Zoom, where it could act as either the Navigator or the Driver in the coding process. 

As the Navigator it will speak to you and analyze your code as you write it to better understand what you're building. It’ll stop you when you start to go down a rabbit hole or if you make a mistake while coding. It’ll keep you on track. 

As a Driver it will be in control of the code that you are writing. We will navigate it by talking to it and describing what step we think we should take next. 

Ideally this experience would be latency free and as accurate (or more so) than a real life pair programming session.

Q: What are the most significant technical hurdles in developing AI systems that can understand and reason with human language? How can engineers overcome these challenges to create seamless and intuitive user experiences?

If you asked me this a couple years ago I would say the machine learning part is the biggest hurdle. That’s all changed now with LLMs and all the tools available for both the developer trying to build the thing and on the user side. 

As for building AI systems into products for users, the biggest technical hurdle that I see is with the reliability of responses. We know LLMs are inherently non-deterministic, but that doesn’t mean we can’t get useful responses, it just means it takes some work. Whether it’s a chat interface, an AI programming assistant, or a backend agent, ensuring accurate and reliable outputs is crucial. At Artium, we use something called Continuous Alignment Testing (CAT). This involves automated testing that runs in our CI pipeline, checking the accuracy of AI responses to specific prompts. For example, if the AI is asked to list products related to entertainment, it shouldn’t return irrelevant items like cookware. These tests help us maintain and improve the quality of AI outputs by providing feedback on how changes in the code affect accuracy. You don’t want a product you build to start hurling insults or misinformation at customers. You need to be able to trust the kinds of responses tools we build will give.

Q: How do you ensure that the AI-driven systems you build are truly user-centric, and what are the key principles that guide this process?

This is where collaboration with our product management team is crucial. It starts with understanding user needs through interviews and experiments, which are then translated into requirements. These requirements flow through the entire system, from ideation to implementation, and all the way to deployment and user feedback. It’s a continuous process of refining and adapting based on both technical challenges and evolving user needs. A strong product team in collaboration with the development team is essential in ensuring that what we build resonates with users and meets their expectations.

Q: As AI continues to advance, how can engineers balance the desire for cutting-edge user experiences with the need for system reliability and user trust? What strategies do you recommend for maintaining this balance?

It’s all about trusting the fundamentals of good software engineering. While AI brings new possibilities, the old-school principles of software development still apply. We still practice pair programming because two heads really are better than one when it comes to writing code. Having a tight relationship between the Product, Design, and Engineering teams keeps churn lower and helps keep code quality high. These teams are agile, able to experiment, fail, and quickly fix issues. Incorporating quality and safety measures into this process ensures that we can deliver reliable software without unnecessary delays. As much as AI is changing things, these fundamentals are what’s going to let teams build new experiences users can trust and rely on.