It is time to evolve AI tools beyond prompt-based interfaces and consider new mental models.
AI tools are changing our lives. New tools pop up almost every day. While many of us are primarily users, some will design them too. Those designers will determine how AI tools are used and implemented. Once again, Designers will have the chance to shape how new technology changes our lives.
A Shift in Interaction
As highlighted by the Nielsen Group, AI is ushering in a new era of user interface design. It is moving us away from traditional command-based interactions to intention-based ones. This means users no longer have to issue specific commands to achieve tasks. Instead, they can express their desired outcome, allowing the AI to handle the necessary steps.
Similar to the early days of the Internet or the advent of mobile phones, we’re entering uncharted territory. I have been designing digital experiences for nearly two decades. Every time, these moments required us as an industry to reevaluate our status quo. We must innovate once again. We must reimagine our mental models, patterns, and guidelines to ensure that AI tools feel as intuitive as websites or mobile apps.
Challenges with Chat-Based Models
Many leading AI tools, like Chat GPT, MS Copilot, and Midjourney, rely on a chat-based interface. These interfaces mirror a one-on-one conversation. Two people sit opposite each other, asking and answering questions. This works well for simple questions like searching for details on a term. In contrast, it falls short of achieving more sophisticated tasks. Even when speaking to someone, it is hard to describe what we want in a few words. We often need to have a longer conversation before we are aligned.
You might recognize this kind of complexity when using Midjourney. Unless you’ve studied prompt engineering, getting it to create an image that shows a specific subject in the right frame using the right style requires a lot of trial and error. Creating a video using Sora is probably doubling the challenge. The same goes for anything more sophisticated using Chat GPT. It could be a day-by-day vacation itinerary, website code, or even a prompt for other AI tools. It is difficult to be specific enough in just one query, especially for more deliberate outputs. We know that since the invention of the command line. And we have seen those challenges with Alexa, Google Assistant, and Siri again in recent years.
A collaborative model
If I compare this to a real-life situation, the setup usually changes. Instead of having a one-on-one conversation, we change the situation. We sit next to each other. We have a piece of paper, a whiteboard, or a screen in front of us. Something we can work on together. We look at multiple options simultaneously. Once we find a good direction, we refine the details, and everybody makes edits.
To unlock GenAI’s magic for the masses, we must change the mental model of generative AI tools. We need to create a UI that mirrors this real-life collaborative approach. To achieve this, here are four key considerations:
1. Providing a permanent canvas
As mentioned above, a shared permanent canvas would be one of the most significant changes. This canvas would be an area that can be updated and does not need to be re-rendered with every prompt. Users should be able to understand what has been changed to evaluate whether the update met the intention. Microsoft already positions Copilot using a similar model. The AI companion sits beside you, and the canvas (your Word or Excel document) is front and center.
2. Let the user compare multiple options
Temp-Mail
This point may be personal. I usually start my creative process broadly. I like to explore different directions — not only when designing screens but also for text outlines or diagrams. During this moment of exploration, I need to compare different directions to consider their pros and cons. The ability to do this directly in the tool would be a huge help. Midjourney already provides multiple options per request. However, those are currently temporary and more or less forgotten with the following prompt.
3. Give the option to add or edit manually
Often, it is easier to make some changes yourself than prompt the AI tool to make the update. The ability to manipulate the output directly enables users to be faster and more precise. Grammarly is a great example. While it can also generate copy from scratch, I mostly use it to improve my already-written text. After its updates, I can continue editing, which often leads to multiple rounds of back-and-forth.
4. Enable users to refine details
Especially if you have something specific in mind, generative AI gets tricky. There are always small details that are just different from what you meant. A re-render might fix the current issue but also could create new problems. You can adjust those afterward in other applications, but you might want to continue using the AI tool, which makes the workflow cumbersome. Especially in combination with the ability to add or edit, this would make many AI tools much more streamlined and efficient. Photoshops’ AI Photo Editor enables you to generate a new image for a select area. It helps to address specific parts of the composition and thereby, layer by layer, create the image you intended.
Summary
AI tools are getting ready to change how we interact with technology fundamentally. Like with other new technologies, designers are poised to shape these tools. AI tools have problems, but we have what it takes to improve them. I suggest changing the mental model from one-on-one chats to a collaborative canvas. This could remove many hurdles and empower users to be more precise and efficient. Let’s shake things up and create tools that let users and AI work side by side smoothly.