How AI Changes the User Experience of UX Designers

By Alexa Polidora, MFA (Interactive Design)

 

Machine Learning (ML) and AI (artificial intelligence) are likely to radically change (UX) design, particularly the design process. In addition, these changes are transforming the role of UX designers. What happens when these changes affect the experiences of the people handling user experiences?

UX Design Without AI

First, let’s establish a “baseline” of how UX design functioned before AI. User Experience (UX) design encompasses all aspects of the end user’s interaction with the company, its services, and its products (Norman & Nielsen, 1998; Salazar, 2023). What sorts of activities do UX designers involve themselves with? A major focus of UX design or experience design is understanding people, their thinking, behaviors, goals, and needs, among other things, According to UX designer Nick Babich (associated with Adobe), UX design typically involves the following activities:

  • Product research and user research
  • Creating personas and scenarios
  • Information architecture
  • Creating wireframes
  • Prototyping
  • Product testing (Rousi, 2023)

Granted, as Rousi (2023) notes, these activities may change “depending on the product and service offering, business model and organization of the business” (p. 2). However, the above activities are (and were) pretty standard for the average UX designer.

A Quick Overview of the UX Design Process

Like any process, UX design follows a set of standard protocols. The Double Diamond (see Figure 1) is a common design process framework often used in UX design, with the major stages of Discover, Define, Develop, and Deliver (Design Council, 2024).

Designers first conduct research to understand the problem and its scope during the “Discover” phase (Design Council, 2024). For instance, UX designers might interview people to gain greater understanding of the problem, the people impacted, and the context in which the problem occurs. As shown in Figure 1, design researchers engage in divergent thinking and associated activities in attempts to understand the problem. They conduct field research including observations, contextual inquiry or interviews, and usability testing as well as secondary research (e.g., literature review, data analytics, competitor analyses) to better understand the problem, people, and context.

Figure 1. Double Diamond Frame (Design Council, 2024). This work by the Design Council is licensed under a CC BY 4.0 license

In the Define stage, design researchers organize, analyze, and interpret the collected data. They engage in convergent thinking to make sense of the information collected and narrow down and define the problem scope. Ideally, by this stage, designers aim to have a clear idea about the problem space and a good understanding of the target audience and the context.

Next comes the Develop stage, which is the initial step in determining a solution to the problem identified in the first two phases, and during this stage, ideation is important (Design Council, 2024; Schrock, 2022). Designers again engage in divergent thinking as they ideate or come up with many possible or divergent solutions to the problem. They evaluate possible solutions and refine them iteratively. During the Develop stage, design teams might generate ideas through diverse activities such as team brainstorming, sketching, prototyping (rapid prototyping), usability testing design scenarios, and user journey mapping.

Finally, during the Deliver stage, prototypes continue to be evaluated and improved. Designers engage in convergent thinking as they narrow down possible design solutions. As designers advance in this stage, they often refine prototypes, so they are of high-fidelity that look and function like a finished product. They deliver a close to final design solution but continue to evaluate it in the field through user feedback and testing.

AI Use in UX

Having established what UX involves without AI, let’s now delve into what UX may be like with AI. We begin by examining how AI tools influence the UX design process. Then, we look at how UX designers talk about AI tools. Finally, we’ll examine some pros and cons of AI use in UX.

The UX Design Process with AI

Bouschery et al. (2023) propose a Double Diamond design model integrated with AI focused on AI and ML methods that can greatly expand knowledge-gathering capabilities during design.

Regarding ML methods, Bouschery et al. (2023) write

ML algorithms perform well on a variety of pattern recognition tasks that are relevant for knowledge extraction. These can range from detecting patterns in visual data, for example, for quality control or analyzing technical samples of an experiment, to identifying novel ideas in online communities or customers with lead user characteristics (p. 143).

Transformer models like GPT-3 are the next evolutionary step after ML methods. These models are helpful because,

Their flexibility and generative capabilities provide ample opportunity for different knowledge extraction practices, allowing NPD [new product development] teams to apply one model for a large variety of tasks. Their context awareness plays a critical role in understanding important connections within a given text and extracting relevant information and knowledge (Bouschery et al., 2023, p. 143).

Text summarization, sentiment analysis, and customer insight generation are three examples of transformer models that can benefit UX professionals by reducing “knowledge-extraction efforts” and exposure to information that a person might otherwise not have known (Bouschery et al., 2023). Additionally, these models can help increase understanding of customer needs and, ultimately, allow design teams to translate insights obtained in the Discover and Define phases to tangible ideas for design solutions (Bouschery et al., 2023).

Because of their few-shot learning capabilities, they can generate adequate responses to a given problem statement and come up with original and useful ideas when prompted with just a few examples of what typical brainstorming results look like. In addition, users of such models can precisely tune them to produce more creative (radical) or more deterministic (incremental) responses–an ability rarely imaginable for humans (Bouschery et al., 2023, p. 145).

Downsides to transformer-based language models include questionable accuracy, insufficient elaboration with some ideas generated by models, and models are only as good as the knowledge on which they were trained, as noted by Bouschery et al. (2023, p. 147).

An important limitation to consider here is that the original training data for language models has a cut-off point after which new knowledge is no longer contained in the training set used for unsupervised learning. Hence, critical information might not be included in the model’s knowledge base. Users have to be conscious of this aspect when interacting with a model. Generally, this natural data cut-off point calls for a continued re-training of models in use. While this aspect might not be critical for applications such as lyric composition or the writing of novels, it is especially relevant in the innovation sphere as innovators should base their decisions on the newest knowledge available. This aspect is even more pronounced in research fields where this knowledge stock expands rapidly. At the same time, the re-training of these models is very easy. These models can very effortlessly acquire new knowledge that can then be incorporated into innovation processes, as long as the information is available in a machine-readable form.

Transformer-based language models can also be prone to biases such as stereotype biases, confirmation biases, and cultural biases because the data on which models are trained have biases (Bouschery et al., 2023; Lawton, 2023).

How UX Designers Discuss AI

Next, let’s better understand how UX designers discuss AI. Feng et al. (2023) examined “… how UXPs [UX practitioners] communicate AI concepts when given hands-on experience training and experimenting with AI models” (p. 2263) and found that UX Designers tend to struggle when describing AI. Designers often lack sufficient understanding of AI capabilities and limitations. Feng et al. (2023) note,

. . .prior work has shown that UXPs encounter numerous novel challenges when designing with AI that emerge from issues including understanding AI models’ capabilities and limitations, calibrating user trust, mitigating potentially harmful model outputs, a lack of model explainability, and unfamiliarity with data science concepts (p. 2263).

Researchers have proposed different solutions to these UX designer difficulties discussing AI such as “human-AI guidelines to offer both cognitive scaffolding and educational materials when designers work with AI” (Feng et al., 2023, p. 2264). Additionally, “researchers have also proposed tools that combine UI prototyping with AI model exploration, process models and boundary representations for human-centered AI teams, and metaphors for generative probing of models” (Feng et al., 2023, p. 2264). Allowing UX designers to create their own AI models helped designers better understand AI systems and bridge the gap between AI experts and UX Designers.

But, Productivity!

Despite such difficulties discussing AI, UX Designers can benefit from AI . . .right? Productivity is great . . .isn’t it? Well, it depends on who you ask. Gonçalves and Oliveira (2023) begin their paper by exploring the positive benefits of AI. First, they view AI as another new form of human creativity:

. . .the evolution of Artificial Intelligence has revolutionized design workflows, offering new possibilities and targeted approaches to computational tools, which play a crucial role in various stages of User Experience (UX) and User Interface (UI) projects. These stages range from automated content generation to advanced data analysis and market insights, enhancing creative and production processes, as well as interaction with the audience through chatbots and virtual assistants (Gonçalves and Oliveira, 2023, p. 2).

Are these developments positive or negative? The answer depends. To help answer that question, Gonçalves and Oliveira (2023) make use of “the advantages, challenges, and potential drawbacks of computational algorithms, as referenced by Madhugiri” (p. 3).

Here are the pros:

  • High accuracy and reduction of human error.
  • Allows you to automate repetitive tasks in different industries.
  • Efficient Big Data Processing.
  • Fast decision making.
  • Improved interaction with customers.
  • Discovery of trends and patterns.
  • Organizes the management of processes and workflows (Gonçalves and Oliveira, 2023, p. 3).

Conversely, here are the negatives:

  • Over-reliance on machines, diminishing human abilities and autonomy.
  • Need for investment in infrastructure and training, making the application of AI more expensive.
  • Data privacy and security concerns.
  • Creative limitations in challenging situations that require innovative thinking.
  • Lack of emotional understanding.
  • Misleading conclusions due to bias in data interpretation and limitations in the models.
  • Lack of flexibility or adaptability of systems (Gonçalves and Oliveira, 2023, p. 3).

“Computational algorithms” have certainly had a positive effect on UX and UI Design, but using AI in UX/UI design brings ethical considerations in addition to usefulness (Gonçalves and Oliveira, 2023, p. 3). For instance, the authors examine how AI’s impact on UX/UI design has necessitated the need for tools such as ChatGPT and Midjourney that optimize workflows and the necessity of understanding how they affect “the evolution and adaptation of professionals in the face of these transformations in creative processes” (Gonçalves and Oliveira, 2023, p. 4).

Where Do We Go From Here?

Gonçalves and Oliveira (2023, p. 6) point out,

…the integration of computational algorithms in UX/UI design is seen as a revolution in user experience, providing improvements in usability, efficiency, and satisfaction through dynamic personalization, predictive and automated interactions, as well as advanced features. However, this technological transition raises questions about the impact on creative processes and the activities of designers.

So much effort is spent on understanding how AI will increase productivity, profit margins, and user satisfaction. However, how do we acknowledge and utilize the usefulness of AI systems while honoring the humanity of UX designers?

AI Systems as Coworkers, Not Replacements

For starters, AI doesn’t have to exile humans from the design process. Rather, AI systems should be used as “assistants” rather than as replacements for humans in the UX/UI design process.

To help establish a positive, working union between UX/UI designers and AI systems, Gonçalves and Oliveira (2023) suggest a few actions.

  1. Join Human-Computer Interaction (HCI) research with Knowledge Discovery/Data Mining (KDD) for Machine Learning (ML), although many UX/UI designers are not experienced with such ML tools
  2. UX/UI designers should familiarize themselves with the responsible use of AI. The authors recommend that designers review documents such as the “Google AI Principles” and the “Beijing AI Principles” [links added for reader benefit].

Gonçalves and Oliveira (2023) note the more optimistic views of designers like Sam Anderson (Intuit) and Andrew Hogan (Figma) who “consider that the involvement of intelligent systems in UX/UI tasks will not replace the role of human designers but rather complement it” (Gonçalves and Oliveira, 2023, p. 7). Additionally, they point out that the creation of new fields due to AI’s inclusion in the design process such as Human-Centered Machine Learning (HCML) and Design for Machine Learning User Experiences (DLUX) that engender involvement of intelligent systems with human designers.

Acknowledging the Digital “Other”

As humans explore the use of AI in UX Design, we will inevitably be forced to address some uncomfortable issues. For instance, in their paper, Sciannamè and Spallazo (2023) aptly describe the implementation of AI in design as “a move from the paradigm of embodiment to alterity” (p. 2). Such an understanding truly changes the nature of human interaction with AI. Along such lines, they add, “AI-infused artifacts may be understood as counterparts, as suggested in Hassenzahl and colleagues’ definition of these products as otherware” (p. 2). They call for the establishment of “new interaction paradigms” so that humans “… see interactive items as other entities rather than users’ extensions” (p. 2). They even go so far as to write

AI-enhanced systems can be proactive and visibly demonstrate their agency to end users by learning, reflecting, and conversing. These systems go beyond delegated agency (Kaptelinin & Nardi, 2009). They can disappoint users, act independently, or – even better – select the ideal solution to the issue at hand (Sciannamè and Spallazo, 2023, p. 2).

If AI systems have agency and are more than merely tools, then such a realization changes human-AI interaction entirely. Yet, is this idea truly the reality of things or simply more of the “theater of the absurd” that Rousi (2023) describes? While this article will not attempt to answer such questions (such philosophical and ethical considerations deserve a paper unto themselves), the fact remains that questions of agency, intelligence, and consciousness are certainly applicable and relevant to how, why, and where AI is used–even beyond such tools’ implementation in the UX design process. The time is likely fast approaching when we, as humans, will be forced to address AI’s nature and ethical implications and what such considerations mean for the nature of consciousness and agency. Eventually, we will face this digital “Other”–whether or not we wish to do so.

 

Bibliography

Babich, N. (2017) “What does a UX designer actually do?”. https://theblog.adobe.com/what-does-a-ux-designeractually-do/

Beijing Artificial Intelligence Principles. International Research Center for AI Ethics and Governance. (n.d.). https://ai-ethics-and-governance.institute/beijing-artificial-intelligence-principles/

Bouschery, S. G., Blazevic, V., & Piller, F. T. (2023). Augmenting human innovation teams with artificial intelligence: Exploring transformer-based language models. Journal of Product Innovation Management, 139–153. https://onlinelibrary.wiley.com/doi/epdf/10.1111/jpim.12656

Design Council. (2024). The framework is fundamental to our work. Framework for Innovation. https://www.designcouncil.org.uk/our-resources/framework-for-innovation/

Feng, K. J. K., Coppock, M. J., & McDonald, D. W. (2023). Designing Interactive Systems 2023. In How Do UX Practitioners Communicate AI as a Design Material? Artifacts, Conceptions, and Propositions (pp. 2263–2280). Retrieved February 15, 2024, from https://dl.acm.org/doi/abs/10.1145/3563657.3596101.

Gonçalves, M., & Oliveira, A. G. N. A. (2023). IX International Symposium on Innovation and Technology. In Blucher.com. Retrieved February 14, 2024, from https://pdf.blucher.com.br/engineeringproceedings/siintec2023/305955.pdf.

Google. (n.d.). Google AI Principles. https://ai.google/responsibility/principles/

Hassenzahl, M., Borchers, J., Boll, S., Pütten, A. R. der, & Wulf, V. (2020). Otherware: How to best interact with autonomous systems. Interactions, 28(1), 54–57. https://doi.org/10.1145/3436942

Hassenzahl, M., Eckoldt, K., Diefenbach, S., Laschke, M., Len, E., & Kim, J. (2013). Designing Moments of Meaning and Pleasure. Experience Design and Happiness. International Journal of Dsign, 7(3), 21–31.

Humble, J. (7AD). What is the Double Diamond Design Process?. The Fountain Institute. https://www.thefountaininstitute.com/blog/what-is-the-double-diamond-design-process

Kaptelinin, V., & Nardi, B. A. (2009). Acting with Technology. Activity Theory and Interaction Design. MIT Press

Lawton, G. (2023, December 5). Transformer neural networks are shaking up AI. Tech Target. https://www.techtarget.com/searchenterpriseai/feature/Transformer-neural-networks-are-shaking-up-AI

Merritt, R. (2022, March 25). What Is a Transformer Model? Nvidia Blogs. February 19, 2024, https://blogs.nvidia.com/blog/what-is-a-transformer-model/

Rousi, R. (2023). Nordic network for research on communicative product design (Nordcode) seminar 2019. In Arxiv. Retrieved February 14, 2024, from https://arxiv.org/abs/2304.10878.

Norman, D. and Nielsen, J. (1998). The Definition of User Experience (Ux). Nielsen Norman Group. Retrieved June 1, 2023, from https://www.nngroup.com/articles/definition-user-experience/

Salazar, K. (2023). User Experience vs. Customer Experience: What’s The Difference? Nielsen Norman Group Retrieved Feb, 2024, from https://www.nngroup.com/articles/ux-vs-cx/

Schrock, D. (2022, February 22). A step-by-step guide for conducting better product discovery. Productboard. https://www.productboard.com/blog/step-by-step-framework-for-better-product-discovery/

Sciannamè, M., & Spallazo, D. (2023). International Association of Societies of Design Research Congress 2023. Design Research Society. Retrieved February 15, 2024, from https://dl.designresearchsociety.org/cgi/viewcontent.cgi?article=1127&context=iasdr.