Freelancer Gold Winner

Freelancer Gold Winner

Freelancer Gold Winner

In UX, Interface & Navigation System

In UX, Interface & Navigation System

In UX, Interface & Navigation System

Student Winner

Student Winner

Student Winner

Apps & Platforms Award

Apps & Platforms Award

Apps & Platforms Award

Reimagining the Conversational User Interfaces of Smart Voice Assistants in the Era of Large Language Models

Reimagining the Conversational User Interfaces of Smart Voice Assistants in the Era of Large Language Models

CODI

CODI

CODI

Home Depot's reliable home helper

Home Depot's reliable home helper

I spearheaded the reimagining of Conversational User Interface (CUI) design concepts in the era of Large Language Models and applied these innovative principles to the UX design of Codi—a smart virtual assistant that helps DIYers diagnose household issues, recommend solutions, and generate step-by-step repair tutorials.

I spearheaded the reimagining of Conversational User Interface (CUI) design concepts in the era of Large Language Models and applied these innovative principles to the UX design of Codi—a smart virtual assistant that helps DIYers diagnose household issues, recommend solutions, and generate step-by-step repair tutorials.

Client

Client

Home Depot

Home Depot

Timeline

Timeline

8 weeks

Aug. – Oct. 2023

8 weeks

Aug. – Oct. 2023

Team

Team

Albert Zhang

Annie Heyward

Revati Naniwadekar

Riley Ren

Albert Zhang

Annie Heyward

Revati Naniwadekar

Riley Ren

Tool

Tool

Figma

After Effects

Illustrator

Premier Pro

Figma

After Effects

Illustrator

Premier Pro

New Era, New Challenge

In the past, interacting with voice assistants often left users frustrated.

These systems could only process simple, one-off commands—and even then, they frequently failed to respond correctly, often defaulting to web searches.

They lacked the ability to retain context, forcing users to repeat themselves over and over.

Complex situations or multi-step tasks were beyond their comprehension.

As a result, these voice assistants were unable to facilitate meaningful conversations.

True conversation is more than just issuing commands. It’s a dynamic, back-and-forth exchange that evolves and improves with context—much like talking to a close friend. However, traditional voice assistants fell short of this ideal.

True conversation is more than just issuing commands. It’s a dynamic, back-and-forth exchange that evolves and improves with context—much like talking to a close friend. However, traditional voice assistants fell short of this ideal.

True conversation is more than just issuing commands. It’s a dynamic, back-and-forth exchange that evolves and improves with context—much like talking to a close friend. However, traditional voice assistants fell short of this ideal.

With the advent of Large Language Models (LLMs) and advancements in AI, voice technology now has the potential to engage in genuine conversations. These systems can understand context, interpret complex scenarios, and assist with multi-step, nuanced tasks.

With the advent of Large Language Models (LLMs) and advancements in AI, voice technology now has the potential to engage in genuine conversations. These systems can understand context, interpret complex scenarios, and assist with multi-step, nuanced tasks.

With the advent of Large Language Models (LLMs) and advancements in AI, voice technology now has the potential to engage in genuine conversations. These systems can understand context, interpret complex scenarios, and assist with multi-step, nuanced tasks.

This breakthrough raises a critical question for designers:

Design
Challenge

Design
Challenge

How can we rethink the design principles of Conversational User Interfaces (CUIs) to leverage the capabilities of LLMs and create voice assistants that's effective and human-like?

How can we rethink the design principles of Conversational User Interfaces (CUIs) to leverage the capabilities of LLMs and create voice assistants that's effective and human-like?

How can we rethink the design principles of Conversational User Interfaces (CUIs) to leverage the capabilities of LLMs and create voice assistants that's effective and human-like?

To design a conversational experience, we applied three approaches.

Solution

Solution

Designing a conversational experience

Designing a conversational experience

Designing a conversational experience

Interactive Conversation

Interactive Conversation

Context-Aware Responses

Context-Aware Responses

All-in-one Conversation

All-in-one Conversation

In the Codi project, I integrated my three design principles for the next generation of Conversational User Interfaces (CUIs). Codi serves as a concrete example, showcasing how these ideas can be brought to life.

Design Principle I

Context-Aware Responses

Context-Aware Responses

Generate personalized responses

Generate personalized responses

Through users' expression, Codi can understand the user's skill level and situation, and then offer more relevant and supportive responses. This personalization is especially valuable for DIYers who may range from beginners needing extra guidance to experienced individuals looking for advanced tips.

Users’ expression

Users’ expression

Users’ expression

Skill level & Situation

Skill level & Situation

Skill level & Situation

Tailored responses

Tailored responses

Tailored responses

Design Principle II

All-in-One Conversation

All-in-One Conversation

Accomplish various tasks in a dialogue

Accomplish various tasks in a dialogue

All-in-one conversation simplifies the user's journey by providing essential tools within the conversation, reducing the need to switch between apps or search for information. For DIYers, Codi offers quicker access to various tools and information.

In Codi, there are two types of conversation cards: chat bubble cards and function cards.

Function cards are designed to enable users to accomplish tasks efficiently—all without leaving the conversation page—thereby ensuring a fluid and uninterrupted conversational flow.

In Codi, there are two types of conversation cards: chat bubble cards and function cards.

Function cards are designed to enable users to accomplish tasks efficiently—all without leaving the conversation page—thereby ensuring a fluid and uninterrupted conversational flow.

In Codi, there are two types of conversation cards: chat bubble cards and function cards.

Function cards are designed to enable users to accomplish tasks efficiently—all without leaving the conversation page—thereby ensuring a fluid and uninterrupted conversational flow.

Chat bubble cards

Chat bubble cards

Function Cards

Function Cards

Design Principle III

Dynamic Conversation Design

Dynamic Conversation Design

Offer more than a reactive experience

Offer more than a reactive experience

Interactive conversation design enables users to tailor the assistant's suggestions to their specific project needs, budgets, and preferences. This flexibility is key in DIY projects where requirements and plans may evolve over time. It also allows for real-time assistance during different stages of a project, enhancing the DIY experience.

When Codi provides a proposal or a series of instructions, users are not limited to merely passively accepting or rejecting it. Instead, users can actively direct Codi to modify the solution or add details to the instructions as needed, all through voice commands.

When Codi provides a proposal or a series of instructions, users are not limited to merely passively accepting or rejecting it. Instead, users can actively direct Codi to modify the solution or add details to the instructions as needed, all through voice commands.

When Codi provides a proposal or a series of instructions, users are not limited to merely passively accepting or rejecting it. Instead, users can actively direct Codi to modify the solution or add details to the instructions as needed, all through voice commands.

Meet Codi

By adhering to three design principles, we have crafted a conversational user interface that simplifies access to resources for our users. Codi, an AI virtual assistant delivered through a chat-based platform, enriches the Home Depot experience by providing support for troubleshooting, guiding customers through their purchasing decisions, and offering personalized tutorials for tackling home improvement projects.

By adhering to three design principles, we have crafted a conversational user interface that simplifies access to resources for our users. Codi, an AI virtual assistant delivered through a chat-based platform, enriches the Home Depot experience by providing support for troubleshooting, guiding customers through their purchasing decisions, and offering personalized tutorials for tackling home improvement projects.

How can Codi help?

Pre-shopping

Guiding Users to Diagnose Issues and Choose the Ideal Solutions

Photo recognition
& 3D scanning

Codi's photo recognition and 3D scanning features enable users to upload contextual information, allowing Codi to deliver more precise and accurate responses.

AI-powered troubleshooting and solution recommendation

Codi analyzes each user’s unique situation with the support of AI, identifies house problem and recommends multiples solutions for users to consider.

In-store

Seamless In-Store Navigation and Expert Assistance

AR navigation
& Export assistance

CODI simplifies users’ shopping experience. Codi enable users easily find products with accurate in-store navigation and get expert assistance from an associate whenever they need it.

Post-shopping

Streamlined DIY Project Management and Personalized Guidance

Codi assists users in managing DIY projects by enabling them to pause and save their progress at any point, and seamlessly resume their activities. Codi also creates detailed, step-by-step tutorials complete with time estimates to keep users on track. Additionally, Codi provides supplementary instructions as required by the user, catering to any specific needs that arise during the project.

Project overview

Step-by-step tutorials

Pause & resume a project

Personalized instructions

Design Process

Design Process

1.

Research and Analysis

Research and Analysis

Associate expertise and contractor network

World’s largest home improvement retailer

Rich DIY educational resources

Home Depot’s extensive resource ecosystem

Home Depot is known for its vast repository of DIY guides and manuals, in-store expertise, and network of external contractors. As the world’s largest home improvement retailer, we saw an opportunity to augment Home Depot’s existing resources by making them more easily available to customers through a smart virtual assistant.

What is the current home improvement experience with Home Depot?

Field research

4 user interviews

4 staff interviews

Existing App studies

Through on-site visits to Home Depot stores, interviews with staff and current customers, and evaluations of existing user task flows, we discovered that

Through on-site visits to Home Depot stores, interviews with staff and current customers, and evaluations of existing user task flows, we discovered that

Despite Home Depot's wealth of resources, aligning these effectively with user needs presents a significant challenge.

Despite Home Depot's wealth of resources, aligning these effectively with user needs presents a significant challenge.

Pain Point 1

Challenges in locating products and finding associates


The store is huge, and sometimes I get lost navigating the aisles.

It’s hard to find an associate. You may have to wait.

And often you’re transferred to another associate.


The store is huge, and sometimes I get lost navigating the aisles.

It’s hard to find an associate. You may have to wait.

And often you’re transferred to another associate.


The store is huge, and sometimes I get lost navigating the aisles.

It’s hard to find an associate. You may have to wait.

And often you’re transferred to another associate.

Pain Point 2

Diagnosis and solution uncertainty


Every time something goes wrong in the house, I struggle to pinpoint exactly what the problem is...

Even when I have a hunch about the issue, I'm often at a loss about how to best address it.


The store is huge, and sometimes I get lost navigating the aisles.

It’s hard to find an associate. You may have to wait.

And often you’re transferred to another associate.


The store is huge, and sometimes I get lost navigating the aisles.

It’s hard to find an associate. You may have to wait.

And often you’re transferred to another associate.

Pain Point 3

Project planning and product specification ambiguity


I have ideas in my head, but translating them into a feasible plan is tricky... I'm never sure which product is right so I end up overbuying materials


The store is huge, and sometimes I get lost navigating the aisles.

It’s hard to find an associate. You may have to wait.

And often you’re transferred to another associate.


The store is huge, and sometimes I get lost navigating the aisles.

It’s hard to find an associate. You may have to wait.

And often you’re transferred to another associate.

Pain Point 4

Desire for continuous assistance during projects


I was constantly second-guessing every step,

wondering if I was doing it right...

I really wish I had someone guiding me throughout...


The store is huge, and sometimes I get lost navigating the aisles.

It’s hard to find an associate. You may have to wait.

And often you’re transferred to another associate.


The store is huge, and sometimes I get lost navigating the aisles.

It’s hard to find an associate. You may have to wait.

And often you’re transferred to another associate.

How might we leverage Home Depot's extensive resources available to assist users with their household problems?

How might we leverage Home Depot's extensive resources available to assist users with their household problems?

Cui

Resources

User Needs

Locating products and finding associates

Diagnosis and solution

Project planning and product specification

Assistance during projects

DIY educational resources

Expertise in-store and external contractors

World’s largest home improvement retailer

2.

Ideation and Design

Ideation and Design

Mapping the potential of a Conversational UI to fulfill user needs

Mapping the potential of a Conversational UI to fulfill user needs

In response to identified customer pain points, we have developed a comprehensive journey map that highlights strategic opportunities for conversational UI (CUI) features and modalities to enhance the customer experience across pre-shopping, shopping, and post-shopping phases.

The key strategies emerging from this analysis are:

In response to identified customer pain points, we have developed a comprehensive journey map that highlights strategic opportunities for conversational UI (CUI) features and modalities to enhance the customer experience across pre-shopping, shopping, and post-shopping phases.

The key strategies emerging from this analysis are:

  • Pre-Shopping Guidance: Facilitating the identification of appropriate solutions tailored to users' unique challenges.

  • Shopping Experience Enhancement: Simplifying in-store shopping process to minimize confusion and decision fatigue.

  • Post-Shopping Support: Providing clear and actionable advice to ensure customers can effectively complete home improvement projects.

  • Pre-Shopping Guidance: Facilitating the identification of appropriate solutions tailored to users' unique challenges.

  • Shopping Experience Enhancement: Simplifying in-store shopping process to minimize confusion and decision fatigue.

  • Post-Shopping Support: Providing clear and actionable advice to ensure customers can effectively complete home improvement projects.

Illustrating User Scenarios with Storyboards

Illustrating User Scenarios with Storyboards

To gain deeper insights into how a conversational UI (CUI) can elevate the customer experience, we created three detailed storyboards showcasing how specific CUI features can address user needs and enhance engagement.

What's Codi like

Defining Codi

Designed around Home Depot's values of trust, honor, and integrity, our Conversational UI serves as a reliable DIY companion, characterized by four key personality traits.

We've named our conversational UI "Codi," a gender-neutral adaptation of the name Cody, signifying "Helper." This aligns with Home Depot's brand and positions Codi as a trusty and user-friendly assistance tool for users.

Approachable

Approachable

Knowledgeable

Reassuring

Empowering

Shaping Codi

Inspired by hex nuts, a common fixture in construction, we crafted Codi’s design after thorough brainstorming and sketching, and refined its appearance to reorient the visual focus, introducing a more modern and minimalist feel.

Animating Codi

To define Codi’s motion states, we utilized a matrix that spans from calm to exciting and from proactive to reactive. This approach allowed us to meticulously align each motion state with Codi's intended personality traits, ensuring a seamless and consistent expression of its character.

Proactive

Exciting

Calm

Reactive

Idle

Listening

Speaking

Notification

Successful

Wake up

Error

Proccessing

Renewing Home Depot’s Visual Identity

Renewing Home Depot’s Visual Identity

We designed Codi’s visual identity by renewing Home Depot’s existing branding, making it aligned seamlessly with Codi’s distinctive personality.

Home Depot's current visual identity

Home Depot's current visual identity

Logo

Typography

Helvetica Neue

primary

Arial

secondary

Colors

Signature

#F96302

Primary

Hunter Green

Secondary

#3E4826

Black

#000000

Tertiary

Cool Gray

Secondary

#868A8D

Dark Gray

#54565C

Secondary

White

#FFFFFF

Tertiary

Imagery and Visuals

Codi's visual identity

Codi's visual identity

Body 1

Regular, 20/28

Body 2

Regular, 18/24

Body 3

Regular, 16/20

Body 4 - Tabs, tips

Regular, 14/16

Headline 1

Bold, 32/40

Headline 2

Bold, 24/32

Headline 3

Bold, 18/24

Headline 4

Bold, 16/20

Black

#

1F1F1F

White

#

F9F9F9

Grey 1

#

C1C1C1

Grey 2

#

A8A8A8

Grey 3

#

747474

Grey 4

#

1F1F1F

Typography- Helvetica Neue

Colors

Iconography

Components

Codi Variation

Primary Orange

#

FF771D

Gradient orange

#

-

Accent

#

76B438

Warning

#

FF3B30

Add

2

+

-

Okay

Okay

Call an associate

Created by Kiki Rizky

from the Noun Project

Created by - Artist

from the Noun Project

Start navigation

Created by Robert Won

from the Noun Project

Confirmation

Package status

Idle

Working

Prototyping Codi's UI

Prototyping Codi's UI

For enhanced usability across different scenarios, CODI's UI is designed for mobile and tablet, offering portability and a larger-screen alternative that facilitates more accessible construction guides while on-site.

Anatomy of UI design

Hey Codi, the pipe under my kitchen sink is leaking. What should I do?

Okay, I can help with that. Could you please take a photo of the pipe and show me where it’s leaking?

9:41

QR Scan

3D Scan

Camera

1

Codi

Home

Shop

Cart

Account

9:41

Mon Jun 6

100%

Hey Codi, I just bought everything I need for painting. Can you walk me through the steps to paint a wall?

No problem. Based on the condition of your wall, here are the steps you'll need to follow for painting.

Paint a wall

Working time | 6.5 hrs

Total time | 2 days

day

hours

3

min

10

Do you want to start now?

Yes

Painting a wall

Mix and load paint

Resume journey

Cancel

2

Hey Codi, I want to take a lunch break. Should I wash my roller?

No need to wash it, just wrap your brushes and rollers tightly in plastic wrap or a plastic bag to keep the air out. This prevents the paint from drying.

Hey Codi, I think I have some unfinished projects.

Yes. You have two unfinished projects. Do you want to resume one now?

Yes. I am ready to finish painting

Okay, I’ll do that.

6

2

3

7

4

5

1

Conversation bubbles

Tool box with QR scan, 3D scan
and cameras

2

Codi

3

Keyboard

4

Collapsible navigation panel

5

History of conversation and projects

6

Speech to text conversion

7

3.

Presentation and Next Steps

Presentation and Next Steps

To bring the concept to life, we created a real-life scenario and made a concept video for presentation, clearly showcasing Codi's features and benefits. The presentation yielded valuable feedback for our next steps.

Moving forward, we look forward to chances to refine and validate Codi’s features through comprehensive user testing with a diverse group of users, from DIY beginners to professionals. This will help us identify improvements and ensure CODI meets a wide range of needs, enhancing the user experience and redefining customer support for home improvement projects.