Project Overview

From the dawn of time, humankind has searched for the perfect way to entertain themselves by traveling. There are technologies to help tourists have a better experience when visiting a new place. However, when the topic comes to the historic heritage, a museum or an ancient palace, it would be more difficult to engage the tourists. 

As a way to address this issue, we proposed a dedicated device to help visitors better understand indoor & outdoor artworks in museums. The final design is composed of AR glasses with a conversational chatbot that serves as a tourist guide to introduce the artworks.

Team

Hyun Tae Park

Yujin Xue

Project Duration

Jan 2020 - Apr 2020

Tool

Sketch, Figma, Protopie, Framer

AR Glasses Display

- UX / UI Design, UX Research, Interactive Prototyping

Augmented Reality museum tour guide for engaging experience

Contributed to user research, ideation, UX workflow development, 

UI design, interactive prototyping, and evaluation

My Role - UX / UI Designer, UX Researcher

Design Challenge

" How might we help tourists get more engaged and better understand artworks in museum? "

AR assisted tour guide Toury

1. Research: Understanding Users & Context

We conducted multiple research methodologies to better understand how museum visitors experience various artworks in museums and what are their pain points. At the beginning of the project, we conducted desktop research to get a general understanding of museum visiting habits. After that, we performed Competitive Analysis to not only understand how current museums are giving engaging experience but also to figure out their technology usage. In addition, we wanted to be informed of our survey questions and direction for further research steps. We then conducted user surveys to understand more about the demographics of visitors as well as quantify their visiting patterns. To get more in-depth qualitative data and figure out visitors' pain points and needs, we conducted semi-structured user interviews as the last research activity. 

Desktop Research / Literature Review

User Surveys

Competitive Analysis

Semi-Structured Interviews

3

Key Findings

By grouping the research results, we were able to spot notable points that would guide us to move on to the next steps.

1

Flexibility in information consumption

Visitors want to see the information they are interested in instead of learning all the information about the exhibit.

4

Hard to locate a specific artwork

If the size of a museum is so huge, then it’s really difficult for visitors to locate the specific artwork they want to see even though they have a guide map.

2

Distracted by handholding devices in museum

Visitors don’t want to be distracted from reality by other visual input from electronic devices while visiting.

5

Unwilling to rent traditional dedicated devices

Visitors are not willing to rent existing dedicated devices because most of them are outdated and uncomfortable to use.

3

6

Insufficient Information in outdoor exhibit

Non-interactive exhibits make the experience boring

There are many artworks that don’t have proper descriptions,  especially for outdoor sightseeing (historic places or outdoor parts of museums).

Visitors don’t feel engaged and get bored if they don’t have background knowledge about certain artworks and if the artworks are non-interactive.

Who is our target user?

From the key research findings and design objectives, we were able to set our main target user of this project. We decided to focus on museum lovers who are willing to rent dedicated devices to augment their experience during their visit. Specifying the target user group helped us put the research findings into the context of the user and direct our focus of this project. 

Travelers who love to visit museums and historic places and willing to rent a dedicated device for a better experience.

" I want to get more engaging and delightful experience during my visit! "

Design Objectives

Visitors cannot get information about the artworks that they are interested in.

It’s difficult for travelers to plan & find the route in the sightseeing spot.

Visitors don’t feel engaged due to the lack of interaction with artworks.

2. Design Ideation: Exploring Design Ideas

AR Glasses

Mobile

Dedicated Device

Public Display

Divergent Design Ideas

After selecting the target device platform, we began brainstorming design ideas based on the research findings while keeping the project goals in mind. The initial focus was on developing new interaction patterns of AR glasses with support of additional controller. We came up with a number of ideas that have different interaction patterns each. And then we moved on to user feedback session to understand our concepts in user's perspective.

Idea 01 AR glasses controls on it with VUI

Idea 02 AR glasses controlled by ring

Idea 03 AR glasses controlled by hand gesture

Idea 04 AR hologram with dedicated device

Feedback Session: Initial Design Ideas

In order to choose the direction that we would pursue in our design, it was important to look at them through an external eye. We hence conducted feedback sessions with potential users who are museum lovers at this stage. We wanted to get their opinion on not only interaction patterns but also overall ideas about leveraging AR experience and voice user interface. The sessions were informal feedback sessions. We briefly walked them through our project focus and research findings, and asked their opinion on each idea. After the sessions, we organized the results into the following three categories:

01 Device Preference

Most of the participants preferred AR glasses over hand-holding devices because, first, they don’t want to be distracted from the actual artworks or sightseeing spots to get more engaging experience. Also, they didn’t want to hold an additional device all the time since tourists are usually carrying stuff like guide books, bags, cameras, and for parents who have children they might have a stroller or have to hold their children’s hands. However, one trade-off is that tourists who already wear glasses will not be able to use the device properly. 

02 Leveraging AR Experience

They thought AR would greatly improve the traveling experience in connecting people with correct information timely and navigating in a sightseeing spot. However, they had a concern that if the interaction method of the AR device, it would be a bit difficult for first-time users to use it properly. In the further step of our design process, we need to carefully strategy the interaction methods for AR glasses that can control the AR interface elements. 

Participants had positive thoughts on incorporating the VUI aspect into the solution. They thought if the VUI could be more conversational and give more proper information, then it would dramatically enhance user experience. However, they also gave us feedback on some limitations that VUI might have depending on the environment. In quiet places such as art museums, users might not be able to have a conversation with VUI. On the other hand, in outdoor places such as historic sightseeing spots, users can actively ask their questions or seek more information via conversation. Considering this difference, we would need to organize the user flow of VUI strategically so that users can use the function properly in different spatial circumstances.

03 Voice User Interface

Overall, they liked the idea of AR assisted sightseeing experiences. Among all the alternatives, idea1 - AR glasses controls on it was the most popular one. because the idea is the most intuitive way to use the device and have fewer trade-off as compare to other ideas.

Physical Interaction of AR device

After finalizing our direction, we moved on to develop the interaction pattern and controls for AR glasses. We tried to keep the interaction as simple as possible because users will not be able to understand the controls if it's too complicated. In addition, based on the developed interaction patterns, we realized that we should limit the depth of AR user flow screens as maximum 3 depths to keep the physical interaction simple.

Notification / VUI

Tap

Swipe

Press

Device Selection

At the beginning of the ideation phase, we first looked up various devices that are either currently used in the museum or can be used to give a more engaging experience and address user needs. We found there are 4 different types of delivery methods. Based on our research, visitors usually don't want to be distracted by hand-holding devices like mobile or traditional dedicated devices. Also, visitors think public displays are more tailored for children so they don't use it that much. In addition, one of the constraints of this project was developing a design for emerging display formats. Thus, we decided to use AR glasses as a way to deliver more delightful user experience in museums.   

Final Concept

With these considerations for the physical interaction and all the previous research steps, we were able to finalize the design concept and features that are most important to give visitors more engaging and delightful museum visiting experience.

3. Design Prototype: Iterative Prototyping

After coming up with core functionalities, we started building the initial low-fidelity wireframes to visualize the user flow. While exploring the designs, we continuously had usability testing sessions with professors, peers, and people outside of the project to get better insights and figure out if there are things that we missed out on. Based on their feedback, we kept iterating the wireframes to have more simple and intuitive usability. 

After quick iterative low-fidelity wireframing, we discussed the significance and relevance of each iteration and shortlisted notable points from the low-fidelity wireframes. Based on the list, we further improved design ideas and details that need to be addressed in the subsequent step. We then created mid-fidelity prototypes to better visualize key functionalities of our design and to conduct usability evaluation sessions with potential target users.

With the mid-fidelity prototype, we conducted design evaluation sessions with 6 museum visitors. Due to constraints regarding the public health emergency, we couldn't test the prototypes in an in-person setting which is important for us because our design has physical interaction with AR glasses. To make up for these limitations, we conducted the testings in a remote setting and utilized the wizard-of-oz testing method with clickable prototypes to test our design in a similar setting to in-lab testing. We used cognitive walkthrough methods with 5 task flows and asked them to think aloud their thoughts while going through the design. Our goal was to receive feedback on a variety of topics that include the overall functionality, the user’s expectations, the ease of use, the error frequency, the user’s satisfaction, and the level of connectedness of each touchpoint.

6 Participants

From the collected data, we highlighted key findings from our evaluation sessions, and discussed next steps for design iteration. The findings were grouped based on related features and ranked by severity of the problem. Some of the changes that we made after the evaluations include:

1

Visitors feel difficult to memorize all the gestures to control the device

Before

After

Issues

Participants had a hard time remembering all the gestures given during the tutorial. Also, the instruction for their action on the bottom of the screen was not well match with the actual direction of the AR glasses touchpad.

Solutions

To solve this issue, we decided to have fewer tutorial steps and show the instruction box on the bottom all the time when users need to take actions. In addition, we changed the wordings so that those fit well into users' context. 

2

Visitors may be surrounded by so many famous artworks and keeps getting a notification

Before

After

Issues

During participants were using the notification feature, we found that they might be surrounded by so many artworks and keep getting notifications which could be annoying for users. Also, we needed to inform users that they can choose not to see the contents.

Solutions

As a solution to this problem, we decided to give the notification based on user's previous viewing history. Also, we added the instruction about choosing no to see the contents.

3

Once visitors are done with their visit, they may just take the glasses off

Before

After

Issues

As a way to show viewing history and rewards to visitors, we had End Visit tab on the menu which makes them see the history only after they are done with their visit. However, in reality, if they want to end their visit, they may just take the glasses off rather than using End Visit feature. In that case, there is no point of having this flow, and showing the history at the end of their visit also didn't make sense.

Solutions

To address these issues, we removed the entire End Visit flow and incorporated viewing history and reward feature into My Trip tap. This will help users use the features more simply and intuitively and also increase the chance to be exposed to users.

Visual Guideline

Icon

Color

Font

Design Principles

Minimalism

Intimacy

Clarity

4. Final Design

 

When a user first puts on the AR glasses, they will get a greeting message and will go over tutorial flow that explains to them how to use the physical interaction and features of the service.

On the menu, there are two different options: My Trip and Settings. On My Trip tap, users can check the summary of their visit and see badges they’ve collected during the visit. On Settings tap, users can control various options related to the display.

Users can locate a certain artwork by using the navigation feature. They can just simply ask VUI the location of artwork then VUI will direct them to the artwork by showing the navigational AR indicator and a map.

Users can get more engaging AR content simply by asking VUI. Then the service will go through the scanning process and show various contents that users might be interested in.

For the outdoor artworks, the process goes the same. Users can get more engaging AR content simply by asking VUI. Then the service will go through the scanning process and show various contents that users might be interested in.

If users are passing by a famous artwork that they might get interested in, then the VUI will notify the users and they can go through the contents of the artwork.

Internalize the pros and cons of remote user testing

Due to multiple constraints regarding COVID-19, we were only able to conduct remote testing sessions. This unique situation gave us the opportunity to understand the strengths and weaknesses of remote user testing. We were able to save our time and resources but at the same time had multiple limitations such as unable to test the physical interaction of AR glasses. After the situation gets better, we would like to conduct usability testing sessions in an in-person setting in order to gain more insights on contextual variables and the physical interaction side.

Address more issues and perform more rounds of design iteration

We identified problems in our design through a number of evaluation sessions. We would keep fixing more usability issues according to the findings for better user experience. Also, at this stage, due to time constraints, we were more focused on developing the broader scope of the design features than the micro details of our design. For future iterations, we will explore more into detail to improve our design.

Develop 3D-based AR prototypes 

It would have been best if we could develop the fully-functioning prototype with actual 3D AR design elements to get a more in-depth understanding of leveraging AR experience in a museum. As the next step, we would like to develop a fully-functioning 3D prototype in Unity so that we can grasp the usability and business scalability of our design better.

Thank You!

Designed with passion by Hyun Park © Copyright 2020