Lora, Voice Assistant for the University of Pittsburgh Medical Center (UPMC)
Voice User Interface for University of Pittsburgh Medical Center
Project Overview
We designed Lora, an intelligent voice assistant that personalizes and expedites the hospital experience at the University of Pittsburgh Medical Center (UPMC). We identified opportunities where voice assistants could intervene to provide personalized, seamless & assistive experience, specifically for frequent local users, non-tech savvies, and international students.
Challenge & Goal
Hospitals are complex environments that present numerous information and unexpected variables. This surge of information often overwhelms patients and visitors, posing a challenge in delivering clear, effective communication and a seamless healthcare experience. Therefore, UPMC needed a conversational voice assistant that provides more humanized, efficient, and personalized hospital experience.
Timeline
Sep - Oct 2022
Interaction Design Studio I
Team
Kristen Cai
Jason Wang
Semina Yi
Andrew Sim
Role
User Research
Concept Development
Interface Design Conversation Design
Prototyping
Tools
Figma
Adobe CC
Miro
Process
The Outcome
Lora is an intelligent voice assistant for UPMC that provides a personalized, seamless, and assistive patient experience from home to the hospital through multimodal interactions.
Why Lora can do
Booking Appointment
Check-in
Navigation
Courtesey Services
Patient Support Services
Test Result Interpretation
Why UPMC?
After exploring businesses with mobile components and kiosks, we chose UPMC as our client. Using a hospital is considered a mundane and often cumbersome task, but there is an opportunity to make it efficient and engaging. Further, because of the persistent COVID pandemic during the project, there was an inherent push to reduce human contact, thus the increasing need for self-service kiosks.
Research
1. Exploratory Research
After finalizing a client, we conducted immersive research at UPMC to get to know the client better and absorb its values to later translate into our assistant’s design. A key takeaway from the visit was that it is easy to get lost in both the hospital campus and buildings.
Fly-on-the-wall observations
Fly-on-the-wall observations
Complicated hospital complex
Complicated hospital complex
Connect front and back of house to keep orders flowing.
Outdated language interpretation system
1-1. User Interview
We thought we would get better responses if people were already in their visitor shoes; hence, we conducted our interviews at UPMC. We drafted an interview protocol↗︎ to guide the informal conversation during the interview.
Our primary research was based on interviews with:
  • 1 physician
  • 1 front desk staff
  • 5 Generation Z visitors
  • 5 Millenial visitors
1-2. Research Synthesis
From the field research and on-site interviews, we could envision possible frustrations from visiting the hospital and the visitors' pain points from the UPMC hospital experiences. We clustered the frustrations and pain points into an affinity diagram.
1-3. Paint Point Mapping
We identified the patients' hospital visit experience from pre-visit, in-hospital, and post-visit as the most salient and significant opportunity space, with crucial leverage points creating ripple effects throughout the UPMC experience. By offloading menial, clerical tasks from staff and providing patients with a dedicated source of transparent feedback and reliable support, our VUI will free providers and staff to focus on expertise-dependent responsibilities, easing points of frustration and inefficiency.
2. Current UPMC User Experience Audit
The resources from the UPMC have a lot of potential but are pretty bare in their functionality right now. So, we audited the current user experience of UPMC.
1) Mobile Pain Points
  • MyUPMC app functions as a directory to the mobile version of their website, where users have to jump between the app and the web browser.
  • The lack of consistency in user experience bothers users in navigating desired information.  
2) Kiosk Pain Points
  • The kiosk system at UPMC was under-utilized and limited by its in-attentive software.
  • Inconsistent screen interface designs require users to think more about how to use the system than what they use it for.
Our main takeaways from the research include:
Based on our user research result, I have gained insight as following.
Unconsolidated experience
Mobile and kiosk services are not supporting one another in the current user journey. Visitors are unaware of the existence of a self-service kiosk.
Difficulty in information access
UPMC campus is complex and lacks consistent signage. Many visitors are unaware of the patient support and courtesy services. Medical terms are difficult to understand and interpret.
Lack of user friendliness and accessibility
Certain demographic prefer traditional phone calls for booking hospital appointment despite the digital scheduling system.
Design Opportunities
We found opportunities for designing a conversational UI based on mapped pain points.
Integrated System
Create a digital system that integrates dispersed ordering and receiving experiences
Personalized Suggestions
Expedite order entry process to minimize error and miscommunication between front-of-house staff and kitchen personnel
Humanized Interaction
Let the chefs keep orders synced with a direct link between front and back of house
Concept Development
1. User Scenarios
With an understanding of the scenarios where Lora could really help hospital visitors, we began storyboarding situations that helped us understand the needs and nuances for which we needed to design. Since the pain points appear from different stages within a hospital experience, we composed the user journey into three phases: Pre-Visit, In-Hospital, and Post-Visit.

‍Focusing specifically on an appointment, check-in, patient service request, navigation, and test result within hospital experiences, we developed Semina and Karen’s combined scenarios to create a user journey map.
Persona
Storyboard
2. User Flow
Based on the third design opportunity, we refined the UPMC experience integrated with a voice user interface.
Scope of Lora
We created a site map to establish the scope of Lora’s feature. Therefore, we were able to identify that users mainly use hospitals and application in three ways.
Lora User Experience
Based on the user research result, I created two user flows to compare the restaurant’s manual ordering system with a digital ordering system, highlighting the improvements offered by digitization.
1. Pre-visit
  • MyUPMC app functions as a directory to the mobile version of their website, where users have to jump between the app and the web browser.
  • The lack of consistency in user experience bothers users in navigating desired information.  
    ‍
2. In-Hospital: Check-in
  • The users can simply tap their mobile phone on the UPMC kiosk.
  • Connected through Lora’s ecosystem, the hospital kiosk automatically receives and confirms the user’s information, proceeding to whether the user wants to add the hospital’s services.
    ‍
2. In-Hospital: Navigation
  • Lora eliminates confusion and helps users find their intended locations
  • Visitors can confidently explore the UPMC campus with Lora's assistance and effortlessly reach their destination
    ‍
3. Post-Visit
  • The users receive test result notifications before the doctor's notes, improving comprehension and preparation for follow-up discussions.
  • Lora simplifies complex medical jargon in records, providing clear explanations.
  • Lora also offers language interpretation services to ensure patients can access information in their preferred language.
3. User Journey Map
We characterized the conversation type between our users and Lora through journey mapping and defining necessary touchpoints within this multimodal interaction.
Define Voice Assistant
1. Existing Voice Assistant Study
We studied other existing VAs to learn how they used movements, transitions, and colors to communicate in different cases.
Implement a system to broaden access points
Create a digital system that integrates dispersed ordering and receiving experiences
Automate order reception process
Expedite order entry process to minimize error and miscommunication between front-of-house staff and kitchen personnel
Connect front and back of house to keep orders flowing.
Let the chefs keep orders synced with a direct link between front and back of house
2. Defining Voice Assistant Characteristics
Since hospital experience embraces unexpected situations and strict rules, we wanted Lora to be reliable, neutral, and informed. To create this personality, we set four keywords that incorporate Lora.
Reliable
Trustworthy and satisfying patient experiences
Caring
Proactive and sympathetic assistant
Empathetic
Smart yet friendly and approachable personality
Calming
Empowering patients to feel more comfortable
3. Form & Motion Ideation
After a few iterations, we decided to keep one element — a cross of blue and purple gradients. We wanted to make Lora playful but not childish and modern but not too serious. So, we excluded other iterations.
We created motions for each state in conversation. Based on UI wireframes, we defined eight conditions and then made transitioning motions between states.
4. Voice Assitant States Matrix
Through transition animations between states, we wanted to further strengthen Lora’s identity by designing the animations to be more human-like. To convey personable and approachable characteristics while maintaining expertise, we positioned each motion state as active and personable but not as too playful.
Visual User Interface
1. Wireframe
2. Finalized UI Structure
We decided to show user dictation at the bottom of the screen and Lora’s response at the top because
  • Integrating user dictation allows users to verify if Lora correctly understands their commands.
  • Integrating user dictation allows users to verify if Lora correctly understands their commands.
Kiosk UI
To help patients have a better check-in experience in the hospital, we introduced a kiosk as our secondary screen to optimize the user flow. Acting as a beacon support or a location pin for special services, Lora’s kiosk is installed in multiple locations within the UPMC campus. Instead of calling the office and needing help communicating where the patients are, they can request the service directly at the kiosk and wait near it before the help arrives.
3. Visual Systems
High-Fidelity Prototypes
Next Steps
Enhancing the accessibility of for different users
We recognize that LORA can be tailored for users with different conditions to enhance accessibility for diverse use cases.
Designing other screens
We have only considered mobile phones and kiosks for this project's scope. Implementing other devices like smartwatches for patients could improve a more personalized and mobile in-hospital experience.
Designing voice interactions in public spaces
We should consider factors like location and privacy when designing appropriate voice interactions that are helpful but not invasive.
Reflections
For this project, we had to assume that the VUI experience existed in a context without data privacy concerns. To successfully enhance the UPMC experience and provide personalized care for each patient and visitor through a voice assistant, the user will have to input their personal health information, which raises a serious concern in today’s technology landscape with constant data breaches and privacy concerns.

Also, not everyone feels comfortable talking to the VUI in public. For example, talking to Alexa, Google, or Siri in a private setting is common, but speaking to a voice secretary in public is rare. So while we aimed to limit voice interaction for in-hospital scenarios, we should be mindful of creating more opportunities for chat user interfaces in public settings.