Beacon
Revamping the Collaborative Workspaces for Data Scientists
Project Overview
We revamped the Workspaces and Lab features in Beacon, LMI’s internal data science platform, to enhance user engagement with the Workspaces, clearly differentiate the individual features, and seamlessly integrate them into acollaborative project management context within an agile environment. Workspaces is a collaborative environment for data scientists and consultants handling sensitive data, and Lab is where data scientists can experiment, analyze data, and develop models.
Challenge & Goal
Two years of Beacon analytics revealed that the Workspaces' click-through rate (CTR) was 10% lower than other features. This posed difficulties in the efficient data collaboration process and project management for data scientists.
Our team aimed to increase user engagement with the Workspaces feature by enhancing its user experience and interface while ensuring consistency with existing functionality and preserving its core purpose.
Timeline
June - August 2022
(10 weeks)
Team
UX Design Lead
Product Manager
Product Design Intern
Role
User Research
Competitive Analysis
Data Analysis
Recruitment
Wireframes
Prototypes
Tools
Dovetail ↗Adobe XD
Figma
Miro
The Outcome
With the insights from the usability testings, we made extensive updates to our design for the MVP version of this platform.
Jira Integration
Integrated real-time visibility of Jira into project progress enables all team members to stay informed about the status of tasks and projects. This improves transparency and accountability, particularly in collaborative data management platforms.
Document Repository
Creating a document repository solves the pain point ofscattered documentation and lack of team centralization. This solution also prevents miscommunication, delays efficiency, and strays the user away from the source of truth.
Lab
I reimagined the Lab to differentiate its features from the Workspaces. I proposed two dashboards
  • A high-level overview of ongoing data experiments for better project management
  • Machine learning training and ML model deployment data to monitor experiment metrics, resource usage, and progress
Discover
1. Heuristic Evaluation
To begin our revamp process, we conducted a heuristic evaluation of the Workspaces feature. Our goal was to identify any usability issues and areas for improvement. During the evaluation, we discovered several critical problems with the interface design.
Match between system and the real world
Some icons in the sidebar may not be immediately recognizable to all users
Consistency and Standards
The inconsistency of the search bar and filter options could make it harder for users to locate the interface quickly
Visibility of System Status
It is not clear what the icons in the top right corner represent, and there is no indication of any system feedback or status
The purpose of Workspaces
To create a single collaborative environment as one central place where teams can store and share notebooks
To utilize the collaborative environment with partnering data storage called Lab, where teams can together view existing code, edit data sources, and track visualizations
We identified ...
Workspaces had been unused because of the similar functionality of Lab, which was meant to be integrated as a hands-on practical data science environment where users can import data and use Workspaces as a collaborative space.
2. Competitive Analysis
To gain a foundational understanding of the area we were going to investigate, my fellow intern and I analyzed the existing competitors operating in the domain. This helped us to become familiar with the domain by conducting analysis of current competitors.
Holistic Dashboard View
Beacon’s workspaces was not catering well to the data scientist in showing a holistic view of their on-going projects
Catalogue Hierarchy for On-Going Project
It does not provide catalogue hierarchy of data modeling, documents, and other necessary files in projects
Problem Statement
How might we increase user engagement with the Workspaces feature by improving its user experience and interface while maintaining consistency with its existing functionality and preserving its core purpose?
3. Qualitative Interview
Remote Qualitative Interview:
We conducted a total of 6 interviews with 6 data scientists over Zoom. We probed for their general behavior with Beacon and preferences and narrowed it down to their most recent experience in the Workspaces.
Interview Protocol
To get responses mostly close to the Beacon usage context from the data scientists, we first drafted an interview protocol to guide the informal conversation during the interview.To capture their moment-to-moment emotions and actions, we divided the questions into 5 phases as touchpoints of the Beacon Workspaces experience.
Define
1. Thematic Analysis
After user interviews, we imported interview records into Dovetail to begin deep diving into analysis and cleaning up data.
‍
We explored the multi-facets of the Beacon Workspaces experiences to search for emerging themes from the qualitative data we collected from the interviews.
2. Synthesis
Holistic Dashboard View
Beacon’s workspaces was not catering well to the data scientist in showing a holistic view of their on-going projects
Catalogue Hierarchy for On-Going Project
It does not provide catalogue hierarchy of data modeling, documents, and other necessary files in projects
Holistic Dashboard View
Beacon’s workspaces was not catering well to the data scientist in showing a holistic view of their on-going projects
Catalogue Hierarchy for On-Going Project
It does not provide catalogue hierarchy of data modeling, documents, and other necessary files in projects
3. Design Opportunities
Difficult project tracking
Some icons in the sidebar may not be immediately recognizable to all users
Detached collaboration
The inconsistency of the search bar and filter options could make it harder for users to locate the interface quickly
Unorganized repository
It is not clear what the icons in the top right corner represent, and there is no indication of any system feedback or status
4. Design Objectives
Differentiate individual features
Some icons in the sidebar may not be immediately recognizable to all users
Harmonize the existing features
The inconsistency of the search bar and filter options could make it harder for users to locate the interface quickly
Update the information structure
It is not clear what the icons in the top right corner represent, and there is no indication of any system feedback or status
Develop
1. Sitemap
I created a sitemap to show the structure of each features grouped under the home. This allowed me to set the scope of the revamp more easily. The visible tags are new designs I revamped and I updated UI design for the selected features.
2. Wireframes
Based on the user research result, I iterated 3 sets of wireframes to test whether the layout and functionality align with the users' needs.
1st Design - Workspaces
I added Jira button on the left side bar menu to provides users direct access to Jira within Beacon.
1st Design - Document Repository
I proposed to incorporate document repository into Workspaces, so that users can find necessary documents and folders while managing projects.Similar with the Lab, I first created card view of document repository.
1st Design - Lab
I first proposed Lab to be a central data hub to differentiate features of Workspaces and Lab and to connect to each other for fluent data science flow. I created card view in order to allow data scientists in quickly grasping complex data sets, helping them visually understanding information of data.
2nd Iteration - Workspaces
I integrated Jira dashboard to enhance the tracking project management through a holistic view. This would also avoid cross platform switching between Beacon and Jira.
2nd Iteration - Document Repository
I created list-view collaboration environment with new organizing tabs that arrange files. I then added hierarchy of the files by title, type, time of view, update, owner, and team members grouped under recently used, shared with me
2nd Iteration - Lab
While card view offers a bigger and aggregate representation, I created list view that  offers an consistent and organized display of multiple items sequentially. This also allows users to filter and sort items based on various attributes, helping users quickly find specific experiments, models, or datasets. Adding intuitive icons also allow users to recognize whether its data, modeling, or experiments.
3rd Iteration - Workspaces
For higher visibility, especially to make Jira dashbard notifiable, I have added saturated color palette in this iteration.
3rd Iteration - Document Repository
I improved contrast and readability of icons associated with different file types in document repository.
3rd Iteration - Lab
This screen of Lab emphasizes content, allowing users to focus on what data matters most.
2. Evaluative Research (Usability Testing + SEQ)
‍Before presenting the hi-fi prototypes, we conducted 30 minutes of usability testing with the same 6 data scientists from the qualitative interview sessions. While conducting evaluative research, we also wanted to answer the following open research questions on our lo-fi prototypes.

After each task, the Single Ease Question (SEQ) was used to assess system usability on a scale of 1-7.
Visibility
Do these design alternatives meet the data scientists’ needs? - what are the needs? 
Behaviors
Did we properly interpret how data scientists would use Beacon and Workspaces?
Motivations
Will the revamp benefit the data scientists enough to use Beacon and Workspaces features? What are the barriers to use?
Expectation
What do data scientists expect? And does the revamp deliver on their pain points and desires?
3. Hi-fidelity design
Workspace wth Jira (1st Design)
Users will first encounter real-time visibility of Jira dashboard. This allows them to access project progress that enables all team members to stay informed about the status of tasks and projects.
Workspace wth Jira (2nd Design)
For more intuitive and humanizing interaction with contents in Workspaces, I gave hierarchy on each icon by inputting lighter blue color palette and real person images of users respectively.
Lab (1st Design)
The first iteration of Lab focuses more on data presentation in a straightforward manner without the added context or usability enhancements.
Lab (1st Design)
Integration of data dashboard add a higher level of detail with metrics and trends that provide immediate insights into the status and performance of various aspects of the lab.
Document Repository  (1st Design)
The interface is straightforward and uncluttered, making it easy for users to navigate and understand. The information is presented in a clear and concise manner, with each column clearly labeled and easy to read
Document Repository (2nd Design)
The second iteration adds more modern and cohesive design with the inclusion of profile pictures, tabs, and detailed access rights, contributing to a streamlined user experience.
Usability Testing Outcome
We put redesigned features of Beacon to test through moderated user testing with 6 data scientists, followed by a short survey focused on evaluating the tool’s value to data scientists. For usability, a Single Ease Question (SEQ) was used.
The average SEQ score for each task was > 6.0
Users expressed appreciation for integrating Jira into Workspaces and liked the visibility of the Jira dashboard
The ability to view data and documents by the ‘List’ view made sorting and filtering necessary files easier in one central repository.
Some users wanted the ability to track ongoing data experiments in Lab
From the beginning, we faced the challenge of designing for a specialized group of users, namely the data scientists with specific technical needs and varying wants. While I enjoyed making design decisions with a targeted audience in mind, it also meant that I needed to be an expert in the domain. I learned from this project that good designs require solid understanding, and sound designers must become experts in the subject matter.
‍One key lesson I learned was: Designing should always consider the target audience first.
Reflection
1. Gauging prototype is necessary towards finishing a project
The initial goal of this project was to increase the user engagement of Workspaces by up to 30%. It would have been great if I could quantify the metric of our revamp through usability testing like A/B testing. However, due to the length of the internship, I could not conduct it. In a future project, it will be beneficial to get enough time to measure CTR after usability testing.
2. Considering existing capabilities already developed
In this internship, I designed for an established product, which required me to gain a holistic view of the product, business, and users. And I was able to think about how my designs could both build upon what already exists and have an impact on future designs of the product.
3. Always think who the users are
When I was ideating and designing features for dashboard and data visualization, my initial thought was having comprehensive data and being able to interactively manipulate the data was the best way to go. However, it turned out that I was not thinking about the data scientists who would be the actual users of the new features.