Aude - Performance Tracking Software · UX Design

by Marlon Needelman | Visit Aude.ai

A case study in using UX principals to iteratively redesign the dashboard with the goal of driving engagement and adoption.

Industry

SaaS - Software Development
Platform - Web Apps.

Role

Project Manager, Lead UX Designer and Prototyper, Contributor to Research and Testing

Team

Marlon Needelman
Denis Koltsov
Jenna Beavers
Izzy Pitceathly

Methods Used

Heuristics, Competitive & Comparative Feature Analysis, User Interviews, Affinity Map, User Persona, User Journey, User Flow, Site Map, Problem Statement, Feature Prioritization, Sketching & Design Studio, Prototyping & Iterating, Usability Testing

The Client

Aude (pronounced: aw-day) is a tool for tracking work performance in a holistic way so that engineers have neutral third-party insights, backed by factual evidence, of the work they accomplish. It captures more than just how many lines of code or bugfixes an engineer has delivered by also seeing interactions with colleagues through digital communication channels.

Executive Summary

The Challenge

Aude has been experiencing low engineer adoption of the tool so they tasked us with figuring out why and proposing solutions to help make the tool more useful, engaging and integral to an engineer’s workflow.

The Solution

Our team identified key issues such as difficulty with:

legibility,
unclear numbers and metrics,
and lack of visual polish.

We tested those issues with users to validate our assumptions and researched other approaches by competitors. We then took those lessons to design a lighter, more approachable interface and communication strategy for the web app.
Some examples of which can be seen below:

It’s nice to see a bit of what the final result looks like, but how did we know where to even begin? The next section reviews some of our assumptions

Hypothesis

First impressions and personal understanding of the problem

We started by evaluating the site ourselves to understand the product better and get an initial sense of what could be going wrong. As a team, we shared our thoughts and opinions to hone in on a starting hypothesis. We boiled it down to three starting pain points. (Click an image to expand)

1. The site contains too much content presented in an overwhelming fashion.

2. Where metrics are used, it’s not exactly clear what they mean.

3. Privacy and settings associated with it don’t seem to be communicated anywhere except for one “Make Private” button.

From that, we hypothesized that the platform struggles to retain attention among ICs (individual contributors) because content feels at once overwhelming and over-simplified and there isn’t a clear understanding of who can see data collected about them. A solution must address these three concerns.
Before we could do that, we needed to make sure we understood the company and it’s competitors better.

Business Analysis

Deep dive into Aude.ai

Before validating our hypothesis, we sought to fully understand the state of the web app, so we conducted a Heuristic Analysis of the company using the Abby Covert Method for Information Architecture (IA) Heuristics. This consists of evaluating the site and its key pages on criteria covering clarity, accessibility, ease of use, error correction and delightfulness to name a few.

Click image to xpand

We noticed the most issues with the Home and My Performance pages, mainly for a lack of clarity and readability. Other pages struggled in similar ways as legibility issues were a recurring theme throughout the site. That said, the site didn’t feel overly error-prone as interactivity is fairly minimal.

Understanding their competitors

We also compared the service to it’s nearest competitors and a few comparators. This helped us better understand Aude’s positioning in the market. The matrices below help us understand, visually, where they fit.

Proactive guidance vs Reactive reporting

How platforms balance insight timing and delivery context.

X-axis Support: Reactive (reporting Past work) → Proactive (Assisting Next Steps)

Y-axis: Support Type: Data Dashboards → Generative AI/Contextual Support

Workflow integration vs Actionability

How seamless the integration with other tools is and the kind of insights that are provided.

X-axis Actionability: Passive → Actionable

Y-axis: Workflow Integration: Not Integrated → Integrated (Slack, Github, Trello, etc.)

Value to organization member

Who does the tool benefit most in terms of value based on the types of insights provided.

X-axis Org Member: Engineer (or IC), Team Leader or Manager, C-Suite Executive

Y-axis: Amount of value gained from the tool: Low to High

Validating the Hypothesis

Not only were our assumed pain points present among the majority of users but we identified several additional pain points, a few are shared below as direct quotes:

“I would expect a bad manager to use this tool to screw me over.”

“If it takes me more time to check that the reported reasoning is accurate than it does to find evidence of my work performance myself, I don’t need this.”

“I won’t be motivated to use this unless it benefits me financially or my career.”

User Research and Defining the Problem

Understanding the Trove of Data

After interviewing 6 people to get their input on tracking tools and the existing site, we gathered and refined our data to highlight key pain points and define an ideal user to represent our focus.

Grouping Quotes into Categories

Our approach to this was to collaboratively examine the data from the interviews and bring them together in an affinity map to highlight key groups.
Below are some selected categories that came out of the exercise, presented from the perspective of the interviewee:

I am lost in the navigation
I am unclear what data the graphs are linked to
I want performance tracking data to include both qualitative and quantitative context

I am concerned by who sees my data
I feel micromanaged by intrusive tools
I like being able to understand the metrics behind the information provided

I am motivated by pay/promotions
I want feedback that is merit-based
I want more links in the UI to make information more accessible.

Creating an Ideal User

From the identified pain points and categories surfaced from the previous exercise, we created an ideal user, often referred to as a User Persona. We do this to have a user in mind in all of our future design decisions.

Zavier perfectly captures the essence of a software engineer that strives to improve and get recognized for his contributions. He is further explored with the following goals, needs and frustrations.

Goals

Needs

Frustrations

Achieve more in less time at work.

Be recognized for his efforts across all his contributions.

Get Promotions or raises regularly.

Performance processes that minimize effort and maximize value.

Needs a simple way to gather evidence of his contributions.

Clear expectations and pathways to earn raises and promotions.

Extraneous work can take more time than it’s worth.

Important context and qualitative impact overlooked.

Reviews don’t seem to be tied to the full scope of his performance.

The Focus

To further refine our understand of our target user we explored the experience he might have using Aude as a new tool within his company over a 14 week period. The journey, pictured below, surfaced key opportunities that we could tackle during the design process.

Given all the information we’d gathered and how we narrowed our understanding of it, we derived a single sentence to live at the core of our design decisions:

Zavier needs a transparent, efficient and customizable tool for measuring his work performance so that he can secure regular raises and stay ahead of his manager’s expectations.

Prototype

It’s time to try it out for yourself!

Sometimes, rather than reading, exploring is the best way to understand what was achieved.

The prototype features a redesigned Home, My Performance and Profile Pages as well as some informational inclusions on the Performance Category and Feedback Pages.

Prototype Link

Design

How did we get here?

Whether you’ve explored the prototype or not, it’s important to understand how we got there. Our process after completing research was to narrow in our focus on achievable and transformative changes.

Navigating the interface, but it’s just words

The following sketch is an exercise called breadboarding. It’s an ultra low-fidelity exercise to visualize what we will make and where it fits into the existing product. It helps us very quickly outline the layout we felt Zavier would need to take through the Aude platform. It also helped us quickly figure out what information would need to be available to the Zavier.

Deciding what key features to pursue and further
visualization of how a user would use them

We followed that up with a Feature Prioritization exercise to identify what features we felt could be present and more importantly, what should be present for a Minimum Viable Product (MVP). Then we developed a couple User Flows that encapsulated the three key features we decided our prototype had to have: Revamped dashboard, Improved Performance page and Action Item selection, and understandable and controllable Privacy settings.

Bringing our heads together

Following that exploration, we organized and led a group sketch brainstorm called a design studio. We included the client representative, a product designer, in the studio to ideate with us and validate our chosen scope of the project. We sketched out ideas that we felt would solve the pain points we were focused on. We then converted those sketches into our first low fidelity wireframes.

(Home, My Performance and Privacy Panel low fidelity frames, click to expand)

Initially we designed lo-fi frames for more pages, but started with mid fidelity frames for the three mentioned before; Homepage, My Performance and Profile Page (Privacy Panel). In a meeting with the CEO and our contact at the company, we decided to further prioritize those three pages over other ideas and keep the project scope narrow and focused. After an initial round of testing we developed the high fidelity prototype in the Prototype section.

Testing the design

What use is a prototype if no one understands it?

Testing of the prototype took place over the course of two rounds and refinements of the prototype we included in the conversion of the mid fidelity prototype to it’s high fidelity. The results we got were a little surprising:

-28%

(86% to 54%)

Avg. Rating of Privacy tools

+18%

(72% to 90%)

Avg. Rating of Action Items

-13%

(76% to 63%)

Avg. chance to recommend to a friend or colleague.

It seems like the prototype got worse between the first and second round of usability testing. But we got a little unlucky. The sample sizes are small, 5 and 4 people in each round respectively, and the second round users simply had less general trust in tools like Aude and were more skeptical of anything to do with AI. This dragged our Privacy and recommendation scores down, but actually showed that the functions of the tool itself were well received, given the huge jump in satisfaction in to our Action Items implementation.

We also had improvements across the board for completion times of tasks. The Privacy section contained three tasks. The Homepage contained one task and the My performance page contained three tasks.

-1 sec

(38 to 37 seconds)

Privacy Tasks

-8 sec

(15 to 7 seconds)

Homepage Task

-15 sec

(37 to 23 seconds)

My Performance Tasks

Delivery and Next Steps

But there’s more that can be done...

After showcasing the entire project to the client with a presentation and delivering the prototype, a detailed report and all associated files we had a detailed conversation about the amount of data we collected but could not act on in this sprint and made several recommendations on other areas that we felt would improve first impressions and adoption of the tool. Below are just some of the ideas that we recommended to the client:

Improved onboarding process, including repeatable tutorial of the various functions of the tool and where to access them.

New users need to fundamentally understand what Aude is good for TO THEM on their very first encounter with it.

Better messaging on their marketing materials and website to emphasize their bottom up approach to improving productivity.

The client wants engineers to be front and center in the ethos of the tool being helpful and empowering. Their messaging needs to reflect that, and balance it with selling how it’s approach to improving productivity is better than their competitors.

Create a Career Progression page that has metrics of performance tied to key incentives for ICs, a tool that would foster a sense of transparency around work and expectations.

Time and again users told us that they care about how tools like this can help them earn pay raises and promotions. This is an opportunity for a business to objectively tie performance goals and career progression together.

Implement the Feedback Hub, a place for peer-to-peer and conversations between managers and their ICs to be saved and reviewable. Users indicated they would especially appreciate documented feedback on specific projects in a space such as this.
Instead of using in-line or superscript citations for connecting to evidence behind reasoning, use icons that represent the app which the evidence pertains to as a way to visually emphasize the place or places an insight was informed by.

Users like the OPTION to know the evidence for what they read on the platform. It helps build trust in what the Aude AI is doing. Making it hidden but clearly accessible through iconic citation representation could be the ideal way to have the best of both..

Conclusion

In closing, Aude.ai reached out to us for insights into why their tool struggles to gain traction among ICs. They got way more than they bargained for as we delivered a massive trove of insights, several impactful but easy to implement solutions, as well as a general roadmap on further improvements that can be made over time.