API Performance Dashboard
In the context of API producers publishing their APIs on Postman, there was a need for a system to monitor consumer engagement and improve their APIs for greater adoption by consumers. I led the design of a consumer engagement and feedback system. I collaborated to design a monthly engagement summary emails and in-product dashboard tailored for API producers. The positive outcome of this effort was reflected in the key performance indicator (KPI) for adoption - the fork count of the producer's APIs increased by 15.2%.
DURATION
2 months
MY ROLE
Product Designer at Postman
METHODS
User Interviews, Data Scraping, Brainstorming, Impact - Feasibility Matrix, Wireframing, Gamification, High - Fidelity Prototyping
PROCESS / V2
Problem Statement
After conducting 1:1 meetings with key stakeholders and performing preliminary research, I refined the problem statement and reframed it using the extended Jobs to Be Done framework:
Goal
Solution
I led the design of the in-product dashboard (V2), evolving it from the monthly summary emails (V1). In both versions, our team provided users with relevant metrics and actionable suggestions to help them improve API quality.
V1 / Monthly Summary Emails
I designed an MVP featuring monthly summary emails for each team, sent to admins responsible for maintaining APIs. These emails provide key metrics and tailored recommendations to help improve API quality.
V2 / In-Product Dashboard
After launching V1, I led the design of the 0→1 in-product dashboard, achieving key outcomes based on user interviews with API producers. As my final project at Postman, I delivered high-fidelity designs and seamlessly handed them off to developers.
5 seconds walkthrough of the solution
Impact
We received positive feedback from the producers on how these summary emails are beneficial for them to understand and measure their consumer’s interaction with their APIs.
PROCESS OF V1
Research
The initial problem statement I received from the PM was vague and fuzzy. I organized 1:1 meetings with the Product Manager, Product Head, Design Manager, and fellow designers to better understand the problem we were addressing, why it needed to be solved, why the current design wasn't working, and what success would look like.
Analyzing current design
To better understand the issues in the current design, I analyzed recorded producer calls, developer relations, and data from social forums such as Reddit, Twitter, and Medium.
I conducted rapid affinity mapping on the collected data to identify key problems in the current system:
Research Questions
After conducting preliminary research, I developed the following research questions to guide me toward solving the problem:
What are quality APIs?
Every producer wants to build a quality API, but quality is a very ambiguous and subjective term.
What are relevant metrics for producers?
Producers need metrics to see whether their APIs are moving towards or away from Quality APIs.
How can we assist them in building quality APIs?
Producers need our assistance in making relevant improvements in their APIs.
Strategy to answer research questions
After brainstorming with engineers, developers, & our chief engineering architect, we crafted following idea:
Defining quality APIs
Understanding consumers' interactions with APIs will help me uncover their pain points. Addressing these pain points from the consumers' perspective will allow me to define quality APIs, answering the first research question.
I conducted contextual inquiries with six consumers to observe how they search for the right API for their use case. This approach helped me identify their key challenges and pain points throughout the process.
I decided to present my findings in the form of consumer journey with stages, actions & pain points:
Alleviating these pain points will help me to define quality APIs:
Suggestions for producers to improve APIs
I also conducted a feature mapping exercise within Postman, using insights from customer analysis to propose actionable suggestions for API producers, which will help answer the third research question.
Prioritize metrics
I facilitated a workshop with my Design Manager, Senior Product Designer, Product Managers, and Developers to prioritize the key metrics for our first launch. Together, we identified which metrics to focus on initially and which to save for the next phase.
Design Opportunities
So, after rationally comparing the 2 ideas as shown above, we decided to go with idea-2. I created the high-fidelity design. I also created the mobile design of the emails.
Visualizing interactions
Visualizing change in engagement and functional metrics over time.
More customization
Giving users control on the way to see their metrics.
Assisting Producers
Providing suggestions coupled with metrics to assist producers in enhancing their API quality.
Ideation
We aimed to start small and decided to introduce it in the form of 'Monthly Summary Publisher Emails.' This approach will enable us to test the idea and iteratively enhance it over time. Consequently, I began creating various sketches to address the challenges within this problem space
Final Design
After conducting usability testing, I analyzed all the data and did the affinity mapping to highlight the following issues in the key flows:
PROCESS OF V2
Research
After launching V1, we conducted user interviews with API producers to understand their pain points, needs, and their current interactions with the provided metrics. This helped me in defining measurable and desirable outcomes.
Iterations
I defined five key outcomes and presented the corresponding designs below. While iterating, one particular outcome stood out as both intriguing and challenging, which I will highlight in this section.
Outcome #4: Investigating Failed Requests
During user interviews, two key use cases emerged: one with high request frequency but moderate failure rates and another with low request frequency but exceptionally high failure rates. Users noted that their prioritization would depend on the specific use case.
This insight led to a key outcome: Enabling producers to easily identify both scenarios, enabling them to strategically prioritize investigations based on their needs.
After reflecting on these ideas and gathering feedback, I realized that separating both frequency and failure rate into two distinct widgets would better achieve the desired outcome
Final Design
This is the final design of the in-product dashboard, with 5 outcomes mapped to each component.
OUTCOME 1
Comparing of APIs
Enable API producers to assess their collections and workspaces using key metrics, quickly identifying which are performing well and which need improvement, with an intuitive way to explore each in detail.
OUTCOME 2
Tracking API Engagement
Help producers gauge consumer interest and API comprehensibility by analyzing views and forks.
OUTCOME 3
Conversion to a Successful Request
Make it easy for producers to visualize how failure and success rates fluctuate over time.
OUTCOME 4
Benchmarking Against APIs in the Same Domain
Offer a clear way for producers to compare their API quality against the top 10% of APIs within the same domain.
OUTCOME 5
Investigating Failed Requests
Identify authentication types with high usage and moderate failure rates, as well as those with low usage but high failure rates, allowing producers to prioritize investigations based on their needs.
Reflections
During my time as a Product Designer at Postman, I took on a hybrid role, stepping into user research responsibilities in the absence of a dedicated researcher. Before designing a solution to help API producers measure and improve API quality, I realized that the first step was to define what constitutes a “quality API.” This project challenged me to take a strategic approach to answering this fundamental question.
I learned the value of collaborating with diverse stakeholders to address research questions. Identifying the right experts and leveraging their insights was critical in navigating ambiguity and building a strong foundation for my design decisions.
Additionally, I recognized that trying to solve every issue at once is neither practical nor effective. Prioritizing the most critical problems and gradually expanding the scope ensured that our solutions were both impactful and scalable.
While designing the in-product dashboard, defining the desired outcomes helped me determine the right metrics, select the most meaningful visualizations, and present them in a way that guided API producers toward improving quality effectively.
OTHER PROJECTS
Out of all the pixels in all the screens, you ended up here. Grateful for your visit!
2024 Ravi Jangir. All Rights Reserved.
Let’s connect