Solution: How do you achieve User Observability?

Add tools and processes to monitor user behavior as part of your observability practice

The Union Between System and User Observability

Connecting system behavior to user insights

The Honeycomb Observability Maturity Model lists five capabilities for a successful observability practice. Along with traditional IT practices of releases, code, technical debt, and resilience, each organization needs user insights to achieve observability.

Everyone involved needs to understand the system’s impact on users. Are customers, employees, and other users accomplishing their goals? Are our systems enabling them or hindering them?

Organizations can better understand user behavior through user experience methods and processes. Insights from field studies and usability tests can help determine requirements and user interface design details, and they can also help with system observability issues.

Source: Observability Maturity Model, Honeycomb. Reprinted with permission.

User Insights as an Organizational Capability

How to drive strategic and tactical decisions based on insights from users

User research involves more than just asking people what they want. Design-centric organizations conduct research to understand what users are doing and why.

Generative research includes field and ethnographic studies to discover the unmet needs that new solutions can address. This helps explain why individuals behave a certain way.

Evaluative research studies how people use existing solutions (using A/B tests and usability studies) to compile metrics on task completion and satisfaction and ultimately improve designs. This helps organizations understand individuals’ true actions.

User researchers focus their time on gaining user insights. Experience designers, on the other hand, absorb insights while designing.

Source: UX Careers Report, Nielsen Norman Group

Examples of User Observability

How user insights support system observability, and vice versa

Beginner: Understanding user impacts as a result of system issues

Some system monitoring tools track application response times and can calculate how they might add up to a “frustrated” user. They can also track a series of clicks on inactive buttons (or “rage clicking”). These are signs of frustration, but what is causing it? To get to the heart of the problem, you will have to talk with these users to understand what they were trying to accomplish and their expectations. Doing so will likely uncover other sources of frustration that the system cannot detect.

Intermediate: Driving investigations into system performance based on user issues

A user researcher performs a usability test on shopping cart functionality, focused on user interface design issues. Users comment on performance issues during the testing. The system observability team then steps in to investigate the issue to see if it is a recurring problem to fix.

Advanced: Doing experiments together

The system and user observability teams both want to understand the relationship between performance and user buying behavior. For example, in which parts of the journey are users most sensitive to system performance and most likely to abandon the process as a result? User researchers can help participate in system tests and collect qualitative data from users about their reactions when parts of the ecommerce system go down. This can help prioritize which parts of the system need beefing up.

Technology Solutions

Tools and platforms to generate insights more effectively

A wide range of products have been developed to help recruit users, observe user experiences, and manage the insights gained from those observations.

The most relevant tools for user observability support active research (surveys, user interviews, and usability testing) and passive research (A/B testing, analytics, and automated feedback).

Standard ways to connect these tools to your existing system monitoring tools have not evolved yet. To get an end-to-end view from what you see on the back-end to what users see on the front-end requires some customization.

Source: 2021 UX Research Tools, User Interviews