Making sense of user research: why do it, when to do it — and a bunch of tools to get you started

What a public-facing organisation wants to say, publish, and create online, is rarely the same thing as what their audiences want to hear, interact with, and accomplish.

This is the reality that many brands struggled to come to terms with in the early years of the web.

User-centered digital design and user experience emerged out of this discordance. The basic premise of user-centered design and UX is this:

It’s only possible to create truly valuable digital products and experiences, once we’ve understood the big picture context of who we’re creating them for, and why.

Otherwise, we’re merely shooting in the dark.

The web is already saturated enough with plenty of online white noise and useless fluff. The last thing we want to do as an digital publisher, is to go and add to the noise, with off-the-mark content and experiences that fail to connect with our audiences.

User research guides us away from that, helping us to instead design experiences we know our users want and need.

What is user research?

User research describes the effort of understanding the big picture context of who our users are and what they need from us — so that we design digital experiences that they value.

A little more specifically, from the perspective of content and UX strategy, user research helps us to:

• understand who we’re designing and publishing for, within which usage contexts
• getting feedback on the usability and effectiveness of our products and services, ‘straight from the horse’s mouth’
• finding out what ‘better’ looks like, from the perspective of those we seek to serve

Understanding user research methods and tools

In practice, user research describes a wide spectrum of activities, providing various insights on who is interacting with our products and services, in what ways, and why.

This chart from Nielsen Norman Group, helps us to make sense of it all:

In short, the chart maps research methods to two dimensions:

• Behavioral vs. Attitudinal. I.e. What people do vs. what people say.
• Quantitative vs. Qualitative. I.e. Objective, measurable feedback that tells us ‘what’s going on’ (analytics data etc.) vs. more subjective insights that suggest ‘why, and how to fix it’.

The chart also provides a third dimension (via the colour-coded key), for understanding the usage context of each research method. That is, whether:

a.) it’s natural or scripted use of the product
b.) the user isn’t using the product at all, or
c.) it involves a hybrid of both of the above

It’s a decent starting point and overview. But if your organisation is still pretty new to user research, you might still be feeling a little confused as to where exactly to start, which techniques to adopt, in what order, and why.

Because whilst some user research methods (such as usability testing), could be considered a reasonable constant throughout the product development cycle, other methods are more unique to specific stages of a project.

To help us understand when and where best to adopt particular methods and tools, let’s take a look at user research along a typical user-centered design process.

Once we’re clearer on the best contexts for adopting different types of user research, we can confidently start embedding it into each step of our design projects.

Sound good? Ok, let’s dive in…

Getting going: embedding user research into the design process

At the beginning of this article, I said that user research is about:

• understanding who it is we’re designing and publishing for
• using user feedback to assist us in benchmarking existing products and services
• getting users to guide us in creating and refining new products and services

Conveniently enough, these goals broadly align with the typical phases of a user-centered design process:

• Discovery: establishing who your users are, their requirements, and key usage contexts
• Design: translate discovery findings into proposed design and content solutions
• Implement and test: validate and refine new solutions

So now for a more detailed look at where and how user research plays its part in that process.

Here we go…

Discovery phase: understanding context, gathering requirements

Discovery phase goals:

• Analyse the existing ‘as is’ reality
• Understand and document user needs
• Identify key strengths, weakness, gaps, and opportunities
• Translate into content and design recommendations

User research goals in discovery phase:

• Get to know users: who they are, their defining traits, behaviours, and needs
• Observe and evaluate usage of existing product/service
• Measure and benchmark UX quality and performance of existing product/service

Typical user research deliverables from Discovery:

• Research/ discovery report, outlining key insights and UX recommendations
• User personas/profiles
• User stories
• Content experience maps

Discovery phase user research methods

Getting to know users

• 1-to-1 user interviews: meet with individual users (either in-person or remotely) to discuss the user’s views, habits, and behaviours in relation to your product or service.
• Focus groups: you facilitate group feedback on your products and services, involving a mix of informal discussion, set activities, and written exercises.
• Ethnographic field studies: meeting with and observing users in their natural environment, in order to understand the contexts in which they’re most likely to interact with your product or service.
• Online surveys: collecting data about who’s using your site and why.
• Google Analytics: specifically, ‘Audience’ metrics such as country, device, and browser usage, plus looking at site search queries, can lend clues to what types of people are using your site.

Observing and evaluating existing product/service usage

• Moderated usability testing: in-person or remotely, a user is asked to complete a set of tasks, with a focus on observing organic behaviours, habits, and assumptions.
• Unmoderated usability testing: online video is used to record users’ organically interacting with your product/service.
• Top task surveys: users are asked to choose and rank the tasks which they most value and want to complete with your product/service.
• Heat map studies: visualisations of the highest density areas of a page being clicked, hovered over, and scrolled.
• Google Analytics: specifically, ‘Behaviour’ metrics such as behaviour flows, site content, and site search data — giving us insight on how a user is moving through the site, what content is most popular, and what language is being used to find and interact with that content.

Measuring and benchmarking UX performance

• Usability testing: focussing on identifying key barriers to task completion, frustrations with existing experience and suggestions from users on how to make improvements.
• Live intercept surveys/polls: customised pop-up surveys or links prompt live feedback on user satisfaction and sentiment (using things like Net Promoter Score rankings), and invitation to offer suggestions for improvement.
• Visual user feedback tools: allows users to highlight and annotate specific sections of a page for comments/feedback/suggestions.
• Google Analytics: with focus on collecting and analysing ‘HEART’ UX metrics (happiness, engagement, adoption, retention, task success)

Design phase: creating ideas, concepts and proposed solutions

Design phase goals:

• Translate discovery insights into ideation for improved user experience
• Produce new design concepts, content, and end-to-end journeys
• Test and validate new concepts with users

User research goals in Design phase:

• Involve users in ideation process
• Test and validate new concepts with users
• Refine concepts based on user feedback received

Typical user research deliverables from Design phase:

• Concept testing report and recommendations
• Designs validated (as far as reasonably possible) and ready for first public release

Design phase user research methods

Involving users in ideation process

• Participatory design workshops: a bit like a more hands-on, advanced stage focus group, users are given materials/tasks and encouraged to work up suggested design solutions.
• Card sorting: a user-led way of developing your Information Architecture, where participants are asked to define how content should be arranged, grouped, and labelled for intuitive browsing and navigation.
• Live intercept surveys/polls: a pop-up form or link prompts visitors to provide suggestions on how a page could be improved.
• Visual user feedback tools: allows users to highlight and annotate specific sections of a page for comments/feedback/suggestions.

Test and validate new concepts with users

• First-click testing: users are presented with early approximations of new product in the form of wireframes/screenshots, with a set of questions that test validity of design concepts and navigation schema.
• Tree testing (or reverse card sorting): a proposed site map (in lo-fi tree diagram format) is presented to users, along with a set of tasks designed to test the validity of new Information Architecture.
• Product/service prototype testing: in-person or remote, users are presented with a functioning, interactive version of the new product/service, testing the complete, stitched together end-to-end user journey.

Implement and test phase: validate and refine new solution

Implement and test phase goals

• Release public version of new product/service
• Monitor and measure performance of new product/service
• Make iterative improvement to new product/service, based on feedback

User research goals

• Continue collecting feedback from users on live product/service
• Feedback and recommendations are regularly reported into product/service owners
• Ongoing iterations and improvements, informed by user feedback

Typical user research deliverables:

• • User feedback mechanisms are set up and embedded into live product/service
  • Regular user research monitoring and reporting set up for new service
  • Long-term user research/testing plan in place for new product or service

Implement phase user research methods

• A/B testing: two versions of a live page are set up, in order to determine which is the higher performing against a conversion/UX metric.

In addition, the implementation phase should see continued use of previously mentioned tools and techniques, in particular:

• Google Analytics
• Heat maps
• Live intercept surveys and polls
• Visual feedback tools

So all together, we’ve got a blueprint for where and when to adopt certain user research techniques, which looks something a little like this…

So, there you have it.

User research can and should underpin all stages of a design project. Although by far the largest spread and density of research techniques occurs in Discovery phase, user research can strengthen and improve your digital platforms during all phases of development — before public launch and far beyond it.

As a project moves from sense-making and requirements gathering, to testing and refinement, there’s always the opportunity to be testing and learning, in order to make your product or service as good as it can be.

You may have noticed that certain methods and tools are repeat visitors to the table. I.e. Things like usability testing and analytics are relevant to the entire end-to-end design process — but their specific purpose varies between project phases.

For example, usability testing during Discovery is all about sense-making and establishing basic contextual scope for a prospective product or service. Whereas by the time we get to Implementation, the very same method is used to achieve something quite different: refinement and improvements to the live product or service.

I hope that provides a good overview of key user research methods, and where they fit into the user-centered design process.

If you have any questions or suggestions, give me a shout.

Contact