Usability Evaluation
Usability Evaluation
Put simply, usability evaluation assesses the extent to which an interactive system is easy and pleasant to use. Things
Book chapter
Usability Evaluation
Your constantly-updated definition of Usability Evaluation and
collection of videos and articles
What is Usability Evaluation?
Usability evaluation assesses how easy and enjoyable it is for users to achieve their goals while using a product. Designers use qualitative and quantitative research methods to identify User Experience (UX) issues. It's a vital step to ensure user satisfaction.
"If a User is having a problem, it's our problem."
– Steve Jobs
Usability evaluation is important in design, but it's not enough by itself. In this quick video, you'll find out why usability evaluation is only one part of the whole picture. When you use it along with other tools designers have, you can come up with better and more useful solutions.
To fully understand usability evaluation, it’s necessary to grasp the concept of usability first. The International Organization for Standardization (ISO) defines usability as:
“The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.”
– ISO 9241-11, Ergonomics of human-system interaction—Part 11, Guidance on usability
Usability Evaluation measures (or in some cases, predicts) this effectiveness, efficiency and satisfaction. You can use usability evaluation methods at any stage of design or development.
Effectiveness checks how accurately users achieve goals in specific situations.
Efficiency looks at the resources used to accomplish goals.
Satisfaction examines how comfortable and pleasant the system is to users.
Let's take an online fitness tracker app as an example. You've downloaded the app to log your daily workouts. Effectiveness measures how accurately the app records the type and duration of your exercises. If it consistently gets this right, that's effective tracking.
Next, efficiency looks at how the app uses your phone's resources. Is it draining your battery too quickly while you log your workouts? Lastly, satisfaction relates to the overall experience. How easy is it to set your fitness goals, and track your progress?
Overall, user evaluation ensures the fitness app is easy to use and meets the users' needs.
When to Use Usability Evaluation
In product development, timing is everything, and knowing when to use usability evaluation is crucial.
Evaluate early on in your design process: An architect always ensures a building's foundation is solid before construction begins. Similarly, usability evaluation is vital in the early design and prototyping stages.
For example, conducting usability tests on an early prototype of a smartphone can flag unclear navigation. Fixing these early issues is like ensuring the blueprint is flawless before construction starts. It saves time and resources.
William Hudson, UX strategist and educator, explains why tree testing and first-click testing can be useful early in the design process.
Evaluate throughout your design process: Like a steering wheel helps you steer and stay on track, continuous user feedback keeps a product on the right path. It ensures the product meets user needs and expectations. That's why getting user feedback throughout the product development process is crucial.
Usability evaluation happens at every step of a project, including during the requirements, analysis and design, implementation, testing and deployment stages.
© Interaction Design Foundation, CC BY-SA 4.0
Evaluate before you launch a new product or relaunch an existing product: It’s essential to do usability evaluation when you're getting ready to make big changes or launch a new product. Think of it like taste-testing a new recipe before you share it with a big group. Usability evaluation helps ensure the product works well and is user-friendly.
Imagine you're upgrading an e-commerce website. You can use remote usability testing to make sure the process of buying products is smooth and error-free. This way, when you release the updated website, it will be easy for more people to make purchases.
Watch this video to learn about the aesthetic usability effect and the importance of visual elements in your design.
Why Usability Evaluation is Important in User Experience (UX) Design
Usability evaluation is like a compass that points User Experience (UX) designers in the right direction. Usability, a subset of UX, is a pivotal component of user experience. It ensures products are easy to use, functional and meets the users’ needs.
User Experience (UX) encompasses the overall emotional and psychological response a user has when using a product. It goes beyond usability and includes aspects like branding, emotions, and design.
Let’s look at some key reasons why usability evaluation is pivotal in UX:
User-centered design: Usability evaluation keeps the design focused on the people who will use the product. It ensures the user’s needs, preferences, and pain points drive design decisions. This leads to a more user-friendly interface.
Problem identification: Usability evaluation methods help find problems with the user experience. Feedback from real users gives us useful ideas for making it better.
Iterative improvement: Regular usability testing helps teams make continuous improvements. They test, improve, and test designs again, which leads to small but valuable enhancements and a better user experience.
Reduced costs: Design teams can use early-stage usability checks to save resources. It's a cost-friendly way to prevent costly changes after releasing the product.
Competitive advantage: Products that go through usability testing often do better than their competition when it comes to competitive advantage. This can make users stick around and give the brand a good reputation.
Alan Dix, professor and bestselling author, walks through three non-negotiable usability guidelines.
Usability Testing vs Usability Evaluation - What’s the difference?
Usability testing is the most used usability evaluation method. It involves observing the behavior of real users. Imagine a group of people trying out a new mobile app while researchers observe to see if any issues come up. The researchers would record observations like a menu that’s difficult to find and how long it took users to complete tasks like logging in.
Usability evaluation includes usability testing, asking users for feedback and examining the product's design. We also refer to these methods as testing, inquiry, and inspection.
The Three Main Types of Usability Evaluation
The three main types of usability evaluation are usability testing, usability inquiry, and usability inspection. Together, these methods provide rich qualitative user insights. Researchers use these insights to gain a holistic understanding of user interactions, preferences, and challenges.
Often conducted before quantitative research, the qualitative research methods used in usability evaluation provide insight into user attitudes and behaviors. Qualitative research methods are typically done with 10 participants or less. Researchers use interviews and focus groups in combination with usability testing to obtain these qualitative user insights.
© Interaction Design Foundation, CC BY-SA 4.0
1. Usability Testing
Like watching people trying to cook a new recipe, usability testing involves observing real users. In the same way a chef in the kitchen can see if an ingredient is hard to find or a step is confusing, UX designers can spot issues by watching how users behave. This qualitative hands-on approach provides actionable insights that help make the product easier to use.
2. Usability Inquiry
Usability inquiry means talking to users to find out what they expect and what they need. It's important to get information from users to make designs people like and help them achieve their goals. Two common ways to do this are focus groups and interviews.
In focus groups, participants come together to discuss their experiences. For example, you can organize a focus group to learn how a group of gamers feels about a new video game’s interface.
Interviews involve one-on-one conversations. For example, a designer could talk directly to a smartphone user to find out what they like and what their challenges are.
Ann Blandford, Professor of Human-Computer Interaction at University College London, explains the pros and cons of user interviews.
Both focus groups and interviews can help UX designers grasp what users want and the issues they encounter.
3. Usability Inspection
Usability inspection means using expert assessments to find usability problems. Heuristic evaluations and cognitive and pluralistic walkthroughs are methods used to test how easy a product is to use.
Heuristic evaluations use predefined principles to check how easy a product is to use. For example, think about testing a mobile app. You'd have a list of guidelines or rules for making a good app, like "clear navigation" and "simple registration." If you had trouble using the menu, you'd use these rules to recommend changes to the app.
The ten Nielsen-Molich usability heuristics are visibility of system status, system match to the real world, user control and freedom, consistency and standards, error prevention, recognition instead of recall, flexibility and efficiency of use, aesthetic and minimalist design, help users recognize, diagnose and recover from errors and help and documentation. These usability heuristics help UX designers measure how user-friendly a digital product is.
© Interaction Design Foundation, CC BY-SA 4.0
Cognitive walkthroughs are when experts pretend to be users. They go through the interface one step at a time to find any problems with how easy it is to use.
An example of a cognitive walkthrough is when a team of UX specialists, engineers, and experts evaluate a new mobile app:
They plan which specific user activities they want to check. This includes tasks like signing in, finding a product, and completing a purchase.
They perform each step individually while putting themselves in the mindset of a first-time user of the app.
The goal is to discover any problems with how easy the mobile app is to use.
Pluralistic walkthroughs are similar to team checks of the interface, where a group of experts, users, and other stakeholders work together to share their perspectives.
For example, a user might say, "I can't figure out how to buy that Nike shoe in size 38." The product manager might interpret that as, "This design doesn't yet match our project goals. It doesn’t give the user the feedback that this particular size is out of stock." This gives a broader view of what needs fixing.
These techniques provide a structured approach to evaluating usability. They also provide useful ideas for improving the design.
Quick Comparison Table: The Three Usability Evaluation Methods
Usability Evaluation Method | Pros | Cons | Examples | Helpful Tools |
Usability Testing | - Reveals real user interactions - Identifies actual user issues - Provides direct user feedback | - Requires user recruitment and coordination - Can be resource-intensive - Limited to the skills and insights of the test users | UX researchers observe users completing a list of tasks on their new food-ordering app. They make notes of potential usability issues. | - Maze |
Usability Inquiry | - Gathers in-depth user opinions - Uncovers user preferences and expectations - Facilitates open-ended discussions | - Highly dependent on user availability - Biased by user’s personal perspective - May not uncover all usability issues | A UX designer wants to gather user feedback and opinions about their new e-commerce website. They conduct user interviews, focus groups, and surveys. | - Zoom for remote interviews - Microsoft Forms |
Usability Inspection | - Expert-driven assessments - Identifies potential issues - Cost-effective and quicker | - May not catch all real user issues - Limited to the expertise of the inspectors - Less user-focused than testing and inquiry methods | A group of UX designers and developers evaluate their new mobile game UI using predefined criteria like heuristics. | - Nielsen's 10 Usability Heuristics - Lyssna (previously UsabilityHub) - UXCheck |
How to Recruit Users for Usability Evaluation
Recruiting the right users for usability evaluations is critical. A well-thought-out approach to recruitment will ensure you gain useful, valuable insights into the usability of your product.
Recruitment Planning Guidelines
Understand Your Users: Before you start recruiting, figure out who your ideal users are. Take the time you need to get clear about details like their age, what they like, and why they might use your product. This will help you find the right people for the usability evaluation.
Look for people who are genuinely interested in what your product is about. You can think about this as if you were starting a sci-fi book club. Would you invite sci-fi enthusiasts or avid romance readers? Sci-fi readers who already enjoy the genre are a better fit. They can be expected to engage and provide relevant feedback during the book club. In the same way, feedback from users who are likely to use your product will provide the valuable insights you’re after.
Craft Compelling Recruitment Messages: First impressions count. Your recruitment message is the first point of contact with potential participants. When writing your message, tell users why you're doing this usability evaluation and what's in it for them. Make it easy for users to express interest, learn more and sign up to participate.
Offer Incentives: Incentives can be a powerful tool in user recruitment. You should experiment with offering tangible rewards, such as gift cards and discounts. You could also offer exclusive access to new features. Incentives attract participants and show your appreciation for their time and input.
Prioritize Diversity: Ensure your recruitment efforts are inclusive. A mix of ages, backgrounds, and experiences gives us a better understanding of what users might think about our product. Inclusive recruitment enhances the richness of user perspectives. It also contributes to more comprehensive usability insights.
Instructor William Hudson talks about user research recruitment and how to identify participants who aren’t a good fit.
Where to Find Usability Evaluation Participants
Existing Users: If you're testing existing products, your existing user base is a goldmine for usability feedback. It's best to avoid assuming they'll join in. Invite your users to join through emails, website pop-ups, social media groups, or even have sales and customer service teams reach out.
Online Platforms: Tap into the large pool of potential participants available online. Social media, user forums, and online communities are a few of the places where you can find potential participants.
Platforms like Reddit provide spaces where users share their experiences. Many of the users found in these online spaces are willing to take part in usability studies. Additionally, consider joining LinkedIn groups or Discord channels where the people you want to reach might gather.
Collaborate with User Research Platforms: Consider partnering with user research platforms that connect UX designers with potential participants. These platforms make it easy to find a lot of different users, which can save time and give you a more diverse group to learn from.
The Nielsen Norman Group discusses five ways to recruit participants for user research.
Common Usability Evaluation Challenges and How to Overcome Them
A lack of budget, time and buy-in from stakeholders can derail even the best-laid usability evaluation plan. Experienced UX designers understand the importance of forward planning when it comes to these potential hurdles. Instead of viewing these challenges as obstacles, see them as opportunities.
Dealing with Small Budgets and Tight Timelines
Tight budgets and timelines can limit the resources available for thorough usability evaluations. However, a small budget needn't stand in the way of valuable insights into user experiences.
Strategies you can use to overcome usability testing budget and time constraints:
Focus on Key Objectives: Identify usability aspects that match project goals. This helps allocate limited resources to the most impactful areas of usability.
For example, an online clothing store would focus on important goals, like making it easy to buy clothing items and search for specific pieces. They would focus their resources on testing the checkout and search functionality.
Lean Methodologies: Adopt efficient usability testing methods like guerrilla testing. These quick, informal approaches provide valuable insights, even with tight constraints.
Open-Source Tools: Make use of open-source usability testing tools like Selenium and OpenSTA to keep costs down. Many effective tools are freely available. They allow teams to conduct usability assessments without a big financial investment.
Managing Resistance to Usability Changes
Resistance to usability changes, often from stakeholders, poses another challenge for many UX designers. It’s necessary to overcome this resistance if you want to improve certain aspects of the product based on usability findings.
Strategies you can use to overcome resistance to usability changes:
Keep Communication Open: Establish open lines of communication between researchers and stakeholders. Clearly explain why you suggest the changes you’ve proposed. It's good to discuss concerns together in a clear and open way.
Show Clear Proof: Back up the suggested changes with clear proof. Use data, feedback from users, and usability metrics. This helps to show exactly how the changes will affect the overall experience for users.
Highlight Long-Term Benefits: Point out the good that will happen in the long run because of user-centric design. Teach stakeholders how valuable it is to fix usability issues early in the design process. Emphasize benefits like more satisfied users, spending less money, and building stronger loyalty to the brand.
For example, you could schedule a meeting with researchers and stakeholders to present the usability findings and proposed changes. During the meeting, you’d explain how the suggested improvements align with the users' needs and expectations, ultimately leading to increased user satisfaction and potentially attracting more customers. You’d address their concerns about disruptions by outlining a phased implementation plan that minimizes downtime and highlights the long-term benefits of the changes.
Get your free template for “Good Questions for Stakeholder Interviews”
Learn More about Usability Evaluation
Read the IxDF’s open-access textbook entry on usability evaluation.
Take our course Interaction Design for Usability.
Read How to Involve Stakeholders in your User Research.
Watch our How to Get Started with Usability Testing Master Class.
Take our course The Practical Guide to Usability.
Read UX research on a budget.
Watch a video about Formative vs. Summative Evaluations on the Nielsen Norman Group website.
Read and learn more about current usability standards.
Read How to Recruit Users for Usability Studies.
Get your free template for “Usability Test Checklist”
Questions related to Usability Evaluation
How do you evaluate the usability of a website?
How do you evaluate the usability of a website?
Usability testing is the most common technique used to measure the usability of a website. In this method, a researcher observes a group of test participants as they perform specific tasks on a website.
Steps to evaluate the usability of a website:
1. Define the target audience: Identify the group of people who will be using the website and what their goals are.
2. Select representative tasks: Choose tasks according to what the target user would do on the website.
3. Find representative users: Recruit participants who are representative of the target audience.
4. Observe what the users do: Ask the participants to perform the tasks. Observe, take notes and record any issues they encounter.
5. Summarize the results: Analyze the data collected, identify any issues and share the valuable insights with the team.
Usability testing methods include in-person testing, remote testing, and automated testing. If you want to identify issues before the site launch, it’s important to test early and often.
Accessibility and usability are closely related when it comes to web design. You can watch the video below and read the article, Usability for all, to learn more.
What is a usability checklist?
What is a usability checklist?
A usability checklist outlines the steps needed during a usability test. The researcher is the only person who uses the usability checklist. They do so to ensure they remember to do or say anything important during the test.
Get your free template for “Usability Test Checklist”
To learn more, read this article for a semi-structured qualitative study (SSQS) checklist.
What is the difference between usability and heuristic evaluation?
What is the difference between usability and heuristic evaluation?
Heuristic evaluation is a specific method within usability evaluation. Experts, often UX professionals, test the product using some predetermined rules or guidelines.
The heuristics are rules of thumb for good design and usability, like Jakob Nielsen’s 10 usability heuristics. Read this article on the IxDF blog to learn how to conduct a heuristic evaluation for usability.
William Hudson, author and instructor in user-centered design, shares what it means to conduct a heuristic evaluation.
What are some highly cited scientific research about usability evaluation?
What are some highly cited scientific research about usability evaluation?
1. Nielsen, J. (1993). Usability Engineering. Academic Press.
This book by Jakob Nielsen is a cornerstone in the field of usability evaluation. It provides a comprehensive framework for usability engineering, introducing key concepts and methods for evaluating and improving the usability of interactive systems.
2. Brooke, J. (1996). SUS: A quick and dirty usability scale. In Usability evaluation in industry (Vol. 189, pp. 4-7). CRC Press.
Designers use John Brooke’s System Usability Scale (SUS) to evaluate the usability of products and services. This paper introduces the SUS and its application as a simple yet effective usability evaluation tool.
3. Lewis, J. R. (1995). IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction, 7(1), 57-78.
Lewis’s paper presents the IBM Computer Usability Satisfaction Questionnaires (CSUQ), which are well-established standardized questionnaires for assessing user satisfaction. It provides valuable insights into usability assessment through user feedback.
4. Tullis, T., & Stetson, J. (2004). A comparison of questionnaires for assessing website usability. Paper presented at the Usability Professionals Association Conference.
This conference paper discusses the comparison of various questionnaires for assessing website usability. It’s important for understanding how to measure usability effectively in the context of websites.
5. Bangor, A., Kortum, P. T., & Miller, J. T. (2009). An empirical evaluation of the system usability scale. International Journal of Human-Computer Interaction, 24(6), 574-594.
This paper critically evaluates the System Usability Scale (SUS) and its use in assessing usability. It discusses the validity and reliability of SUS, which is essential for usability practitioners.
If you’d like to cite content from the IxDF website, click the ‘cite this article’ button near the top of your screen.
Is usability testing formative or summative?
Is usability testing formative or summative?
Usability testing is both formative and summative. Formative usability testing takes place during the initial stages of the design process to improve the product design. Summative testing takes place after the product is out to see how well it works and how easy it is to use.
Learn more about formative vs summative usability evaluation on the Nielsen Norman Group website.
What are some recommended books that cover usability in UX well?
What are some recommended books that cover usability in UX well?
1. Krug, S. (2006). Don’t Make Me Think: A Common Sense Approach to Web Usability. New Riders.
Steve Krug’s book is a classic in the field of web usability. It offers a user-friendly and common-sense approach to web usability.
2. Norman, D. A. (2013). The Design of Everyday Things. Basic Books.
Don Norman’s book is a foundational work in UX design. It explores the fundamental principles of design and usability, emphasizing the importance of user-centered design and understanding the psychology of users.
3. Barnum, C. M. (2010). Usability Testing Essentials: Ready, Set... Test! Morgan Kaufmann.
Carol M. Barnum’s book is an excellent resource for those looking to understand and conduct usability testing. It covers various usability testing methods, techniques, and best practices.
4. Krug, S. (2009). Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability Problems. New Riders.
This is another practical book by Steve Krug, focusing on usability testing and problem-solving. It offers a step-by-step guide to conducting usability tests and improving user experiences.
5. Garrett, J. J. (2011). The Elements of User Experience: User-Centered Design for the Web and Beyond. New Riders.
Jesse James Garrett’s book provides a comprehensive framework for understanding and implementing user-centered design. It breaks down the elements of the user experience and how they relate to the design process.
6. Lidwell, W., Holden, K., & Butler, J. (2010). Universal Principles of Design. Rockport Publishers.
This book compiles 125 design principles, including many related to usability, in a visually engaging and easily digestible format. It’s an excellent reference for designers and usability professionals.
7. Buley, L. (2013). The User Experience Team of One: A Research and Design Survival Guide. Rosenfeld Media.
An ideal read for individuals working in smaller teams or as the sole UX designer. It offers guidance on effectively integrating usability practices into projects, even with limited resources.
You can find a full list of the top UX Design books, catering to both beginners and experts, in this article.
What are the 5 usability evaluation criteria?
What are the 5 usability evaluation criteria?
The five usability evaluation criteria are:
1. how quickly you can learn it,
2. how efficiently you can use it,
3. how well you remember how to use it,
4. how often you make mistakes,
5. and how much you like using it.
These factors aren't always equally important for every project. They can help designers make products that are user-friendly.
Short on time or resources? These 5 Simple Usability Tips are easy to implement and won’t break the bank.
What is usability in HCI (Human Computer Interaction)?
What is usability in HCI (Human Computer Interaction)?
Usability is a cornerstone concept in Human-Computer Interaction (HCI). It focuses on making systems easy for people to use. Systems should be simple to learn, have fewer mistakes, and be pleasant to use.
The five usability factors in HCI are:
1. how easy it is to learn,
2. how efficiently it works,
3. how well you remember how to use it,
4. how often mistakes happen,
5. and how satisfied you are with the experience.
To learn more about the relatively new discipline of Human Computer Interaction, start with our foundational course.
Can I conduct usability testing remotely?
Can I conduct usability testing remotely?
Remote usability testing is a widely used research method. UX researchers use a special platform to capture how test participants use a digital product. The most common types of data they collect are screen and voice recordings.
In moderated remote testing, UX researchers watch and talk to users as they do specific tasks. Moderated testing works well for tricky tasks where more talking and asking questions will help with testing.
In unmoderated testing, the researcher shares a set list of tasks, and the participant does them on their own. Remote usability testing is a practical and money-saving way to observe real people do real tasks.
You can learn more by taking our popular Conducting Usability Testing UX design course.
How will AI affect UX research?
How will AI affect UX research?
UX researchers can use AI to make work processes easier and give users better experiences. This includes using AI to look at data, figure out how people feel about a product or system, and use AI-powered virtual assistants to do simple tasks. UX researchers will then have more time to come up with questions, set the right goals, and get better results.
It’s important to know AI tools can bring in human biases when people use AI-powered results in decision-making. Read this article to learn more about how designers can adapt to new technologies like this, read this article.
AI will impact all areas of UX design. Let’s hear what Don Norman, author and co-founder of the Nielsen Norman Group, thinks about Artificial Intelligence.
Literature on Usability Evaluation
Here’s the entire UX literature on Usability Evaluation by the Interaction Design Foundation, collated in one place:
All open-source articles on Usability Evaluation
Four Assumptions for Usability Evaluations
Usability is a vital part of the user experience as a whole. It’s vital for UX design teams to be able to assess whether
- 499 shares
- 7 years ago
System Usability Scale for Data-Driven UX
Have you ever wondered how top designers, researchers, and product developers consistently create user-friendly, efficie
- 352 shares
- 3 mths ago
Open Access - Link to us!
We believe in Open Access and the democratization of knowledge. Unfortunately, world class educational materials such as this page are normally hidden behind paywalls or in expensive textbooks.
If you want this to change, , link to us, or join us to help us democratize design knowledge!
