Evaluating Your Designs

In the previous sections, we explored the four phases of the design thinking process, which are summarised below. 

  1. Empathise - Understanding the customers and their context.
  2. Define - Formulating a meaningful and actionable problem statement.
  3. Ideate - Collaborating to generate novel, and unconventional ideas to solve our design challenge.
  4. Prototyping - Creating a mock-up of our design concept to be evaluated by potential customers.
The five phases of design thinking adapted from the Interaction Design Foundation.

Now we've reached the fifth stage of the design thinking process and that's the test mode. Note well that this might not be the final stage, as you may uncover insights that lead you back to the empathise, define, or even ideation mode, in order to refine your proposed design solutions and make them better. Again it is needful to say that the design thinking process is not always linear as multiple stages may overlap. In this case, you may find that prototyping and testing often occur at the same time. 

What is User Testing?

Testing provides an opportunity to evaluate your prototypes and gain direct feedback from your prospective customers. In the test mode, you'll empathize with your potential customers and assess their experience with your design solutions.

You may assess the following:

  1. What do they like or dislike?
  2. What do they find difficult or frustrating or confusing or misleading?
  3. Are they able to successfully complete the tasks?
  4. Where do they get stuck?
  5. How are they navigating to complete their tasks?
  6. How long does it take to complete a specific task?
  7. What are they saying?
  8. What do they want?
  9. What are their suggestions for an improved user experience?

At this point, it is imperative to seek feedback and clarification through your “Why?” questions. In this way, you'll dive deeper into the problem-solving process and uncover insights to refine your designs and provide a better user experience for your customers. 

In test mode, it is best to evaluate your prototypes in the users' natural context. For example, if you are evaluating a wearable fitness device, it is ideal to incorporate the users' everyday routines in the evaluation. In addition, if you're exploiting a narrative technique, then scenarios depicting real-life situations can be recreated at this stage. 

Why do we test?

Why is it important to test with your users?

  1. To identify usability issues before you launch your product and uncover unexpected insights that could lead you back to the drawing board. Ultimately, this reflective process will be beneficial to refine and improve your final product.
  2. As mentioned earlier, in the test mode, you'll be able to empathize with your customers and unveil deeper insights into what works and doesn't work with the product.
  3. It provides an opportunity to revisit and refine your problem statement. It is quite possible that your user evaluation results could suggest that your solutions did not adequately address the problem, you made incorrect assumptions about your customers' behaviour or you didn't frame the problem in the right way.
  4. By identifying bugs and errors at an early stage, you can save time and money. By exploiting an iterative approach, you can revisit the issue/s and address them based on new customer insights and feedback rather than fixing them after development. User testing will provide customer feedback that will help you to make informed customer-centered decisions that will satisfy their needs as well as help you to avoid expensive development errors.
  5. You'll improve customer -satisfaction, -retention, and -advocacy -  By testing directly with your customers, you'll improve your chance to accelerate their acceptance of your product or service. In addition, you may form good relationships and as such, it is highly likely that customers may come back and refer you to their networks (colleagues, friends, and family), i.e., once your design solution satisfies and exceeds their expectations.

When should you test with users?

Don't wait until you're ready to release your product on the market to test your designs. That's a dangerous move!  To meet your customers'  needs and expectations,  you'll need to test early and quite often.

You may deploy low-fidelity prototypes (e.g., paper prototypes) to determine if your initial ideas are worth further exploration. 

Once your team and customers are onboard with your initial prototypes, you'll add more details and make your design concepts more realistic through clickable wireframes. As customer acceptance with your low-fidelity prototypes improves, you'll move on to testing with more visually appealing,  content-rich, and interactive prototypes (i.e. your high fidelity prototypes). In so doing, you will provide a richer and more interactive experience for your customers and gain richer and deeper insights into their overall experience and usability of your design solution.

Testing Methodologies 

  1. Guerilla testing - this is basically going to a public space (e.g., a snack bar) and asking people to evaluate your prototypes. This is quite relevant for testing your preliminary concepts and you may leverage your low fidelity prototypes in this test. You'll randomly test your participants and you can offer to pay for their drink or snack in return.
  2. A/B testing - also referred to as bucket testing or split-run testing and involves running randomised experiments between two prototypes (A and B) to determine which prototype performs the better. Review this article to gain a thorough perspective on how to run an A/B test.
  3. Usability testing - the act of evaluating your product with your users. It helps you as the designer to identify how users use and misuse your product or service, identify the areas of your designs that need improvement, and also helps you to better empathise (understand how they think, feel, and experience your designs) with your customers. There are several types of usability tests including quantitative and qualitative evaluations as well as in-person and remote evaluations. Read this article, to learn more about usability testing and its various types.

User Testing Guidelines

  1. Devise a clear plan and scenario It is important to set specific goals e,g.,  What's the setting? What tasks do you want your customers to complete? How many tasks do you want them complete? - Be sure not to overwhelm the users with too many tests. What do you want to learn from the user test? What are the expected outcomes of your user test?). This is done so that you can build the right prototype that addresses your design challenge and select an appropriate testing methodology.
  2. Recruit your participants i.e., enlist participants that represent your target group. Offering an incentive can boost motivation to participate in your study and prompt their interest to participate in further evaluation of the prototypes or final product.
  3. Evaluate in the context of use. In other instances, try to replicate the customers' routines and natural environment through scenarios that match their situation.
  4. Specify the roles for your testing procedure. In this case, you'll want to specify the following as inspired by the Standford d.school bootleg bootcamp Testing with Users method card.
    • Host/Facilitator - The person who will guide the users to transition from reality to the situation where they will interact with and evaluate the prototype. The host translates the scenario, explains what the prototype is about, and how it works without sharing too many details. He or she is also responsible for building rapport with the user, making the user feel comfortable so that they will share their honest opinions and feedback as it relates to the product. The host will also clarify the users' assumptions and uncertainties and often leads the questioning process.
    • Players - You'll want to assign people to different roles to create and enrich the experience of interacting with your prototype.
    • Observers - It is vital to assign a team of observers in your evaluation session. In this way, they can capture how the user experiences and interacts with the prototype. Videotaping the interactions can be useful to identify unconscious or unobserved behaviours (e.g., facial expressions, gestures, and general body language) and interactions with the product.
  5. Say less and Show more - provide a minimal context and allow your user to interact with your prototype without too many explanations of your design rationale.
  6. Listen and let them actively share their experience throughout the evaluation.    Here the host can ask Why? questions or ask the user to think aloud as they execute their tasks. You can ask questions like what do you think is happening here? What do you think this does? What's your first impression of this feature? What do you expect to happen here?  What does this icon/button/menu mean to you?
  7. Observe how your users use and experience the prototype. Observation can be useful to capture inconspicuous behaviours and interactions with the product. 
  8. Clarify with questions -  It is vital to follow-up with your customers. Ask them questions related to how the products make them feel, what they find difficult, confusing, and be sure to ask why.
  9. Don't forget to document your findings and observations. This is useful to help you reflect, analyse, and synthesise your findings. Ultimately, it helps to make evidence-based and customer-driven decisions about the final design of the product.

Do you stop at user testing?

After you have run several tests and noted your observations and findings. You'll need to spend some time in reflection with your teammates. You'll analyze your findings and determine how you can improve your design to address the design challenge. Your test results will reveal what works well and what doesn't and could lead you to revisit the define, ideate, as well as prototype modes. It may take several rounds of refining your solution before you reach an ideal solution that fits in your customers' context and operates as expected. 

Guess what again? Even after your product is released on the market, you'll need to run more tests to meet new requirements,  improve product features, address system errors, or modernize your product or service offerings. 

Additional Resources 

  1. https://careerfoundry.com/en/blog/ux-design/user-testing-design-thinking/
  2. https://www.nngroup.com/articles/usability-testing-101/
  3. https://xd.adobe.com/ideas/process/user-testing/top-7-usability-testing-methods/