In my previous post, I looked at the benefits and strategies behind validation testing with real users.
Here’s a quick list of do’s and don’ts as you consider this method for testing your product:
Client’s perspective: I know I need to test my product, but I’m not sure what to test.
Set Goals
DO know the overall objectives and goals for testing. What hypothesis are you trying to prove or disprove? Are you concerned about onboarding? Testing a new navigation? Know what you are trying to accomplish and let the goals guide the rest of the testing decisions.
DON’T string together tasks that don’t make sense to the user in order to clear a backlog. If you can’t provide a coherent task flow, then maybe it’s not time to test yet.
Client’s perspective: I know I need to test my product, but I’m not sure who to test.
Recruit Well
DO test with a wide variety of real users. What does this mean? In an enterprise situation, you should include various levels of experience and roles. More junior members can give you a fresh perspective, while more senior or managerial figures can provide more context about legacy processes.
DON’T test with your team or expert users who have exposure to the behind-the-scenes process. Since they are invested in the product, their opinions may be biased, whether they realize it or not.
Client’s perspective: I know I need to test my product, but I’m not sure how to test.
Make It Real
DO strive for the highest fidelity prototype. When you give participants something that feels real, they can more quickly access the right mindset and give natural responses.
- Using prototyping software that mimics code, like Axure, or actual working code, like HTML sites, speeds up this process.
- Seemingly small touches to recreate the user’s workstation – like using monitors at the same resolution, the right OS, and even style of mouse – can help ease the transition from “testing” to doing your job.
- Props can be useful. If you know that the user typically performs a certain action in response to an email, include that as part of the testing flow.
DON’T use wireframes or art boards. Basically, avoid anything that puts too much strain on the user by not providing guidance about the interaction experience.
Client’s perspective: I know I need to test my product, but what if something goes wrong?
Test the Test
DO run through the test as if it was real, a few days before testing begins, with a “real” fake participant. A subject-matter expert is a good choice because he or she has the closest mental model to the actual user and can help troubleshoot the test before it is live. If you are testing with the general public, try to select a dry-run participant who most closely matches your recruiting criteria.
DON’T skip this step. A thorough dry-run is part of the process of vetting the test. Testing can be unpredictable; eliminate as many potential issues as you can ahead of time.
Client’s perspective: I know I need to test my product, but I’m not sure how to get results that will impact my project.
Measure Strategically
DO combine qualitative and quantitative methods to capture the full story. Qualitative techniques will help you understand the “why” behind the user’s actions, while quantitative tools give you a way to measure across participants. When balanced together, quantitative methods can provide a quick visual snapshot that reinforces the qualitative findings.
DON’T overdo it and attempt to quantify every part of the test. Testing fatigue is sure to set in as participants tune out and stop thinking about what they are trying to accomplish.
Validation testing can provide a tremendous strategic advantage and help you understand how real users will experience your product while you are still in the design/development phase. In return, this can help you avoid costly rework, decrease your time to market, and avoid costly developer spending.
