Tips for a successful usability interview

1. Ask the right questions

Always aim for open-ended questions.

• Product questions: Do you like our travel app idea? => What and why questions: What travel apps or websites do you use? Why? (try to uncover data and problems that people have)
• Leading questions: Would you prefer checking-in online? (people will always say “yes”) => How do you check-in? Please describe what you did (work out for yourself if checking in online would be better for users)
• Avoid future focused hypothetical questions: If our app were available tomorrow would you use it? => Specific questions about the past: Tell me about the last time you booked a flight
• Don’t ask closed questions: Did you check in online? => Ask open-ended questions: How do you check-in? Please describe what you did (this will provide rich data)

If you don’t ask anything else, ask the following three questions

• Why did you visit our app/website/product today? (People are free to write whatever they want, it is an open-ended question. By asking this question you learn people’s goals). - I came to the website because I wanted to do X or Y or Z. This is one of the most important questions.
• Were you able to complete your task today? People can answer yes or no. And if they answer no, we ask them to tell us, why not. What we are learning here is if people are learning to do what they want to do on our product. And if not, then what are we doing wrong, what roadblocks are we putting in their way? This is extremely important information.
• What would you change about our website? What improvements would you make? (We are not asking design questions: what features would you include? What would you change about the layout? We are just asking about improvements on a very general level. It is a great question because people will tell you about a mix of things (problems, possible features, what they would like to do etc). So this question gives people the opportunity to tell us more information, whatever didn’t fit in the first two answers.

2. Don’t ask the wrong questions

   Direct yes/know closed questions (Did you notice the advertisements on the right?) Just because you asked, you drew their attention to them and compromised the data. => Instead what and why questions, ask it in a round-about way: What are you looking at? What are you looking for?

   Leading questions: Would you prefer the times to be included? (If you ask them, they will say yes) => instead approach it in a round-about way: What information do you need to continue? Why is that important? (We can draw out that information without biasing the user).

   Future-based(pointless, people are not good at predicting the future): So get them to complete the task in the here and now: Show me how you would book the flight. Don’t waste time asking them about their potential behaviours in the possible future. => Ask about specific things they did in the past (last time they booked a flight online: where were you going? Who were you going with? What were the dates? How long did it take you to complete the booking? By zoning in on the specific details of the task you will begin to paint a picture of the problem. But not too much detail. They won’t remember buttons they clicked. You should keep your questions relatively high-level.

   Design-related questions: What would make this screen better? What do you think of the colors? How would you arrange the icons? All you will get are answers about topics that most people don’t have strong opinions about. Design is our job, what we are doing now isn’t design. We are doing research. We need to understand the problem and later based on the data we found, we can design a solution. => Instead keep the questions focused on what the problems are now: Show me how you would book the flight. And the design will work itself out. Asking for page design feedback (will not produce good data). The best way to get feedback on your screens is to observe your users using them. Then you will be able to assess if they perform well or not. All you will get by asking these questions is opinions of people that don’t know much about design and most likely don’t have strong opinions on the topic.

   Asking hypothetical questions (would you use our product if we added these three features?) If you had a discount, would you use it? => Instead we need to find out how the software is working now and what we need to fix.

   Asking for product feedback (especially if it's a startup): what do you think of my product? Do you think it is a good idea? Would you use it? But asking for product feedback isn’t the purpose of the interview. We are trying to understand the problems our product can help solve. These are two completely different things. A lot of startups fail because they ask the wrong questions during the research phase: would you use the product? A lot of people say yes and it could give the founders a lot of false hope. So, do not ever ask for product feedback during interviews, instead, try to understand if people have a problem that your product can help them solve. And then it is up to you, the product designer, to decide if it is worth solving the problem and creating the product. Also do not ask about people’s future (would you buy it?) - people are not good at predicting the future. Despite the fact that people said that yes they will buy - no one virtually did.

3. Avoid assumptions

Assumptions on their own don’t carry any weight. They are nothing more but ideas in our heads (that are not necessarily based on any factual information). We know our industry way too well. We know our product inside and out. It is easy to assume that we know what our users want, what they would find difficult or easy, enjoyable or frustrating. But what seems obvious to us, may not be obvious to our users. So, when you feel like you have a question about your product that you don’t have an answer to - don’t assume the answer. Instead, do a quick round of usability testing and observe your user actually using the product. Of course, sometimes assumptions are inevitable. But they have to be supported by facts, test results, research results etc.

4. Don’t interview your colleagues (or any person who knows your product well)

Just like you, people in your organization know too much about the industry and can bring in bias into the design decisions. If you listened to all the suggestions made by the people you work with and incorporated them into the design, then you would be designing the product for them! But we are designing for the user.

You might hear people say to “If I used this, I would want…”, “I don’t like this feature…”, “I think we should add this feature…” Even if it is your CEO that says these things to you, do yourself a favor and avoid falling into that trap. As a UX researcher (or UX designer) it is your responsibility to advocate for your user. Say: “Thank you for sharing your opinion. We will look into that.” Make a note, run a round of research and see if your colleague’s assumption is confirmed. But always remember: your co-worker is not the target audience of the product. Your boss is not the target audience of the product. You are not designing the product for them. You are designing the product for your user.

5. You are not your user!

Again: you know way too much about your business, about your product, about what went into creating it, reasons for having certain features and skipping the other ones. Most importantly, you know your website/app/software. You know how to use it, where to go, what to click. Even if you have a product aimed at a very broad audience, you, of all people, are not the user. So, go ahead and do some research. It allows you to see the product through the eyes of your actual audience, which is very important at all stages of product creation.