The Challenges of Traditional User Research Method

Share the knowledge

User research is an important aspect of web, mobile and product design. Whether you’re working on a portfolio website, online store or web application, making your product intuitive and easy to use is key.

There are several tools to accomplish this, including surveys, phone calls, usability tests and more. Below we assess the tradeoffs and challenges with traditional user research methods.

1) Cost

Some traditional user research methodologies, like usability tests, can get very expensive. Companies typically pay each participants between $50 to $200 to participate in usability tests. Even online solutions like UserTesting costs about $49 for each participant. If you wanted 10 participants in a usability test, that can cost you up to $500 to $2,000. For startups and small businesses, this price might be too steep.

2) Time

Another challenge with some traditional user research method is time. For usability tests, expect to spend about three hours to prepare for a usability test, an hour for each participants and another 2 hours for each participant to analyze the results. For 10 participants, expect to spend about 3 to 5 days on a proper usability test. That’s 3 to 5 days you could spend selling, building your product or designing new features.

3) Response Rate

The third challenge with some of the traditional user research methods is the response rate. Surveys have an average response rate of less than 1%. That means that if you sent out 100 surveys to your customers, most likely only one of your customers will respond. You could increase this response rate with incentives (for example a $5 Starbucks gift card), but that increases the cost and potentially skews your results.

4) Quality of Data

One of the ways to overcome the challenge with low response rate is to share your survey with everyone – your family, friends and even strangers. However, you risk degrading the quality of your data. It’s key to only target qualified users to give you feedback. You need to make sure that the participants in your panel fit your product’s target audience persona. However, this takes time time and money to cherry pick qualified participants.

5) Data Analysis

Finally, after gathering both qualitative and quantitative data from your user research questions, you need to analyze the data to look for patterns and insights. You can use tools like cohort analysis, regression models and even semantic pattern matching. The challenge of this is that it requires non-trivial knowledge to employ and interpret results from data analysis tools like these.

Tradeoffs with Traditional User Research Methods

Each traditional method – like surveys, phone calls and usability tests – have advantages and disadvantages. Make sure to weight the benefits with cost, time, response rate, data quality and data analysis.

For more tips on user research and user feedback, subscribe to the ListenLoop email list below or follow us on Twitter (@ListenLoop).

Join Our Email List