Seven questions to ask when determining how to invest in immersive user research
Most designers I know feel they have to advocate, bargain and in some cases beg to do immersive user research. This sentiment exists because product/client team members have different opinions about:
- The value of immersive research.
- The amount of time and money that should be dedicated towards research.
- The most appropriate research methods given a particular problem space.
- The most appropriate time in a product’s life cycle to conduct research.
The above bullets were distilled from both personal experience and the “Competing Forces,” section in the 7th Chapter of Don Norman’s The Design of Everyday Things: Revised and Expanded Edition.
With this post, I’ll discuss some of the questions we (Handsome) consider when determining time, budget and methods around user research engagements. These questions center around, Leanness, Documentation,Empathy, Harm, Quality, Beauty and Buy In.
Leanness: How much money and time do we have to work with?
If you’re pressed for time and money, here are some practical suggestions that can help conserve resources.
- Don’t pay participants. You’d be surprised how many people will talk to you for free. If you’re not convinced here’s an article highlighting how paying people can decrease satisfaction (known as the over justification effect).
- Don’t hire a recruiter. Hiring a recruiting agency can cost between $75-$300 per participant. Non-expensive ways to recruit participants include: emailing friends and family, @replying on twitter and walking up to strangers in public places (I’ve done all of these). Also, I suggest getting involved in the design community in your city. Most designers are sympathetic to user research on tiny budgets and will open up their address book if you ask them nicely (even if they don’t know you that well). One downside to not hiring a recruiter is that finding participants can take an excessive amount of time.
- Interview on the weekends. When you’re not paying people, it’s important to conduct interviews during times that are most convenient for your participants. Don’t be afraid to suggest meeting on weekends. I’ve found prospective participants will decline to meet before suggesting a weekend time.
- When going back through audio recordings, take notes directly on sticky notes. Instead of writing down data-points from the audio recording on an excel sheet, tagging individual cells, then going through the excel sheet and transferring those data points to something that can be externalized and manipulated (like sticky notes)…go through the audio and write directly on sticky notes.
Documentation: How will we keep track of what we’re learning?
Documentation in research is important because it codifies what has been learned and what is believed to be true in a concrete way that is visible to the entire product team. Teams who do not document their findings run the risk of making costly mistakes as a result of forgetting what was learned during earlier product stages. This is especially true of companies where new designers and managers are hired who do not have the tribal knowledge of the founding team.
The keys to effective documentation are:
- Visibility: Is everyone on the product team forced to take note of the artifact in a way that facilitates a significant depth of processing. For one client, I posted pictures of personas on the office refrigerator.
- Value: Does the product team see the artifact as essential to building a great product? To communicate value, it’s sometimes worth tactfully showing a product team how a lack of understanding of users has lead to bad product decisions and how a shared understanding of users will lead to good product decisions.
- Evolution: Documentation should evolve as a product team’s understanding of users becomes more refined as users engage with the actual product. To keep teams accountable to evolving their documentation, I’ve found it helpful to build a checklist of reflection questions that will help teams determine whether an artifact meant to provide insight into key users should be updated.
Empathy: How are we going to cultivate empathy in our design team?
The primary role of research is not just to identify pain points to be eased. It’s also to create a shared experience that grounds the product team in empathy. (Wright & McCarthy, 2008; Kouprie & Visser, 2009). I love the quote below from Jeremy Rifkin where he asserts that empathy is not just something that’s nice to have but rather the substance for rational thought:
If empathy did not exist, we could not understand why we feel the way we do, or conceptualize something called emotion, or think rationally. Many scholars have mistakenly associated empathy with just feelings and emotions. If that were all it was, empathic consciousness would be an impossibility. Reason, then, is the process by which we order the world of feelings in order to create what psychologists call pro-social behavior and sociologists call pro-social intelligence. Empathy is the substance of this process.
(Jeremy Rifkin, An Address Before the British Royal Society for the Arts).
Harm: How much harm will we cause if we get the design wrong?
If you’re designing something to be used by individuals where the unsuccessful execution could lead to significant emotional or physical harm, it’s wise to invest more in up front research. This is often the case for designs in the medical, transportation, non-profit and military space.
“First do no harm” is a good starting point for everyone, but it’s an especially good starting point for designers. We have to remember…every move, every decision; every curve we specify is multiplied—sometimes by the thousands and often by the millions. And that every one of those every has a price.
(Alan Cochinov, A Manifesto for Sustainability in Design)
Quality: What threshold of quality do we need in order for customers to experience core product benefits?
Making an investment in research can often lead to a higher quality minimum viable product (MVP). Some companies assert the first product should always lack in quality (otherwise you’re not moving fast enough). However, this lack of quality is detrimental if the users cannot experience any benefit because your team will learn very little and your users will be irritated. (Summarized from Eric Reis’ The Lean Startup).
Beauty: How beautiful do our research deliverables need to be?
As a design researcher, I love putting together decks in Keynote that have full bleed images with provocative user quotes perfectly positioned in the shadows of said photo. While dedicating weeks to make this type of deliverable beautiful can sometimes help sell design ideas, it’s been my experience that the investment can be overrated. After the initial presentations and applause, the huge printouts of photos go in a landfill and the beautiful deck that costs anywhere between $1,000-$20,000 is put “on the server.”
Our goal is not to create a deliverable; it’s to change something in the world—to create an outcome.
Buy in: How many people on the product team will have regular intentional contact with users?
The more people on your product team who regularly connect with actual users, the less you need to invest in explaining the value of research or the validity of research insights. With everyone having contact with users, it’s easier to develop a shared understanding of the problem space and thus efficiently make judgments around the most appropriate user-focused design.
I used the words “intentional contact” intentionally. It’s critical the team is aligned on research goals and to collectively commit to a rigor around how those insights are gathered and synthesized. Sometimes individuals and teams can fall into the trap of only doing “research” to prove they are right. To make sure your team is aligned for all research initiatives, I suggest the following:
- Create research plans, discussion guides, and task sheets that include: the goal of each initiative, questions that will be asked and initial hypothesis/assumptions of the researcher.
- Record the audio (and video if possible) of all interviews in the event a test/conversation results in feedback that seems inconsistent.
In summary, research is messy yet critical. It needs to be executed with intention but does not need to be an unwieldy investment. It needs to be appropriately documented but not excessively ornate. And it needs to be something the entire product team commits to conducting regularly.