A few years ago, I sat in a meeting where another researcher was presenting findings from a study. The room was filled with all kinds of people who use research to make decisions: designers, product owners, developers, and more. The screen showed one chart after another, sometimes graphed as bars and sometimes showing trend lines.
One of my colleagues raised their fingertips off the table to catch the presenter's attention, and spoke up. "That spike in views from two weeks ago is interesting. Do you have any theories about why it's there?"
I held my breath as I waited for the presenter’s answer. What our colleague didn't realize when they asked their question was that they were directing it to the wrong person. In the last two issues of Finding Out, I have shared stories with you of discovering what researchers can do for the people we research and for the people who use our research. Today's story is about the third group of people affected by our work: our fellow researchers.
My career so far has focused on leading studies where I directly observe the people who participate, whether I run the sessions in person or over the internet. Other designers attending the meeting had experience with this kind of research, too. We listen to the memories that our participants share, or we notice how they solve tasks. The stories we gather help us better understand why and how people do things.
If the motivation behind the increase in views was worth understanding, we were the ones best suited to find out.
Other researchers, on the other hand, focus on studies where they indirectly observe people. Just like the presenter in today's story, they use counts and timestamps and locations gathered with the help of computers or other instruments. These gathered numbers help us better understand what and when and how many people do things. While this data can help us narrow where to look for audience motivations, there are few contexts where that kind of data could thoroughly answer why a sudden increase in views happened.
These two kinds of research—often called qualitative and quantitative, respectively—are complimentary. Together, they can help us build a full picture of the way people behave. Unfortunately, I have rarely witnessed or taken part in collaboration between qualitative- and quantitative-focused researchers. Instead, perhaps through inertia alone, they exist separately at best and in competition at worst.
When I held my breath in that meeting room, I wondered whether this would be another occasion when the strengths of qualitative research would be overlooked, and suspicions would be treated as fact.
"I do have theories," said the presenter. "But I'm certainly not going to tell you them."
Gasp! I was amazed that the presenter refused to answer. He went on to explain that he hadn't researched why the spike was there, and that he didn't want to suggest anything without looking into it. Further, he said he probably wouldn't be able to determine the reason anyway, because we hadn’t set up the website to gather the data needed to support any conclusions about the spike.
What this presenter did required discipline. We humans love stories, and sometimes I wonder if we love guessing even more—even the researchers among us. But the presenter resisted that urge. And in this context, it was a service not only to the people who use his research and the people whose behavior he studied, but also to people like me who practice research alongside him.
While this story highlights the sometimes-rivalry between qualitative and quantitative research(ers), the tension created by the inadvertent stretching of research findings can happen in plenty of other ways. If a fellow usability study practitioner agrees to ask participants what colors they prefer, it makes it harder for someone else to refuse to do so in their own study. If someone who runs surveys agrees to include a poorly worded question, it makes it more difficult for another survey moderator to recommend edits to future questions.
When researchers instead do as this presenter did, and use our methods within their bounds, however, we make it easier for our peers to do the same. Together, we take the small, sometimes difficult steps to protect our practice. We quietly, one presentation at a time, lead in what it means to find out.
Thanks for joining me for the third of four Finding Out issues introducing the roles of people in UX research. Next issue will conclude this topic for now by discussing how all three groups fit together. If you missed the earlier two issues and want to catch up, check out the archive of Finding Out.