Ghosts of my research (repository) past
Finding answers when I was looking for other answers
Believing I was looking into why research repositories irked me so much, I learned one reason why I so often struggled leading UX research at my organization.
When I first heard about them, research repositories did not appeal to me, to say the least.
The basic definition I heard then was along these lines: a research repository allows people in your organization to find and reference either entire past research reports or individual pieces of data from studies.
Either way, to have one at our organization, someone (implied: me) would have to maintain the repository to make it possible for our peers to find the reports and data.
I’d have to organize it, describe the items, and put them somewhere that people could access them. I’d have to explain how to get access to the repository, and I’d probably have to do trainings, and I’d definitely have to answer questions about the process and the tool as they came up. I would try to involve my research and design teammates in these efforts, too, and that would be an effort in itself. We wouldn't get something for nothing; I would have to invest additional effort on top of my other responsibilities.
I don’t mind effort! We call it “work,” after all. And I’m an information architect, for goodness sake. You know I love a well-organized database (or spreadsheet, for that matter).
But the benefits of research repositories, as I understood them, just didn’t seem enough motivation to invest that effort.
I was not interested in making individual pieces of data (think: quotations) available from studies. I believe that a participant’s action is only meaningful within the context of a particular study. When I select participants, I select individuals whose relevant experience and behaviors balance one another. This is how I get as complete an understanding of the research topic as I can.
By comparing one person’s difficulty with another person's difficulty, with a third person's ease and fourth person’s ease, I start to see which reactions are because of the product and which reactions are flukes. I would never examine one moment from a study in a vacuum, or perhaps even worse, I would never compare a moment from one study to a moment in a different study as if the two moments were equal. To do so is to take away the very context that we design studies to provide.
Therefore, a research repository that made individual pieces of data available for mixing and matching, no matter how well tagged, was completely unappealing to me.
If a research repository, on the other hand, were about extending the usefulness of reports with their context intact, I could imagine that a research repository might be worth the effort.
So, in preparing to write about my long-held, yet poorly-informed scorn for research repositories, I set out to find some of the current thinking on them. Researcher extraordinaire Steve Portigal replied to my tweet, recommending a report by the ResearchOps Community. It turned out to be a excellent introduction to their much needed work figuring out what people have been doing with repositories and where they’ve found challenges and success.
Their definitions of not one, but four distinct kinds of research repositories brought me thoroughly into the “Ohhhhhh, I see, yes, this could be useful after all, in the right organizational context” camp. But what really got me thinking was a discussion of the organizational contexts themselves.
The article writers suggest that the type of research an organization tends to conduct can hint at what they hope to get out of their repository. “Organisations focused on evaluative research tend to value research as evidence,” and “[o]rganisations focused on generative research tend to see research data as a reusable asset for making decisions moving forward.” The way I interpret those phrases is this—
If a team mostly conducts usability studies, then they probably want their repository to show that they had reasons for changing the product the way they did and that they are making regular improvements.
If a team mostly conducts exploratory interviews, then they probably want their repository to capture meaning that can inform design and business decisions over and over again.
Absorbing and interpreting this distinction from the article, I now realize that I worked at our organization in a potentially transitionary moment.
We were mostly conducting usability studies, but we were longing to share conclusions that would inform decisions over and over again. This interest was showing in my peers’ curiosity toward research repositories, as well as in our newly started open research meetings and one peer's habit of looking through old usability study videos for new insights.
Instead of seeing the connection that the ResearchOps Community suggests between research method and use, however, our organization was trying to apply our usability studies to broader contexts, contexts they weren’t designed to serve.
I was uncomfortable with that at the time. I knew that our team was trying to use evaluative research as if it were generative research. But I hadn’t thought of the counterpart. I hadn’t thought of a valid reason one might reference usability research after its ability to support decisions had expired.
If I had considered these two values side by side—that evaluative research can serve as product history and that generative research can serve as product guidance—I hope I could have led more productive conversations about why we were using and sharing research the way that we were.
Regardless of how it would have gone, however, I feel a bit more at peace with my understanding of what happened. I wasn’t struggling because I wasn’t trying hard enough, and I wasn’t struggling because I was RIGHT and they were WRONG. It was because we hadn’t aligned our goals nor our understanding, and we didn’t know it.
And while that still sucks, I feel grateful that I understand it more than I did before.
Mentioned in this issue: The tweet where I asked for current resources about research repositories, and Steve so kindly replied; and the ResearchOps Community’s introduction to their excellent study on research repositories (don’t you love a study that studies research??), called “Research Repositories: A ResearchOps Community Program of Work.” Oh and, yes, indeed, the introductory sentence of this issue is an almost unrecognizable riff off of a line from Paul Simon’s “Gumboots.”
About this issue’s image: I planted a cilantro start in this pot earlier this year, but only its dried stems remain. The plant’s roots couldn’t absorb the water in the large pot quickly enough. Now, thriving at the edge of the pot is Genoese basil, grown from seed that fell from last year’s plant and remained in the soil until it had the chance to sprout.
Thanks for reading Finding Out! Subscribe for free to receive new posts and support my work.