New
Why we raised $4M and what we’re planning for 2020
Learn more →

Research repositories in 10 minutes

20 Mar 2020Research9m read

At UX Australia Design Research 2020, Dovetail’s CEO, Benjamin, presented a talk called ‘Research Repositories in 10 minutes’. He spoke about what a research repository is, what problems they are trying to solve and covered some principles to guide decision-making if people were planning to set up one for their organization. Below is a lightly edited transcript of the talk.

What is a research repository

What is a research repository?

It started in 2017 with the article Democratizing UX published by Tomer Sharon, the then Head of User Experience at WeWork. In this article, which went on to become extremely popular, he coined the terms atomic research and nuggets. A couple of years later, Matt from Microsoft published an article called ’How Microsoft’s human insights library creates a living body of knowledge’. This was released in June 2019 after Microsoft built its internal system, HITS. In the article, he talks about collaboration with researchers and how insights should be created throughout the project rather than a big cliff to climb up at the end of the project.

The period of June, July, and August of 2019, was a very hot time for research repository stories. Sarah from GitLab wrote ‘Why we built a UX Research Insights repository’ and then Etienne from Uber wrote ‘The Power of Insights: A behind-the-scenes look at the new insights platform at Uber’, Kaleidoscope. Rosenfeld Media also held a video conference on the subject and Etienne shared that Kaleidoscope was built in people’s spare time over months. So these are the internal tools that organizations have invested in. After hearing about these platforms and also talking to our customers, we landed on the following definition of what a research repository really means. We think it’s a centralized, searchable database of research and insights that the entire company leverages to make better decisions. The whole point is to allow everybody to uncover user research insights and explore the context that led to them.

Why set up a research repository?

The question is why does everyone seem to want to set up a research repository and what exactly are the problems they are trying to solve? After talking to our customers and reading through the articles, I’ve summarized some of the problems to share with you.

Problems repositories try to solve

Problems repositories try to solve

Teams are siloed

Research is conducted differently across teams and departments, especially in large organizations. Different methodologies, tools, and formats for insights can cause silos. Here’s a quote from a customer interview we conducted last week:

We want to bring research through the entire business to all of the fragmented areas so that we can cross-pollinate our research. So, really about trying to break down those silos between teams.

Research is repeated

New team members join and ask the same questions over and over. Similar projects are run repeatedly. Sherif, a Product Manager at Atlassian said:

The problem is how to help team members build a key insight that existing members have realized. The cost of relearning is expensive, especially in growing organizations.

Reliance on tribal knowledge

Undocumented knowledge means you have to know who to talk to. When that person leaves the organization, they take their knowledge with them. We refer to this as tribal knowledge, and relying on it in any organization is dangerous. Another quote from a customer:

We want to stop relying on one person who has been here for seven years. People in your company are known as know it alls and often pestered for coffee.

Reports aren’t standardized

Because everyone works differently and we’re in a gestation period for the discipline of user research, research reports aren’t standardized. We’ve seen Powerpoints, blog posts, Confluence pages, and more. Without a consistent format, it can be difficult to leverage the insights from the past. This is what Tomer Sharon was talking about – having consistency for the formats makes it easier to store, search, and leverage that past research.

Data is spread across tools

Lastly, research data is spread across a variety of tools. Most people are familiar with Google Drive, Dropbox, Airtable, Word documents, and of course, sticky notes. While these tools are all great in their own right, they are limited in that you can’t really search them. When they pull that wall of sticky notes down, they are gone. Sticky notes also require people to actually be in the room, which is not so good for remote work. Another customer indicated:

We had developed a treasure-trove of research over the years and felt limited by traditional file-sharing platforms which only allowed us to put the research into the one place but not explore and digest the information in an intuitive and user-friendly way.

These appear to be the most common problems that motivate people to implement a research repository. We also found an article from Jakob Nielsen of Nielsen Norman Group from 2005, indicating that when people have to locate existing usability reports, they’ll fail, or they won’t know what to look for because there is no single place that lists usability results. Worse, if past project owners are the only source, you risk losing this when they leave the company or their roles are reassigned. While in this example, Jakob is specifically talking about usability reports, he predicted the whole trend 15 years ago, talking about the lack of a single place and the institutional memory when people leave the company or move projects.

5 principles to guide your decisions

5 principles to guide your decisions

Let’s say you’re sold and you want to set up a repository, and you feel your organization will benefit from it. Here are a few principles to keep in mind to guide the process.

Principle 1: Retrievable

We all know that researchers are busy and often outnumbered. The whole purpose of a repository is to help scale a research team to service the entire organization. This means that the repository has to support the ability to retrieve answers to questions in a self-service way. As a tangible example of that, something like search should be effortless, free of complicated queries, filters, or views.

Principle 2: Approachable

We also know some stakeholders don’t feel comfortable interacting with research, especially if there is clunky software that requires training. We should also encourage everyone to make data-backed decisions. The head of research for one of our customers shared their Dovetail project with the board of their company, like a big public company, and had the board kind of playing around with it. She recounted that it was a huge win because the members of the board felt connected to their end-users.

Principle 3: Traceable

Evidence helps to build trust in the research team. I’m sure you are familiar with bringing in a stakeholder—for example, a developer—into a usability testing session or interview and seeing their first-hand reaction to observing a user. Traceability between the insight and where it came from is very important and helps to keep biases in check by providing that extra context for people to explore the data on their own.

Principle 4: Accessible

We know that researchers want to build participation to create empathy with end-users. It’s a very common thing that researchers like doing, but it’s counterproductive if you have the research data locked away somewhere. The idea of the repository is to create a level playing field regarding access to data. Tangible examples are features like open by default permissions and one-click sign-on for easy authentication.

Principle 5: Secure

Lastly, we live in the age of regulation like GDPR and the California Consumer Privacy Act. Your repository will store sensitive data, so it’s important to consider storage policies, deletion policies, features like encryption and anonymization, and further requirements this space.

A few other things to think about

There are a few other things to think about which might prompt you to ask further questions. Things like, how will you get buy-in from the org? The repository won’t be successful only one or two people using it. Who is going to have access? If you are going to allow everybody to have access, will they create insights or just be viewers? How will you migrate past research data? Will you pick a time and say that from here on out we’ll be using the repository? What kind of data will you store? So, for instance - interview transcripts, video recordings, usability testing notes, maybe survey responses and NPS? And who will manage the taxonomy? And guard it and make sure it doesn’t turn into the wild west? Who will standardize and be the guardian of the data?

Benson Low, UX Research Manager at REA Group asked :

Have you seen the adoption of repositories grow in the last 12 months? There is certainly a lot of talk about them, but are you seeing their adoption?

In response, Benjamin indicated that the folks at Dovetail have been inundated with people talking about finding ways to set up their repository, looking for tools to help them out, etc. Most organizations can’t afford to do what the big guys have done and build their own internal platform. They appeared to be either big teams working on the project or longer-term part-time projects that the researchers initiated and received developer assistance to help them actually build-out. Organizations trying to solve their research problems with tools makes a lot of sense. It seems to be a reaction to the general rise of research inside companies. As a designer, Benjamin highlighted the parallel to what happened to design when designers managed to get a seat at the table.

Another question that was asked was:

When beginning to set up a repository and using a tool like Dovetail, do you have a suggestion about whether you should bring in the past information? Like research that you already have? Or does it make more sense to start from scratch? Doing the former and bringing everything, your historical data in can feel overwhelming what are your thoughts on that?

There are so many different methodologies and formats of data including video, text, transcripts, notes, survey responses, both structured and unstructured. Importing it and transforming it into a format that makes sense for whatever tool you’re deciding to use is difficult. We see a lot of customers just draw a line in the sand and say we’re going to start from here. Or maybe they do a gradual rollout and use it for interviews but not for survey responses. When importing data, often unstructured qualitative data, you end up realizing just how much you actually have. The question is whether it’s really worth it, which is up to each organization to decide how valuable that stuff is.

Start a free trial

Analyze data, collaborate on insights, and build your research repository.

Try free for 7 days

Try now for free

Try free for 7 days

Trusted by companies likeSee all →

shopify.comsquare.comsketch.comtarget.comvmware.com

Product

Data analysisResearch repositoryCollaborative researchCustomersIntegrationsPricing

© Dovetail Research Pty. Ltd.

Made in Australia