At Advancing Research 2020, co-chair of the ResearchOps community, Brigette Metzler hosted a session in the Dovetail lounge to chat about ‘Research in the time of COVID: How libraries can help’. Below is a condensed version of the discussion.
My name is Brigette Metzler and I’m the Lead User Research Librarian for Services Australia. At Services Australia, we designed and built our library. I certainly have a strong perspective that libraries are really helpful and hopefully, you do too.
I’m lucky enough to co-chair the ResearchOps community with lovely Holly Cole and 11 community directors. The ResearchOps community is a group of about 5,000 people from 62 countries. We talk about Research Ops, and also research of course, and advancing research.
Just in the past year, outside work, I’ve conducted about 120 meetings, workshops, chats, and interviews about Research Ops. About 30 of those were interviews I did with people leading research and research operations for a global project on research repositories/libraries that we’re doing in the community. We have spent a lot of time understanding the lifecycle of research and research ecosystems within organizations.
I believe that pace layers can help us conceptualize and understand how we work.
Fast learns, slow remembers. Fast proposes, slow disposes. Fast is discontinuous, slow is continuous. Fast and small instructs slow and big by accrued innovation and by occasional revolution. Slow and big controls small and fast by constraint and constancy. Fast gets all our attention, slow has all the power.
This is a quote by Stewart Brand from 1999 from the Clock of the Long Now project. The important point about the quote is that “fast learns, slow remembers”, and that’s pertinent to libraries, and pertinent to research.
If we look at those pace layers that Stewart Brand put together, I’ve married them alongside research because I think we can use pace layers to help us understand research, and also understand how to manage research as a librarian.
Often we hear the complaints that researchers using slower and more in-depth research methods, like generative and generative longitudinal research, often struggle with constant pressure to reduce the cadence of their research. I think now more than ever, we’re probably seeing that pressure reach gigantic proportions. Using pace layers to build understanding helps us to communicate the value of the slower deeper layers, and to identify the friction between the layers.
On the other hand, evaluative research is the one that makes all the noise. It’s quick, it’s high demand, and it’s high turnover. Researchers doing evaluative research often struggle with people saying that what they’re doing isn’t really research and that it has no value.
These are the eight pillars – this is how we understand the connection between Research Ops and research, and you can see knowledge management plays a large part.
When we look at the layers from an Ops perspective, you can see that knowledge management is hugely important in the generative and generative longitudinal layers but also in the descriptive and evaluative research. It’s important in all of them but the key thing I want to get across is the difference in how we respond to them in each of the respective layers. If we can put them together then it helps us to make a strategy for scaling research. As a librarian, if I put them together it helps me understand things like research shelf life, what level of data management I need to apply, what the problems are, and how to manage those things more effectively.
A key thing I’ve learned from the research repositories project is that there seem to be two types of libraries. One is a research library that sees research as an asset and the other is sees research more as evidence. There is a very clear delineation in the way these libraries are set up and what they need when putting them together.
This is a case study of a team of about 50 researchers who are doing mostly longitudinal and generative research. The more contextual the research, the greater the use and reuse capacity of the research.
This team is more focused on managing participants, consent and ethics, and probably wanting to run a research library. Their environmental challenges are about time. So they won’t need to evangelize research as such, but they have a lot of pushback about the time that it takes and the subsequent cost.
They manage the melody of long and slow with the needs of the business through a rigorously managed panel, good participant experience, and by building their base of research to a level that others can dip into it as needed.
This is where libraries come in. All of that contextual data generate hefty needs around consent and data maintenance, so they’re likely to want a digital librarian and a great relationship with their legal team. That contextual research is a super-power however, and so the value proposition for them in terms of libraries is seeing research as an asset that can be reused. Their research data is something that they add to over time, and grows in richness as they do so.
Reuse of this ever-growing dataset is where their libraries can show the most value and so, we see this reflected in the effort and investment in their libraries.
Many tech companies tend to lean towards fast research and are much more likely to have decentralized researchers working across design, development, and product teams. They’re wrestling with time constraints, getting insights to the right people when they need them, and still want to concentrate on research as a team sport.
With this group, there’s a heavy focus on tools and infrastructure, and their needs are quite different when it comes to a library. Consent is lightweight, most of the research is de-identified right from the start, they’re generally not thinking about a library, and if they are, then it’s held in whatever system the developers use like Confluence or Jira.
Modularizing research is tempting, but also troublesome. It’s hard to put aside that kind of effort for research that’s generally valued for evidence rather than as an asset that can be reused. So in times such as these, that evidence is important because you’ll be moving super fast trying to respond to a brand new situation. You want to know why you made the choices that you made and you want a trail.
I’m assuming that we’re all under a lot of pressure to get research done really quickly these days and doing it online is even harder. So where do we start? My advice, whether you’re buying a platform or creating a library yourself, is to start with a small taxonomy. In all of the interviews I’ve done, everyone has said “Oh, we just went too broad with a taxonomy and now we’re scaling it back”. As a person who’s done that myself, I can say that’s just sort of what happens.
If you can, try to reuse other people’s taxonomies and vocabularies. There are a bunch up on the web. One of my favorite ones is Sage. Sage has a fantastic vocabulary that you can actually hook into. It has what research artifacts are, and they’ve got terms for going and doing the research and then collecting the research. They’ve got heaps and heaps of terms that are already defined that you can reuse.
Uber have done a lot of presentations about Kaleidoscope. The key takeaway for me is that they set time aside to work on their taxonomy as one of the first things they did. You’ll find the same with Tomer Sharon and Polaris. The first thing they did was sit down for three months and work out the taxonomy. Even if you go and buy a product (like Dovetail), one of the first things you should do is sit down and work out your taxonomy.
With HITS at Microsoft, we don’t get to see their taxonomy but, and I’m pretty sure we’ve had this conversation together before (with Matt Duignan), their taxonomy also went too broad and came back and refined over time.
Primarily, when we’re discussing taxonomies, we are looking at a few key things. We’re looking at find-ability and filter-ability. Having a taxonomy allows you to tag stuff and then also allows you to understand how you understand your research by way of forcing you to develop a conceptual framework. It’s an opportunity for you to connect your research, as a data set, to other data sets.
A great place to start is to think about what types of data sets already exist in your organization, how they interact with the work that you do in your organization, and what words can be used that will be easily identifiable internally. So, most of all, the act of actually defining what you’re there for, what you do, and how you understand what your research is. You’ll always think it’s such a waste of time defining the user journey, but let me assure you, three years down the track, you’ll find it really, really important.
Aim for simplicity in these times. You don’t actually need to have all your research in the one spot, you just need to know where it is, how to find it, and what’s in it. In some ways, you need the tags more than the research, which is probably a heretical thing to say. But if you can, try to make a habit of recording where it is. Ideally, you want to move your data to a central place to query, or if you’ve got a record of it tagged in a certain way in a central place, to ensure you can query it and discover it.
Aim to avoid creating more research debt. I know this is a hard lesson. When starting a library, you should expect to go too big at the start because that’s what everyone does. Getting that balance between the ease of adding, ease of finding, and ease of reusing is hard. People want to use their own words, they’ll want to make a million tags, and they’ll want a lack of structure. Then the people who are creating the data sets—your librarians—they want that whole picture of research, they want more tags, but they also want extensibility so there’s a tension there. They also want all that goodness that’s a human-centered conceptual model. View that taxonomy as the biggest opportunity your organization has to become more human-centered.
Analyze data, collaborate on insights, and build your research repository.Try free for 7 days
ProductData analysisResearch repositoryCollaborative researchVideo and transcriptionCustomersIntegrationsPricing
Use casesUsability testing analysisCustomer interview analysisNet Promoter Score analysisSurvey response analysisSupport ticket analysisRemote user research
© Dovetail Research Pty. Ltd.
Made in Australia