New!
A new way to tell stories with video highlights and transcription
Learn more →

Cool, calm and collated: Getting organized with your qualitative research data analysis project

31 Mar 2020Research11m read

Creating order from chaos is part of every researcher’s job description, so it’s no surprise that we’ve also developed some tips and tricks over the years for systematizing our work. Here are some examples of how people in our design research community get themselves organized.

Get into a good rhythm and the right mental zone

As knowledge workers, our most important tool lies between our ears. Working to a research cadence and establishing helpful habits and routines makes our work more sustainable and successful. These could be small things like daily routines that keep us on track, or big picture priorities that make us better researchers.

Establish a cadence. It’s easy to focus on the task at hand, but more difficult to juggle other functions that contribute to a successful analysis project. Having regular and frequent touchpoints helps retain focus and provides a steady stream of progress. Product leader Wren Lanier, says:

Setting up a weekly cadence accelerated our learning and built momentum behind strategic experiments. It forced us to keep our work small and iterative.

Use repetition and task order. Having a routine for things can help form healthy habits, and be a soothing way to trust your competency. Setting flexible traditions is key for reflection and maturing.

Clear your mind. Turn off distractions as much as possible when you need headspace to focus. It’s not just about efficiency, it’s about mental health too.

Interruptions driven by task switching and an incessant stream of data have high psychological costs.

Dr. Gloria Mark, Professor of Informatics at the University of California, Irvine, says:

These are both stressful and costly, they cause higher levels of stress, frustration, mental effort, a feeling of time pressure and mental workload.

According to a recent study she conducted, it can take an average of 23 minutes to regain focus on a task after being called away.

Mental ability and focus-time are precious commodities during analysis, and it helps to shut yourself away in a room so you can give things your full attention. Write down any stressful thoughts looping around your mind, reduce web usage, resist your devices, ignore your emails and message alerts, silence your phone, and stay away from social media. Everything you need for doing your analysis exists within these four walls.

Make time to think and reflect. David Travis has spent 20 years training and coaching user researchers and noticed that there was one particular behavior that stood out amongst the most successful user researchers.

They didn’t just attend training courses and do the work day-to-day. They consciously and deliberately reflected on their work. To really learn from our experience we need to analyze it, deliberately and consciously.

Reflection is powerful because experience alone does not always lead to acquiring skills. Thinking critically about your work as a researcher and analyst is about assessing your performance. Articulate and record your experiences and thinking in writing, such as a journal, and revisit it regularly. Ask yourself some revealing questions. Why did you do things that way? What theory underpins that approach? Are there alternatives you could have used?

Make intentional plans. Take fifteen minutes at the start of the day, or the end of the day before, to make a rough daily plan for your research and analysis tasks, and to understand what you’re aiming for. It makes it easier to assess and adjust things when new priorities inevitably develop.

Man looking at wall

Collate to make your data visible

With so much raw data, things get messy and it’s easy to lose sight of what we’ve collected (or what we’re still missing). Gathering it all together and storing it in one place makes it much more accessible and approachable.

Set up a standardized digital filing structure. Having a directory for each research participant usually works best. It’s an object-oriented approach that allows you to quickly and easily group things by their cohesive unit—the people you’re learning from. It’s not the only approach you can take, for example, you might instead consider filing things by date.

Alexander Khmelevsky from design agency Clay, says:

If the work is coming in on a daily basis, the results should also be so. That’s why we store daily results in a folder named with the date, like ‘2018–11–25’. This way, the daily research material can also easily be assembled into a presentation.

Collate your materials. There will be documents, session notes, observations, drawings, artifacts, photos, videos, and email correspondence you’ve amassed from your various primary research methods. Don’t forget to include academic literature and website clippings if you’ve conducted secondary research too.

Include relevant participant data. Make sure you have a participant profile that allows team members to quickly scan a participant’s key traits, even if recruiting screeners, permissions and personal data are kept safely elsewhere. You’ve got a natural behavioral persona right in front of you, the starting point for further thinking. You might include demographic or segmentation data too. Be very mindful of your GDPR obligations and practice good anonymization or pseudonymization techniques to ensure compliance and respect for privacy.

Study and convert field notes. Rewrite and expand the entries you scribbled during sessions or in breaks between participants. This is a good opportunity to edit them for clarity and accuracy, and to add comments that articulate how you understand things.

Transcribe audio recordings from interviews, contextual inquiries, walkthroughs and other events in the field. Check the transcriptions for any errors, and edit them for accuracy. Add comments and tags where these will help clarify and categorize. You’re on your way with analysis already!

Photograph and transcribe handwritten journals and diaries and other image records into indexable and digitally searchable formats. Having a digital image serves as a sharable artifact, and as a backup. If your research repository includes the ability to import and tag images, you can start doing this too.

Download open-ended responses from surveys and other qualitative inputs. Online tools often allow CSV or spreadsheet exports that give you full access to the raw data. Clean these up by beginning the process of filtering, summarising and labeling, while protecting the integrity of the original data itself.

Use the power of your team’s collective mind

Caitria O’Neill, a senior UX researcher at Google, jokes:

You’re tired. You just interviewed seven people back-to-back in a small room lined with one-way glass. You’re already starting to lose track of who said what. Your team is in the observation room, hangry and quickly losing focus. This is a perfect time for a debrief.

Debrief together. The analysis of qualitative information shouldn’t be a solitary activity, and this begins right at research collection. Consolidate your notes together, and as soon as possible after running research. This helps the team learn from each other, care for each other, builds consensus, and helps triage any emerging issues.

Team debriefs work best when someone takes charge to facilitate the conversation, but teams will quickly learn to work together.

Look for meaning, not just facts. The goal is to explore how fresh inputs from the research contribute meaning and understanding, more than just regurgitating what happened in the session.

O’Neill says:

It helps the group move past superficial insights to actionable insights.

So, for example, rather than noting that a participant ‘didn’t like the button’, move towards thinking about something more insightful, such as ‘why people were worried about leaving the site’.

Discuss the key things you’ve found, and talk about any highlights and patterns that might indicate an insight being formed from various findings. Capture and structure all of this collective thinking on whiteboards, or on walls with sticky notes.

Use collaboration to start documenting. You might also share things asynchronously with a research collaboration tool that supports your team’s way of working. Popular UX research collaboration tools include Evernote, Miro/RealtimeBoard, Trello, Slack, Google Apps, and Dropbox.

Think individually, act collaboratively. Working together enables sense-checking amongst the team of people conducting the research. They act as sounding boards for possible interpretations and opportunities for further research, which leads to healthy cross-pollination of ideas and the raising of new approaches. It’s easier to align on these things if you’ve had debriefs after each participant. It keeps everyone abreast of the latest data, provides a reason to summarise learnings, and is often a time where observations and highlights suggest tags or themes that are emerging.

Establish a command headquarters for your research

Having a thinking space of your own will save the need to pack down and set up every day. Making it your research happy place will help you switch into research mode the instant you enter the door. The physical layout actively encourages your latent synthesis skills.

Think of it as a ‘mind palace’, a Sherlock-Holmes-esque aid to memory and data visualization. Make this a physical space that presents an ordered version of what you carry in your mind. It will act as an evolving mosaic that documents your exploration and growing understanding.

Jon Kolko, the founder of the Austin Center for Design, explains how abductive thinking and sensemaking are key tools in reaching synthesis.

The quantity of data that must be analyzed is often too large to hold in attentive memory at one time, and so a designer will externalize the data through a process of spatialization.

Simply through arranging the information physically, we can go a long way towards finding and making novel connections.

Reserve a work area. Having a dedicated place you’ll be comfortable in, such as a meeting room or board room, a cluster of desks in the corner. It should have plenty of room to lay things out: tables, sideboards, walls. Air circulation is good! Book it for at least a week and make sure it is somewhere your data can be left alone without needing to pack down every day.

Assemble a toolbox of useful aids. Thinking tools such as index cards, Blu Tack, pads of sticky notes in a couple of different colors, small sticky tabs for labeling, highlighters, sharpies, scissors, sticky tape, and any other stationery that you find personally helpful for getting data up and out and categorized into a physical space.

Prepare your blank canvas. You want lots of flat work surfaces, whether they be glass walls, windows, whiteboards, foam, or cork. Some people use interactive whiteboards as a useful way of straddling the physical-virtual divide.

Have your collated research data on hand. Easy enough with a laptop and a printer. It can sometimes help to have almost everything printed out, not just transcripts, but that is often pretty wasteful. And if you do bring your laptop into the war room, watch out for the potential distractions!

Supply brain food. Pound for pound, your brain uses more energy than any other part of your body, and while it doesn’t necessarily use more power when thinking than when resting, it’s still responsible for around 20% of your body’s overall energy use. Thinking can be tiring! For good memory, concentration and brain health, you need to sleep well and have a good daily diet, but for quick boosts to your alertness, mood and concentration look to caffeine, berries, seeds, nuts, and chocolate for good options.

Expect to go a bit bonkers. You’ll be so deep in engaging with the data that you’ll come out clutching a sentence clipped from a printed transcript, eyes gleaming, and wonder why others aren’t quite as excited about the insight you’ve just surfaced that is destined to ignite the rest of the project.

Soccer goal on water

Stay connected with your research purpose

A clear goal for your research allows you to focus your questions and investigations and, in turn, produce specific themes in what you learn. You can use this kind of focus to predict likely tags you might connect with relevant parts of your notes and transcripts.

Nikki Anderson, the founder of the User Research Academy, understands the importance of good objectives to good insights.

Since I have started really paying attention to, and honing, my research objectives, my interviews have visibly improved and my confidence in interviewing has greatly increased. Besides that, it is a fun exercise to deconstruct a problem statement into smaller puzzle pieces that move you toward your goal!

There are many rabbit holes to explore, and you want to ensure you pick the right ones that maintain your focus. Go back to your research goals. Trace them through to the questions you asked in research sessions and, in turn, to the answers you received. Think ahead and continue these threads in the way you tag and label your research as you begin analysis.

Move forward by reducing and reprioritizing data

Joy Frechtling, an expert in mixed-method evaluations at the National Science Foundation, says:

Beginners often fail to understand that even at this stage, the data do not speak for themselves. A common mistake many people make in quantitative as well as qualitative analysis, in a vain effort to remain ‘perfectly objective,’ is to present a large volume of unassimilated and uncategorized data for the reader’s consumption.

A good argument for ‘less is more’, as long as your clippings and summaries are tightly aligned with your research purpose. Data reduction, sometimes called ‘data concentration’, involves selecting, focusing, simplifying, abstracting and transforming your raw data. For manageability, the amount of data needs to be condensed and, again, this should be guided primarily by your research goals. Qualitative data have a personal association for the researcher beyond just the words that have been recorded; they represent real people, places, and events so it can be difficult to sort and discard things. The acid test must be how the pieces address the salient research questions.

Maintenance required

Maintenance is required to stay on top of your organized research. You don’t head to the gym for a workout, achieve fitness, and then stop exercising. Being organized is the same.

Rather than waiting until after you’ve finished collecting all your research data, it’s good practice to store and organize things from the get-go. As well as satisfying the neat freak of your inner librarian, you can use it as an opportunity to begin analysis early.

At the end of each research day, or even at the end of each session, practice good data organization practices. Store and organize as you go, including getting stuck into analysis tasks while you’re still researching.

Start a free trial

Analyze data, collaborate on insights, and build your research repository.

Try free for 7 days

Try now for free

Try free for 7 days

Trusted by companies likeSee all →

shopify.comsquare.comsketch.comgitlab.comvmware.com

Product

Data analysisResearch repositoryCollaborative researchVideo and transcriptionCustomersIntegrationsPricing

© Dovetail Research Pty. Ltd.

Made in Australia