Are you rushing from field work to findings?
Author's note: This article is intended for 1) stressed-out researchers who need supporting material to share with their managers, 2) the managers, and 3) organizations that want to commit to research, but feel pressure to rush through data processing.
I listened not too long ago to a podcast by a well-known and respected design agency where they recalled a pivotal moment after returning from extensive field research. They started, as many do, by spreading all the collected quotes and observations onto big poster boards. Then they got a bit stuck. Overwhelmed by the amount of information literally staring them in the face, they cheerfully flipped their data boards around and decided to just go with what they remembered. It was great, they said. So much easier.
It’s always easier to stick with our preconceived notions, especially when we encounter even the slightest bit of support for them in the field. The unconfronted path is always faster. But being unchallenged, we learn nothing. The entire research process (or, in the case of the agency from the podcast, the data accumulation procedure) becomes pointless.
Spending time with your data is a critical step in understanding and making sense of what you saw and experienced in the field.
Here are a few ways I like to spend time with my data.
First: Be Accurate
I love spending time with my data. If at all possible, I listen to the audio recording of every interview, ideally within a few days, and amend the notes I took during the interaction. When time constraints just won’t allow me this luxury, my minimum acceptable re-listen rate is one to three. I pick one-third of the population and review their audio. My method of choosing is not scientific — if I was going to be scientific, I would insist on time for reviewing it all. Instead, I pick the “most interesting” (or sometimes most complicated) from among my sample.
Why is listening to the audio so important?
Bottom line, for accuracy. Memory is fallible. Don’t ever think it’s not.
Bonus point: “Re-experiencing” the interview this time as a passive observer is always eye-opening. Think about watching a movie. The first time around, you are captivated by events as they happen — you have no idea what’s coming next and the progressive reveal is (hopefully) very engaging. You are actively in each moment. The second time around, you already know what to expect. Little details you missed the first time begin to emerge. My favorite example here is the movie The Sixth Sense. I won’t ruin it if for some reason you’ve never seen it — but trust me that you miss a ton of clues about the big reveal on that first viewing. Now apply that phenomenon to an interview or participant observation session. Eye opening, right?
Don’t Forget: Making Introductions
It’s always insightful seeing how different people view the same thing. A team de-brief on each participant interview lets you see the data from someone else’s eyes. I favor the “user shrine” approach, of which there are many variations. I keep the rules loose on this: I draw an avatar in the center of an 11x11 sticky note (or sheet of paper) and bullet-point out the most interesting things. This is your chance to go with your gut. You get this one time. Making this activity collaborative means you can check each other.
Why is the team data introduction (i.e., user shrine activity) helpful?
First, if the full team was not part of data collection, this step helps get them onboarded. Second, if the full team *was* part of data collection, this step helps externalize all those little thoughts each team member is carrying around and assuming everyone else is thinking, too.
Next: Get Well-Acquainted
Did I mention I love spending time with my data? I read and re-read my notes. I read my teammate’s notes. I consolidate them both into an Excel document with annotations about why I think something might be important or what its deeper meaning may be (a process called “memoing” in grounded theory). I do this whenever I find a little time as fieldwork is progressing. You never know when a seemingly minor quirk from a participant visited early in the process will suddenly come up again with a much later participant. Being acquainted with your data will allow you to identify this similarity and follow that intriguing new thread.
At some point, you do need to filter out data that isn’t relevant to the particular problem you are working on. I curate very heavily when preparing for an affinity diagram. (The curation process deserves a post of its own.) This might sound like I’m making the same mistake as the agency group I pointed out who turned their boards around and went on memory, but there’s a few key differences.
- I’m not curating on memory, I’m making informed decisions, and …
- I’m going to revisit the full data set later.
All that information that gets left out of the affinity diagram is not discarded. I reread the consolidated spreadsheet notes later on when it’s time to create deliverables. Theses notes still have good stuff for building meaty personas, detailed journey maps, and for just telling that compelling story.
Fluency in Your Data
If this sounds like a lot of time to spend on the data part of a project, it is. Becoming familiar with your dataset is the best way to advocate for your population. And no matter what sort of research you're doing -- UX, CX, market, public health -- you're ultimately going to serve as an advocate. Being able to speak fluently about the individuals (with their identities properly concealed) that you interacted with and about the field experiences you had with them will cultivate empathy among your team, stakeholders, clients, and policy makers. This is how you make a difference.
... Unless you just wanted to go ahead and use the notions you had to start with, in which case, leave the research to someone else.
Where did the example agency go wrong? Why were they so overwhelmed by their boards? It seems clear to me that they missed at least one step in the data review process, and I see three places where this may have happened:
1) I do not think they were well-acquainted with their data. When you know your data well, the affinity process is far less overwhelming. It’s still a big endeavor, but you are less lost when placing those first, most difficult, data points. It’s all much less enigmatic.
2) I’m not sure they had “introduced” their data to the full team. If you have team members with no connection to the data, they can have a hard time being anything more than a sounding board — a useful thing, but not what this team needed at that time.
3) The curation process is a frequently overlooked step. If you try to affinitize (or otherwise synthesize) every bit of data you came across, you are going to have a very painful time. Thoughtful and effective curation can only come after the other data familiarization steps are complete, so make sure you at least re-read the entire set of notes before you work on data inclusion for synthesis.
As Always: Ethics
Remember to follow best practices when it comes to collecting, handling, and storing participant data. Keep identities confidential, use the data they give you respectfully, and do not sell personal, private, or identifying data.
If you enjoyed this piece, please help others find it by sharing. Thank you!
Are you a craft specialist being asked to generalize? Our team benefitted from using both design and research specialists. Check out a Defense of Specialization here.
As Principal Experience Researcher at projekt202, Kelly Moran utilizes an innate curiosity and unceasing desire to ask “why” to understand how people use products and services to accomplish their goals — whether those goals be work or play. Check out more of Kelly's expert insights and writing online.
projekt202 is the leader in experience-driven software design and development. We are passionate about improving the experiences that people have with enterprise and consumer digital touchpoints.