There are a variety of research methods available when you want to better understand the behavior of your customers or users. Behaviors are always most meaningful in their natural context, but sometimes it’s not practical to directly observe people when and where those behaviors are occurring. There are now a variety of reliable digital tools that allow you to collect behavioral data remotely, using an established method called a diary study. With a little extra prep work, you can even set up your own custom diary study using freely available tools and resources.
Diary Study Basics
Diary studies are a way to collect data from study participants (the users, customers, or other people who match the criteria you’re targeting) over time. In a typical diary study, participants self-report their behaviors, frustrations, opinions, desires, and aspirations at defined intervals or in response to carefully designed prompts or tasks. While going out to participant locations and observing them first-hand is typically my first choice when striving to understand behaviors, there are several reasons why diary studies can sometimes be a more appropriate choice. For example:
- When the behaviors or actions of interest happen sporadically or in unplanned moments
- When these behaviors or actions are part of a series or flow which takes place over time
- When the research aims to understand change over time
- When there is reason to worry that observers would influence behavior or opinion to an unreasonable extent (with private activities, for example)
- With sensitive populations that cannot tolerate direct observation
Diary studies can take place over a few days or across several months. Depending on what type of behaviors or actions you need to capture, tasks may be assigned at predefined intervals or they may be triggered by certain events. Diary respondents may be asked to complete tasks several times a day, only as events come up, or at any other interval of time. They are in many ways a middle ground between a fully controlled lab-based study, and an open-ended in-field observational study.
Benefits of Diary Studies
There are several benefits to using a diary study to understand your target population. These studies often surface topics that a team has not thought to pursue in other, more tightly controlled research, because the team does not know it exists as a phenomenon. This can include topics and phenomena that participants have not brought up in interviews because they simply did not come to mind in that interview moment.
Diary studies are also minimally intrusive, and capture naturalistic (in-context) data. The ability to conduct them with 20, 30, or even more participants simultaneously makes them very efficient. Also, they provide a way to collect data over a period of time (what we call longitudinal studies) without committing a research team to an intensive field commitment.
One final benefit to diary studies is the ability they give your team to “get smart” about the population before going out to the field when used as a first step in a multi-phase discovery project. The team can head out already knowing about specific instances they can probe into in more depth. A positive byproduct of getting to know the respondents before we meet for face-to-face observation is the breaking down of some initial barriers. Arriving with a foundation of rapport already laid down means we can get right into meaningful observation shortly after arrival.
The Process – Going Digital
The traditional method uses paper notebooks sent out to participants with instructions and prompts already included. The research team then waits anxiously while diary respondents (hopefully) complete all their diary tasks over the prescribed period of time and then sends them back. At this point, the research team is flooded with data and begins the process of sorting, organizing, analyzing, and synthesizing — often finding along the way that some of their tasks were misinterpreted and that a few of their participants were either a bad match for the study or were not able (or willing) to provide the level of response necessary to be useful. If this scenario has happened with several tasks or for numerous participants, the yield is a less robust data set than anticipated, which can mean the need to relaunch the study with revised tasks and stricter recruiting criteria. Careful preparation can mitigate the risk, but people are endlessly surprising in the ways they interpret instructions, and it is generally advised to run a pilot test on all tasks and then over-recruit to ensure an adequate final sample size.
Enter the digital age and the ability to remotely moderate a diary study in-flight. Tools such as D-scout, Indeemo, and YouXJournal (free!) as well as free options available from the Google suite of online products can help you avoid or quickly mitigate many of the pitfalls associated with paper diary studies.
Note: When it comes to the digital options mentioned above, you often get what you pay for. I have successfully used free Google products to run diary studies, but they require heavy planning and moderation, and, at the end of the day, you’re on your own in terms of quality assurance. D-scout and Indeemo may seem like they carry hefty price tags, but both provide set-up support and in-project technical support for both moderators and participants. I haven’t used YouXJournal yet, but I have to assume that, with the free price point, you’d be responsible for troubleshooting and technical support. If you have the time – go for it!
With digital diary studies, you have the ability to monitor responses as they come in and intervene or redirect where necessary. Often you will very quickly identify some participants who need to be replaced. This allows you to end up with a higher quality dataset at the conclusion of the study – which really is priceless.
Diary studies are a useful alternative to collecting in-context data when field work isn’t a good option. As with all projects that collect data on human beings, do not forget to keep ethical considerations in mind. Most importantly:
- Ensure that you have what we call informed consent —participants know you are collecting data and they know why. Do not collect any data without permission.
- Use images, quotations, and voice recordings respectfully, and with permission.
- Do not sell participant’s information.
- Store data responsibly, or delete it at the end of the project.
Useful References on Diary Studies
Palen, Leysia and Marilyn Salzman. “Voice-Mail Diary Studies for Naturalistic Data Capture under Mobile Conditions.” Department of Computer Science, University of Colorado, Boulder.
Rieman, J. (1993). “The Diary Study: A Workplace Oriented Research Tool to Guide Laboratory Efforts Collecting User-Information for System Design.” Proceedings of the ACM INTERCHI’93 Conference on Human Factors in Computing Systems, p.321–326.
As Principal Experience Researcher at projekt202, Kelly Moran utilizes an innate curiosity and unceasing desire to ask “why” to understand how people use products and services to accomplish their goals — whether those goals be work or play. Check out more of Kelly's expert insights and writing online.
projekt202 is the leader in experience-driven software design and development. We are passionate about improving the experiences that people have with enterprise and consumer digital touchpoints.