Step 2: Develop your Data Collection Plan
Now that you have your data collection instruments, you need to decide:
--Who will apply each instrument
--When will you collect data
--Where they will apply the instrument(s)
--How the data will be collected
Important: Each of these aspects has important implications for the quality of your findings.
Use the Worksheet: Data Collection Form as you move through Tasks 1 to 3 of Step 2.
Worksheet: Data Collection Form
Task 1: Decide who will collect the data
If you are using methods that involve direct contact with people, it is fundamental to choose the right persons who can elicit the most complete and honest answers or observations possible. Persons who:
- Understand and are sensitive to the issues to be discussed
- Will not judge the participants
- Will be trusted by them and
- Will make the respondents comfortable
Characteristics may include people who:
- Have no vested interest in the results and can be objective and unbiased
- Are not in a power relationship over respondents, especially if the latter are highly vulnerable or marginalized people
- Possess qualities the respondents are likely to trust within a specific social and cultural setting – in some cases this means a person who is similar to the respondent, but in other cases it means a person who is respected but different; in some settings to talk about sex, the interviewer needs to be a married person of the same sex.
Tips: When Is Different Better?
- Do not have prejudices that could interfere with data collection
- Have good inter-personal skills and can set people at ease
- Are knowledgeable and comfortable talking about the topic area
- Are good listeners
- Are trustworthy and have good ethical standards and responsible work ethic
- Responded well to training and have demonstrated they will follow data collection instructions and application protocol
For advice you should consider when staff are the ones who collect the data, Tips: When Staff Become Data Collectors
To calculate how many people you may need, Tips: How Many People Will You Need to Collect the Data?
Logical Framework: Fill in Column 5 of the Logical Framework
Option 1: 
Download Word version of the log frame in order to work on your own computer. We highly recommend you download it to your PC before working in it. The Word version is best for modifying the form to suit your needs, for example, adding rows and columns.
Option 2: 
Download PDF version of the log frame in order to print. PDF files are best for viewing and printing.
Option 3: 
Fill out the form Online. To enable you to view the entire form at a glance, we have restricted the number of characters to be entered into each text box. Therefore, we only recommend using the online form if you do not need to enter a lot of information. In order to use the online form, you will first need to create an account, and then log in
Task 2: Decide when the data will be collected
Many factors can affect the time in which you can collect data.
- Seasonal concerns include outdoor temperature, labor patterns, harvest time, rainy season, crime levels, or festivals; anything that would alter the way people behave, and their availability.
- Time of day may alter accessibility of participants. , For example, workers may only be available on their lunch hour or after work. People who arrive early to a location venue may differ from those who come in late. You may only find certain people at home at certain times of the day, but they may have tasks that make them unwilling to participate at those times.
- Time of month or week may also change behavior and availability, for example in relation to pay day, market day, religious observances.
Important: Such factors vary for different populations in different places, so be aware of local barriers to data collection in your area and work around them.
Logical Framework: Fill in Column 4 of the Logical Framework Online
(Website registration required)
Or Download to print:
Word; PDF
Task 3: Decide where data will be collected
Where you collect data can influence:
- Access to the data
- Quality of the data (completeness, honesty)
- Risks to the people from whom you are collecting data
- The extent to which your sample will reflect the evaluation’s priority population
Some common issues to take into account in SRHR work are:
- Where are you likely to find the kind of person or document you are seeking? For example, to find out-of-school youth, you may need to look in the home for girls, but on the streets, in clubs or the military for boys.
- Is there enough time to collect the data with the respondent or is the situation too rushed.
- Does the setting provide enough privacy? For example, asking certain intimate questions in an open waiting room could greatly limit the truthfulness of responses. Where should people fill in a self-assessment survey so that they don’t fear someone looking over their shoulder? Women, may not be open if their children, friends, or partner are within earshot. Men might not feel comfortable talking about sensitive topics in public places.
- If in a clinic or other service, you need to make it clear that the answers will in no way affect the quality of services the person will receive; people need to trust they will not lose their right to care or other benefits if they criticize the program.
- Community meetings may be fine if the information you seek is not too sensitive, but you may need a more individualized approach where answers could cause rifts in the group or discomfort to some.
- Make sure that by going to find a certain person, or just by being seen interviewing someone, you are not placing that person in risk by disclosing a hidden identity, raising concern among neighbors, etc.
Task 4: Make sure the data will be collected well
Even if your instruments are excellent, the way data are collected can affect the quality of the data obtained.
Make sure:
- Instructions for how to apply the instrument are clear
- Data collectors are trained and adequately supervised
- Data collectors are encouraged to reflect critically to clarify procedures and avoid repeating mistakes
- There are opportunities for data collectors to meet and discuss challenges, concerns, and successes with recruiting respondents and using the instruments
=> Develop clear guidelines for data collection
Before you train the people who will collect data, decide on the protocol, or set of rules, for how things need to be done, since you will need to train them on these aspects, as well as making sure they know how to facilitate or interview well.
The guidelines should include:
In the case of individual or group interviews or surveys, you will want to get either verbal or written consent. Tips: When to only get Verbal Consent? In the case of observation, consent is usually given by the person giving access to the site, and the adult responsible for underage people. However, the observer should also gain the trust of the participants, and get their permission through the way she or he builds relationships in the field site. For observations in a public place, there is no one to get consent from.
You might want to use a simple example of an individual informed consent form to help you to adapt to your own needs and context.
- How the data collector should present her or himself to the respondents or people being observed including
- how to greet the respondents
- what clothes to wear
- why they are there, and how the information will be used
- how they should represent the program/agency in the community
- Instructions about whether or not the interview or events will be taped, filmed or photographed. Tips: To Record or Not to Record?
- What procedures to follow when data can’t be collected since a person or document is not found. Make sure data collectors follow the procedures and keep track of cases that were missing. Tips: What to Do When No One Is Home?
- Specific procedures for safe guarding people’s rights to anonymity, confidentiality and/or privacy.
- Guidelines for prematurely ending a data collection event if conditions, quality of interaction, safety, or state of the respondent are such that the validity of data collection is seriously jeopardized
=> Train the people who will collect the data
If the instrument is not applied correctly, people may not want to respond or you may end up with data you can’t use, or that is not as accurate as needed. To make sure this doesn’t happen, you need to:
- Provide the right kind of training
- Give clear and enforced rules on how to go about collecting data
- Provide on-going support and discussions with data collectors.
All persons who collect data should receive some training about the project and their role, even if the data collectors are staff members or people with data collection or M&E experience. Tips: What to Do If You Have Never Trained Data Collectors Before?
At a minimum, this training should include:
- Evaluation objectives
- Review of data collection techniques
- Thorough review of the instruments
- Practice using the instruments
- Skill building exercises on interviewing and interpersonal communication. Tips: What Skills Need to Be Built?
- Discussion of ethical issues (including privacy, confidentiality, anonymity, data security, and informed consent)
- Considerations of professional boundaries (e.g. if a respondent starts to cry, is it okay for the interview to hug them?; if a respondent asks interviewer if she has experienced XYX during pregnancy; if a respondent invites interviewer to come inside for coffee after the interview is complete)
- Discussion of their feelings about participating in the evaluation, e.g. about addressing sensitive topics, and, if they are program staff, about the program and how they feel if they find out the program is not as great as they thought it was
- The importance of following instructions and the application protocol
- Exercises to help them become analytical observers, and to make useful and insightful field observations
Once data collectors have been trained:
- Have them practice their new skills by role playing with each other
- Observe each data collector doing a real survey, interview, focus group or observation session (and do one or more yourself even if you won’t be collecting data later on)
- Review the transcript of the first few in-depth interviews or focus groups and provide specific feedback
- Use a checklist of what you are looking for, based on what they learned in their training
- Provide reinforcement of good behaviors and corrective training if things are not done the way they need to be
- Do not use data collectors who do not meet minimum expectations you established at the beginning of the training
Task 5: Pilot test your tools and data collection plan
Before investing the time and resources in your data collection, do a pilot test using the instruments you have created or chosen, with:
- stakeholders and
- people who are similar to those in your sample, but who will not actually be in your sample
Important: Make sure you pilot test on people, events or documents that are similar to those in the actual data collection.
A pilot test will help you find out:
- How long actual data collection takes
- Whether or not you can collect the data where and when you planned to do so
- If there are problems with the way the instruments work in the real world
=> Test your data collection plan
- With two or three people from each of the major subgroups in your sample.
- Or with two or three people from the most diverse sub-categories in your sample.
Example: If your sample is of out-of-school youth, and the sub-categories included are girls vs. boys, employed vs. unemployed, and living on the street or living at home, but your resources are very limited, you might pilot your data collection plan with groups who are hardest to reach. Your pilot test might include: two young boys who live at home, two young girls and two boys who live on the street.
=> Make sure:
- Data collectors time themselves so you can adjust the plan accordingly
- Data collectors tell you about any confusing questions or problems they had in understanding how to complete the instrument application
- You observe the interviewers in action
- You undertake some practice data collection yourself to get the feel of any problems still present
- You observe the interviewers in action
- You review their field notes
- You check to make sure they filled in the instrument(s) correctly
- You invite data collectors to give you feedback about how things went and any observations they have about the instrument and its application
Important: Create a collaborative working environment so that data collectors feel comfortable sharing their concerns with you and your M&E Team. Their observations and work quality are key to the success of the M&E Process.
=> Check that your instruments are reliable and valid.
These concepts are very important to evaluation but often scare people away from even trying to assess complex social concepts. It is unlikely that you will have the resources to undergo extensive testing of the instruments you will be using but you can do the following:
To check reliability:
- Schedule two different data collectors to collect data on the same person
- Have the same data collector repeat the data collection on the same person, document or event the next day
- Review the paired data collection instruments to see if you come to the same conclusions.
Example: In an instrument that is looking at the attitudes among politicians towards legalization of inheritance rights for women, the instrument would not be reliable if it showed one time that a certain person strongly agreed and the next day that he or she had no opinion or disagreed.
To check validity:
- Make sure you have broken down any complex social concepts addressed into measurable components (If not, return to Module 4)
- Make sure that each question only asks one thing. Do not use “and” or “or,” as you will not know which part of the question the person is answering. Tips: Writing a Good Question
- Make sure that the people responding to the instrument understand your questions
- Make sure that the questions or observations fit the realities of the local setting
- You may ask more than one question at different times in the interview about the same thing and see if they agree
- You may ask the person being interviewed for his or her qualitative appraisal of how he or she feels on a certain aspect you are trying to assess.
Example: If you are applying a list of questions about decision making in the home (as a component of gender equity), ask the respondent if she or he feels that decisions in the home are shared equally between herself and her partner or not.
- Use more than one data source to obtain information about key aspects of the evaluation.
Example: If the evaluation wants to assess the level of girl’s empowerment to make life decisions, you may want to apply a checklist of decisions each girl feels confident in making, but also ask the girls’ teacher to rate them on the level of empowerment they demonstrate; and maybe even talk with their friends or family to ask about the same aspects of empowerment.
