Step 3: Identifying the M&E Audiences and Questions

 

The choices you make in designing your evaluation will be shaped both by who is interested in your findings (your audience), and also the main questions that you need to have answered. One of our guiding M&E principles is that program staff should be one of the primary audiences for the evaluation- you should be doing it for yourselves in order to learn from and improve your work. However, we recognize that funders in particular, but also other audiences, will be interested in, and may even demand evaluation results.

figure out the primary audiences for the evaluation
determine what questions they need you to answer
find out what kinds of information would be most useful to them

For further clarification on the difference between stakeholders and audience members see Tips: Are Stakeholders and Audience Members the Same People?

Hold a meeting with your M&E Team and complete Task 1 and 2. (Include key stakeholders who are not part of your team if feasible and appropriate)

Task 1: Determining your Audiences and Understanding their Needs

Tips: Understanding Your Audience’s Needs

Think about your audiences- those people who will be interested in your evaluation findings. What questions will they most want to have answered? Some audiences (such as public officials or directors of public programs, economic advisors, and some donors) might be more interested in questions that are focused on the magnitude of change, before and after the program, and are more convinced by ‘hard numbers’ or quantitative data. Others (such as journalists and colleague organizations) will seek to answer questions that are focused on how the change occurred, and what it means to participants, and will be more convinced by qualitative data such as people’s stories and their own words. Your M&E Team and agency will likely want both kinds of information. You should try to get input from some stakeholders, especially your program staff and your program funders, in thinking about your M&E audiences and their needs.

Tips: Whose Attention do you want to get?

You may want to use the Audience Information Needs Worksheet to keep track of your audiences and their interest in your evaluation.

Worksheet: Audience Information Needs

Task 2: Define the evaluation questions: Monitoring (process) and Evaluation (impact)

You will need to answer questions that have to do with both monitoring and evaluation:

-Monitoring questions focus on process: how your program is/was delivered.

-Evaluation questions focus on impact: the change that your program contributed to, or produced, in the target population.

When you write your evaluation questions, keep in mind:

-Your target problem

-Your program objectives and activities

-Your audiences

Some Common Evaluation Questions:

• Monitoring- Process:
  • Which program strategies were actually implemented?
  • Were activities implemented as intended?
  • Were there any factors that changed or hindered the delivery of the program?
  • Who participated? The intended participants? Others?
• Evaluation- Change/Impact:
  • What impact did the program have?
    • Did it have the intended impact?
    • Was there any unexpected impact?
    • Any negative impact?
    • What was the difference when comparing before and after program participation?
    • Was there any difference between those who received the intervention and those who did not?
  • At what level did the program have impact?
    • Individual
    • Couple
    • Family
    • Community
    • Institutional (i.e. capacity building, policy change)
  • Which aspects of the program had the intended impact?
    • Which activities produced the change you intended?
  • What was the experience of program participants?
    • From their perspective what was the most significant aspect of the program?
    • If the participants experienced a change- do they think it was due to the program? If so, what was the most significant change for them?
  • Did certain program participants benefit more, or differently, than others?
    • What factors shape the program’s impact on participants?
    • Was there any difference in impact between: Women and men? Adults and youth? People of different socio-economic status? People of different ethnic groups?
    • Was there any difference in the experience or impact of the program for those who are marginalized in some way and those who are not?

Task 3: Focus your Evaluation

Now you need to decide which audience’s needs and which evaluation questions you most need to, and can, answer.

• What questions do program staff and stakeholders most want answered?
• How would the Team prioritize the different information needs, and which are feasible to meet based on the time, money, and human resources available at this point?

Create the list of questions you will answer, and share it with your primary audience(s) for feedback and input. You may even find that this will help you to raise more funds for your M&E efforts, particularly if the donor is your primary audience.