Resolving Concerns about the STEPS Approach
Contents
A. A self-evaluation approach
Concern: Will the findings be seen as suspect if the agency evaluates itself?
Bring in some external colleagues onto the Team; ensure effective collaboration with outside stakeholders.
M&E Team.
Concern: Will staff and clients feel pressured to say nice things about the program instead of sharing their real opinions?
Sensitive interviewing techniques should overcome this problem.
Concern: Is staff reluctant to question what they have always been doing and therefore will be uncomfortable doing self-reflection?
Talk about the importance of generating a culture of self-reflection, make sure evaluation findings are not used to jeopardize staff, involve staff actively in the M&E process.
B. The relationship of M&E to program planning
Concern: Prior experience showed them that nothing ever changed when evaluations were done, so why will it be different this time?
Ensure that the agency’s director supports the M&E process and stays involved; by involving program staff and members of other programs it will be far more likely that M&E findings feed into program changes.
Section in M7 on using findings for program changes.
Concern: Team members may fear that the people who plan the programs won’t listen to them.
Involve program planners from the start – required by donors as well, have to do it.
C. The utility of involving stakeholders in M&E
Concern: Some Team members fear that they will be losing authority if they involve people from the community.
Discuss the benefits of interaction with the community; ensure that such interaction is planned in ways to avoid embarrassing staff or criticizing individuals; maintain a professional but friendly demeanor in interaction with the community.
D. Using both quantitative and qualitative approaches
Concern: People may see qualitative data as inferior and a waste of time.
Clarify the different kinds of questions that qualitative data can answer vs. quantitative data; point out their utility for communicating the program’s successes.
Make sure you will use the kind of data collection that fits the questions you will be asking.
Section on quality vs. quantity.
Concern: Some Team members may be skeptical of the “numbers game” and prefer to use more personal, nuanced (qualitative) approaches.
If you want to show change, it will be essential to use some quantitative data;; avoid an excess of charts and numbers in your presentations; make sure you tailor the kind of data you collect to the interests of the audiences you are trying to reach.
See Module 7: Analyzing Data.
See Module 1: Getting Started.
E. Monitoring and evaluating changes on level of immediate, intermediate & final results
Concern: Some members may be impatient to show final results right away.
Your Team will create a more in-depth understanding of the problem you are trying to solve and this will show the importance of being patient.
Team members will trace out the links between what your program is doing and the final result.
Understanding the problem section.
Concern: A concern may be raised that the donor, or political audiences will not be interested in seeing immediate or intermediate changes but will demand final results right away.
By having a clear picture of how your program relates to the final goal, you will have stronger arguments against such demands; involving the donor and political stakeholders from the outset and understanding their concerns but showing them the causal framework that underlies your M&E plan.
Theory of Change, understanding your target issue; audiences, disseminations sections.
