Print Page: Word; PDF

Step 3: Interpreting your Findings

Now that you have the findings from your analysis, you will need to describe what they mean. This will be particularly challenging if the findings are different than you expected. You will need to take certain actions to interpret your findings so that they make sense to your

Important: Remember that the findings don’t mean anything by themselves! Their meaning comes from the rigor of the evaluation, the reality of the program and of the lives of the people involved.

Task 1: Understand aspects within your findings that are particularly surprising to you and your M&E Team

These may be due to:

• Independent factors that interfered with your program having the desired results. All SRHR programs take place in the real world. This means that other things that had nothing to do with your program may also have influenced your results.
  Example: A program designed to help parents speak more openly and positively about sexuality with their children may have less positive results than expected because a new radio soap opera instilled shame and fear of sexuality in the parents or, on the other hand, because both the program and the comparison group were exposed to a new enlightened TV program that made them all more ready and willing to talk about sex with their youngsters.
• Problems with the way the data were collected. Evaluations are not foolproof, and you might find that something about the data collection or analysis led you to false results. If this was the case, you will need to find out what went wrong in the way data were collected. Go back and check the original data collection forms, discuss the reality of data collection with the people who collected the data, and check for tabulation errors. Of course, if no such errors are found, you may need to face the fact that the results were disappointing and use them to make improvements in the program.
• It’s too soon to show results. Your evaluation and findings may be premature. Perhaps the program was only in progress for six months and you are hoping to find major changes that will show up later. Remember that SRHR programs are influencing complex social processes that take time to change. Make sure to continue to monitor the program and measure immediate results that can lead to the intermediate results you are hoping to find.
• You have not specified the immediate results properly and therefore you are not seeing the changes you could expect to see or that you had hoped for.
• Factors that were not taken into account in planning the program. If the theory of change or causal pathway for the program was deficient, there may have been important gaps that were not taken into account making the program less effective than hoped. You should work with program staff to identify what is missing and make the changes needed.
• Lack of effectiveness of your program. If you thoroughly analyze the results and are convinced that the evaluation has rigor and validity (the results really do tell you about impact) and you find no benefit from the program, you should make significant changes to your program or stop conducting it. The history of the public health and development fields are full of examples where programs were halted or modified for just this reason.
  
  • Perhaps your program is a good one but is too small an intervention to have the result you had hoped. For example, if you are trying to overcome homophobia, one set of posters or radio spots may not be sufficient to have the effect you had hoped for despite being on target, very catchy and culturally relevant. In this case, the program should continue its efforts but join forces with other groups to make a greater impact. The evaluation should also identify more immediate and intermediate changes that can demonstrate the program’s effectiveness in a more gradual and partial basis.
• The results are too good to be true. Although it is difficult to do, if your findings are far more positive than you had expected, resist the natural temptation to assume they are all due to the program. Ask yourselves if something else was happening that influenced the findings. It’s better to be cautious than overly confident unless your analyses give you evidence to show the results are real and they are related to the program.

=> Meet with key stakeholders

• Convene small discussion groups of the program beneficiaries. Ask them for their perceptions of the program and of the questions they were asked about the program. Show them the results you have found and ask them if they agree or disagree and what the findings mean.
  
  Example: The M&E Team decided to call a focus group with some of the young people who had participated in sex education program. The findings had shown higher levels of sexual activity after the program. They asked them to look at the survey instruments and to tell them if there were different ways they had answered them before and after the program. Most of the youngsters said they had lied on the baseline survey since they feared the information would be used against them. On the evaluation questionnaire they were more honest since the program had created a higher level of trust. This information will certainly need to be reported so people don’t think the program was a failure when it had really succeeded in helping young people be more comfortable talking about sex, a key to healthier sexuality in the long run!
• Meet with staff who were involved in the program. Ask them what the findings mean to them.
  
  Example: The staff of a sex education program might have realized that the young people had originally been afraid to reveal their sexual experiences at the beginning of the program but opened up during the participatory discussion. The staff may also have observed that the young people answered the baseline questionnaire when they were still in primary school, but were answering the evaluation questionnaire in middle school, where sexual development and exploration are normal parts of the development process, and possibly where sexual relations are more acceptable or encouraged by the new and older peer groups.
• Meet with stakeholders from the community who are knowledgeable of your program and its participants in order to get their interpretation of the findings. They may be able to more fully explain the context of and reasons for surprising results.
  
  Example: While you may think that your social support program for high risk pregnant women had little effect on how frequently the women came for prenatal care, stakeholders may know that the women went to a different clinic that had opened with very low costs, and the women had attended more consultations than those in the non-intervention group, but not at your clinic.

=> Compare your findings with those that others have obtained

• Review the literature about similar programs
  • try to find examples that are similar to yours
  • look for findings that contrast with yours in ways that make it necessary for you to explain the differences. In such cases, look carefully at the data sources, the instruments used as well as differences in the programs and the populations served
• Present your findings to colleagues in other organizations that work in similar areas
  • Have they observed the same kinds of surprising findings as you have? What interpretation did they give to the findings?

Task 2: Undertake analyses that can help you know if the findings are the result of your program

• Use your monitoring data to know if there were different levels of exposure for different subgroups and how that relates to the findings.
  
  Example: If your results showed that girls and boys had greater knowledge of HIV in your evaluation findings, but your monitoring data show that very few girls ever showed up to your youth center, you will not be able to attribute the positive findings to your program. In a focus group with young people who you interviewed during the evaluation, you might find out that, at the same time as your program, there was a popular new TV program that included lots of HIV education, and that the girls watched it faithfully (since they were kept home after school by their parents).
• Use your monitoring data to tell you if other things were occurring that could have influenced your results.
  
  Example: When going over reports from the field you may find that data collectors observed increases in domestic violence among the women who were attending your literacy classes as an unintended immediate negative result of the program. While the program will have already fed these observations back into the program and tried to address this problem, it may have been too soon for your new motivational classes with the women’s husbands to have had a positive impact on your final or intermediary results.

Back to Top