reading evaluation reports
things to look for in evaluation reports
The Clearinghouse databases contain many evaluation reports.
Find articles and reports on the Clearinghouse Research and Resources database
Find good practice evaluations listed on the Clearinghouse Good Practice database
things to look for in evaluation reports
Not all evaluations are equal. Some are done well and some not so well, so you need to have a critical eye when reading evaluation reports. Evaluations can be limited by their scope, time, resources, access to users or staff, data available, evaluator abilities or by parameters set by funders.
These are some questions you might want to consider when reading an evaluation report:
-
did the evaluation look at the entire program or service or only parts of it?
-
did it measure the outcomes or only the process?
-
whose goals did it focus on, those of the users/target groups, service or program staff, or funders?
-
was there a control group or a comparison before and after implementation; i.e. how do you know if the intervention was responsible for the change?
-
was the evaluation conducted while the program or service was still operating, or after implementation, noting that memories can fade and workers can move on?
-
who was included in the sample; i.e. did it represent the diversity of the stakeholder or client group?
-
what questions were asked?
-
did the data tell you enough about the program or service or outcomes?
-
was the data validated by different sources; was the data anecdotal only?
-
did the evaluation consider other factors that might have influenced outcomes; e.g. the impact of individual staff members, policy changes, legislative changes, social or economic conditions?
-
are the conclusions of the evaluation supported by the data?
-
did the evaluation consider if outcomes are sustainable?
You may also want to evaluate research reports. The following resources provide detailed advice about assessing the value or worth of research reports:
Guttmacher Institute 2006, Interpreting Research Studies, In Brief, no. 2. New York & Washington
This brief aims to demystify social science research and provides key questions to ask when evaluating a research report.
Research in Practice 2010, An introductory guide to evaluating research
Beeman SK 2002, Evaluating violence against women research reports, VAWnet, The National Online Resource Center on Violence Against Women
Robert Harris 2007, Evaluating internet research sources
