Evaluation

Monitoring and evaluation

Monitoring and evaluation are fundamental aspects of good program management at all levels and can be effective tools to enhance project planning. Monitoring and evaluation:

  • Provides data on program progress and effectiveness;
  • Improves program management and decision-making;
  • Allows accountability to stakeholders, including funders;
  • Provides data to plan future resource needs;
  • Provides evidence on effectiveness that could help to secure continued funding or additional funding for new initiatives that build on previous work; and
  • Provides data useful for policy-making and advocacy.

What is monitoring?

Monitoring helps project managers and staff to understand whether the projects are progressing on schedule and to ensure that project inputs, activities, outputs and external factors are proceeding as planned.

What is evaluation?

Evaluation assists organisations to assess to what extent the projects have achieved pre determined objectives. In the longer term, evaluation also attempts to link a particular output or outcome directly to a program after a period of time has passed.

Evaluation has been defined by Weiss1 as:

“Evaluation is the systematic assessment of the operation and/or the outcomes of a program or a policy, compared to a set of explicit or implicit standards, as a means of contributing to the improvement of the program or policy”.

Trochim’s2 definition of evaluation:

“Evaluation is the systematic acquisition and assessment of information to provide useful feedback about some object”

These definitions highlight the evaluation process as being systematic, and emphasises it as acquiring and assessing information.

Note that the latter definition does not make reference to measuring the worth of the object. This is important as what is worthy to one person in a particular context may not be judged as worthy by another person in a different context. Therefore evaluation involves:

  • observation and measurement, and
  • comparing these findings to a set of criteria which are considered by the user as indicators of good performance.3

Why evaluation?

It is important to evaluate programs for many reasons. Firstly you need to make sure at the very least, that the project is not creating any unintended harm. Secondly evaluation enables you to determine if the project is making a positive contribution. Through evaluation you can demonstrate to funding bodies, key stakeholders, participants and the community that the project has achieved what it set out to achieve, therefore gaining credibility. This in turn will assist your endeavours in procuring future funding.3

Furthermore, evaluation is a tool for improving and learning. You can learn what elements made the project run as intended, how it can be replicated, how challenges can be overcome in the future and how to make the project sustainable.2,3 Evaluation also contributes to, and builds on existing research to assist and influence public policy.2

Green and South4 very succinctly summarised the four main reasons evaluation is conducted for:

  • accountability;
  • learning;
  • program management and development; and
  • ethical obligation.

The context of evaluation

It is important to establish the context in which the peer-based program operates at the evaluation design stage. This is essential as contextual factors influence program implementation and outcomes, and need to be considered to ensure the most appropriate evaluation tools are used.2

It is important to keep in mind that the planning of the evaluation needs to take place at the same time as the program is being planned.  Too often program evaluation is tacked on the end and evaluators find there are no pre-program data by which to assess the planned outcomes. For example, a pre-program questionnaire which measures participant perceptions of their self-esteem, confidence etc. can be implemented so that when the post program evaluation is conducted there will be pre-program information by which to measure change.

Another important aspect to be incorporated into the program planning process is to build in the amount of funds needed to conduct the evaluation. This can usually be estimated as 10% of the total budget.

Find out more about an evaluation framework for peer-based youth programs.

References

  1. Weiss, C (1998) Evaluation: methods for studying programs and policies, Ch. 1, pp. 1-19, Prentice Hall, N.J.
  2. Trochim 2006
  3. Hawe, P., Degeling, D., Hall, J. (1990) Evaluating Health Promotion: A Health Worker’s Guide, MacLennan & Petty, Sydney.
  4. Green J. & South J. (2006) Evaluation: key concepts for public health practice. Open University Press.