Types of evaluation

Many types of evaluation exist, consequently evaluation methods need to be customised according to what is being evaluated and the purpose of the evaluation.1,2 It is important to understand the different types of evaluation that can be conducted over a program’s life-cycle and when they should be used. The main types of evaluation are process, impact, outcome and summative evaluation.1

Before you are able to measure the effectiveness of your project, you need to determine if the project is being run as intended and if it is reaching the intended audience.3 It is futile to try and determine how effective your program is if you are not certain of the objective, structure, programing and audience of the project. This is why process evaluation should be done prior to any other type of evaluation.3

Process evaluation

Process evaluation is used to “measure the activities of the program, program quality and who it is reaching”3 Process evaluation, as outlined by Hawe and colleagues3 will help answer questions about your program such as:

  • Has the project reached the target group?
  • Are all project activities reaching all parts of the target group?
  • Are participants and other key stakeholders satisfied with all aspects of the project?
  • Are all activities being implemented as intended? If not why?
  • What if any changes have been made to intended activities?
  • Are all materials, information and presentations suitable for the target audience?

Impact evaluation

Impact evaluation is used to measure the immediate effect of the program and is aligned with the programs objectives. Impact evaluation measures how well the programs objectives (and sub-objectives) have been achieved.1,3

Impact evaluation will help answer questions such as:

  • How well has the project achieved its objectives (and sub-objectives)?
  • How well have the desired short term changes been achieved?

For example, one of the objectives of the My-Peer project is to provide a safe space and learning environment for young people, without fear of judgment, misunderstanding, harassment or abuse. Impact evaluation will assess the attitudes of young people towards the learning environment and how they perceived it. It may also assess changes in participants’ self esteem, confidence and social connectedness.

Impact evaluation measures the program effectiveness immediate after the completion of the program and up to six months after the completion of the program.

Outcome evaluation

Outcome evaluation is concerned with the long term effects of the program and is generally used to measure the program goal. Consequently, outcome evaluation measures how well the program goal has been achieved.1,3

Outcome evaluation will help answer questions such as:

  • Has the overall program goal been achieved?
  • What, if any factors outside the program have contributed or hindered the desired change?
  • What, if any unintended change has occurred as a result of the program?

In peer-based youth programs outcome evaluation may measure changes to: mental and physical wellbeing, education and employment and help-seeking behaviours.

Outcome evaluation measures changes at least six months after the implementation of the program (longer term). Although outcome evaluation measures the main goal of the program, it can also be used to assess program objectives over time. It should be noted that it is not always possible or appropriate to conduct outcome evaluation in peer-based programs.

Summative evaluation

At the completion of the program it may also be valuable to conduct summative evaluation. This considers the entire program cycle and assists in decisions such as:

  • Do you continue the program?
  • If so, do you continue it in its entirety?
  • Is it possible to implement the program in other settings?
  • How sustainable is the program?
  • What elements could have helped or hindered the program?
  • What recommendations have evolved out of the program?3,4

References

  1. Nutbeam & Bauman 2006
  2. Trochim 2006
  3. Hawe, P., Degeling, D., Hall, J. (1990) Evaluating Health Promotion: A Health Worker’s Guide, MacLennan & Petty, Sydney.
  4. South Australian Community Health Research Unit n.d.c