Evaluation framework

Structuring peer-based program evaluation1

Peer-based youth programs are diverse but they share some common objectives:

  • To provide a safe space and safe learning environment for young people without fear of judgment, misunderstanding, harassment or abuse
  • To increase knowledge and skills within a specific area, e.g. sexual health; parenting skills for young parents; mental health; diversity issues
  • To increase social connectedness and create a sense of belonging within supportive peer networks
  • To provide positive adult and peer role models for young people who may only have experienced negative influences from adults and peers
  • To provide activities and opportunities which help young people to develop a positive self concept, self acceptance and high self esteem.
  • To prevent the onset or further development of mental health problems including depression, anxiety, self harm and suicidal ideation
  • To increase confidence and to develop and enhance communication and social skills
  • To empower young people to build personal resilience and improved help-seeking behaviour through increased knowledge of available support, reduced stigma associated with mental health difficulties and help-seeking, the development of positive coping strategies and the development of skills needed to access help
  • To foster future thinking and optimism and to widen young people’s understanding and beliefs of the opportunities available to them.

Peer-based youth programs are implemented within a range of settings and with a range of populations showing some common attributes of youth most at risk. Peer-based youth programs work through a complex interplay of environment factors, peer group factors and program factors to deliver short term impacts on the individual. A process of normative socialisation takes place where new members to the group learn the behavioural rules/norms of the group (the “code of conduct”) through observation of positive role models, experimentation, and feedback received from the group. The short term individual impacts contribute to long term outcomes as well as indirect impacts on others. A range of external and moderating factors (i.e. the “context” in which the program operates) influence program implementation, impacts and outcomes.

The Evaluation Framework for Peer-Based Youth Programs describes each of these components in more detail and can be applied to all types of peer-based youth programs including after-school programs, online support services, drop-in spaces and youth camps.

Using the Evaluation Framework to measure program effectiveness

Program effectiveness is often measured by assessing the extent to which programs meet their objectives. Good program objectives are measurable and based on quantifiable indicators. Some programs may already have key performance indicators (KPI’s) that they have agreed with their funding source. If KPI’s do not exist, program staffing teams are advised to identify and agree on the specific local indicators which will be used to measure program effectiveness before selecting evaluation tools.

Indicators define what you will measure and programs should set local targets for selected indicators e.g. 50% young people complete program. Evaluation tools are how you will measure selected indicators e.g. surveys tools, observation, group discussions, and interviews.

The Evaluation Framework for Peer-Based Youth Programs outlines the components you might consider when identifying specific local indicators for your program and includes some example indicators as a starting point. Indicators can be numeric e.g. the number of young people who finish school. Indicators can also be based on observed behaviour e.g. young people showing improved social skills. Behavioural indicators can also be quantified e.g. the number of young people showing improved social skills.

A good way to decide on what indicators you will measure is to ask some simple questions:

  • How will we know that the program is having a positive effect on participants?
  • What changes would we expect to see in young people as a result of participating in the program?
  • What features of the program are important in achieving intended effects and therefore need to be monitored?
  • What might indicate that the program is not having its intended effects?

Check out the Evaluation Framework Tool

References

  1. Lobo, R., G. Brown, B. Maycock., A. McManus. 2010. Development of an evaluation framework and evaluation approaches for peer-based youth programs – Interim Report. Perth: Western Australian Centre for Health Promotion Research, Curtin Health Innovation Research Institute.