Course for mentor in the sector of eco-industry

Here is a method to help formulate evaluation questions. Look at the logic model and start with the following five elements:

• Who - Who was your strategy initiative, or programme intended to benefit?
• What - What was the effort intended to do? What was the context within which the effort took place and how could it have affected its implementation and outcomes?
• When - When did activities take place? When did the desired changes start to occur?
• Why - Why is the effort important to your organisation or community? Why might it be important to people in other organisations or communities?
• How - How is the effort intended to affect the desired changes or bring about the desired outcomes?

Here are examples of evaluation questions to ask for the previous sample scenario:

Another method for formulating evaluation questions is to consider different aspects of your strategy, initiative or programme and generate questions about each of these aspects.

By using the logic model, and knowing the questions you want answered, you can develop a measurement framework, as an evaluation planning tool. Developing a measurement framework will allow you to determine how to assess progress toward achieving outcomes and answer the evaluation questions.
With a measurement framework for your effort in hand, you get a clear picture of how to conduct your evaluation. The measurement framework provides another opportunity for stakeholders to further define outcomes. With it, you can consider what the outcome means in more concrete terms.

Key Components of the Measurement Framework
Seven key components make up the measurement framework:

• Outputs are direct products of activities and may include types, levels and targets of services to be delivered by the strategy, initiative or program.
• Outcomes are the immediate, intermediate and long-term changes or benefits you need to document. These outcomes should be the same ones identified in the logic model.
• Indicators are markers of progress toward the change you hope to make with your strategy, initiative or program.
• Measures of change are values - quantitative and qualitative - that can be used to assess whether the progress was made.
• Data collection methods are the strategies for collecting data. This could include quantitative methods, such as conducting surveys or analysing existing data, or qualitative methods, such as conducting interviews or a document analysis.
• Data sources are the locations from which (e.g., national database, programme survey), or people from whom, (e.g., programme participants), you will obtain data.
• Data collection frequency is how often you plan to collect data.

How to Use a Measurement Framework
Once you have identified your outputs and immediate, intermediate and long-term outcomes,
you can list each output and outcome on the measurement framework in the first column. After you have listed each one, you can make a clear plan for assessing progress toward that particular output or outcome. This involves moving across the rows of the measurement framework from left to right to identify indicators, measures of change, data collection methods, data sources and data collection frequency for each outcome. Please note that as you complete the measurement framework, some components could contain overlapping responses. For example, the data source for two outcomes may be the same.

When developing the evaluation questions, the following four categories for measuring mentoring programmes should be considered;

• Relationship processes
• Programme processes
• Relationship outcomes
• Programme outcomes

The following table illustrates the areas of evaluation for these categories.

Category	Evaluation
Relationship processes	What happens in the relationship? For example: how often does the pair meet? Have they developed sufficient trust? Is there a clear sense of direction to the relationship? Does the mentor or the mentee have concerns about their own or the other person's contribution to the relationship?
Programme processes	For example, how many people attended training? How effective was the training? In some cases, programme processes will also include data derived from adding together measurements from individual relationships, to gain a broad picture of the situation
Relationship outcomes	Have mentor and mentee met the goals they set? Some adjustment may be needed if circumstances evolve, and the goals need to change slightly.)
Programme outcomes	Has the programme, for example, increased retention of key staff, or raised the competence of the mentees?