M3.06. Preparing the evaluation of the mentoring process

2. Principles and methods for evaluating the mentoring process

2.1. Focus of Analysis

The focus of analysis is the subject of the evaluation analysis, including; an organisation, staff, a programme model, a policy or strategy, etc. The evaluation focus in not limited to one subject and can incorporate one or more of these subjects at the same time.

Depending on the scope and timeframe of the effort and expectations of all the key stakeholders, you can decide the focus of the evaluation.
When the focus of analysis is the individual:

  • You are documenting the changes that individuals experience.
  • Your evaluation will usually want to assess two things:
  1. the degree to which the strategy, initiative or programme is being implemented accordingly and
  2. if the individual participants experienced the desired outcomes.

When the focus or unit of analysis is the organisation:

  • Evaluation of changes in your organisation's priorities, policies and practices.
  • The evaluation process should be used to collect and analyse data on key outcome indicators such as how successful the organisation has been in;
    • Engaging and sustaining the involvement of different groups of people
    • Providing the resources for a mentoring programme
    • Developing the key documentation and guidance (e.g. policy and strategy) for the programme
    • Providing and/or supporting growth and opportunities for mentees

When the focus of analysis is the mentoring programme:

  • To evaluate these new or adapted initiatives or programs, you could conduct a study to understand:
    • How the effort was implemented.
    • What elements made it effective.
    • What worked and didn't work and why.
    • What knowledge, skills and other capacities are required of the staff to implement the effort.
    • Most important, how to refine the effort.
  • Much of the data collected could be qualitative. When the initiative's or program's elements have become final and stable, you can conduct an outcome or summative evaluation and combine the use of a quasi-experimental design using the methodology above to assess its effectiveness.
A quasi-experimental design assesses the causal effects of a programme by comparing two groups of participants (a "treatment" group and a "comparison" group) or by comparing data collected from one group of participants before and after they participated in the program.