M3.06. Preparing the evaluation of the mentoring process
2. Principles and methods for evaluating the mentoring process
Evaluation can be done using formative or summative assessment.
| Formative assessment: |
Formative assessment is a continual form of monitoring, throughout the mentoring programme. |
| Summative assessment: |
Summative assessment is assessment which is conducted at the end of mentoring programme. |
Formative and summative evaluation can be done using two key methods of evaluation: Process and Outcome.
Process evaluations focus on whether a programme is being implemented as intended, how it is being experienced, and whether changes are needed to address any problems (e.g., difficulties in recruiting and retaining mentors, high turnover of mentees, high cost of administering the program).
Outcome evaluations focus on what, if any, effects programs are having. Designs may, for example, compare goals to outcomes or examine differences between mentoring approaches. Information of this sort is essential for self-monitoring and can address key questions about programs and relationships.
Process and outcome evaluations can be done using either:
- Qualitative assessment, or;
- Quantitative assessment
| Quantitative assessment: | Quantitative methods are those that express their results in numbers. They tend to answer questions like "How many?" or "How much?" or "How often?" |
| Qualitative assessment: | Qualitative methods don't yield numerical results in themselves. They may involve asking people for "essay" answers about often-complex issues or observing interactions in complex situations. |
 Qualitative and quantitative methods are, in fact, complementary. Each has strengths and weaknesses that the other doesn't, and together, they can present a clearer picture of the situation than either would alone. Often, the most accurate information is obtained when several varieties of each method are used. That's not always possible, but when it is, it can yield the best results. |
Evaluation assessment methods
When determining the evaluation strategy, the mentoring relationship must be considered. The purpose of the mentoring programme basically is to promote the growth and satisfaction of the participants.
Mentoring relationships are successful and satisfying for all parties involved when certain factors are established, and both the mentor and the mentee take active roles.
The following factors are suggested as key indicators for mentoring programme participants to assess the effectiveness of their mentoring outcomes.
1. Purpose:
- Both partners are clear on the reasons they're meeting
- The roles and objectives have been discussed and agreed by both parties
- Both partners will recognise when they've completed their purpose.
2. Communication:
- The mentor and mentee communicate in the ways they both prefer
- They get back to each other in the timeframe they've agreed upon.
- Both parties listen attentively, and remember information
- Information exchange and conversation is two-way
- Nonverbal language is monitored (e.g. body language supports verbal conversation)
3. Trust:
- Information is welcomed and kept confidential.
- Commitments are honoured, and cancellations aren't made without compelling reasons
- Neither partner talks negatively or others, or is unfairly critical
- As the relationship develops, and trust develops, information sharing increases
4. Process:
- Meetings are regular and at a time which suits both parties
- Sessions are usually an appropriate length
- Both parties should enjoy the meetings.
- The mentor and mentee should be aware of the four stages of formal mentoring (building rapport, direction setting, sustaining progress and ending the formal mentoring part of the relationship) and should be working through them.
- Both parties should be satisfied with the pairing, and should regularly evaluate the pairing to ensure compatibility
5. Progress:
- The mentee of the partnership has identified appropriate life goals and is making significant progress towards building competencies to reach those goals.
- Both parties identify interesting learning experiences and process the results of these together.
6. Feedback:
- Both parties discuss and agree on the feedback format
- Feedback should be given in an honest and tactful manner and as frequently as agreed upon
- Feedback should be welcomed and neither party should be defensive, but instead take immediate steps to apply it.
Using these 6 key parameters, evaluation questions can be determined, and a measurement framework can then be created to identify the data sources; frequency of data collection; and the qualitative and quantitative measures of change. This information, along with data collection methods, analysis strategies and plans for reporting and communicating the findings, are compiled to form the evaluation plan.
After the evaluation plan is completed, data collection can start, followed by data analysis.
Some analyses can be linear and straightforward. For example, if the goal of the evaluation is to find out the impact of your organisation's mentoring program, then the sequence of activity is more or less linear, as noted below.
- Collect data from the stakeholders before the programme starts.
- Collect data halfway through the programme or at the end of the programme (or both).
- Analyse the data using statistics (e.g., percentages) and coded qualitative data.
|
The next few stages after data collection and analysis relate to communication and interpretation of findings and making informed decisions about improvements and next steps. The reporting, interpretation and reflection of evaluation findings will be discussed in-depth in the "Conducting an Evaluation of the Mentoring Process" section.