M3.07. Conducting evaluation of the mentoring process

2. Use of Quantitative Methods

Determining if you should use quantitative methods.
Depending on the answers to the questions raised, quantitative methods such as surveys or assessments could be right for you. Or qualitative methods such as interviews or focus groups might suit your needs better.
Types of questions that quantitative methods can help answer
Generally, quantitative methods are concerned with what, who and when. Therefore, you should consider quantitative methods if your evaluation questions include inquiries about who participated and benefited from your program; what the impact of the programme was; what changes were brought about by your programme; and when the changes occurred.

Examples of quantitative data include:

  • Number of people who participated in programme activities over the course of the year (participation rate)
  • Whether or not programme participants developed new knowledge and skills (percent change in knowledge and skills before and after participation)

Primary audience for the findings
Quantitative methods generate data that appeal to people who prefer information that quantifies impact and provides the "bottom line."

Potential respondents and sample size
Qualitative methods can be advantageous because they can be more cost effective. For example, a survey can be distributed to 50 people, and completed in their own time, whereas a 30-minute interview with each of those 50 people would take considerably more time.
However, you must also consider the disadvantages of the survey method versus face-to-face interviews. Conducting interviews or focus groups with fewer respondents may actually be preferable because it could yield better quality data, even if it more time consuming.

Amount of time for data collection and analysis
Quantitative methods are useful if the amount of time to collect and analyse data is very limited. A survey with close-ended questions (i.e., you provide response options that respondents can select) that ask people to rate something takes less time to administer than scheduling and conducting interviews and focus groups. The data from questionnaires takes less time to analyse - calculating frequencies, averages or percentages generally take less time than reviewing, coding and analysing qualitative data (i.e., notes from discussions and interviews). Quantitative methods are equally useful if the programme is long-term, since you can compare baseline data with subsequent data.

Budget and other resources
In general, under comparable conditions (i.e., amount of time for data collection, sample size), quantitative methods can be less expensive than qualitative or mixed methods (which use a combination of quantitative and qualitative methods) for several reasons. Secondary data such as records of training course attendance, can be obtained usually with very little time or cost. Surveys can be administered electronically, which also costs less than the typical resources which may be needed to travel to an interview or focus group. An analyst needs less time to calculate frequencies and percentages than to read and code text taken from interview and focus groups. Also, a different set of skills is required to do these calculations than to code text and generate themes.