Home

Introduction

About this site

Informed Consent

Survey Overview

Current Survey

How To Get Started

Survey Results

Focus Groups

Bibliography

Contact Us

Choosing Your Sample

Should you send the survey to all of your peer-tutor alumni or a smaller sample? If you have a modest number of alumni (say, 50 or fewer), you may well want to send surveys to all of them, so that you will have a good number of responses to analyze. Even if you have more alumni, you may still choose to survey them all, so that you’re staying in touch with all of your alums, learning from all of them, and getting the broadest possible perspective on the peer-tutoring experience at your school.

From experience, though, we’d urge you not be too ambitious or in too much of a hurry. It’s a lot of work to track down names and current addresses for hundreds of alumni, nudge all of them to respond, thank each respondent individually, and do any meaningful kind of analysis of even 50 narrative survey responses. And unlike many surveys, where a 30% response rate is considered good, this survey has had, at least for us, response rates in the 80% range. And although not all of our alumni are loquacious in their responses, many write detailed responses, which deserve careful analysis. To see how rich these responses can be, see the samples we’ve posted here.

So to spread out both the work and the joy involved in this research and to fit it into your already crowded work life, you might want to use a batch approach and/or select a sample of alumni to survey.

Batch Approach

This is exactly the approach we’ve taken, as we’ve piloted the survey and tried to keep this research manageable.

1. You could start by gathering names and addresses for as many of your tutor alumni as possible.

2.  You then send surveys to c. 20 alumni, spread across all the different years of graduation. If you’ve got 20 years worth of alums, you might choose one from each year, perhaps the first alphabetically from each year. We’ve found that 20’s a good number: enough to get some responses fairly quickly, which is enormously motivating; enough to fill the spare bits of time you can devote to this project; and enough to offer some variety of perspectives.  

3. Then after you’ve solved problems with out-of-date addresses, sent reminders and thank-yous, and savored and shared the responses, you’ll be ready to send another batch of 20. We’ve been sending out a new batch of 20 every six months or so.

A Sampling Approach

If you’ve got a large number of alumni, you might want to select a subset (or sample) of them to survey. For your sample, you’ll want to resist the understandable urge to choose the names you best remember or know you can contact easily, or your favorite former tutors. Instead, you should aim to select a representative sample. As you know, psychologists, pollsters, and other social-scientists have various methods for selecting a representative sample out of the entire population they’re interested in studying—methods that allow researchers to have confidence in the conclusions they draw from analyzing only a subset of the population. With this research, we don’t expect to be reporting specific margins of error for this research the way political pollsters do, but we are convinced that representative is good.

To have confidence that samples are representative, they are supposed to be random. In this case, “random” doesn’t mean haphazard; rather it means that each possible alumna/us has an equal chance of being selected.

Some options you might consider for choosing your sample:

1. If you’re not especially concerned about achieving balance in the years since graduation, you can do what’s called “systematic sampling.” If your list of alumni is in a spreadsheet, simply sort your list alphabetically by last name, number each sequentially, and then choose every n th name—if, for example, you have 100 alumni and want to have a total sample of 20, you’d simply choose every 5 th name. (To be random, this method assumes that an alphabetical arrangement doesn’t lead to some kind of hidden bias in the sample.)

2. If you want to achieve some proportional balance by certain criteria—by gender, for example, or by year of graduation, by major, by number of years of tutoring experience, by geography—you can do what’s called a “stratified sample.” If, for example, 60% of your tutor alumni were female, you might want to use a method to ensure that you have a stratified sample in which c. 60% of the alumni you send surveys to are female. Within those groups, you’ll want to choose individuals randomly.

3. If you have some other specific goals for your research, you might select a sample to achieve those goals. If, for example, you want to compare different eras in your tutoring program, you might want to stratify your sample to make sure that you have a good sample from each distinct era. Or if you have a small number of tutors of one gender or race, and you’re especially interested in hearing diverse perspectives on the peer-tutoring experience, you might oversample particular groups of alumni. Again, within those groups, you’ll want to choose individuals randomly.

To help you choose a representative sample of your tutor alumni, you might want to seek advice from someone in your institutional research office; experts in social-science research that gather and analyze data about students for your school, college, or university. Or we’d urge you to consult a faculty colleague who specializes in survey research in sociology, psychology, business, or other social-science fields. As we’ve piloted this research project, we’ve enjoyed that kind of collaboration on our campuses; not only have we gotten some good advice, but we’ve also discovered colleagues who are very interested in our research projects and are eager to help.

The Web also offers convenient ways to learn more about the basics of sampling techniques:

A brief and easy-to-understand introduction to sampling, from Coventry University

StatPac , a statistical software and online survey company, offers a clear one-page introduction to sampling methods.