Nearly all after-school and summer programs have their kids take surveys, and nearly all programs suffer through the process.
Surveys so rarely live up to their promise, and it gets harder and harder to convince staff and youth to complete this annual chore. In my experience as a professional evaluator, surveys almost always have three big flaws that limit their usefulness. Fix these flaws to turn survey hell to survey heaven!
Step One: Organize survey questions by theme
Think of the three or four key themes that you want your surveys to address. Consider themes like program quality, stakeholder satisfaction, and participant outcomes. Write your key themes down, then map the questions to these themes. You’ll find that, at least to start, you have lots of questions in one theme, and almost none in the other. That’s OK! Revise the items until you have a balance of questions in each theme. (I aim for 2-3 questions per theme.)
What about the survey questions that don’t link up to any theme? First, I’d encourage you to drop those mismatched questions. If you find that you have a lot of “orphan” survey items, you may need to revise or expand the themes in the survey.
To make analysis a breeze, save a copy of your survey with the appropriate theme listed next to the question.
Step Two: Find and fix confusing survey questions
There are a few common mistakes we all make when writing survey questions, and we all need to fix them before fielding a survey. Ask yourself:
Will the respondent know the answer?
For example, we ask parents to weigh in on the quality of their kids’ homework help, or ask kids to report logistical details about their out-of-school time program. Think about how likely your audience is to have experience with – and an opinion about – what you are asking. Cut or modify questions that are likely to elicit lots of “I don’t know” responses… or well-intentioned fibs.
Am I asking two – or three – questions in one?
For example, we ask kids whether their summer program was fun, engaging, and educational. What if it was educational, but not particularly engaging? What answer should the young person pick? The easiest way to find two-in-one survey questions is to use the Find function to look for the word “and.” Almost always you will need to re-write your item – or break it into two.
Step Three: Check your reading level, weep, and revise
You spent a long, long time learning to become a facile reader. I am proud of you – really! Reading is awesome, but hard. It’s very easy to forget what you read in 5th grade, much less if you were a 5th grader who struggled to read.
For youth surveys, you absolutely, positively must run all of your instructions and each survey item through a reading level calculator. Use the reading level calculator to estimate the age-level at which your survey question is written, and revise until it is close to your survey takers’ reading level. Keep a box of tissues handy, as this is a painstaking, yet essential, process.
And that’s it! You’ve changed your path from survey hell to survey heaven. Congratulations! I’ve listed some of my go-to survey resources below for additional education and delight.
To find existing surveys for youth programs:
- From Soft Skills to Hard Data (Forum for Youth Investment)
- Youth Development Evaluation Toolkit (Colorado Trust)
- Assessment Tools in Informal Science (PEAR)
- To calculate readability scores: https://readability-score.com/
- To learn more about crafting surveys, without earning a PhD: The Survey Playbook: How to Create the Perfect Survey (Matt Champagne)
For breakfast I had fruit, yogurt, and oatmeal.
Author Profile: @coreynewhouse