Surveys are an important data collection tool used that nonprofits and other organizations can use get a pulse on the satisfaction, thoughts, and feedback of their participants and consumers. Surveys are also an excellent way to capture information about your organization’s key outcomes. Surveys can be relatively quick, easy, and cheap to create and administer, but it can be harder to make sure they are well-written. A well-written survey is critical to getting the data you actually need.
A Quick Note About “Burden”
One of the most important concepts in writing a good survey is to think about burden. Burden represents both the cognitive load needed to complete a survey and the overall experience of the survey taker (aka respondent or participant).
A good survey considers and tries to limit how much a respondent has to think through each question and works to create clear, concise and easy to understand questions. A good survey aims to ask only the essential questions to prevent your participant from becoming bored, or worse, quitting before the end. Digital and paper surveys should be easy to navigate and use.
Ten Survey Writing Tips
Below are ten tips to reduce the burden on your survey participant so they stay engaged and willing to provide you the information you need. When writing a survey, it can be helpful to use this as a checklist once you have a draft to make sure you are following the basic rules of survey writing.
1. Ask only one thing at a time.
Your questions should only ask one thing at a time, so both you and your participant are crystal clear on what is being asked. Having two items or concepts in one question is known as a double-barreled question. For example:
Double-barreled question: Do you think our staff are accessible and friendly?
If a participant finds staff accessible but unfriendly, then they don’t know how to answer. If someone responds negatively to this question, you won't know if the issue is with how accessible or how friendly the staff are. If both concepts are important for you to measure, then it's best to split this into two questions.
Ask only one thing at a time. Do you think our staff are accessible? Do you think our staff are friendly?
2. Simplify language.
As a rule, and depending on your audience, surveys that will be taken by adults should be written at an 8th grade reading level. You can find handy tools online that can assess your language, but it's important to use familiar terms and simple language. You do not want to include any acronyms, jargon, or SAT-level words in your survey. For example, take a look at a complicated versus simple version of this question:
Complicated: Do you typically agree that the CTA is unimpeachable in their approach to ethics?
Simplified: Do you agree that the Central Transit Authority is trustworthy?
Even more simplified. Do you agree or disagree with this statement: I trust the Central Transit Authority.
Try to simplify as much as possible, and edit your survey at least once, but preferably
twice to try to streamline your questions.
3. Avoid negatively worded questions.
Write your questions with a positive frame of reference rather than negative whenever possible. It can be hard for our brains to make sense of a question that is negatively worded.
For example:
Negative wording: Do you agree or disagree that students should not be allowed in the homework center after 8pm?
Positive wording: Should students be allowed in the homework center after 8pm? Agree or disagree?
In the negative wording example, your brain has to do a lot more work (i.e. a higher burden) to understand the statement “Students should not be allowed…” and then pick your answer. It is less cognitively burdensome to respond to “Should students be allowed in the homework center after 8pm?” and feel confident that you are answering it appropriately.
4. Be clear on time frames.
If you are ever asking someone to reflect on or provide answers to something they did during a specific time frame, be clear on the time frame. It is much easier for someone to respond to something that happened within a recent, specific time frame than in the past or vague time frames. For example, which would be easier to answer:
No timeframe: How many times do you drink alcohol?
Unspecified timeframe: On average, how many times a week do you drink alcohol?
Specific recent timeframe: How many times did you drink alcohol in the past 7 days?
These questions ask similar things but could get you very different answers. The first question, which is likely a write-in response, could net a wide range of answers and explanations. The second question will provide you with an individual's typical experience, which you may want, but you have no control over the period of time upon which they are reflecting - it could be the past week or an estimate of weekly alcohol consumption past 2 years. However, the third question provides you with a specific, recent response to alcohol use, which may be more important if you care about recent or current use. Think carefully about what type of information you might want when creating wording for time specific questions.
If you are asking about something in the past, be aware that it is harder to answer questions that rely on memory. For example, if you asked me, how often do you visit the library?, then I could provide a pretty good answer about my typical behavior (weekly!). If you asked, how many times did you visit the library last summer?, then I would struggle and make a guess based on my normal behavior. I also might consider summer when my kids are out of school - from June to mid-August, while someone else may define it based on the actual season change, starting in June. All of this can introduce unwanted messiness in your data, so being clear with timeframes is best.
5. Make questions as short as possible.
It can be tempting to try to pack a lot of information into a single question, but this approach can be overwhelming to respondents. Try to shorten and streamline questions. This applies to both the question and answer options.
For example:
Long Question: People use social media for a variety of reasons, including communicating, entertainment, getting access to news, and socializing. How much time in a typical week do you spend on Instagram, Facebook, TikTok, Reddit, Twitter, and Snapchat?
Short Question: How much time do you spend on social media sites in a typical week?
The long question includes reasons why people might use social media, but given how ubiquitous social media is, this likely doesn’t need to be explained. The longer question also includes a list of social media sites, when the specific sites are not necessarily important. It might also leave out currently popular sites and make the respondent wonder if you are intentionally excluding those sites.
The short question gets to the point quickly and asks respondents about their social media use in a typical week. It doesn’t attempt to define the specific sites and allows the respondent to self-define those in order to answer the question. However, if you want to define specific sites (e.g. focus on LinkedIn only), then be clear about that.
6. Avoid leading or loaded questions.
You want the language you use in your survey to be as neutral as possible so that you aren’t leading or prompting your respondents to answer in a specific way. Below are some examples of leading or loaded questions, with the leading words bolded:
Loaded: How would you rate the world-renowned keynote speaker Dr. Z at the Annual Gala?
World-renowned places value and esteem on the keynote speaker, and may cause participants to answer more positively than they would otherwise.
Loaded: Do you believe we should force community members that attend events to provide their personal contact information?
Force is a strong, negative word which may bias participants to answer negatively
Leading: Don’t you agree that our food pantry provides the best selection of fresh product in the area?
Don’t you agree is leading someone to agree with your statement, even if they do not. And depending on the answer options, it could be challenging to figure out the correct response.
A more neutral wording would be: “To what extent do you agree with the following: Our food pantry provides the best selection of fresh products in the area?” Response Options: Strongly Disagree, Disagree, Neutral, Agree, Strongly Agree
7. Avoid built in assumptions.
It can be easy to assume that a survey participant knows something that you know - so be sure to check these assumptions when writing your survey. For example, you might put a survey on your Facebook page that asks participants “How easy is it to get to our building for events?” This assumes that they know where your building is, either causing participants to have to search for this information or feel like they cannot answer at all.
Another example: “Tell us about your experience voting last Tuesday.” This assumes that someone is eligible to vote, registered to vote, and did actually vote! Unless you know these things about your survey population already, you may get a lot of responses that are not very useful if a lot of people just respond “I didn’t vote.” Be careful about these assumptions and who may see and take your survey, and edit accordingly.
8. Triple check everything.
Mistakes and typos are inevitable, so check your survey many times. As with writing anything, including this blog post, you can’t always catch everything but you should try. A few steps to take when checking your survey:
Re-read for clarity, spelling and grammar
Check answer options to make sure they make sense with your question
Have an “outsider” read the survey. I think of this as the random stranger test - find someone, a partner, friend or family member, who is not affiliated with your organization or program and ask them to read the survey. They will catch things that are unclear, identify jargon and often find typos that your eyes glaze over.
9. Be inclusive.
It is important to take care to be inclusive and thoughtful in your survey questions and responses. Try to think through the different constituents who might be answering your survey and their range of possible responses. Are you considering people from a wide range of backgrounds and identities? Are the race and ethnicity categories you included able to represent everyone who responds? Are you making assumptions about your population? Are you using current terms and language?
Every single question that a survey respondent sees should have a valid, truthful answer for them, even if that answer is sometimes “Don’t Know” or “Prefer Not To Respond” or some version of “Other.” When in doubt about the best answer categories, err on the side of asking open-ended questions so respondents can provide the answers. While this can take longer to analyze, it will provide robust, inclusive answers. You can use these responses to craft answer categories the next time you create a survey.
Pay attention to the accessibility of your survey. For online surveys, this includes whether it can be taken by an individual who uses a screen reader. Fortunately, most survey software now includes checks and guidance for this type of accessibility.
You may also want to consider whether a paper or online survey would be better for your audience. How tech savvy are they? Do they have access to reliable internet? Or would it be better to ask people to take a paper survey?
Finally, does your need to be translated into different languages or provide assistance for individuals who may not be fluent speakers of the language your survey is in. Aiming to be as inclusive as possible respects your survey participants and makes it possible for them to respond in an honest way, while feeling respected.
10. Cut, cut, cut.
My number one piece of advice, even though it is listed last, is to make your survey shorter! Unless it’s 5 questions or fewer, consider cutting some questions. It is so easy to think of a survey as an opportunity to ask as much as possible of your participants, to gauge info on things you are curious about, and get as much detail as possible.
Once again, think about the burden on your participant. If you have ever invited to take survey will only take a short 20 minutes, I’m guessing you have thought what I think: 20 minutes isn’t short! Even 10 minutes can feel really long when you are taking a survey, and our attention can easily be pulled elsewhere - an incoming email, text message, the lure of a more interesting update on Twitter.
Be ruthless in cutting your survey down. Think about what is “nice to know” versus “critical to know.” And finally, keep your participant in mind. You want valuable information from them, so make it as short and easy as possible.
Get Started Creating Great Surveys
Surveys are one of multiple tools that nonprofits and other organizations can use to gather data and understand your results. Surveys can capture insights that are not easily obtained from demographic analysis or examining outcomes, program efficiency, or other metrics.
A good way to get started with writing better surveys it to edit an old survey your organization has used in the past. Examine each questions side-by-side with these 10 tips. Take a crack at making each question a little shorter, clearer, easier to answer, and more inclusive - or maybe cut the question completely. When you are done, you'll have a survey that will do a better job getting the data you want and your respondents will have an easier time giving it to you.
Guest Writer
Reporting your impact is hard when you’re juggling spreadsheets. countbubble makes it easy so you can focus on your mission.
countbubble is powerful yet simple software that enables nonprofits to collect and report data on their services. Get rid of the spreadsheets and give countbubble a try.
Feel free to email us at contact@countbubble.com to start a conversation, or sign up for email updates.
Founder, CountBubble, LLC
Comments