Writing Your Online Survey – 7 Items to Caché

One of the most important factors, if not THE most important factor when fielding your online survey is: quality of responses. How can you ensure that you will receive quality? How do you reduce your drop rates? (By the way, “drop rate” is the percent of respondents who leave a survey before completing it.)

How can you get aaaaall that data you need without boring the participant? Below is a list of the top seven items to keep in mind when designing and writing your online survey.

1. Length of Interview (LOI)

Everyone’s time is valuable; the client, the researcher, and the participant(s). Yet, there are online surveys currently being administered with length of interviews (LOIs) of 25 minutes, 30 to 40 minutes, and even up an hour long. The longer the survey, the higher the chance of losing your participant, the lower the quality of your responses due to fatigue and lack of attention.

So is there an ideal time? Should all online surveys be microsurveys? The answer is never black and white as it always depends on your research objectives. According to several studies and sources, the average attention span of humans continues to decrease. We lose focus and get distracted more than ever before due to our exposure to technology. Keep your survey under 15 minutes. Better yet, aim for 10 minutes since quality begins to diminish between the 10 and 15-minute mark.

2. What Kind and Types of Questions Should Be in Your Survey?

The answer: it depends, duh! As with most market research, it always depends on other factors like goals, clients, audience, and so on. We suggest you mix it up so that the survey does not become mundane. Answering the same question format over and over and over…and over, will cause your drop rate to increase as well as cause the panelist to enter in autopilot mode, decreasing quality of responses. Here are some ideas for spicing up your survey.

• Images
• Heatmaps
• One to three open ends (no more than this – a participant does not want to write essays for you)
• Gamification
• Scales like ratings, likerts, and multiple choice
• Drop downs, box checks, single choice, matrixes, and so forth.

Bottom line: You need to do research for your research to ensure you are getting the data you need.

3. Mobile or PC Friendly?

Make your surveys mobile and PC friendly to maximize the number of responses. According to recent studies, 77% of Americans own a smartphone, and approximately 87% of households in the US has a computer at home. Tailor questions to fit both devices. Add logic to your survey that filters respondents based on their device and allows them to see questions based on the device they are using.

4. Incentives – Money Talks.

Make your incentive attractive to your audience. Are you surveying consumers or B2B? Teens? Parents? There are several options: retail gift cards, VISA gift cards, check, sweepstakes and drawings, products, cash, and on and on. Be sure to always have your audience in mind, and the commitment you are asking from them. Do not expect your quotas to be filled giving out $1 for 15-minute surveys. Do this and you will struggle. Furthermore, your participants are likely to use gibberish, provide false data, and straight-line. (By the way, “straight line” is industry speak for respondents who go down the survey and respond the same answer for every question.)

5. Order of Questions: Behavioral vs Demographics – The Debate Continues.

Some researchers will tell you to place demographics upfront to capture that respondent data in case they drop off, while others will tell you to place demographics at the end as to not scare them off by asking personal information upfront. We find a combination works best.

Place non-invasive demographic questions at the beginning of the survey and save the more sensitive ones such as income, ethnicity, phone numbers, and addresses for the end. In between the easy and hard demographic questions, ask your behavioral questions. Furthermore, it is also best to not terminate/disqualify participants immediately after a demographic question. Add a “buffer” question to reduce and prevent panelist becoming upset.

6. KISS – Simplicity and Ease of Understanding In Your Survey

Ah, the classic KEEP IT SIMPLE STUPID (KISS). When working in research, data, and analytics, we sometimes get too granular with survey questions and forget to just keep it simple. Be sure that your questions are not confusing or require difficult mathematics. Keep it clear and concise with your directions, and do not make assumptions. If you cannot get around a complicated or wordy question, provide examples and be careful to not introduce bias.

7. TEST TEST TEST…And Then Soft Launch…And Then Full Launch!

There is nothing more annoying and frustrating than finding an error in your survey. Whether that be logic or a typo, those mistakes can cause difficulties in the analysis of data, or worse, it can cause for data to be thrown out. Be sure to test every skip, logic, terminate, qualifier, and read as if you are the one intended for taking the survey. Stop, and take your time when testing.

If possible, ask a colleague to test the test as you can easily overlook errors when programming. Once tested, soft launch with a small group to ensure you find any other errors. If data looks good, proceed with full sending your surveys out.

As researchers, we strive for quality data and need to remember that without participants, we would not have data. Always take a look at your survey through the eyes of your audience (while keeping in mind of your objectives of course).

 

This blog post was written by FDR’s Research and Innovation Center’s Consumer Research Manager, Victoria Holloway. The article is one in a series of articles on conducting research. If you are looking for a facility or even full-service market research services, contact us.