“Paper … the garbage version of the internet.” (Ze Frank)
So before you end up with a big pile of paper surveys (and per the mining company survey mentioned in “Paper Surveys (Part 1)” a significant amount of rock dust) lying around the home/work office floor, let’s talk about setting up the survey and arranging for data entry.
The art of paper survey design.
Assuming you are not entering thousands of surveys and that there is an accompanying online version, I’d suggest making the paper survey look as much like the online survey as possible – to the point of printing the online version to PDF if that works.
Yes this means more page turning by the data entry person and so some added costs as a result. But it also means that:
- you won’t have two different versions of the survey (with the associated chance of error)
- the data entry person will be less likely to mix up what response goes where
Additionally, and here’s where the paper and online survey may differ, you’ll want to add small data entry codes next to each response option (ex. Yes=1, No=2) to make the data entry fast, easy and less error-prone (data entry people LOVE number pads).
Oh and if you’re thinking of adding “secret codes” to each of the paper surveys for help identify the respondent or their demographics (ex. location), it might work if you absolutely have to. But be cautious as people tend to figure that out pretty quickly, and will either not complete the survey, not answer honestly, or tear off the offending corner of the survey before sending it in.
Data Entry – What to enter?
Assuming that you’ve done the above re-numbering response options, the scaled results are easy to enter.
But what about the written comments? Unless you are purposely trying to obfuscate the source of each comment AND your data entry person is a content expert, we highly recommend typing comments “as is” – typos and all – using “[sic]” where applicable to tell the difference between a respondent erorr [sic] and an entry error.
It’s a dangerous path to go down in allowing a data entry person (data-enterer?) to start making decisions that will affect the data. This is true both in reality, and conceptually if people become aware that this is being done.
Data Entry – Who is entering?
Now how are you going to process all those paper surveys – complete with written comments where the handwriting makes the respondent clearly identifiable – and in a way that is still seen to be confidential?
You COULD send them to someone internal to the organization who has no stake in the game and who wouldn’t recognize anyone’s turn of phrase or handwriting. But they would also have to be relied on to keep the completed surveys confidential – even when someone in upper management orders them to send on a PDF scan of one or more of the surveys on the threat of termination.
Given that, and remembering that employee impressions of confidentiality are just as important as actual confidentiality, the best solution of course is to have the surveys sent directly to a third party to process and enter.
A note on language: If your survey is in more than one language and you have questions that require a written response, you’ll want to keep that in mind when hiring for data entry. Yes it’s possible to type in any language without knowledge or context (assuming a common alphabet), but your typos will go way up.
Data Entry – How is it being entered?
Rather than typing the results into Excel or Word, if there is an accompanying online version it’s usually better and easier to type paper responses directly into that. This reduces data entry error due to the parallel layout and common scale values and helps make sure that the right response goes in the right place.
To allow for this, ask the people running the survey to create one or more “open” survey links that allow multiple submissions, and you’re good to go. (You can even track the data entry of each person by providing them each with unique links.)
So to recap:
- Make the paper survey look like the online survey (if there is one)
- Add response codes to the options (1,2,3,…) to ease data entry.
- Try not to use “hidden codes” unless it’s critical
- Add verbatim comments “as is” – using [sic] where applicable
- Use an external data-entry service if at all possible
- If it’s a multi-language survey ensure that the data-entry person knows each.
- If possible have the responses typed from paper directly into the online version of the survey
As always, give us a shout if you have any questions on this, and next up: “Employee Survey Pro Tip Series 13 – Testing your survey links and avoiding the spam bucket“
