Online Surveys - Some things to consider

[Jean Douglas, CMC]

The world of surveying has opened up considerably with the availability of internet hosting and data collection services. A quick search on surveys results in a fairly extensive list of survey hosting and data management websites. Colleagues and clients have sung the praises of some of the more popular sites; describing them as easy to use, fast to set up and oh yes, cheap!

Well, there are a few things to consider before establishing a working relationship with an online survey supplier/host.
First and foremost, read the fine print! What are you signing up for and what does it mean? Here are some questions you need to answer:

• Where is the survey hosted? In what country does your data reside? This is an important question as it is relevant to Privacy Laws and Security Issues. For example, if you are surveying a Canadian population and the data server is not in Canada, then each invitee must be advised of this and you must secure their permission before they complete the survey. Survey data hosted in the US is subject to the Patriot Act, so if you intend on surveying anyone in another country, you should be advising them of this and securing their permission.

• Who owns the data? Does the survey host have the legal ability to use the data or sell it? You won’t know this without reading the fine print (Terms of Agreement and Privacy Policy). What prevents them from doing so if they are located in another country? Could they sell the data to your competition? What about data from essential and sensitive services (e.g., the military, research) that another country’s government might find interesting? Is the survey supplier owned by a marketing firm that is benefiting from your data in ways you are unaware of?

• Who owns your mailing list? If your organization considers its client and/or employee list to be privileged or confidential, then using the host’s invitation tool to send out email invites may be compromising the security of this information. Another thing to keep in mind is that when you send out individualized email invites, you must not promise the respondent anonymity, all you can offer is confidentiality.

• Who owns the survey content? If you are using a copyrighted survey or one that your firm considers proprietary, then make sure you haven’t signed away a trade secret. Some websites claim the right to use, in any way they wish, any or all content posted on their website.

• Do you have the authority to accept and sign the Terms of Service/Agreement for your organization? Most organizations asked to sign a multi-page contract would run it past legal first. For some reason, most people do not see the online contract in the same light as a paper contract. What exactly have you signed away? Are you placing you or your company in jeopardy? The wording of these contracts is providing the provider with protection more than it is its users. How legal is it? Especially when most retain the right to change the terms as and when they wish.

• Are there other security risks? Have you asked your IT department if sending out invitations and a web link through the company email server is safe? Organizations are spending a lot of money on security and firewalls – it would be a shame to be the one that provided a hacker a security breach.

• What will your invitees see? Some sites retain the right to display third-party advertising. Do you want your employees and/or clients looking at competitor ads while completing your survey?

• What assurances do you have that the data collection and the analyses provided are correct? Survey programs are not typically written by statisticians. Make sure that the results you get are correct. Are there any cautions about sample sizes and confidence intervals?

• Are you measuring what matters? The entire area of survey content must be addressed. Some survey hosts provide a database of survey items to use. Some are good and others not so much. Easy access to survey tools is a bit of a double-edged sword. Survey design is a science. Make sure that whoever designs the survey knows what they are doing. Being able to type and construct sentences isn’t enough.

• What do you do if something goes wrong? There is a reason that survey consulting firms charge what they do for survey hosting; they have taken into consideration most if not all the items listed above. A good survey consulting firm will advise you of security and privacy issues, have tested (many times) the program in all its functionality (data invitations, collection, storage, analysis, reporting and year over year comparisons), accurate translation services (e.g., peer review by experts), and security measures (e.g., server redundancy/backup in the event of the servers going down). Plus you generally know where to find them if there is a problem.

This doesn’t mean that there are not many fine online survey hosts out there. But, a simple reminder, buyer beware.

[Susan Robinson]

Jean,
this is a very valuable and informative post. Thanks for sharing it!
Susan

[Lalida Pimpakarang]

Hi Jean,

I am following your all of comments in this network. I'm really interested about employee engagement topic.And now,I'm doing my thesis that title is "The driven factors influencing employee engagement in Thailand".
I am a student of MBA of Maefahluang University in Thailand.
Moreover, I lack some information about how to measure employee engagement...as your comments in "How to measure employee engagement" forum.

Could you mind to give me some suggestions about
a. your set of engagement questions as your items follows:
1. I think [organization name] is an excellent employer.
2. I like working here.
3. I get a sense of personal satisfaction from my job.
4. I like what I do in my job.
5. I am proud of the way we do business.
6. I care about the future of [organization name].
7. I would recommend [organization name] to a friend as a good place to work.
8. I am personally motivated to help [organization name] be successful.
9. When I talk about working for [organization name], I do so with pride.

If I would like to use your items in my paper,how do they use? They can measure of employee engagement (answers about say,stay and strive),can't they? And could you mine to tell me about calculating reliability scores and predictive analyses.

b. About SurveyMonkey, I plan to use it also because my friend used to use it and it has a demo that I can pre-using. In my case, I never use it before and if you have more suggestions about it.

Finally, thank you for all comments that you gave in forum, they are really meaningful for me.And thank you for advance if you can help me about my questions. Anyway, my e-mail is "[e-mail]".
I am look forward to your reply.Thank you.

Best Regards,
Lalida Pimpakarang

[Jean Douglas, CMC]

I have recently completed two online member surveys where the surveys were hosted by SurveyMonkey - on completion my browser was instantly redirected to Survey Monkeys website where they are soliciting business. My concern about this? Well for one thing, I don't think it is professional looking to be sent to another firm's website after completing a survey. All my other reasons are listed in my earlier blog posting.