A view from the other side

Chris Mawn Article

McNair yellowSquares prides itself on treating it’s “surveyvillage” panel members as actual people and avoids at all costs referring to them as “respondents”. Because you know what? They are actual people! 

We wanted to find out more about our panel members – who is really behind the email address and the “Gender Age Location Non-user/User” quotas.

We invited a random cross-section of our panel community to our online discussion platform. The “mySpeak” platform has been designed to be a familiar and comfortable environment for anyone that has used any kind of social media.

Our discussion was set over a 3-day period with 3-4 tasks per day. Tasks included asking our members (or as we refer to them our “villagers”) to record a video to introduce themselves, share pictures, describe other kinds of activities or hobbies and their day to day life. But more importantly, what they really think and know about market research, what they enjoy or hate when participating and what they want from surveys and research in general.

Loyalty

Only a very few panel members will remain faithful to just one panel. Most are members of 5-6 other panels although only really highly active in 2-3 of them. On inactive memberships they tend to just ignore the majority of the survey invites rather than unsubscribe (just in case the “dream” survey comes along!)

Motivation

I’ve long suspected that incentive alone cannot possibly be the sole motivator for members completing survey and for the vast majority we spoke with they genuinely said it was important to be heard.

“The thing I like the most is I get to have my say. In the real world (my job), I never get a voice. But with survey panels, I get to say exactly how I feel and it helps not just me, but researchers who study trends etc.” Male, 38, Melb

However, unsurprisingly and fair enough, the key motivator (certainly within panel) IS still incentive. 

“I like to express my opinions a lot but I wouldn’t do them (surveys) without the incentive” Female, 24, Perth

So, do panels pay their members enough incentive?

It seems Australian panels tend to reward their members from 10c to $1 per 5 mins of survey. The members we spoke with agreed that a value of around $1.25 per 5 mins seemed fair and reasonable. However, a source of ongoing frustration was increasingly long screeners in surveys where members would spend 3-4 mins (and reports of 10 mins) completing a survey to be then screened out or quota failed and given nothing. 

The panel members seem to all agree on the same pain points when completing surveys. The top 3 being Matrix/Grid questions, surveys on financial products and being ‘forced’ to rate or rank a product or brand they were not familiar with.

The majority of participants in the discussion said they recognise a survey URL and/or survey provider and will make a decision as to whether they commence the survey or not (regardless of incentive) – knowing that they are usually either mislead on “length of survey” or the survey will likely have technical issues, require “Flash” (yep some still do….) or not work well/at all on a mobile device. 

Overall, panel members we spoke to in Australia tended to prefer the domestic/Australian based panels (such as TEG, Pureprofile and our very own surveyvillage) and found international panels’ surveys more “glitchy” and less engaging. A common complaint was they would be asked their gender, age and location several times before even commencing a survey, likely because of survey routers.

The ongoing discussion of survey length continues to rear its head and appears to be a factor in why some are not as accurate as they could be when completing surveys. However it would seem it’s not just length of interview rather “Length of interview” + “Incentive” that is the combination to be perfected.

“Yep sometimes rush those long boring surveys especially when I feel that I’m not getting adequately compensated for my time.” Male 50, Syd

It’s not just time and money that contributes to respondents taking less care, it’s also poor survey design and layout.

(on rushing/straight lining) “I have done it in the past where the survey has not been engaging or it has dissolved into gridfest. Too many options on a single page displayed in a completely non-engaging way. That goes on and on over pages making you start to lose the will to live. You know the ones” Male, 45, Sydney

What does the ideal survey experience look like for a respondent?

  • No longer than 15 mins (consider chunking larger surveys)
  •  MUST work (well) on mobile and tablet devices
  • Interactive questions types – make it fun! 
  • Minimal / no grid questions
  • Non-repetitive 
  • No lingo or “business” speak
  • Fast/responsive survey – not glitchy
  • Screen outs should only occur within first couple of questions
  • Rewarded sufficiently $3.75-$4 for 15 mins (and overwhelmingly panelists WANT to be paid by PayPal!)

It was comforting to hear that ALL participants in the discussion trust us as an industry, and trust that their privacy and security is looked after. They also consider Market Research important, necessary and love being a part of it.

This is good news and very fortunate for us as an industry, as we would not exist without “respondents”, yet I still think we need to do more to recognise, acknowledge and value their actual contribution. 

With continued pressure to reduce the cost of panel sample, will come the pressure to also reduce incentives. But by reducing incentives there is the risk of reducing response rates and data quality. To prevent this we need to respect the people that complete our surveys with appropriate rewards and realistic survey lengths, to ensure they stick around and keep our industry alive.

**

Chris is Online Director of McNair yellowSquares Pty Ltd – an independent Australian online fieldwork agency based in Sydney.

For the purposes of transparency and full disclosure the surveyvillage panel pays between 80c and $1 per 5 mins of survey and 10c for short screen-outs/quota fails for standard mainstream surveys. We use the services of Gift Pay to reward our members. An edited version of this article was originally published in the AMSRS “Research” publication. All members used in this article have given consent for their images and comments to be used.