All of us that work for or with financial services organizations understand the mission-critical nature of protecting data assets that belong to the firms we work with and the customers they serve. This responsibility extends to the primary market research that is conducted by and for financial services firms.
Unfortunately, over the past decade as more and more research is conducted online, the allure of survey incentive payouts has given rise to fraudulent and “professional” survey respondents. The issue is magnified in B2B research due to the larger incentive payouts that are made to encourage participation across these niche and smaller populations.
Therefore, it is now more critical than ever for market research buyers to understand the scope of the issue and to partner with firms that employ robust sample quality and fraud prevention controls.
Vetting out fraudulent survey takers.
Fraudulent survey takers are individuals who take a survey more than one time and/or use false credentials and responses to qualify for a survey. Many bad actors use multiple IP addresses, computers, or browsers to mask their identity, and some configure bots to do the work for them.
Radius and our select and trusted panel partners utilize several mitigation approaches to root out fraudulent survey takers. Many of these fall under the category of respondent verification. To verify that respondents are real people who we can trust, we use several authentication factors by verifying name and address, IP address and geolocation. We look at performance ratings on past studies, and blend panel sources to mitigate biases or threats, and use sophisticated software to assist in these efforts.
It is now more critical than ever for market research buyers to understand the scope of the issue and to partner with firms that employ robust sample quality and fraud prevention controls.”
Software we use to vet respondents:
CleanID is an API that collects data from respondents and runs algorithms utilizing geolocation, IP address, time, and language to develop a fraud profile score and flags to assess panel acceptability.
We also use:
– Relevant ID/Imperium
– Research Defender
– Queens ID
– Torfac, Global Checks
– Survalidate
– Serif
– Iovation
Adding protections for B2B studies.
We use additional methods for B2B studies such as phone recruitment from known professionals to target potential respondents via employment profile and conduct in-depth interviews with industry analysts specific to certain providers. We verify workplace via a phone call, LinkedIn and additional social media and workplace email confirmation. In interviews, we ask standardized screening questions per industry, with red herring markers.
Methods to identify frequent survey takers.
Individuals who are legitimate survey takers, but speed through multiple surveys in a short amount of time to maximize incentive payments represent another threat to data quality. The danger is that frequent survey takers are speeding through surveys just to complete them without providing thoughtful or legitimate responses.
We root out frequent survey takers by building quality checks into our surveys including Turing (Captcha) and two-question tests, red herring responses, trap questions and open-ended questions.
We closely monitor interviews as they come in to identify speeders via survey completion time, and we stay in field longer to allow for higher, more diversified responses. Once the responses are in, we de-dupe respondents and review the number of unanswered questions, “don’t know” and refused responses. Additional flags that we look for include straight-lined or inconsistent responses. We also use third-party fraud prevention technology.
Questions to ask your research provider to ensure quality.
Research buyers must be aware of these issues and evaluate their partners on how they protect against data quality problems and fraud. Here are some actions you as a buyer can take immediately:
1. Proactively ask your research providers for more information on their sourcing and processes for ensuring quality.
- How are you sourcing your panel of respondents?
- What methods are you using to detect fraud and validate the authenticity of these respondents?
- What is your process when fraudulent activity is discovered?
- What is your definition of a frequent survey taker?
- Are you monitoring and controlling the number of surveys taken by an individual?
- What limits do you have regarding the number of surveys taken?
2. Demand full transparency on these issues from research partners.
3. Make sample quality and fraud prevention a critical decision factor when choosing partners.
4. Recognize that speed and cheaper budgets may negatively impact sample quality and fraud prevention controls.
Please contact us to discuss approach to data quality and fraud prevention in more detail.
