Data Quality Approach
Toluna supports all industry-wide data quality initiatives. We believe it is important to provide our clients with assurance that we have our own standard operating procedures in place to ensure data quality.
We have defined this process as SmartSample™. SmartSample™ ensures that Toluna respondents are Real, Valid, De-duplicated, Engaged, and Representative.
Respondents are GeoIP and postal-code validated using postal address files to check for a valid postal code format. In addition, each respondent completes a CAPTCHA confirmation process. New panelists are required to double opt-in. The process is as follows: Step 1: A prospective panelists completes a panel registration form, which includes contact and demographic information. This is the first opt-in. Step 2: An automatic email is sent requesting verification of registration by clicking a link that confirms their log-in details. Step 3: Once the prospect has clicked the link he or she is officially a full fledge panelist. This is the second opt-in. Step 4: After double opt-in and our controls and validation, panelists become Survey Eligible for email invites.
Toluna blocks individuals with known “fake” or “disposable” domains. Toluna also uses GeoIP and postal code validations to ensure that individual members provide a valid postal address. In the United States, members are further validated using Verity, a third-party identity validation service that confirms members’ name and address.
For our proprietary healthcare professional panels in the UK, France, Germany, Italy and Spain, we collect web URLs and phone numbers for the physician’s practice to validate. Our US physicians are AMA (American Medical Association) member validated when they join, and each time they take a survey.
Respondents can’t enroll in the Toluna community, or take a survey more than once, fraudulently or accidentally. To prevent this from happening, Toluna uses a Match 2 process to flag similarities among Toluna panelists upon enrollment. This automated process clusters individuals and helps to identify members who may have attempted to register for the Toluna panel more than once. In addition to this process, Toluna has developed Duplicate Respondent Detection™ technology. This flash/ cookie-based technology is used during the panelist-registration process, and at the beginning of every Toluna survey. This is coupled with a third-party digital fingerprinting technology, Imperium’s RelevantID™, which is used at the beginning of every Toluna survey.
Toluna’s process for preventing respondent duplication is proactive: respondents are prevented from participating in surveys more than once.
Toluna’s goal is to provide clients with the highest quality data. To do this effectively, respondents must be engaged in the surveys that they complete. Toluna provides clients with data that has been subject to many checkpoints, helping to ensure respondents have provided truthful and thoughtful responses to survey questions.
Every data file delivered to clients is analyzed for the quality of open-ended responses and spelling checks are applied; respondents who participate in less than a third of the mean response time are also eliminated from the data set, as a standard practice. Finally, cleaning scripts are used to ensure that data follows logical skip patterns, name and address information matches respondent enrollment data, and more. In addition, we include red herring questions on surveys we program.
Representativeness is achieved by selecting a sample that reflects the population-of-interest accurately. This can be achieved through the appropriate application of simple or stratified random sampling, quota sampling, or by using more advanced techniques, an individual respondent selection method that ensures that the characteristics of each respondent for each project, irrespective of originating sample source, match those of the target population of interest. At Toluna we offer all of these techniques and test them to ensure the results are representative of the population of interest.
In addition to using appropriate sampling techniques, representativeness can be achieved by weighting the data to be representative of the population-of-interest. Toluna experts understand the appropriate weight function to apply and appropriate variables to use in the weighting algorithm.
In addition to these processes, we measure client experience and satisfaction on each project. Our goal is to provide clients with high-quality service and our satisfaction rates are typically 95% and higher. When low client satisfaction scores are received, we investigate root-causes to provide a corrective action in the shortest possible time frame.
Clients Satisfaction: 98%
After each project, Toluna launches automatically a survey to measure the client satisfaction. Each answer is analyzed and leads to an improved process if needed.
With an unique combination of innovative technology, online expertise and reactive teams, 98% of Toluna Clients are globally satisfied in 2015, with detailed satisfaction for Quality and Timelines of Sampling, Programming, Data Processing or Project Management over 95% or 96% (3979 respondents).