How to Score Recruitment Sources and Verify Panelists at Signup

Recruiting quality panelists is largely a measurement problem, and that measurement should start when someone signs up.

For the broader recruitment process—including panel sizing, recruitment channels, and onboarding—see OnGraph’s guide to building and managing an owned research panel. This guide focuses on what happens after someone decides to join: how to verify new panelists, compare recruitment sources, and measure the quality of the people each source brings in.

Who You Recruit From Changes Who You Recruit

Not every recruitment source produces the same type of panelist.

A 2025 study published in Political Analysis looked at professional survey-taking using actual web-browsing data instead of asking participants about their own behavior. The study classified someone as a professional survey taker if they made more than 100 survey visits per active day.

The results were very different across sources:

• 34.7% of respondents from a marketplace router met the definition.
• 7.6% of respondents from a managed panel met it.
• 1.7% of respondents recruited through Facebook advertising met it.

That’s a roughly twentyfold difference between the highest and lowest sources.

Being a professional survey taker does not automatically mean someone provides poor-quality data, and the study does not make that claim. What the research does show is that the recruitment source has a major effect on who enters your panel.

Cost shows a similar pattern.

A 2023 PLOS ONE study compared several recruitment platforms and found that the share of high-quality respondents ranged from 67.9% on Prolific and 62.0% on CloudResearch to 26.4% on MTurk. The cost per high-quality respondent was $1.90, $2.00, and $4.36 respectively, compared with $8.17 for a managed Qualtrics panel.

The interesting part is that MTurk, CloudResearch, and Prolific all paid participants the same $0.96. The difference in cost per usable respondent came from how many respondents actually passed the quality requirements.

The data is from 2023, before the recent increase in AI-generated responses, but the lesson remains useful: recruitment source should be treated as a quality metric, not just a cost metric.

Most panel platforms track the cost of acquiring a new panelist. Fewer track fraud rate, first-survey completion, or long-term retention by recruitment source.

Those numbers are much more useful when deciding where to spend the next recruitment budget.

What Verification at Registration Actually Looks Like

Email confirmation alone is not enough to verify a panelist.

The large panels that publish their verification practices use several checks during registration.

Toluna’s ESOMAR 37 response describes its signup quality checks as including:

• Double opt-in
• CAPTCHA
• Blocking disposable email domains
• Checking for duplicate email addresses
• Checking duplicate contact details
• Detecting unusual registration patterns

Dynata also describes a separate enrollment process. Its documented checks include RegGuard, Real Mail, Verity, RelevantID, device and IP anomaly detection, IP reputation checks, and machine-learning analysis of open-ended responses.

These checks happen at enrollment and are separate from controls used later in the survey router.

Prolific provides another useful example. It says it performs more than 110 verification checks before someone gets access to its participant pool. It also uses live identity verification through a third-party provider, requires a mobile number registered in the participant’s country of residence, and uses an onboarding study with attention checks.

Prolific also reports admitting roughly 13% of applicants from its waitlist.

That shows an important point about panel recruitment: registration does not have to be a wide-open funnel. It can be a quality filter.

For teams evaluating panel management software features, this creates two practical requirements.

First, verification needs to happen during registration, not only when someone enters a survey.

Second, the platform should save the verification signals it collects. A fraud score that is discarded after signup cannot help you understand later why a particular recruitment source produced poor-quality panelists.

What Signals Should You Check?

Modern panel verification can use several types of signals together.

Verisoul, for example, documents network and device signals that can identify things such as:

• Proxy and VPN usage
• Datacenter IP addresses
• Country mismatches
• Timezone mismatches
• Device and browser links between accounts
• Duplicate accounts

Its email intelligence can also classify email domains as personal, business, education, government, disposable, relay, or inactive.

IPQualityScore publishes fraud-score thresholds as well. Scores of 75 or higher are considered suspicious, 85 indicates suspicious activity, and 90 or higher represents abusive or malicious behavior.

The exact thresholds will depend on your verification provider and panel, but the broader approach is more important: don’t rely on a single signal.

Email, IP, device, phone, location, and account-linking signals can work together to identify suspicious registrations.

Price the Qualified Panelist, Not the Recruit

Cost per recruit is easy to calculate, but it is not necessarily the number you should optimize.

A better metric is the cost of acquiring a panelist who:

• Completes registration
• Passes verification
• Completes their profile
• Takes their first survey
• Continues participating over time

In other words, measure cost per qualified panelist, not simply cost per registration.

A 2024 experiment published in JMIR Formative Research shows why recruitment incentives need to be evaluated alongside quality.

The study tested three Facebook and Instagram recruitment approaches. The group that received no incentive produced zero fraudulent responses, but the cost per complete was $58.26.

Adding a $5 gift card reduced the total cost per complete to $10.84. A $15 gift card resulted in a cost of $17.38 per complete. Fraud was 4.7% and 4.5% in the two incentive groups.

The $5 incentive therefore offered a much lower acquisition cost while introducing a relatively small amount of fraud that could be detected through verification.

There was another important finding. The no-incentive group was 65.4% White, compared with 42.0% and 44.3% in the incentive groups, and it was also considerably wealthier.

So removing incentives may reduce some fraud, but it can also make recruitment slower and less representative.

The right approach is not simply to remove incentives. It’s to design the incentive and verification process together.

Connect Verification With Recruitment Costs

There is another useful capability to consider when evaluating recruitment systems.

Some verification providers can automatically reverse an affiliate conversion when a registration is identified as fraudulent. This means a recruitment source that sends bad traffic does not continue getting paid for those registrations.

This creates a direct connection between verification and recruitment spending.

Instead of simply reporting:

“Source A has a high fraud rate.”

you can build a process where poor-quality traffic also affects how much that source gets paid.

That’s what makes source-quality measurement useful beyond reporting.

Recruitment Protocol Matters Too

The recruitment process itself can affect both quality and cost.

A peer-reviewed field study in Nigeria changed its Facebook recruitment protocol and reported significant improvements:

• Device duplicates fell from 21.1% to about 2.3%
• Duplicate phone numbers fell from 15.4% to 0%
• Straight-lining fell from 7.4% to 3.7%
• Cross-wave attrition fell from 76% to 7%
• Mean cost per ad set fell from €48.49 to €29.72

The important takeaway is that better recruitment controls do not always mean higher costs. In some cases, improving the recruitment process can improve both quality and acquisition efficiency.

Specialist Audiences: Verify the Credential, Not the Claim

Healthcare and B2B panels need another layer of verification.

If the value of a panelist comes from their professional role, asking them to type their job title into a registration form is not enough.

For US healthcare professionals, the NPI registry operated through the CMS NPPES service provides a useful verification source. The registry is free and public and can return information such as:

• NPI
• Credential
• Active status
• Specialty taxonomy
• State license number

That means a healthcare panel can verify a claimed professional identity during registration without relying entirely on self-reported information.

Specialist healthcare panels go further. M3 describes three-factor verification covering identity, qualifications, and specialization. Sermo requires license or NPI verification before granting access to its community.

Industry guidance also supports collecting additional information for professional respondents. The ESOMAR/GRBN sample quality guideline lists fields such as full name, business address, business telephone and email, professional identification numbers, and specialty.

For B2B panels, a similar process can include:

• Business email verification
• Professional profile matching
• Company and firmographic checks
• Employment verification

There is an important caveat here.

Older ESOMAR/GRBN guidance referenced research questioning whether identity validation improves data quality for general consumer respondents. That research found little difference in response patterns between validated and unvalidated participants.

So identity verification should not automatically be treated as a universal solution for consumer panels.

The case is much stronger for healthcare and professional panels, where the respondent’s professional identity is part of what makes that person valuable.

Verification Does Not Stop at Signup

A panelist who passed verification two years ago may not have the same profile today.

People change jobs, locations, household circumstances, and other profile information. That’s why verification should not be treated as a one-time event.

The panel should continue checking quality after registration.

One vendor-published test of 1,044 completed responses that had already passed an initial sample-provider screening found that some fraudulent respondents caught by device forensics had already passed behavioral evaluation. The combined system flagged 84% of low-quality respondents.

Because the study was vendor-published, its results should be considered in that context. The broader lesson is still useful: device checks and behavioral checks can catch different problems.

For panel operators, a practical approach is to:

• Re-verify panelists periodically
• Refresh profile information on a defined schedule
• Repeat device and duplicate checks
• Maintain a quality score for each panelist
• Carry that score across surveys instead of starting from zero each time

This is also something to consider when evaluating panel management software challenges and solutions.

The Recruitment Source Scorecard

If you manage a research panel, you should be able to review each recruitment source in one report.

A useful monthly scorecard can include:

1. Registrations Started and Completed

Track how many people start registration and how many finish it.

A high drop-off rate can itself be a signal about the quality of the traffic you’re buying.

2. Verification Failure Rate

Measure the percentage of registrations rejected during verification and identify the signals responsible for those failures.

3. Cost per Qualified Panelist

Calculate recruitment spend divided by the number of people who pass verification and complete their profile.

Don’t divide spend by registrations alone.

4. First-Survey Completion Rate

Track how many newly recruited panelists actually complete their first survey.

This is one of the first indicators that a new panelist is genuinely interested in participating.

5. 30/60/90-Day Activity

Measure whether each recruitment cohort continues participating after signup.

6. Fraud and Quality Flags per 1,000 Completes

Track quality problems after panelists join and attribute them back to their original recruitment source.

7. Twelve-Month Retention

Long-term retention can completely change the ranking of recruitment sources.

A source that looks cheap at signup may become expensive if most of its panelists disappear after a few months.

NORC’s published AmeriSpeak funnel provides a useful example of why recruitment, profiling, retention, and completion rates need to be considered together. These stages compound. For example, a 10% recruitment rate followed by 60% profiling, 30% retention, and 60% completion produces a true response rate of roughly 1%.

8. Reward Liability per Active Panelist

Track how much the panel continues to spend on rewards for each active panelist.

This gives you a better view of the ongoing cost of maintaining a recruitment cohort.

Two sources can have exactly the same cost per recruit and still perform very differently across these metrics.

Without this scorecard, those differences remain hidden and the recruitment budget tends to keep going to whichever source looks cheapest at the top of the funnel.

Where OnGraph Fits

OnGraph builds recruitment and verification features into its panel management platform instead of requiring teams to manage everything through separate tools.

The panel management software supports multi-region recruitment, registration-based profiling, GDPR-compliant data handling, fraud protection, and reward automation. Its survey panel integration connects the panel with survey fieldwork.

For teams building a new panel, OnGraph also provides guidance on how to build a survey panel website.

For custom or white-label requirements, OnGraph provides market research software development and panel management software development services.

The Bottom Line

Recruitment quality is not only about choosing the right channel.

It’s about measuring what each channel actually produces.

A strong panel should verify people when they register, keep the signals collected during verification, connect later quality problems back to the original recruitment source, and periodically re-check existing panelists.

Most importantly, measure cost per qualified panelist, not just cost per recruit.

A recruitment source that looks cheap at signup may turn out to be expensive once you account for verification failures, survey completion, fraud, retention, and rewards.

The panels that track those numbers can make better recruitment decisions and build a much more useful respondent base over time.