You Already Record Every CATI Interview. Now You Can Check Every One.

Your phone room records every interview. Your QC team checks 5–15% of them, the range AAPOR's falsification task force recommends and most agencies run. Nobody picked that ratio because it's enough. It's what reviewer time allows: checking an interview against its recording takes a person about as long as the interview itself.

That limit is gone. Software can now check every recorded complete for the problems that make it unusable, within minutes of the recording arriving, and hand your team only the interviews that need a listen. This post covers what that costs next to checking by hand, what gets checked, and what your QC team does with its hours instead.

What checking by hand costs

A 12–15 minute interview takes a reviewer about 14 minutes to check, once you include noting what they found. For a 1,000-complete week:

The last row assumes one interview in five gets flagged and each flag takes three minutes to settle, listening at the times the software gives. Both numbers are deliberately high; the point holds well before you reach them. The software itself is priced per interview, a fraction of what 14 minutes of a reviewer's time costs.

So for less listening than a 10% sample takes today, every interview gets checked. The team stays the same size. It just stops spending most of its hours on clean calls.

What the sample was costing you is the part it missed. At 10% checked, an interviewer who mishandles one complete in twenty has roughly an 80% chance of getting through the week unnoticed. The full sampling math is in our CATI quality control guide.

What gets checked on every interview

Nothing new to a QC team. The difference is that it now runs on every complete instead of the one in ten someone gets to:

• No interview. Almost no speech, a respondent who barely speaks, or a recording too short for the questions that have answers.
• Cut short. The respondent is gone and the answers keep coming: the data file runs minutes past the end of the recording.
• Questions not asked, answers recorded. Checked against the route each interview's own answers take, with lists and batteries checked as a whole.
• Answers that contradict what was said. Opposites only (yes for no, a 2 for "about a nine"); a shade apart, or a don't-know on either side, doesn't count.

Only gross problems flag an interview, such as three questions not asked or three contradicting answers. A single slip stays in the record without costing anyone a listen. Each flag comes with the question and the times to listen to.

The patterns a sample can't show

A sample can't do this at any rate. A 10% sample gives you four of an interviewer's forty completes a week, too few to see a habit. Checking all forty shows it:

• The same questions missing across one interviewer's interviews. Three demographics absent from most of their completes, yet answered every time.
• Contradictions concentrated in a few interviewers. Spread evenly across the room, they point at the questionnaire instead.
• One interviewer flagged on most of their interviews, visible in the first days of field instead of at delivery.

The answer export adds checks that need no audio: one interviewer's interviews overlapping in time, answers copied from another interview, and an interviewer with far more of the survey's fastest interviews than chance allows.

What it doesn't replace

• Your QC team's decision. A flag is a time to listen at. Nothing is final until a reviewer confirms it.
• Back-checks. A screener read correctly to an ineligible person sounds fine on the recording. Contact and eligibility still need re-contact.
• Coaching on delivery. Question reading, probing and clarification are a different review, with feedback to each interviewer; that's Deep Quality Control.

Interviews without a recording, or split across several, are reported as not checked, never counted as clean.

Try it on a project you've already delivered

The easiest test is a finished project: the questionnaire, the answer export and the recordings. Every interview gets checked, and you can compare the result with what your sample found at the time.

Whatever you use, keep the record: for each delivered complete, whether it was checked, what was flagged and what your team decided. When a client asks how the data was verified, that record is the answer. Checking every complete this way is what Integrity Screening does.

---

Sources and refresh policy

• AAPOR/ASA, Interviewer Falsification in Survey Research (2003): the 5–15% verification recommendation
• IQCS Standards 2023: the 10% validation minimum
• VeriCATI, CATI quality control guide: reviewer time per interview and the sampling math behind the 80% figure