Here, AI-led interviews mean conversations with current employees about their work, not interviews with job candidates.
Thirty-two of forty survey respondents rate their workload favorably. In this fictional service team of 50 people, that looks reassuring. Later, in optional AI interviews linked to those responses, six of the favorable respondents describe client changes that regularly keep them working after 9 p.m. at month-end.
The interviews contain a specific concern that the headline score doesn't show.
A leader could decide the survey failed. Another could decide the interview encouraged complaints. Both judgments arrive too soon. Someone can consider their workload manageable overall and still want a recurring late-night scramble fixed.
When a score and a conversation appear to disagree, neither should win by default. Leaders can investigate a concrete problem before settling which answer better represents an employee's overall view. The useful questions are what was reported, how widespread it is, and what response the evidence justifies.
AI interviewing makes that question more pressing because it can produce detailed accounts alongside familiar scores. Those accounts deserve attention. They do not automatically establish how common a problem is, why an employee answered differently, or which organizational response will help.
A concern can coexist with a favorable rating
A September paper by Thilo Tamme and colleagues examined a consultancy’s survey-first, AI-voice-interview process. In 18 of 44 first sessions, a favorable rating coexisted with a substantive concern on at least one matched theme. Counting only the top response, “Agree,” as favorable reduced that figure to nine sessions. The 132 matched observations were three themes per session, not 132 employees. Workplace study
This is a small, voluntary, single-organization study. It has no comparable human-interview condition, randomized channel order, or independent truth criterion. The authors explicitly caution that transcripts are not ground truth.
The authors go further: their model of bounded disclosure interprets softened and redirected concerns as self-protection, and recommends treating favorable scores as a ceiling on contentment. They address the probing objection directly: each theme began with the same neutral question and participants received the same probing approach, so they argue prompting alone cannot explain who showed a gap.
That interpretation deserves consideration. A shared interview protocol, however, doesn't isolate the effect of changing from a rating to an adaptive conversation. My narrower conclusion is that a specific concern deserves examination without diagnosing why a particular employee gave a favorable score.
Before calling it a contradiction, compare the questions. Did both ask about the same period? Did the rating concern the overall job while the conversation focused on an exception? How much prompting preceded the concern?
A person who likes their team can dislike one scheduling practice. A person who generally has enough capacity can struggle during closing week. Neither answer has to be false.
Asking changes what can be answered
The interview is part of the measurement process.
An experiment published in Public Opinion Quarterly in August 2026 analyzed 2,243 US adult opt-in participants across five randomized survey arrangements, including fixed open-ended follow-ups and text-based AI interviews before or after closed questions. The data were collected in December 2024, on immigration and tariffs. AI elicited more reasons; AI follow-up also increased changes in later repeated ratings relative to closed questions alone. Fixed follow-ups produced some similar-direction changes, with less consistent evidence. Randomized comparison
The interviews used CloudResearch's Engage platform; co-author Leib Litman is the company's Chief Research Officer. The study used political topics and LLM-assisted coding. It cannot tell an employer how employees will respond. It does provide a reason to examine the questioning and its order before treating the later answer as a more accurate reading of an unchanged attitude.
Detail can help people articulate an exception. Reflection can also change the answer they would give. A listening process should leave room for both possibilities.
The people who finish matter
A transcript collection also leaves out people who never complete an interview.
In a separate randomized study conducted by survey-panel company Verasight with AI-interview platform Outset, Verasight reported completion of 99.4% for written responses and 40.5% for external AI audio/video interviews. Most loss occurred at the platform handoff. Participants had not been warned about the spoken format, and completers differed on prior attitudes and response preferences. Verasight report
Those percentages are not a forecast for an employer’s program. They show why implementation and participation belong in the evidence. A method can collect richer accounts while hearing from a narrower group.
The Public Opinion Quarterly experiment did not detect comparable AI-related rolloff. Its text-based interviews, participant pool and implementation differed from Verasight’s spoken, externally hosted condition. The comparison doesn't isolate what caused the different completion results.
For a buyer, a compelling sample transcript is therefore only part of the demonstration. Ask how many people were invited, how many started, how many finished, and what alternatives were available to those who preferred another format.
Keep three questions separate
First, what was reported?
Six favorable survey respondents described late client changes and after-9-p.m. work at month-end. Preserve the accounts and their context. Check that the interview summaries accurately reflect what was said, including qualifications. Repeated mentions by one person should not become several employees in a count.
This gives the organization a concrete issue to examine. It gives no basis for labeling those employees dishonest, disengaged, or secretly dissatisfied.
Second, how widespread is it?
The example establishes 32 favorable responses among 40 completed surveys. It does not establish that 80% of the whole team is satisfied, or that everyone who rated workload favorably has the same experience.
Nor do six accounts establish the proportion of the team affected by late work. We have not specified how many people were invited to interview or completed it. Even with those denominators, voluntary participation and different prompts would limit a workforce prevalence claim.
To learn more, ask the team a consistently worded question about a defined period: on how many days in the last month did late client changes require work after 9 p.m.? Include an option for no such days, and report nonresponse. Keep the broader workload question if that broader judgment still matters.
The new question measures a more specific experience. Its answer should not quietly replace the original score in a trend chart.
Third, what response is warranted?
The six accounts could justify examining a bottleneck without establishing that most employees experience it. Under existing authorization and privacy rules, review aggregate scheduling or change-request records for the relevant period. Such records can help check timing; they may miss informal work and cannot reveal how people felt.
If the evidence supports the account, try an earlier client-change cutoff with an explicit exception route and shared late coverage. Set a review point, then examine late work, missed client commitments and whether the burden simply moved to another team.
If the sources disagree, retain the uncertainty and ask what each can actually observe.
Useful listening does not require a majority
There is a reasonable objection: leaders cannot investigate every qualification in every interview.
They should prioritize. Consider the potential harm, the specificity of the account, the cost of checking it, and whether a reversible response is available. A rare, serious concern may warrant prompt attention through an appropriate process. A vague suggestion may call for clarification before resources are committed.
Prevalence helps with some decisions, such as staffing a service across an organization. It is not a prerequisite for taking every concern seriously. Conversely, a vivid anecdote alone is a weak basis for a sweeping reorganization.
The important discipline is stating why this evidence supports this action.
Make the interview earn its place
Before expanding AI interviews, I would run a bounded comparison around one operational question. Offer voluntary participation and a workable alternative. Explain whether responses are linked to individuals, who can read them, and what decisions the material will inform. Linking the survey and interview made the paired comparison possible in our example. It also creates a privacy tradeoff and may affect willingness to speak. Decide whether that comparison is needed, and limit who can access the link. Unlinked feedback can still surface useful problems.
Compare the AI conversation with a well-designed fixed open-ended question or the existing interview process. Keep the topic and reporting period comparable. Where appropriate, vary order across groups so a survey-first sequence does not become an invisible assumption.
Inspect actual follow-up questions. Have reviewers identify concrete concerns before seeing the associated ratings, then examine the pairings. Count invitations, completions, participant time and review effort. A small pilot can expose failure modes; it cannot establish a reliable population rate or validate the entire method.
Ask which useful issues the AI conversation adds beyond the simpler alternative, and whether that gain justifies participant and review time. Longer transcripts and confident summaries are insufficient.
For the fictional team, the goal is to reduce the month-end scramble without moving the burden elsewhere. A successful trial would give leaders a reason to keep the change. They don't need a verdict on which employees meant their favorable ratings to start investigating.