The Question Behind the Question: Auditing Assessor Prompts for Fairness, Access and Coverage

In vocational assessment, the question bank is often treated as a quiet administrative aid: a list kept alongside the assessment plan, dipped into when a professional discussion needs structure or a witness statement needs clarification. That understates its influence. In professional discussions, witness testimony and supplementary evidence conversations, questions are not neutral containers. They shape what a candidate or witness notices, recalls and feels able to say. A poorly framed prompt can narrow an account, exclude a competent candidate or introduce variation between assessors that is difficult to defend.

Auditing the question bank is therefore a quality assurance activity, not a trap-setting exercise. For UK providers working within inclusive assessment and compliance expectations, the aim is to make questioning open enough to capture genuine practice, structured enough to cover the criteria and consistent enough to withstand internal quality assurance and external scrutiny.

Begin with the purpose, not the list

A useful audit starts by restating what the question bank is for. It should elicit the candidate's own account, test the breadth of the assessment criteria and allow proportionate follow-up where evidence is thin or ambiguous. It is not a script to be read without thought, and it is not a coaching tool. The assessor's task is to create the conditions in which evidence can be produced, not to supply the answer through the question.

That distinction matters most in conversations. In a portfolio, the evidence is already fixed. In a professional discussion, the question itself becomes part of the evidence-generation process. If the prompt says, 'You did check the risk assessment, didn't you?', the candidate is being invited to agree rather than explain. A fairer prompt is: 'Take me through how you decided what checks were needed before the activity.' The second version tests the same criterion without supplying the expected answer.

Four fault lines to look for

Leading prompts

Leading questions imply the assessor's preferred answer, contain an assumption or bundle several points together. They are especially risky in witness testimony, where the assessor should not contaminate an independent account by introducing details the witness has not offered. A question such as, 'How did you manage the handover after the incident?' assumes an incident and a handover. A more defensible approach is to ask the witness to describe what happened in their own words, then probe only where clarification is needed.

Coverage gaps

Many question banks grow organically. Questions are added after a difficult assessment or removed when a unit changes, but the whole set is rarely mapped back to the criteria. The result can be uneven: one criterion is explored through six prompts while another has none. Build a simple coverage matrix with columns for criterion, question, evidence type, whether the question is core or optional, and the intended follow-up. Gaps and over-weighting become visible quickly.

Inaccessible wording

Accessibility is not only about reasonable adjustments. It is also about the everyday clarity of the question. Long, double-barrelled prompts, unexplained acronyms and culturally specific references can obscure what is being asked. A candidate may understand the vocational task perfectly but stumble over the wording of the question. Review questions for plain English without stripping out technical language that is genuinely part of the standard. Where a technical term is essential, clarify it neutrally rather than replacing it with something vague.

Inconsistent follow-ups

Assessors often improvise follow-up questions. Some of that flexibility is healthy, but wide variation can create fairness problems. One candidate may be asked for a specific example; another may be asked a closed question that invites a yes or no. Develop an optional probe menu with neutral follow-ups such as, 'Can you give me an example?', 'What did you do next?' and 'What was the outcome?' These keep the conversation moving without turning it into an interrogation.

A five-step audit

First, sample and label. Select questions actually used in the last assessment cycle across professional discussions, witness testimony and supplementary evidence conversations. Record where each question came from and which criterion it is intended to serve.

Second, build the coverage matrix. Map each question to the criteria and evidence requirements. Identify duplication, gaps and prompts that collect opinion rather than evidence of practice.

Third, run a read-aloud test. Ask two assessors to review the questions independently and flag leading language, ambiguity, double-barrelled wording and inaccessible terms. The read-aloud step is important because questions that look reasonable on paper often sound very different in conversation.

Fourth, role-play a response. One assessor plays a candidate with limited but sufficient evidence; another uses the question bank. Check whether the questions allow the candidate to say 'I don't know', ask for clarification or offer an alternative example. If the conversation feels like an interrogation, the fault may lie in the sequence and tone of the prompts rather than in the candidate's competence.

Fifth, compare records. Sample completed assessment records to see how the questions were actually used. Note where assessors skipped core questions, added leading probes or accepted a single-word answer without follow-up. Produce a short action log with owners and dates.

Rebuilding without turning assessment into interrogation

A defensible question bank does not need to be rigid. It needs to be deliberate. Rewrite prompts using open stems: 'Describe how...', 'Walk me through...', 'What prompted you to...', 'How did you decide...' and 'What would you do differently?' Keep the number of core questions proportionate to the criteria. Provide optional probes rather than long lists of mandatory prompts. Define when the assessor should stop and move on.

Candidate briefing also matters. Candidates should know the format of the conversation, that they may ask for clarification and that adjustments are available where needed. That is not the same as publishing the exact question bank. It is a matter of making the process understandable without compromising its integrity.

Governance and CPD

Give the question bank an owner, a review cycle and a change log. Include it in internal quality assurance sampling and standardisation sessions. Use role-play recordings to compare how different assessors use the same prompts. Review the bank after appeals, complaints or patterns in assessment decisions. For assessors, this work belongs in CPD: it sharpens judgement, supports inclusive practice and provides evidence of reflective professional development.

The question behind the question is always the same: does this prompt help the candidate show what they can do, or does it show what the assessor has already decided? An audit that keeps asking that question will produce a bank that is fairer in use, stronger in coverage and far easier to defend.