Calibrate Call Reviews Before Using Scores for Coaching

A score helps coaching only when reviewers use one meaning for every rating. Without a shared meaning, one manager might reward a seller for asking several questions. Another reviewer might expect a question, a buyer response, and a related follow up before awarding a high mark. Both people hear the recording. Both people believe their decision follows the scorecard. The scorecard gave each person too much room to fill gaps with private standards.

Begin calibration by naming the decision connected to a score. A manager feedback conversation needs a practical and stable standard. A report used across several regions needs closer agreement, since leaders might compare scores from different reviewers. Ask the quality owner how scores will be used, who reads them, and what decision follows from a score. Those answers shape how detailed each rating definition needs to be. Do this work before asking reviewers to debate a recording.

Select calls with useful differences. Choose one short call where evidence for a behavior is clear. Choose another where a seller starts a useful move yet stops before enough evidence appears. Give every reviewer the recording, rubric version, evidence sheet, and scoring deadline. Reviewers work alone first. Conversation before independent scoring often creates agreement around the most confident voice instead of agreement around the call.

The evidence sheet should ask for a time point, seller language, buyer response, score, and criterion sentence. A note such as "strong discovery" leaves no trail for a later discussion. A note such as "at 5:40, Jordan asked how approval would move through the committee, then asked who owned the final review" gives every reviewer a place to look. Evidence notes also expose a separate problem. A reviewer sometimes rewards an outcome they like rather than the behavior described by the rubric.

In a fictional calibration meeting, quality director Ana starts with a concern. "Managers tell me scores depend on who reviewed a call." Consultant Theo replies, "Let us choose discovery and listen for evidence before discussing any totals." Ana reads the existing line aloud: "The seller asks effective questions." Theo asks each participant to write a score and cite one moment. No one speaks during the first ten minutes.

At minute four, fictional seller Jordan asks, "Who else will review our proposal?" The buyer names a regional lead and an operations lead. Morgan gives full credit because Jordan identified two stakeholders. Lee gives partial credit because Jordan does not ask what either person needs. Theo does not ask for a vote. He asks, "Does full credit refer to identifying people, learning each person's role, or both?" The group sees two behaviors inside one sentence. Their next rubric draft gives each behavior a distinct place.

Ana asks whether every rating needs an objective rule. Theo answers, "Use observable evidence. A reviewer should point to language, sequence, or a response instead of a feeling about call quality." Some judgment stays in a human review. The aim is a visible path from a recording to a score. A scorecard does not measure a seller's worth or predict a buyer decision. Keep the scope narrower. Review selected behaviors which support a chosen coaching focus.

Build anchors from calls already assessed during calibration. For discovery, a full anchor might show a question tied to a stated problem, followed by a question about effect, owner, or timing. A partial anchor might show a relevant first question with no follow up. A no evidence anchor might show no question related to the stated issue. Add a short explanation beside each excerpt. Reviewers need to understand why an example earned a mark, not memorize a preferred sentence.

Limit each session to a small number of criteria with wide score spread. Ask reviewers where the language forced them to guess. Write the uncertain phrase, suggested change, person testing the change, and date for a fresh review. The enablement consultant guides discussion. The quality owner approves rubric edits. Managers receive notice before reports use new wording, since a silent change makes score trends hard to interpret.

Another fictional moment tests discipline. Lee says, "I lowered Jordan's discovery score because Jordan interrupted the buyer." Theo asks, "Where does interruption appear inside discovery?" Lee answers, "Nowhere, though the interruption affected the call." Theo records listening as an issue for a separate criterion. The discovery score follows current wording. This separation protects a valid observation without changing a rule during one discussion.

After scoring, compare patterns before focusing on individual disagreement. A small difference across many scores might point to a rating boundary needing one clearer phrase. A broad split usually signals different definitions. Ask reviewers, "What evidence led you to this mark?" Then ask them to point to the supporting rubric sentence. "Why did you score low?" turns a review into self defense and rarely produces better language.

Write a decision note before people leave. Include the rubric version, call names, participants, agreed meanings, unresolved questions, and date for retesting. This note helps a later reviewer understand why wording changed. Share revised anchors with managers before managers prepare coaching. A manager who receives new scores with no explanation might treat a wording change as a sudden shift in seller performance.

Test the revised criterion on unfamiliar recordings. Familiar calls invite remembered ratings. Ask reviewers for a score and a complete evidence note. Look closely at notes which use vague words or cite no buyer response. Revise wording when evidence notes still diverge for the same reason. Resist adding a line for every possible call behavior. A crowded scorecard slows reviews and sends managers into coaching conversations with too many targets.

For fictional practice, give Ana and Lee a ten minute recording and separate evidence sheets. Ask both people to score discovery without discussion. Pause at 6:10, after a buyer mentions an approval committee. Each person writes the next seller question, records the buyer response, and marks the exact anchor used. Compare their marked anchor sentences. If Ana points to stakeholder identification and Lee points to follow up depth, Theo splits the criterion. Ana sends the new anchor note to managers before Thursday's coaching session.