How to Build a Call Review Scorecard (Gong, Modjo, or a Spreadsheet)

One out of fifty sales managers cares about the quality of a call. Fifty out of fifty count calls. They count because counting is easy. A scorecard is the cheapest tool that forces the harder question, and most teams that own Gong or Modjo still don't have one worth using.

This is for the VP of Sales or sales manager who has the recordings, has the dashboards, and still can't tell you whether their reps run good discovery. After seven years and 1,400+ manually reviewed B2B SaaS calls across Benelux and DACH, which form the dataset behind my 2026 EU Sales Call Benchmark, the pattern is consistent. Teams are far better at building the infrastructure of sales than at inspecting what actually happens inside a call. The scorecard is how you close that gap.

A call review scorecard is a structured rubric a manager uses to rate a recorded sales call against a small set of defined skills, scoring each 1–5 and capturing one concrete, timestamped observation per skill. It is not a compliance checklist, and it is not your activity dashboard. That distinction is the whole game, because most "scorecards" sold as templates are really just call-counting in a prettier table.

1Score Quality, Not Volume

Here is the test for any metric on your scorecard: if it does not provoke a coaching action, it doesn't belong. Dials made, talk-to-listen ratio, calls logged. These get tracked because they are trackable, not because they change what a rep does next Monday. An A call is profoundly different from a B call, and no amount of counting tells you which one you just listened to.

My EU benchmark scores each core skill out of ten across the teams I review. Those averages are sobering, and they double as the reference points for your own scorecard. A rep's score isn't just a number, it's a number against a known European baseline.

EU Sales Call Benchmark, average score out of 10 per skill (1,400+ calls, 140+ teams, Benelux & DACH).

2The Five Criteria That Actually Predict Deals

Resist the urge to score everything. A scorecard with twenty-five line items gets filled in mechanically and coached on never. The criteria that earn their place are the ones where my benchmark data shows reps consistently fail and where improvement visibly changes win rates. Five skills, not twenty-five:

3Score 1 to 5, Never Tick-Box

Binary scoring, where pain is captured yes/no and the DMU is understood yes/no, massively oversimplifies the nuanced nature of discovery information. A rep can technically "ask about pain" and learn nothing. The fix is a 1–5 scale per criterion, with the bands defined in advance so two managers reviewing the same call land in the same place.

Define each band concretely. For discovery, a 1 is "asked no pain questions," a 3 is "surfaced a stated pain but no impact," and a 5 is "quantified impact and tied it to a business consequence the buyer confirmed." Without written bands, scoring drifts, calibration collapses, and the scorecard is worthless within a quarter. Be honest: does your current review have written bands, or does "good discovery" mean whatever the manager felt that day?

4Can AI Score Your Calls? Watch for AI Sales Psychosis

AI Sales Psychosis is the false sense of control that appears when Gong, Modjo, and AI notetakers tell reps and managers "don't worry, you've captured the pain" when the buyer gave only the faintest signal. I named it because I kept watching it happen. When I compared AI analysis of calls against the manual reviews that formed the basis of my 2026 benchmark, I found a discrepancy of more than 30%.

The failures were specific. The AI marked Pain criteria green because the customer nodded along while the rep recited common pains. It marked the DMU "well understood" when the buyer's actual answer was "yeah, I probably need to involve my manager at some point." Large language models are pleasing by nature. They soften gaps, make weak discovery look structured, and turn assumptions into CRM data. Suddenly everyone believes the deal is better qualified than it is, right up until the buyer chooses the status quo, a competitor, or simply vanishes.

You can make AI scoring far more honest with one prompt instruction: require the model to quote the exact transcript evidence behind every judgment, both the question the seller asked and the information the buyer actually gave. If the AI can't produce the line, the criterion isn't met. That single rule turns an AI scorecard from a confidence machine into a genuine reviewer.

5The Tool Is Only as Good as the Team's Discipline

Conversational intelligence platforms like Gong can supercharge coaching, but they carry three failure modes worth naming before you build inside one. First, the big-brother effect: reps quietly switch off recording during sensitive moments, which skews the very data you're scoring. Second, vanity-metric drowning, where managers have so many numbers they trust none of them. Third, the maintenance burden, since scorecards in these tools need constant upkeep and neglected ones rot.

None of this is an argument against Gong or Modjo. It's an argument that the platform doesn't create the discipline; the manager does. A scorecard in a spreadsheet that a manager actually uses every week beats a beautifully configured Gong scorecard nobody opens. Start where your discipline is, not where the software is.

6Sampling, Not Surveillance: The Cadence That Makes It Stick

You do not need to score every call. To get a real grasp of call quality, take random samples on a weekly or bi-weekly basis and record them in the scorecard. Sampling is sustainable for a manager carrying seven reps, and it sidesteps the surveillance dynamic that makes reps perform for the recording rather than for the buyer. The cadence runs on a weekly loop:

1. Sample. Pull two or three random calls per rep each week, not the rep's hand-picked best one.

2. Score. Rate the five skills 1–5 against the written bands. Capture one specific observation per skill, with the timestamp.

3. Coach one thing. Bring a single skill to the 1:1. The fastest way to make coaching useless is to dump all five scores on a rep at once.

4. Make them practise it. Don't tell the rep what to do, have them do it. The scorecard finds the gap; role-play closes it. AI role-play tools like Jam (wejam.ai), which I co-founded for exactly this, let reps rehearse the weak skill at volume without burning manager time.

That last step is the one most teams skip, and it's the difference between feedback and coaching. Telling a rep "your discovery scored a 2" changes nothing. Running a five-minute role-play where they re-run the impact question until it lands is what moves the next call.

Where the Scorecard Fits a Coaching System: COMPASS

I built COMPASS after giving my own team genuinely boring coaching. I mostly banged on about one issue until they zoned out and answered Slack messages mid-session. The framework is a corrective for managers like the one I was. Good coaching is Continuous, Organized, Measured, Practical, Appealing, Specialized, and Strategic.

The scorecard is the Measured pillar, what makes coaching objective instead of a manager's gut feeling. But a scorecard without the Practical pillar (role-play, not lecture) and the Specialized pillar (tied to each rep's individual goals) is just measurement, and measurement on its own develops nobody. This is also why AI sales coaching tools that score calls but never drive practice tend to disappoint: they're all M, no P. The cadence and the 1:1 structure that surround the scorecard get their own treatment in the manager coaching cadence.

The scorecard tells you which call was a B and which was an A. Coaching is the only thing that turns the B into an A, and you can't coach what you never scored.

Where to Start

The scorecard is the easy part. You could build the five-skill rubric above in ten minutes. The hard part is the manager habit: sample, score, coach one thing, make the rep practise it. If you want help turning a scorecard into a coaching system your managers actually run, that's what my coaching work is built to do. And if you're not sure where your team's biggest quality gap sits, start with a baseline.