A CFP review rubric is a short, weighted list of criteria — each scored on an anchored scale — that every reviewer applies to every submission. It exists so that “we picked the best talks” is a claim you can defend, to your committee and to the speakers you turned down. Write it before the call opens, keep it to four or five criteria, and put words (not just numbers) on what a 1, 3, and 5 actually mean.
Here’s a template you can copy today, and the two habits — anchoring and calibration — that separate rubrics that work from rubrics that decorate.
The template
| Criterion | Weight | What you’re actually asking |
|---|---|---|
| Relevance & audience fit | 30% | Is this what our attendees came for, at the right level? |
| Clarity of takeaway | 25% | Can you state what the audience walks out knowing or able to do? |
| Speaker evidence | 20% | Does the abstract show the speaker can deliver — experience, specifics, a real story? |
| Originality | 15% | Would a regular attendee of this event have seen this talk before? |
| Format & practicality | 10% | Does it fit a track, a slot length, and our rooms as proposed? |
Score each criterion 1–5, multiply by the weight, sum. Adjust the weights to your event — an academic meeting might swap “originality” for methodological rigor and crank it to 30%; a practitioner conference might push “clarity of takeaway” higher. What matters is that the weights are decided before anyone reads a submission, because weights chosen afterward have a way of ratifying whatever the loudest reviewer already wanted.
Anchors beat adjectives
A scale labeled “1 = poor, 5 = excellent” isn’t a rubric; it’s a mood ring. Anchor at least the 1, 3, and 5 with sentences a reviewer can match against. For clarity of takeaway:
- 1 — After two readings, you can’t say what the audience gets from this talk.
- 3 — There’s a takeaway, but it’s generic (“learn best practices for X”).
- 5 — The abstract states a specific, checkable outcome (“you’ll leave able to set up X to do Y, and know when not to”).
Anchored scales are what make scores comparable across reviewers — which is the entire point of having more than one reviewer. This is the same logic behind formal peer review, scaled down to fit a conference timeline.
Calibrate on five real submissions
Before the full review round, have every reviewer independently score the same five submissions — ideally a deliberately mixed bag — then compare numbers together. You’re not looking for agreement on which talks are good; you’re looking for agreement on what the numbers mean. Thirty minutes of “you gave that a 4 and I gave it a 2 — why?” fixes more scoring drift than any amount of rubric wordsmithing. Do it once, at the start, and your averages become meaningful.
Decide the blind-review question on purpose
Hiding speaker names during the first scoring pass reduces halo effects and makes room for new voices; it also hides “speaker evidence,” which is a real criterion — track record predicts delivery. A workable middle: blind the first scoring round, then reveal identities for the shortlist discussion. Whatever you choose, choose it before the call opens and say so publicly — submitters deserve to know how they’re being read. (More on structuring the pipeline in how to run a CFP without a spreadsheet.)
Argue only where reviewers disagree
When scores come in, resist re-litigating the whole pile. Sort into three bands: clear accepts, clear declines, and the middle. Then spend the committee’s meeting time only on submissions where reviewers disagree strongly — a 5 and a 2 on the same talk means someone saw something the other didn’t, and that conversation is worth having. Averaging the disagreement away is faster and worse: it quietly buries both the risky-brilliant talks and the polished-empty ones.
The failure modes, quickly
- Too many criteria. Eight criteria don’t add rigor; they add noise and reviewer fatigue. Four or five, weighted, beats ten unweighted.
- The rubric doesn’t match the form. If you score “speaker evidence,” your submission form needs to ask for it. Every criterion should map to a field a submitter actually filled in.
- One rubric for every format. A 10-minute lightning talk and a half-day workshop can share criteria but not anchors. Adjust the anchors per format, keep the weights.
- Scores as the whole decision. The rubric builds a defensible shortlist; the final program still needs human judgment about balance across tracks, topics, and voices.
Where the tooling comes in
None of this requires software — a rubric is thinking, not technology. Tooling earns its keep in the plumbing around it: getting each submission in front of the right reviewers, collecting scores against the same criteria instead of five color-coded spreadsheets, and turning acceptances into sessions without re-keying. That’s the job of the abstract peer review workflow in AgendaForge, where review happens in the same pipeline the agenda is built from.
But the rubric itself? Steal the table above, anchor three points on every scale, calibrate on five submissions, and argue only about the disagreements. That’s the whole method.