You send the RFP to a longlist of agencies on a Thursday. You spend three weeks doing other work. Eleven responses arrive in your inbox at the deadline. Then you have eight working days to read all of them and tell your partners which three you want to put on a pitch stage.
If you read them in the order they arrived in, you’ll like the third one and grow to resent the eleventh. If you read them all once and try to score them in your head, the most recent will dominate the conversation. If you skim them and rank by gut feel, the partners will ask you for the working and you won’t have any.
The way to read eleven RFP responses without losing your mind is to do the scoring before you read them.
The scoring matrix is the work
Open the RFP you sent. Every question has a weight already, between 2 and 5. The 5s are the questions that decide the engagement. Budget. Timeline. Security certifications. Integrations. CMS recommendation. The 4s are the questions that matter a lot. Account management. Team. Accessibility. Brand consistency. The 3s are the standard questions. References. Documentation. Training. The 2s are the baseline ones. Company details, VAT numbers, office locations.
Build a spreadsheet. Forty one rows for the questions. Eleven columns for the agencies. Each cell takes a score of 1 to 5 and a sentence of rationale. Cumulative weighted total at the bottom. Sort descending.
You can resist the urge to build the matrix and just dive in. I’ve watched marketing leads do this and pay for it later. The first response feels exhaustive. The second response feels similar but with a different style. By the fourth, you’ve started skipping. By the seventh, you’re skimming. By the eleventh, you can’t remember whether agency three had the SOC 2 or whether that was agency six.
The matrix takes a morning to build. The reading takes a week. The reading without the matrix takes a fortnight and produces a result you can’t defend.
Score with rationale, not with stars
The temptation is to give every answer a number and move on. Don’t.
A score of 4 with two sentences of rationale beats a score of 5 with no rationale at every partner meeting you take that score into. Partners are trained to interrogate the evidence behind a recommendation. If your evidence is a star rating, they’ll discount it. If your evidence is a paragraph of why this agency scored 4 on the integrations question and that agency scored 2, they can argue with it on the merits.
The rationale also forces you to read the answer properly. You can give a score of 4 without engaging with what the agency actually said. You can’t write two sentences explaining the score without engaging with it. The rationale isn’t a record of what you decided. It’s how you decide.
What the eleven responses will tell you
When you’ve read all eleven and tabulated the scores, you’ll find they cluster into three groups.
The top tier, roughly four out of five, will answer the question that was asked. They’ll be within the word limit. They’ll quote a fixed price with hours broken down. They’ll name actual references with contact details. They’ll hold the certifications they’re claiming.
The middle tier, roughly three to four out of five, will answer the question they wished you’d asked. They’ll have good case studies that don’t quite match your brief. They’ll quote a price but lean toward a range. They’ll be adequate but undifferentiated.
The bottom tier, below three and a half out of five, won’t have read the brief properly. One will skip the timeline question entirely. One will say “this is not an official budget” and leave the cost section blank. One will refuse to provide references until a contract is signed.
The clustering happens reliably across engagements. You’ll see it. The matrix lets you describe it precisely.
The shortlist conversation is shorter than you think
With the matrix in front of you and the rationale documented, the partner meeting takes about two hours. The first half hour is the partners reading the top of the matrix. The next hour is questions about specific scores. The last half hour is a conversation about the three to four agencies clustered at the top, and which three to put on the pitch stage.
The disagreement happens on the data, not on impressions. That’s the whole point.
If the partners don’t have the matrix in front of them, you’ll be in that meeting for half a day and you’ll leave it without a shortlist. You’ll have a list of agencies the partners liked the look of, which isn’t the same thing.
The matrix is the work. The reading is the easy part once the matrix exists. Read the full case study Eleven responses. Three on the pitch day. One agency appointed in eight weeks.