Reference Checks in IT Procurement: Why Most Organisations Get Nothing Useful (And What to Ask Instead)

You’ve spent three months running an IT tender. Shortlisted two vendors. Scored their responses. Sat through demos and technical deep-dives. Now you’re at reference checks.

So you call the three contacts the vendor provided. You ask if they’re happy with the service. They say yes. You ask if they’d recommend the vendor. They say yes. You tick the box and move on.

A year later, you’re dealing with implementation delays, scope creep, support gaps, and a vendor relationship that doesn’t match what was sold. Somewhere in the back of your mind, you’re wondering if those reference conversations could have told you this was coming.

They could have. But only if you’d known what to actually ask.

If you’re involved in technology sourcing decisions, whether in IT leadership, procurement, or transformation, reference checks can surface real risk. Most just don’t.

Why most reference checks are useless

Vendors don’t give you random references. They give you their best relationships. Clients who like them. Projects that went well. Contacts who’ve agreed to take calls and say nice things.

That isn’t dishonest. It’s just how the process works.

The problem is that most organisations ask questions designed for friendly references. Questions like “Are you satisfied?” or “Would you recommend them?” invite simple yes-or-no answers. In a reference call arranged by the vendor, the answer is almost always yes.

At this stage of the process, you’re not trying to confirm basic competence. You already know the vendor can deliver at some level or they wouldn’t be shortlisted. What you’re trying to understand is how they actually operate, where friction shows up, and whether that friction matters in your context.

That requires different questions.

What reference checks should actually do

A useful reference check does three things.

First, it confirms capability in a specific scenario. Not whether the vendor is good in general, but whether they’ve delivered something structurally similar to what you’re proposing. Comparable scale, similar technology, similar constraints.

Second, it surfaces behaviour under pressure. What happens when scope changes, timelines compress, or something breaks? The goal isn’t to catch anyone out. It’s to understand response patterns so you know what “normal” looks like once the contract is signed.

Third, it helps you calibrate expectations. If every reference mentions that the vendor is technically strong but slow to respond, that may not be disqualifying. But it tells you where governance, escalation paths, or commercial protections may matter more.

If your reference checks aren’t doing those three things, they’re mostly noise.

Asking for the right references in the first place

Before you get to the questions, you need to speak to the right people.

Most vendors default to providing their largest, most recognisable clients. Those references look good on paper, but they’re only useful if your environment is genuinely comparable.

A mid-sized organisation with limited internal IT capacity will get very little insight from a reference at a large enterprise with a dedicated infrastructure team. The delivery model, escalation paths, and commercial leverage are different.

Be specific about what you need. Ask for references from organisations of comparable size, similar IT maturity, and ideally a similar operating context. If you’re running 500 endpoints, a reference from another organisation in the 400–600 range tells you more than one running 5,000.

Recency matters too. A reference from a client onboarded five years ago reflects how the vendor used to operate. Ask for references where onboarding occurred within the last 12 to 18 months. That gives you a view of current delivery teams, processes, and capacity.

If a vendor struggles to provide references that meet those criteria, that’s information in itself.

The questions that actually matter

Start with context, not satisfaction.

Ask the reference to describe the engagement. What were they trying to achieve? What was in scope? How long did it take? This tells you whether the reference is genuinely comparable.

Then move into specifics.

“What surprised you about working with this vendor?” often surfaces things that weren’t obvious during the sales process.

“Where did they struggle?” is more useful than asking whether there were problems. Every vendor struggles somewhere. The question is whether those areas overlap with what matters to you.

“If you were doing this again, what would you change about how the engagement was set up?” frequently reveals gaps in governance, contract structure, or expectations.

“How did they handle changes to scope or timelines?” tells you how the relationship behaved under pressure.

“What’s their response like when something breaks?” matters more than headline SLAs. It tells you what day-to-day support actually feels like.

“Would you use them again, and for what?” is more revealing than a blanket recommendation. It clarifies where the vendor adds value and where they don’t.

And one of the most useful questions of all: “What didn’t we ask that we should have?”

What to listen for in the answers

Pay attention to hesitation. Not silence, but qualification. “They’re great, but…” is usually where the useful detail sits.

Listen for specificity. Vague praise is meaningless. Concrete examples tell you how the vendor actually operates when things don’t go to plan.

Watch for patterns. If multiple references independently raise the same strength or limitation, it’s probably real. Neither automatically disqualifies the vendor. Both inform how you structure the relationship.

Also pay attention to what isn’t said. If questions about support responsiveness drift back to technical capability, that’s a signal. If delivery questions pivot to relationship management, that’s another.

When to go off-list

Formal procurement processes usually restrict reference checks to contacts provided by the vendor. That’s reasonable from a probity perspective.

But for higher-risk or higher-value ICT procurements, informal calibration through peer networks can add context. A quiet conversation with someone who has worked with the vendor before may surface insights that don’t appear in a formal process.

This isn’t about bypassing governance. It’s about understanding how vendors behave outside rehearsed reference calls.

How this fits into the broader procurement process

Reference checks often sit at the end of an evaluation, when a preferred vendor is already emerging. That creates confirmation bias. You’re listening for reassurance rather than risk.

Treating reference checks as a scored evaluation criterion, with defined questions and consistent weighting, helps counter that bias. They become another data point, not a box to tick.

The same discipline applied to technical compliance and commercial evaluation applies here too.

What happens if you skip this

Nothing immediately.

The vendor is appointed. The contract is signed. Delivery begins. For a while, everything looks fine.

Then issues surface. Support doesn’t scale as expected. Change requests take longer than anticipated. Account teams rotate. The relationship resets.

None of this would necessarily have disqualified the vendor. But much of it could have been anticipated and managed differently if the signals had been surfaced earlier.

That’s the real cost of weak reference checks. Not bad decisions, but decisions made with less information than was available.

A final note

Reference checks won’t tell you everything. They won’t eliminate risk or guarantee outcomes.

But they do offer insight into how vendors actually operate, not just how they present themselves in a tender response. In long-term ICT procurement relationships, that difference matters more than many organisations expect.

This article provides general commercial and procurement commentary only and does not constitute legal, financial, or professional advice.