A reference call goes like this. The vendor connects you with a restaurant owner in another state. They tell you the setup was easy, the team likes it, the orders come through fine, and they wish they had done it sooner. You hang up twenty minutes later feeling good and knowing nothing you did not know before the call.
That call was not useless because the operator lied. It was useless because you asked questions with an obvious right answer, and because the vendor picked someone who was going to give it. Reference calls are worth doing, but only if you go in with questions the happy path cannot absorb.
The reference you were given is a selected sample
Any vendor with more than a handful of accounts is handing you their best one. That is not dishonest, it is what a reference list is. The information you actually want is about the median account, and the median account is not on the list.
So ask for one reference and find one yourself. Post in an operator group, ask your POS rep who else in the area is running voice AI, or call a restaurant of similar size in a neighboring market and ask the manager directly. Five minutes with an operator who has no relationship with the vendor is worth more than an hour with a curated one.
If you are early in the process, the vendor evaluation questions worth asking the salesperson are a separate list. This one is for the operator.
Ask about weeks one through three, not about today
An operator who has been live for a year has forgotten the rollout. Push them back to it, because the launch is where most of the pain lives and where your next sixty days will be spent.
Good version: "Walk me through the first week. What broke?" Not "was setup easy." The first question gets a story. The second gets a yes.
Follow with how long menu work took on their side. Every voice deployment needs somebody at the restaurant to verify the menu, the modifiers, the delivery zones, and the hours, and that person is usually the GM doing it after close. If the answer is "a couple hours," ask what they had to fix in week two. If they say nothing needed fixing, ask what happens when they add a special. The onboarding checklist covers what should have been on their list.
Then ask the question nobody volunteers: what did your staff do the first time the agent got an order wrong in front of a customer? The answer tells you how the vendor handled an early failure, which is far more predictive than how they handle a demo.
The bad-day questions
These are the ones that separate a working deployment from a tolerated one.
- What happened the last time your internet or your POS went down while the agent was answering, and where did the orders go?
- How long does it take to get a human at the vendor on a Friday night, and have you ever needed to?
- Has the system ever taken an order it could not actually fulfill, like an item that was 86'd, and how did you find out?
- What percentage of calls still end up with a person, and are those the calls you want a person on?
- What have you asked them to change that they have not changed?
That last one does most of the work. Every real deployment has an open request. An operator who cannot name one is either not paying attention or not using the system enough to have opinions.
On the outage question specifically, compare what you hear to the vendor's written commitments. Marketing pages and uptime language in a contract are frequently different documents.
Money questions belong to the owner
Ask what they pay, and then ask what they paid in month one. The gap between those two numbers is where setup fees, overage, integration surcharges, and per-minute billing live. Plenty of operators quote you a monthly figure that is really a base rate they exceed every month.
Ask whether the invoice has ever surprised them, and by how much. Ask what happens to their bill in December when volume doubles. If the vendor bills per minute, ask whether they have ever looked at the minute detail, because most people do not until something looks wrong.
Then ask a harder one: has it made you money, and how do you know? Most operators cannot answer this cleanly, and that is fine. The useful part is watching whether they reach for something concrete, like a count of after-hours orders they used to lose, or whether they reach for a feeling. Our own pricing is a flat monthly starting at $250 specifically so this question has an answer, but a reference should be able to tell you the shape of their bill regardless of who they buy from.
What a credible answer sounds like
An operator who is genuinely running the system will contradict themselves a little. They will say the agent handles ninety percent of calls fine and then, three minutes later, complain about how it handles a specific item with a weird name. Both things are true. That texture is the signal.
Be suspicious of unqualified enthusiasm. Be equally suspicious of an operator who cannot describe how the orders reach their kitchen, because that person's manager is running the system and they are just answering the phone call you made.
Listen for whether they talk about the vendor's people. A restaurant that has been live a year and cannot name anyone at the vendor has never needed help, which is either great news or a sign they gave up on asking. Ask which it is.
The exit question
End every reference call the same way: what would have to happen for you to switch?
It is the only question that reliably surfaces current weaknesses without asking anyone to speak badly of a partner they are on a call to endorse. Answers range from "nothing, honestly" to "if they raise the price again" to "if we ever get a real second option on our POS," and each of those tells you something different about how locked in they feel.
If you are the one considering a move rather than a first install, the mechanics of changing voice AI vendors matter as much as the reference does, because the cost of leaving is part of the cost of joining.
Write down the three most specific complaints you hear across your reference calls. Then take those exact scenarios into your own demo and make the vendor reproduce them live, using the approach in what to test in a demo. A reference tells you where to point the flashlight. The demo is where you look.