Vendor evaluation in this category goes wrong in a predictable way. The operator watches four polished demos, all of which sound impressive, and then decides based on which salesperson followed up or which pricing page looked friendliest. The demos all sound good because sounding good is the easy part now.
A better process filters hard on one thing first, tests a small number of vendors on your actual menu, and pays close attention to how the vendor behaves when something breaks. That last signal is the most predictive and the least measured.
Step one: filter on your POS, ruthlessly
Before you look at anything else, ask each vendor a specific question. Is my exact POS, on my version, a currently live integration that another restaurant is running today?
Accept only a direct yes with a named integration. "Compatible with most systems," "we can build it," and "it's on the roadmap" are all no. So is "we integrate through a middleware partner" unless they'll tell you which one and what it does to your modifiers.
This one filter usually cuts your list in half or better, and it prevents the most expensive mistake in the category: buying a system that answers your phone beautifully and then can't put the order into your POS as a usable ticket. We make the case for weighting this above everything else in why POS integration depth matters more than voice quality.
Step two: define what you're buying before you look
Write down the two or three problems you're solving, in your words. "We miss calls between 6 and 8." "The host stand phone is destroying our seating flow." "Nobody covers the phone after close." Then check each vendor against those, not against a feature grid.
Feature grids are where evaluation goes to die. Every vendor's grid has checkmarks on every row, because the rows are written to be checkable. Your problem statement isn't.
Step three: demo on your actual menu
Don't accept a demo on a generic template menu. Ask each shortlisted vendor to load your real menu and let you call in. A vendor that won't do this is telling you something.
Then run the same scenarios against each one so the comparison is fair. Our demo testing guide has the full list, but at minimum: a normal order, an order with three modifiers and a substitution, a mid-sentence correction, an item you 86'd that morning, an informational question, an ambiguous item name, and a request for a human.
Have the same three people place the calls at each vendor, with different voices and different speaking speeds. One person testing tells you how well the system understands that one person.
Step four: look at the ticket, not the transcript
This is the step operators most often skip, and it's where systems separate.
After each test order, go look at what landed in the POS. Right items, right modifiers, right price, right order type, routed to the right printer or station? A system can hold a flawless conversation and then flatten your modifiers into a note field that your line cook has to read as a paragraph.
If you can't see the POS side during a demo, ask the vendor to screen-share their end and show you the created order. If they can't show you that either, you've learned something.
Step five: run a real pilot
Demos are curated. A pilot is not. Route real overflow traffic, meaning calls your staff didn't pick up, to the agent for two full weeks including your busiest shift, against criteria you wrote before starting.
The design details are in designing a voice AI pilot. The thing to emphasize here is the part that's about the vendor rather than the product: when something breaks during the pilot, how fast do they fix it, and do they explain what happened honestly or reassure you vaguely? You're buying a relationship that will outlast this month's feature set, and the pilot is your only sample of it.
Step six: the questions that aren't about features
Ask each finalist:
References on my POS, in my segment, at my size. A multi-unit quick service reference tells a single-location full-service operator very little. Ask for the specific match, and actually call them.
What happens when you're down? Not the uptime number, the caller experience. See SLAs and uptime and human handoff and failover.
Who updates my menu and how fast? If a price changes at 4pm, can your manager fix it in two minutes, or does it require a support ticket?
What's the complete fee schedule? Everything, including anything not on the pricing page. The models are broken down in pricing models.
What are the exit terms, and can I take my data? Read the notice window, the auto-renewal date, and the data-export terms before signing, not after.
How are card payments handled? Tokenized to a processor, or touching the vendor's systems? Full list in the security questionnaire.
What to weight, and what to discount
Weight heavily: POS integration depth, what the resulting ticket looks like, menu update speed and who controls it, failure behavior, transcript access, and how the vendor responded when something went wrong during the pilot.
Weight moderately: price, contract terms, analytics quality, multilingual support if your customers need it, escalation design.
Discount: voice naturalness beyond a reasonable threshold, feature counts, dashboard aesthetics, logo walls, and any percentage quoted without a stated methodology. If a vendor cites an accuracy figure, ask how it's measured and on what mix of calls. A number without a denominator isn't a number.
Reading the sales process as data
A few patterns are worth noticing, because they tend to persist after the sale.
A vendor who answers a hard question directly, including by naming a limitation, is more useful than one who answers everything with confidence. A vendor who won't put terms in writing before signing won't become more forthcoming afterward. A vendor who resists a real pilot on live calls either doesn't trust the product or doesn't want you looking closely. And a vendor who tells you their product isn't a good fit for your situation has just given you the most valuable data point in the process.
The bottom line
Filter on your exact POS first, define your problem in your own words, demo on your real menu with multiple voices, and always look at the resulting POS ticket rather than the transcript. Then run a two-week pilot on real overflow traffic and watch how the vendor handles the first thing that breaks. Weight integration depth and failure behavior above voice quality and feature counts. Most of the information you need is available before you sign. It's just not in the deck.