The Filter
AI vendor pitches in 2026 are slicker than ever. The same demo can hide a serious product or a thin wrapper around an API. The CEO's job: cut through the polish and find out which one this is, in under an hour.
The 9 Questions to Ask
- What does the AI do that a generic GPT call wouldn't do? (If the answer is fluffy, this is a wrapper.)
- How do you handle hallucination? Show me a real example.
- How do customers measure ROI? What number do they report?
- What happens to my data — training, storage, deletion?
- Show me your three most recent customer churn cases and why.
- What's the time to first value, in days?
- What integrations are in production, not on the roadmap?
- If your API provider raised prices 2x, what happens?
- Why will this vendor still exist in 3 years?
- • Customer-specific KPI examples, not generic case studies.
- • A clear answer to “what makes you defensible.”
- • A willingness to share a customer reference call.
Demo Red Flags
- The demo only works with their pre-loaded data.
- The output is impressive but unverifiable in 60 seconds.
- The vendor can't explain failure modes.
- The vendor avoids your specific use case.
- The pricing structure changes mid-conversation.
Pricing Decoding
Three pricing patterns and what they really mean: per-seat (predictable, scales with team), usage-based (scales with success but unpredictable), annual contract(lock-in plus discount — valuable if you'll really use it).
Data Handling Questions
The non-negotiables: written BAA where applicable, no training on your data without opt-in, data deletion on contract end, encryption in transit and at rest, SOC 2 Type II in regulated industries.
The Post-Sale Reality
Talk to two current customers without the vendor in the room. Ask what broke in onboarding, what's on the roadmap that they're waiting for, and whether they'd renew. The honest answers come out in 10 minutes.
The Decision Worksheet
- Problem clearly defined, with a baseline number.
- Vendor answered all 9 questions credibly.
- Two customer references checked.
- Pricing modeled at 0.5x, 1x, and 2x expected usage.
- Internal owner identified.
- Success metric and review date set.
The best CEO move on an AI vendor pitch isn't skepticism — it's specificity. Generic skepticism reads as “not a buyer.” Specific questions read as “serious buyer.”
See 9 questions that reveal a bad fit.
FAQ
How long should a pitch take? 45 minutes. Anything longer is over-selling.
What about pilot programs? Useful if they have clear success criteria and a real timeline. Useless as a delay tactic.
Should our CTO be in the room? Yes for technical use cases. CEO leads the strategic questions.