Chelsea AI Ventures Ten questions to ask before you fund an AI pilot
Put these to the vendor, or to your own team, before the money moves. Each one comes with what a good answer looks like and the failure it prevents. Ten minutes to read, no sign-up.
The checklist
MIT's 2025 survey of enterprise generative AI found that 95% of pilots produced no measurable profit-and-loss impact. S&P Global found the average company scrapped 46% of its proofs of concept before production in 2025. The pilots that made it answered the questions below before they started. Most proposals answer two or three.
-
What does the process cost or produce today, in a number?
A good answer
a baseline with a date and an owner. "Four analysts clear 300 claims a day at a 6% error rate."
What it prevents
a result nobody can measure. RAND's interviews put a misunderstood problem first among the five causes of AI project failure, and a UK survey in March 2026 found that only 41% of companies using AI had a clear idea of what success would look like.
-
What does "it worked" look like, and who signs it off?
A good answer
one or two metrics with a threshold, written down before the build, and a named person who decides.
What it prevents
a demo that impresses everyone and changes nothing.
-
What is the no-go condition, and what does stopping cost?
A good answer
"If precision is under 90% on the test set at week four, we stop, you keep the report, and you pay only for the weeks used."
What it prevents
pilot purgatory, where a pilot with no stated end is renewed quarter after quarter because nobody defined failure. Gartner expects 30% of generative AI projects to be abandoned after the proof of concept.
-
Where is the data, and has anyone counted it?
A good answer
the tables or documents by name, how many labelled examples exist, who owns access, and whether anyone has opened a sample.
What it prevents
month two spent discovering that the data is in PDFs, split across three systems, or missing. Gartner lists poor data quality first among the reasons projects are abandoned.
-
How will you know the system is right before anyone trusts it?
A good answer
an evaluation set of known inputs and expected outputs, scored before and after every change, with the failure cases listed.
What it prevents
a language model that makes something up in front of a customer. If the proposal has no evaluation set, the first test is the production launch.
-
Who runs it after the consultants leave?
A good answer
a named person on your side, in the room from week one, with the code, the documentation and the retraining procedure.
What it prevents
the system nobody can change six months later. It is the most common complaint about consultants in the practitioner forums we read: the deliverable worked on handover day, and nobody could maintain it.
-
Whose cloud, whose model, whose bill?
A good answer
the deployment target by name (your cloud account, a private VPC, your own hardware), the model licence, and the monthly running cost at your volume. Also the most sensitive class of data the system will touch, the person in your security team who approves a new supplier and how many weeks that review takes, and whether a pilot may start on sample data while it runs.
What it prevents
a per-token bill that grows with success, data leaving the jurisdiction, and a vendor you cannot leave. Also a finished pilot that waits months for a security review nobody scheduled, or hosting your security team refuses once the data class is known. Among German companies using AI, 66% name data protection as a hurdle.
-
What is the price, and which part of it is fixed?
A good answer
a fixed price for a defined scope, with a written list of what would change it.
What it prevents
estimate-to-invoice drift. Reviews of large delivery shops on Clutch describe eight-month deliveries planned at five, and Clutch's own summary of one of them reads "initial estimates sometimes differ from final project costs".
-
Which existing system does it have to plug into, and has anyone looked at the interface?
A good answer
the system named, its API or export format checked, and the person who owns it consulted.
What it prevents
a working model that cannot reach the workflow it was built for. The integration is usually most of the work and most of the delay.
-
Why AI at all, and what is the simplest thing that would do?
A good answer
a sentence on what a rule, a database query or an existing tool would achieve, and why that falls short.
What it prevents
technology looking for a problem. RAND names chasing the latest technology as one of the five root causes. If a spreadsheet does the job, the proposal should say so.
How to score it
Count the questions the proposal answers with a number, a name or a date.
-
Eight or more
fund it.
-
Five to seven
fix the gaps before the money moves. The usual gaps are questions 1, 3 and 5, and closing them takes about a week.
-
Under five
you are being sold a demo, and a demo is what will be delivered.
This is the list we work through on the first call, and it is where our own scope comes from. A written baseline, a success threshold, a no-go clause and an evaluation set are in every fixed-price proof of concept we quote. Book a free 30-minute scoping call if you want a second opinion on a proposal, or take the readiness assessment first. We write up what we learn from pilots on our Substack.
Sources. MIT NANDA, The GenAI Divide: State of AI in Business 2025 (95% of pilots with no measurable P&L impact). S&P Global Market Intelligence, via CIO Dive, March 2025 (46% of proofs of concept scrapped). Gartner, July 2024 (30% abandoned after proof of concept; data quality first among the causes). RAND, The Root Causes of Failure for Artificial Intelligence Projects, 2024 (five root causes). Studio Graphene survey via consultancy.uk, March 2026 (41% with a clear idea of success). Bitkom, September 2026 (66% name data protection). Clutch reviews of Future Processing and STX Next.
Have a proposal on your desk?
Bring it to the call. Thirty minutes, and you leave with a view on which of the ten questions it answers and what to ask for on the rest.