Before approving an AI pilot, leadership should be able to answer one question:
If this works, what will we do differently?
Will we respond to customer requests faster? Give our specialists more capacity? Reduce the time between receiving a complete file and making a decision?
If the answer is unclear, the pilot needs a better business case.
The industry is still working through that gap. McKinsey’s 2025 State of AI research describes widespread adoption alongside continued difficulty moving from pilots to scaled impact. Its research also associates workflow redesign with stronger business results. That is an association, not a guarantee that adding AI to a process will improve it. (McKinsey)
My perspective on this is commercial. A technology investment earns its place when it improves an outcome the business cares about, within conditions the organization can actually support.
Consider a manufacturer preparing quotes from customer specifications.
An AI demonstration might produce a first draft quickly. That is interesting. The business needs to know whether the complete process improves: gathering specifications, finding relevant past work, checking assumptions, getting engineering approval and returning an accurate quote.
If the draft takes less time but creates more review work, the benefit may disappear. If engineering capacity is the real bottleneck, faster drafting may simply build a longer queue.
I would start that pilot with a short agreement between the commercial leader, the people doing the work and the technology team:
The outcome
What should improve for the customer or the business?
The baseline
How long does the process take today, including review and rework?
The owner
Who is responsible for changing the process if the pilot succeeds?
The decision
What evidence would justify expanding, revising or stopping the investment?
Those questions make the technical work easier to evaluate. They also make it harder to declare success based on a good demonstration alone.
For organizations handling sensitive information, there is another decision to make early: which data can the system use, and under what conditions?
A quoting workflow might involve customer drawings, negotiated pricing and proprietary manufacturing knowledge. A useful pilot needs to test representative work under approved conditions. A demonstration using simplified documents may leave the hardest questions unanswered.
This is where a private-AI approach can become commercially relevant. If a valuable workflow requires data to remain within infrastructure the customer controls, that requirement should shape the project from the beginning.
It should also be compared with what the organization already has. Existing enterprise software may provide an acceptable solution. A private deployment should earn its place by meeting requirements and delivering value—not simply by carrying the word “private.”
NIST’s AI Risk Management Framework connects evaluation to the intended use, business context and risks of a system. Its framework calls for determining whether the system achieves its stated objectives and whether deployment should proceed. (NIST)
In practical terms, keeping data in a controlled environment still leaves important operating questions. Who can access it? Who reviews the output? What happens when the system is wrong? Who maintains the workflow after the pilot team moves on?
Those questions belong in the business case because their answers affect cost, adoption and accountability.
The same discipline applies to return on investment. Hours saved are a starting point. Leadership still needs to decide what those hours make possible.
- Can the same team handle more qualified requests?
- Can customers receive answers sooner?
- Can experienced employees spend more time on exceptions that require judgment?
Include integration, review, training and ongoing support in that assessment. Then decide whether the improvement is worth the full cost.
That is the conversation I want to help Kirk bring to business leaders: identify a valuable workflow, establish the conditions for using its data, and agree on the evidence that would justify putting it into production.
Before funding the next pilot, write down the decision its results will support.
A successful experiment should leave the business ready to act.
Schedule a call to learn more
