The pilot works. The answers look convincing, the team can see the potential and the project appears ready to move. Then the AI meets the real operation: incomplete data, permissions, exceptions, partial results and several confirmation points before anyone gets a usable outcome.
Across IZZY’s work on different client tasks, we see the same pattern repeatedly. It does not mean the demonstration failed. It means the demonstration proved a capability, while production has to prove that an organisation can operate that capability safely, usefully and at a cost worth paying.
The question is no longer just “can the AI do this?” It becomes: is this the right form of automation, what does it cost to operate, who controls the decisions, and what happens when it is wrong?
IZZY’s decision rule: use the least complex mechanism that can deliver the business outcome reliably. If deterministic automation can do the job more reliably and economically, do not build an AI agent. If human review multiplies confirmations and branches until the gain disappears, reduce autonomy before adding more controls.
Answer in 60 seconds
- Start with a real business workflow and its current cost, not a tool looking for a use.
- Compare the manual process, deterministic automation, AI assistance and an agent that can act.
- Where possible, introduce AI into a small but real part of the process with bounded consequences.
- Test normal, incomplete, conflicting and sensitive data, plus the real human-review workload.
- Define access, evidence, unacceptable failures, ownership, stop authority and fallback.
- Broaden use only when the value survives the cost of review, security, monitoring and maintenance.
A successful pilot is not proof of security, regulatory compliance or commercial return.
In this article
- Why production changes the decision
- Deterministic automation, AI assistant or agent?
- Define what the AI is allowed to do
- Record the decision, including why AI is the right mechanism
- Check the complete data, supplier and security path
- Why “a human will review it” is not enough
- Earn broader scope, or stop
- Conclusion: AI has to earn its place in the workflow
- Frequently asked questions
- Sources and limitations
1. Why production changes the decision
A demonstration answers a narrow question: can the system produce an interesting result from selected examples?
Production adds harder questions:
- does the result remain useful when the data is messy or incomplete?
- can important errors be detected before they cause harm or rework?
- who decides when several outputs look plausible?
- what happens when the model, data, integration or permissions change?
- does the business still save time or money after review and maintenance?
- can the team stop the system and continue the work another way?
The NIST AI Risk Management Framework connects governance, context, measurement and risk treatment. It is voluntary, non-sector-specific and intended to be adaptable. It does not produce an automatic release verdict or prove that a control works in your environment.
The right unit of governance is not the model on its own. It is the operating system around it: people, process, data, instructions, model, tools, suppliers, destinations, review, monitoring and recovery.
The operational ownership test
Before moving further, a named person should be able to explain:
- the business outcome being pursued;
- what the system is not allowed to do;
- the evidence supporting acceptance;
- the failures that require a stop;
- who can actually stop or restore the workflow.
If the answers exist only in the head of the person who built the pilot, the organisation does not yet own the operation.
2. Deterministic automation, AI assistant or agent?
The decision is not simply “AI or no AI”. The same job may have four viable approaches with very different costs and uncertainty.
| Option | It may fit when | Watch for |
|---|---|---|
| Human process | Volume is low, decisions are rare or context varies heavily | Delay, key-person dependency and manual error |
| Deterministic automation | Rules, inputs and outputs are stable and testable | Unhandled exceptions and rule maintenance |
| AI assistance | Inputs are unstructured and a person still owns the outcome | Review time, source quality and output variation |
| AI agent | The system must choose steps or use tools within a defined boundary | Permissions, unwanted actions, branching, recovery and monitoring |
Deterministic automation may be less impressive in a demo, but it is often easier to test, explain and maintain. Choosing it is not a retreat from AI. It is evidence that the mechanism was selected for the workflow rather than for the demonstration.
Price the complete operation
The model or API fee is only one line. The real operating cost also includes:
- discovery and integration;
- data preparation and quality work;
- evaluation before and after release;
- human confirmation and correction;
- security, permissions and incident handling;
- quality, latency and cost monitoring;
- maintenance when the workflow or supplier changes;
- manual fallback, export and supplier exit.
AI earns its place when it removes meaningful cost or friction, or improves the process enough to justify these controls. If intermediate confirmations absorb the expected gain, narrow the AI’s role or return to deterministic automation.
What IZZY will not recommend
We will not recommend an agent because a clean pilot looked impressive, treat an approval button as a substitute for a workable control path, or widen access before a bounded real workflow has survived the full operating cost. A decision not to build is a valid production decision.
3. Define what the AI is allowed to do
Autonomy is not the business outcome. Give the system only the authority required to create the value.
| AI role | What it may do | What stays human-owned | Evidence before broader use |
|---|---|---|---|
| Prepare | Retrieve, classify, summarise or draft | Use and publication of the result | Source checks, input quality and review of realistic samples |
| Propose | Recommend a decision or action | Final choice and its consequences | Normal and edge cases, useful explanation and genuine rejection |
| Act within limits | Execute authorised, reversible actions | Sensitive actions, exceptions and stop authority | Least privilege, abuse tests, logging and proven recovery |
These roles can coexist in one workflow. AI may prepare a case and suggest a response but never send it. It may update a low-impact field while requesting approval for a customer-facing or financial change.
Use a small but real operating boundary
A clean sandbox can hide the problems you need to find. Where lawful and safe, choose a bounded part of the real workflow with five properties:
- an observable business outcome;
- real or faithfully representative data;
- limited and reversible consequences;
- an available owner;
- a manual route that continues to work.
The boundary might cover one request type, one document class, one team or one low-impact action. Its appropriate size depends on data sensitivity, consequence, technical maturity and the organisation’s ability to control the work.
4. Record the decision, including why AI is the right mechanism
An SME does not need a general document about “AI” before it can make a sensible operating decision. It needs one current record for the workflow in question.
Start with the mechanism, not the selected model. The record should make a decision not to use AI just as inspectable as a decision to approve it.
| Dimension | Decision to make | Minimum evidence | Stop signal |
|---|---|---|---|
| Mechanism | Should the work remain human, become deterministic, use AI assistance or allow an agent to act? | Feasible options tested against the same outcome, workload and complete cost | AI adds variation or control cost without enough additional value |
| Outcome | Which cost, delay or failure should change? | Baseline and outcome measure | The gain disappears after review and maintenance |
| Ownership | Who accepts, monitors and stops the workflow? | Named roles and effective authority | Nobody can make or execute the decision |
| Data | What enters, leaves and remains stored? | Data-flow map, input rules and destinations | Sensitive data or use rights are unresolved |
| Quality | Which outputs are acceptable or unacceptable? | Real cases, edge cases, refusals and evaluation criteria | A severe error cannot be detected before consequence |
| Confirmation burden | Which partial results, approvals and branches does one case create? | Representative end-to-end paths, decision count, wait time and rework | The control path removes the expected gain or creates unowned states |
| Human review | Can each reviewer make the required decision under expected workload? | Time, competence, visible evidence and rejection authority | Rubber-stamping, delay or reviewer overload |
| Security | What can the system access or change? | Identity, least privilege, secrets and logs | Excess access or an untraceable, irreversible action |
| Operations | What is monitored and who responds? | Signals, thresholds, permitted response and escalation | Drift or incident has no assigned response |
| Fallback | How does work continue without the system? | Tested manual path or known-safe configuration | Dependency without a usable recovery route |
| Change | What requires the decision to be reopened? | Trigger list and owner | Silent change to model, data, tools or permissions |
The record is not a certification. It makes the decision inspectable and revisable. A missing answer does not always require a company-wide stop; it may mean reducing the data, authority or operating boundary.
5. Check the complete data, supplier and security path
A statement about model training answers only one data question.
“Your data is not used to train the model” does not necessarily mean “nothing is retained, logged, reviewed or sent to another processor”.
For the exact product, account, configuration and region in use, record:
- whether prompts, outputs, files, feedback or connected data may be used for training or improvement;
- normal and exceptional retention periods;
- the conditions for human access;
- memory, personalisation, analytics and safety processing as separate uses;
- subprocessors and external models that receive the data;
- deletion, export and exit options;
- whether important promises are contractual.
Public policies show what a supplier says. They are not a substitute for a suitable contract, verified settings, customer-accessible evidence or technical assurance where the consequence requires it.
Secure the action, not only the conversation
An AI that reads a document and an AI that changes a CRM record do not create the same exposure. Check:
- the identity under which the system operates;
- the resources, records and actions it can reach;
- environment separation and secret handling;
- manipulative inputs, attachments, links and external content;
- where generated data can be sent;
- useful logs without unnecessary copies of sensitive content;
- access revocation and the stop mechanism;
- restoration to a known state.
The NCSC and its international partners’ secure-AI guidance covers secure design, development, deployment, operation and maintenance. For product-level detail, see IZZY’s guides to managing AI-agent permissions and the controls required before an agent can act.
Where personal data is involved, the CNIL’s recommendations on developing AI systems show how purpose, minimisation, retention, information, individual rights and security apply, and state their own scope: they cover the development phase, not deployment, which needs its own assessment. The CNIL’s AI compliance hub links the rest of its material. The EU AI Act and GDPR can create role- and use-specific duties. Organisations operating elsewhere must assess the rules, roles and sector duties that apply to their actual workflow. This article does not classify a system or establish compliance.
6. Why “a human will review it” is not enough
Human review is often presented as the universal answer to AI uncertainty. In operation, it can become the weakest part of the system.
An AI workflow rarely produces one clean final answer. It may classify an item, request information, propose several actions, receive a confirmation, move to another branch and ask for approval again. Every partial result and confirmation adds a decision point. The number of possible paths, uncertain states and error opportunities grows quickly.
Trace the confirmation burden
Follow one representative case from its first input to a usable outcome. Record every partial result, confirmation, branch, correction and wait state, not only the final approval.
For every checkpoint, define:
- why confirmation is needed;
- which evidence and sources the reviewer can see;
- the available choices and their consequences;
- what rejection, editing or no response means;
- the time and competence required;
- what is retained as evidence;
- who owns an unresolved result.
A human in the loop is a control only if that person can understand the result, spot the relevant error, take the necessary time and reject it without bypassing the process. Under real workload, an approve button can become a ritual.
Measure more than generation speed:
- total time to a usable result;
- number of retries and confirmations;
- corrections after approval;
- abandoned or escalated cases;
- workload transferred to another person;
- maintenance of rules, data and evaluations.
If AI makes one step faster but the complete workflow slower, the design has not created the expected efficiency.
7. Earn broader scope, or stop
A broader operating boundary is justified when evidence from the real process shows that:
- value remains positive after complete operating cost;
- important failures are known and detectable;
- reviewers retain genuine capacity and authority;
- data and access stay within the agreed boundary;
- actions are traceable, limited and reversible;
- stopping and fallback have actually been exercised;
- material changes reopen the decision.
Broader scope may mean more data, users, cases or authority. Avoid changing several dimensions together if you need to understand what caused a drop in performance or control.
When to reduce or stop
Reduce the AI’s role or stop the workflow when:
- a severe error cannot be detected before consequence;
- confirmation, correction or incident cost removes the benefit;
- data, rights or ownership cannot be established;
- the reviewer cannot genuinely reject the output;
- the required permissions are disproportionate;
- no safe recovery route exists;
- a material change has not been re-evaluated.
Bring in qualified specialists when the workflow involves sensitive data, decisions about people, employment, health, finance, insurance, education, critical infrastructure, intellectual-property rights, highly privileged actions or sector-specific duties. IZZY can map the workflow, evidence and technical choices; that does not replace qualified legal, regulatory or cyber-security conclusions.
Conclusion: AI has to earn its place in the workflow
Moving from pilot to production does not start with a supplier or a promise of autonomy. It starts with a real workflow, an expected economic result and a clear operating boundary.
Test where the work actually happens. Count confirmations and retries. Check the data, access and security. Give someone the power to stop. Then let evidence decide whether the boundary should become broader.
The right answer may be an agent, limited AI assistance, deterministic automation, or no change. Avoiding unnecessary complexity is a useful outcome too.
Bring one real workflow. We’ll give you an honest read.
Bring the current workflow, the outcome you need, the options already considered and any evidence from the pilot. IZZY will compare deterministic automation, an AI assistant and an agent, then give you an initial, bounded read on which option justifies its cost and exposure, or whether nothing new should be built.
See how IZZY builds AI products and agents.
Frequently asked questions
There is no universal score. The workflow needs a defined business outcome, owner, permitted data, acceptance evidence, realistic review, limited permissions, monitoring and usable fallback. The required depth depends on consequence.
Choose the simplest option that solves the job reliably. Deterministic automation fits stable, testable rules. AI becomes useful for unstructured inputs, variation that is costly to encode or decision support. An agent is justified only when autonomous action creates more value than the control it requires.
Yes, when the use is authorised and the environment, access, minimisation, retention and consequences are controlled. Otherwise use carefully prepared representative data. Artificially clean examples can hide the defects that will matter in operation.
Not by itself. The reviewer needs the right evidence, enough competence and time, and real authority to reject or stop. That capacity must hold under realistic workload.
Separate training, retention, human access, memory, safety processing, subprocessors, deletion and exit. Check the exact product, account, settings, region and contract in use.
When the benefit survives the full operating cost, important failures are detectable, review holds under load, and stop and recovery routes work. Broader use should follow evidence from the operating workflow.
Sources and limitations
- NIST: Artificial Intelligence Risk Management Framework 1.0
- UK NCSC and international partners: Guidelines for Secure AI System Development
- CNIL: AI system development, recommendations to comply with the GDPR
- CNIL: AI, how to comply with regulations
- European Union: Regulation (EU) 2024/1689 - Artificial Intelligence Act
- European Union: Regulation (EU) 2016/679 - General Data Protection Regulation
Sources checked 5 September 2026. This synthesis combines institutional guidance with aggregate IZZY practice across different engagements. It does not present an individual client case or quantified outcome. It is general operational guidance, not legal advice, a regulatory classification, a security audit or a performance guarantee.