Applied AI insights

Evaluation and adoption

Where does work get stuck? A practical AI workflow guide

Work gets stuck where records, decisions and accountability no longer move together. A useful AI workflow assessment identifies that point, tests whether a focused system can help and preserves human authority over consequential decisions.

Assess one bounded workflow using real records, named ownership, clear controls and measurable outcomes. Then decide whether to proceed, reshape the scope, defer automation or stop.

How to assess where work gets stuck

An operational workflow is the sequence of people, records, systems and decisions involved in completing a defined task. A bottleneck is where work repeatedly slows, accumulates, fails or demands disproportionate human effort.

The assessment should examine the work before selecting the technology. Define where the workflow begins and ends, which records support each decision, who remains accountable and how improvement will be measured. The aim is not to justify AI. It is to make a reasoned operating decision.

  1. 01Define the workflow boundary. Identify the trigger, completion point and systems involved. A narrow boundary keeps attention on a real operational problem instead of drifting into adjacent processes.
  2. 02Name the accountable owner. This person owns the operational result. Authorised reviewers retain approval rights for consequential decisions.
  3. 03Examine representative records. Review ordinary cases, difficult cases and known exceptions from the real workflow, not only polished process maps or convenient examples.
  4. 04Trace identifiers and discrepancies. Establish which supplier, order, delivery and invoice identifiers align reliably, which need interpretation and which must be reviewed by a person.
  5. 05Set one measurable outcome. Define an operational improvement, such as reducing the human effort required to prepare complete cases while retaining approval controls.
What a practical workflow assessment should establish
Assessment areas, evidence and reasons for review
Assessment area What to confirm Evidence needed Why it matters
Workflow boundary A clear trigger, end state and set of systems Process map and representative cases Prevents scope from expanding into unrelated work
Ownership A named accountable owner and authorised reviewers Role definitions and approval limits Establishes decision authority
Inputs Source records are available and representative Emails, orders, delivery records, invoices and metadata Tests whether the real workflow can be evaluated
Identifiers How order, delivery, supplier and invoice identifiers align or fail to align Field mappings and exception samples Reveals matching and routing difficulty
Controls What the system may interpret and what it must never authorise Approval matrix and control requirements Keeps consequential decisions with authorised people
Outcome One operational improvement can be measured Baseline volumes, quality data and handling-time data Supports a meaningful decision
Evaluation reference Verified answers exist for representative cases Cases independently resolved and approved by authorised reviewers Enables comparison with known outcomes before production

A process map shows how work should happen. The records show how it actually happens.

Separate rules, AI and decision authority

Rules and AI serve different purposes. Rules suit stable, explicit conditions, such as confirming that a mandatory field is present or that invoice totals agree. They should remain versioned, traceable and connected to clear exception routes.

AI can help interpret ambiguous language, variable document layouts and possible relationships between records. Its outputs still need confidence thresholds, source evidence and review routes. AI interprets ambiguous material; rules validate known conditions.

A combined workflow may interpret a supplier document, apply established checks and prepare the result for an authorised reviewer. It should not quietly take authority that belongs to finance, operations, procurement or another accountable function.

A useful boundary: the system may extract, compare, classify, flag and prepare. An authorised person accepts, corrects, rejects or approves consequential outcomes.

Agent Aligned’s applied AI workflow guidance covers other operational tasks where interpretation, validation and authority need to remain distinct.

Illustrative operating scenario

Consider a supplier email and document-reconciliation workflow covering orders, deliveries and invoices. This example shows how a focused system could organise evidence and prepare cases for review. It does not describe a client deployment or promise a performance result.

Trigger and source inputs

A supplier email arrives with an invoice or delivery document attached. Its arrival creates a case and starts the bounded workflow.

The case may draw on the supplier email, purchase order, delivery record, invoice, approval limits and identifier mappings. Each source remains identifiable so that a reviewer can trace a proposed value to the record from which it came.

Exact matching may not be enough. An order may use a formal supplier name, an email may arrive from a shared trading address and an invoice may use an abbreviated entity name. An invoice reference may contain a prefix omitted in another system, while one delivery record may cover only part of an ordered quantity.

System role and conflict boundary

The system extracts candidate values, aligns identifiers where the evidence supports a relationship, applies known validation rules and prepares a reviewable case. It can flag missing fields, mismatched totals and possible links between records. It does not hold approval authority.

When source records conflict, the system should retain both values and route the discrepancy with its source evidence to an authorised reviewer. It must not silently choose a value, approve a case, post a transaction or make a payment.

Exceptions and human approval

Low-confidence cases, missing records and unresolved identifier differences should move to a review queue. The queue should explain why each case needs attention, display the relevant source material and direct it to someone with appropriate authority.

An authorised person reviews the evidence and accepts, corrects or rejects the proposed outcome. The action should be logged with the evidence available at the time, creating a traceable record of the decision.

Primary measure: before production, the confident error rate can show whether apparently reliable outputs are still incorrect often enough to create unacceptable operational risk.

The system remains bounded: it prepares evidence and applies defined checks, while people control exceptions and consequential actions. Teams considering a specialised logistics use case can also read Agent Aligned’s guidance on logistics document-to-cash workflows.

Evaluate a focused system before production

Test the system against a verified evaluation set before it affects live work. This is a representative group of cases whose correct answers were independently established and approved by authorised reviewers before testing began.

The set should reflect the work the system will encounter, including normal cases, difficult cases and known exceptions. If missing documents, irregular layouts or inconsistent identifiers occur in practice, include them in proportions that support a meaningful judgement. A collection of easy, complete cases will overstate readiness.

Keep configuration examples separate from evaluation cases. Configuration examples help design instructions, mappings, rules and thresholds. Evaluation cases test the configured system on material it was not tuned against.

Define measures before testing

Confident error rate
The percentage of outputs above the agreed confidence threshold that authorised review later finds incorrect.
Confidence threshold
The minimum level of system confidence required for an output to follow a specified review path.
Median human handling time per correctly completed case
The middle review time when correctly completed cases are ordered from shortest to longest. The median limits the influence of a few unusually long cases.

Report quality and effort separately. Lower handling time cannot compensate for unacceptable errors. Strong accuracy on a small sample does not show whether the workflow will remain manageable at realistic volumes. Decision-makers need both measures, sample sizes and the conditions under which the test was conducted.

Test controls and rollback

Evaluate the controls as well as the outputs. Access should follow approved roles. Logs should record inputs, proposed values, validation results, reviewer actions and relevant system versions. Reviewers should be able to trace each output to source evidence.

Escalation paths should cover missing records, low confidence, failed integrations and cases outside the designed scope. A rollback plan should explain how the workflow returns to an approved manual or earlier operating state if the system, integration or control fails. Test that path before production.

Incomplete records and unstable processes limit what an evaluation can prove. If teams bypass the documented process, definitions change during testing or key records are unavailable, process repair or a narrower scope may be the better next step.

Where personal information or regulated records are involved, assess lawful processing, access, retention, providers and accountability with appropriate professional input. Agent Aligned’s article on POPIA, data residency and sovereign AI offers context, but does not determine compliance for a particular workflow.

When to defer, reshape or stop

Defer automation when no accountable owner can define the desired result or accept responsibility for operational decisions. Establish ownership and approval authority before designing the system.

If representative records are unavailable, pause the build and collect evidence from the real workflow. Synthetic or unusually clean examples may support early exploration, but they cannot show how a system will handle normal variation and exceptions.

If the intended outcome cannot be measured, reshape it into a narrower operational objective. The team should be able to state what improvement it expects to observe without relying on a vague ambition to use AI.

An unstable workflow may need repair first. Frequent policy changes, undocumented workarounds and shifting approval paths make dependable behaviour difficult to configure and evaluate. A narrower, stable part of the process may still be suitable. If no valuable, controllable scope remains, stopping is a responsible decision.

Decision guide for a workflow opportunity
Proceed, reshape, defer and stop decisions for AI workflow opportunities
Decision Use it when Next action
Proceed The workflow is bounded, owned, supported by representative evidence and linked to a measurable result. Build and evaluate a focused workflow with defined controls and exception routes.
Reshape The problem is worth solving, but the scope is too broad, records are inconsistent or the proposed system carries too much authority. Narrow the workflow, change the objective or strengthen the control boundary.
Defer Missing ownership, evidence or process stability can reasonably be addressed. Repair the prerequisite, collect records and reassess.
Stop The workflow lacks a worthwhile outcome, cannot be controlled appropriately or would transfer unacceptable responsibility to a system. Record why further investment is not warranted.

Agent Aligned’s delivery phases

Agent Aligned uses three phases to move from an operational bottleneck to a controlled production system. Each phase creates a decision point between understanding the problem, building a focused workflow and sustaining it in normal operations.

Bottleneck Diagnostic

This phase typically takes about one to two weeks and produces a proceed, reshape or stop decision. It defines the workflow boundary, examines representative records, identifies control requirements and determines whether a focused system has a credible operational purpose.

A proceed decision supports a bounded build. Reshape narrows or changes the opportunity. Stop records why further investment is not warranted.

Build and Evaluation

This phase typically takes about four to six weeks for a focused first workflow. The system is built around approved inputs, decision boundaries and exception routes, then tested against the evaluation reference.

The purpose is to expose limitations and unresolved risks rather than conceal them behind an impressive demonstration. Production readiness depends on evidence from the defined task and its controls, not broad claims about the underlying technology.

Sustaining Production

Production systems become part of the operating workflow. Quality indicates whether completed work remains correct. Latency shows whether cases move through the workflow within an acceptable period. Cost includes the resources needed to run, review and support the system. Adoption reveals whether authorised users can and do work with it as intended.

These signals need to be considered together. An accurate system that is too slow, expensive or poorly used may still fail operationally.

Reviewer corrections are evidence for investigation. They should not alter production behaviour without authorised testing, approval and controlled release management.

Prepare for a Bottleneck Diagnostic

Bring enough evidence to examine one real workflow without turning the diagnostic into a broad transformation programme.

  • A named workflow owner who can define the operational result and coordinate decisions
  • The trigger that starts the workflow and the intended outcome that ends it
  • Representative source records, including routine and difficult cases
  • Known exceptions and examples of how they are handled
  • Approval limits and the roles authorised to accept, correct or reject outcomes
  • Available baseline volume and handling information for the current process
  • Systems, integrations, access restrictions and other practical constraints

Handle sensitive records through approved channels and share them only with people who need access. Consider personal information, commercial confidentiality and retention obligations before assembling the material.

South African buyers in KwaZulu-Natal can review regional information for Agent Aligned’s Durban office.

Sources and further reading

About the author

The AgentAligned Editorial Team writes from the delivery side of applied AI engagements: the practitioners who design, build and run governed workflows for South African operations. Articles are reviewed before publication and updated when practice changes. More about how we work.