Skip to content

Case study · Custom AI Systems

Automating the work between documents, decisions and teams

Operational work is a loop run by hand: a document arrives, someone reads it, decides what it is, copies fields into a system, and forwards it on. Stallwart turns that loop into an orchestrated workflow: the system classifies and extracts, routes the work, and a person approves what matters, with an audit trail behind every step.

The approach
  1. 01Document intake
  2. 02Classification
  3. 03Extraction
  4. 04Routing & decisions
  5. 05Human approval
  6. 06Audit trail & analytics
Client
a professional services firm
Problem
Operational work lives in inboxes and spreadsheets: manual intake, re-keying, and no audit trail.
Approach
A workflow platform: intake, classification, extraction, routing, human approval, exceptions, audit.

The problem

A lot of operational work is the same loop run by hand: a document arrives, someone reads it, decides what it is, copies fields into a system, and forwards it to whoever handles the next step. It is slow, it is inconsistent between people, and when something goes wrong there is no record of who decided what or why.

Off-the-shelf tools automate the happy path and fall over on the exceptions, which is where operations actually spends its time. The whole loop, exceptions included, has to become a structured workflow before any of it can be trusted.

The reality behind the problem

The obvious fix is a script or an RPA bot per task: one that reads the inbox, one that keys the fields, one that clicks the next button. Each works in isolation and then breaks the moment a step depends on the one before it. Operational work is not a set of independent clicks, it is a multi-step process that carries state: what was decided, what is still pending, what has to happen if a later step fails.

The harder reality is partial failure. A real process spans several systems, and any of them can be slow, down, or return something unexpected halfway through. A bot that assumes every step succeeds leaves work stranded in an unknown state, with no safe way to retry and no clean way to undo what already happened. The thing that actually needs building is the orchestration around the steps, not the steps themselves.

What was assumed

This process is too messy and judgment-heavy to automate.

What Stallwart asked

Which steps are rules, which need judgment, and can the system orchestrate between them while keeping state?

The desired outcome

The target is not a faster bot. It is an orchestrated workflow that holds the state of every item, keeps a person in control of the consequential calls, and can always explain and, where needed, reverse what it did.

  • Each item moves through the process as a tracked unit of work, with its current state always known.
  • Rule-based steps run automatically; steps with real consequences wait for human approval.
  • A step that fails partway can be retried safely, without duplicating work already done.
  • Actions that change the outside world can be reversed or compensated when a later step fails.
  • Inputs that do not fit the expected pattern are detected and routed to a person with context, not dropped.
  • Every action, automated or human, is recorded, so any item can be explained after the fact.

The system we designed

Stallwart builds a platform that takes documents in from the channels work already arrives through, then classifies each one and extracts the fields that matter. Each item becomes a tracked unit of work with its own state, and an orchestrator moves it from step to step, running the routine decisions that are genuinely rules and waiting where a person is needed.

Anything that needs judgment, or any input that does not fit the expected pattern, is routed to a person for approval or handling rather than forced through. Steps that touch other systems are made retryable and reversible, so a partial failure can be recovered without duplicating work. Every action, automated or human, is written to an audit trail, and analytics show where volume concentrates and where exceptions keep appearing, so the process can be tightened over time.

How it works

  1. 01

    Separate rules from judgment

    The first step is deciding which parts of a process are genuinely rules and which need a person. The system automates the rules and routes the rest, instead of pretending the whole thing can run unattended.

  2. 02

    Design for exceptions, not just the happy path

    Inputs that do not fit the expected pattern are detected and routed to a human with context, rather than dropped or forced through. Handling the exceptions is what makes the automation trustworthy day to day.

  3. 03

    Make every step auditable

    Each action, automated or human, is recorded. An audit trail is what lets an operations team trust the system with real work and answer, later, what happened on any given item.

The workflow

Intake to approved action, with the exceptions handled.

  1. 01

    Document intake

    From the channels work arrives through

  2. 02

    Classification

    What is this, and what path does it take

  3. 03

    Extraction

    The fields that matter, structured

  4. 04

    Routing & decisions

    Rules automated, work sent onward

  5. 05

    Human approval

    Judgment and exceptions handled by a person

  6. 06

    Audit trail & analytics

    Every step recorded, patterns surfaced

Routine steps are automated; judgment and exceptions route to a person, and every step is recorded.

Architecture

What sits underneath.

Underneath the workflow are four layers. The orchestrator holds the process together; the others handle what comes in, who decides, and what is recorded.

01

Intake, classification, and extraction

Documents arrive from the channels work already uses, and each is classified by type and has its relevant fields extracted into structured output, so the rest of the system works from clean data rather than raw attachments.

02

Orchestration and state

Each item is a tracked unit of work with a persisted state. The orchestrator moves it step by step, knows where every item is at any moment, and resumes exactly where it left off rather than restarting from the top after an interruption.

03

Approval and human-in-the-loop

Consequential steps pause for a person, who sees the item, the extracted fields, and the context of the decision, then approves, edits, or rejects. Exceptions route here too, so judgment and odd inputs reach a human instead of being forced down the automated path.

04

Observability and audit

Every action, automated or human, with its inputs, outputs, and who or what performed it, is written to an append-only record, and operational metrics surface throughput, where items wait, and where exceptions concentrate.

The hard parts

Where the engineering judgment was.

01

Human approval gates on consequential steps

Operations carries real consequences, so the system asks for approval where a wrong automated decision would be costly, and acts alone only where the rule is clear.

The trade-off

A person stays in the loop on the important steps, so this is supervised automation, not a black box.

02

Route exceptions instead of widening the rules

Trying to encode every edge case makes the automation brittle. Detecting an exception and handing it to a person keeps the automated path simple and reliable.

The trade-off

Some volume still reaches a human by design, which is the point: they handle judgment, not re-keying.

03

Orchestrate stateful steps rather than chaining scripts

A multi-step process that spans systems needs to survive a failure halfway through. Holding each item's state in an orchestrator lets a step retry or resume without losing track of what already happened.

The trade-off

An orchestrator is more to build and operate than a set of scripts, but it is what makes partial failure recoverable instead of silent.

Reliability & guardrails

How it avoids the wrong call.

The guardrails are about what happens when a step does not go cleanly, so a failure never leaves work in an unknown or unrecoverable state.

Retries that do not duplicate work

Steps that touch other systems are made idempotent, so a retry after a timeout or transient error repeats safely instead of creating a second copy of the same action.

Reversible and compensating actions

Where a step changes the outside world, the system records how to undo or compensate it, so a failure later in the process can roll the item back to a clean state rather than leaving it half done.

Approval gates on consequential steps

A person must approve before the system takes an action with real consequences, so a wrong classification or extraction is caught by a human before it propagates rather than after.

Failure isolation, not silent drops

An item that cannot complete is held in an explicit error state with its context, visible to an operator, instead of being dropped or retried forever in the background where no one can see it.

Productionization

What turns it from a demo into software.

What turns the workflow from a demo into software operations can run every day, rather than a script that works until the first unexpected input.

Durable state and resumability

Each item's state is persisted, so a restart, deploy, or crash resumes in-flight work from where it stopped instead of losing it or starting the process over.

Observability into the live process

Dashboards and metrics show how many items are in each step, where they are waiting, and where exceptions concentrate, so operators manage the process rather than guessing at it.

Versioned workflows

Changes to the process are versioned, so items already in flight finish under the logic they started with and new rules apply going forward, without breaking work mid-stream.

Replay from the audit trail

Because every action is recorded, an item can be reconstructed and, where needed, replayed, which makes debugging a specific failure a matter of reading the record rather than reproducing it by hand.

What changed

The change is in what the operations team actually does. Instead of reading, keying, and forwarding every item by hand, they review the ones that need judgment and approve the consequential steps, while the routine path runs itself and stays recoverable when something upstream misbehaves.

The process stops being a set of fragile scripts that no one can explain and becomes a workflow with a known state for every item and a record of every decision. When something goes wrong, there is an item to inspect and a trail to follow, not a silence to investigate, and the automated path can be tightened as the analytics show where work and exceptions concentrate.

  • Each item moves through the process with a known state, not as a one-off manual pass.
  • People spend their time on judgment and approvals, not re-keying fields.
  • A step that fails partway can be retried or reversed, instead of stranding the item.
  • Any item can be explained after the fact from an audit trail of every action.

Ownership & handover

What you receive, and keep.

At handover, the platform and everything needed to run and change it are yours.

  • The orchestration workflows, the intake, classification, and extraction components, and their source.
  • The human-approval interface and the rules that decide which steps require it.
  • The audit-trail store and the analytics that surface throughput and exceptions.
  • Runbooks for operating the workflows, handling stuck items, and recovering from failures.
  • Documentation of the state model, the retry and reversal logic, and how to version a workflow change.

What we learned

The insight that generalizes: automating operations is an orchestration problem, not a scripting one. The value is not in making a single step faster, it is in holding the state of a multi-step process, keeping a person in control of the consequential decisions, and making every action recoverable and explainable. Build the orchestration first, and the individual automations become safe to add.

This work connects to

  • Custom AI Systems
  • AI operations automation
  • Intelligent document processing
  • Workflow orchestration
  • Human-in-the-loop systems

Frequently asked

What is AI operations automation?

It is turning a repetitive operational process, such as document intake, classification, data extraction, and routing, into a structured workflow where the system handles the rule-based steps and routes judgment and exceptions to a person, with an audit trail behind every action.

How is this different from traditional RPA?

Rule-based automation follows fixed scripts and breaks on anything unexpected. An AI operations workflow can classify and extract from inputs that vary, and it orchestrates multi-step work with persisted state, so it detects exceptions and routes them to a human and recovers from a partial failure rather than failing silently.

Does automating operations remove human oversight?

No. The system automates the routine, rule-based steps and keeps approval gates on consequential decisions. People handle judgment and exceptions, and every step is recorded, so oversight increases rather than disappears.

What happens when a step fails partway through a workflow?

Each item carries its own persisted state, so a step that fails can be retried safely without duplicating work already done, or reversed through a compensating action if a later step cannot complete. An item that still cannot finish is held in an explicit error state with its context for an operator to resolve, rather than being dropped or silently retried.

Have an operational loop that runs on inboxes and re-keying?

Walk us through the process. We will scope the workflow that automates the rules and routes the judgment to your team.

Last updated: October 13, 2026