Three lessons from optimizing agent behavior as something you train, not something you write.
By Mitul Tiwari, CTO and Cofounder, Numos AI
Background
A smart engineer opens the prompt. Adds a clarifying sentence. Runs a few examples. Output looks better. Ship it. That's how most agent behavior still gets written, in prompts and in skill documents alike, and the loop has no memory: no baseline, no score, no record of what the sentence replaced.
In finance, that habit has a specific cost. A wrong journal entry doesn't throw a compile error. It lands in the GL and surfaces in an audit six months later. There is no unit test for "this reconciliation is correct," only for “this reconciliation ties”, and those two are different claims. If you want that signal, you build it. So when behavior changes, somebody has to be able to say what got better, what got worse, and how they know. "Output looks better" is not an answer a controller can take to an auditor.
Anthropic recently described recursive self-improvement as a ladder of delegation, and put the last human-held rung at research taste: knowing which problems are worth attacking.
From inside a production finance AI stack, the ladder looks the same and the top rung looks different. What a human can't hand over is the definition of correct. Everything below it, we've been automating. Agents emit tapes: typed traces of what was asked, what was reasoned, which tools ran, and what came back. An optimizer reads them, diagnoses the failure, and rewrites the instruction. A validation gate decides what ships. The result is a system that improves itself.
What's broken
Strip away the tooling and the old workflow has four structural problems.
Behavior is authored by hand, without provenance. Forty prompts across six domains, each carrying sentences added by whoever had context that week. Nobody remembers why paragraph three says what it says, so nobody dares delete it. Instructions accumulate like sediment.
Improvement is unmeasured. "Output looks better" compares a new version against a felt memory of the old one. Whether last month's change helped is unknowable, because the version it replaced was never scored on anything.
The logic isn't addressable. When behavior is scattered across services, prompt fragments, and conditionals, there's no object an optimizer could hold even if you had one. You can't train what you can't address.
The most useful feedback gets discarded. Every run produces a rich account of what the system did and where it went wrong. Conventional logging files it away once the bug is closed. Reinforcement learning, the sophisticated alternative, compresses that to a scalar. The single most informative artifact in the system is treated as exhaust.
For a finance product, all four converge on the same failure: no change made this way is defensible. Not to an auditor, and not to the controller who has to sign the close.
Where things actually stand
Two things changed in the last year, and they changed independently.
The optimizers got good. DSPy (my explainer) made the case that prompts should be compiled rather than authored. GEPA pushed it further and showed that reflective text-space optimization can beat reinforcement learning, outperforming GRPO (my explainer), an RL baseline, by up to 20% while using up to 35X fewer rollouts. No weights move. It just reads traces and rewrites text.
The unit of agent logic got bigger. Agent behavior stopped living exclusively in code and started living in skill documents, a workflow, its edge cases, and the judgment calls that go with it, written down in one addressable place. These skill documents are trainable now with SkillOpt, which treats a skill file as the external state of a frozen agent and optimizes it with the training discipline you'd apply to model weights.
On our side, the substrate was already in place. Every Numos agent runs and emits a tape (TapeAgents talk and article), which includes typed traces of prompts, reasoning, tools invoked, and output of the tools. Each domain has real eval datasets and scoring functions, because finance work has to be checkable. And we moved agent logic out of code and into skill documents, and we shipped an agent optimizer as a part of our product offering. This makes the next rung addressable. Three lessons from addressing that.
1. The tape is the training signal
A tape is a typed, structured trace of the prompts received, the reasoning performed, the tools called, the data observed, and the actions returned (TapeAgents talk and article). We built tapes to capture every trace an agent produces and feed it back into optimization. The same artifact earns its keep twice. The optimizer reads a tape as a trajectory: what was tried, where it went wrong, what the failure looked like in full. Also, a controller reads the same tape, in plain language, to see why an accrual landed at $40K, without paging engineering.
Text-space optimization doesn't want a reward. It wants the account. Hand a model the whole trace, in language, and ask it to diagnose the failure and revise the artifact. Language provides "a much richer learning medium" for LLMs (GEPA, Agrawal et al.).
Which puts the burden somewhere teams don't expect: your scorer is part of the optimizer. A metric that can only return 0.0 or 1.0 teaches nothing. Our scorer for the billing-period task, which pulls the service period off an invoice, returns the predicted date range beside the expected one, plus the ground-truth notes explaining that the invoice covers a quarterly service period. That's not a score. That's a diagnosis, and it's what the loop actually learns from.
This generalizes past prompts, which is why it's a lesson rather than a footnote. SkillOpt runs a forward pass with a frozen model, a backward pass where a separate optimizer model reads successful and failed trajectories, then an update step proposing bounded edits to the skill document. Different artifact, same appetite: traces plus feedback that explains itself. Whether the unit under optimization is one instruction, a skill encoding an entire close workflow, or eventually the agent's own scaffolding, the input is the tape.
So the foundation doesn't get rebuilt as you climb. We spent time on tapes and articulate scorers because observability and optimization go hand in hand, and finance demanded them. It turns out that's the prerequisite list for self-improvement at every rung. Teams that skipped the audit trail because nothing forced them to now have nothing to optimize against. The shortcut and the dead end were the same decision.
2. Train the prompt instead of writing it
Let’s take the billing-period task from the previous section: one prompt, a golden eval set, an exact-match metric, and a scorer that explains each miss. Our production prompt already scores 100% on that set, which is the right outcome for a production prompt and the wrong one for testing an optimizer, because there is nothing left to improve. So we gave it something to fix. We seeded the run with a deliberately wrong prompt:
Find the invoice date (the date printed on the invoice) in the document below, and return it as both the start date and the end date.
That's the mistake a rushed human makes too: conflating when an invoice was issued with the period it covers.
It closed the gap, purely from scoring feedback: predicted-versus-expected dates plus ground-truth notes. The rewritten prompt redefined the target as the service and coverage period, explicitly not the invoice, due, or issue date, then went further and derived a billing-schedule-table rule that the harder purchase-order cases need. Nobody had written that rule down for it.
Two consequences, both now product decisions rather than script decisions.
- Headroom ships with the task. Every task in the optimizer offers a choice of starting prompt before a run begins: the production prompt, or a deliberately weak one. You validate a self-improving loop the way you test a smoke alarm, with smoke, and that shouldn't require an engineer.
- Runs have to be legible afterward. An inspector lists every golden example with the pass or fail the winning prompt actually produced on it, recorded at eval time. Aggregate score is not enough; the failure that matters is the one where the average improves while a specific class of cases quietly regresses.
Tasks stay separate for the same reason. Each owns its own eval set and its own metric. Month-end close is not one task, and an optimizer chasing a single averaged score will happily trade away the specialist that finally cracked intercompany elimination for a generalist that's mediocre everywhere. Averaging is how an institution forgets things nobody noticed it knew.
3. Skills driven agents and optimization
A prompt is a thin slice of what an agent knows. The richer unit is the skill: a document holding a whole workflow, the preconditions, the escalation rules, the treatment of the awkward third case that only shows up at quarter end.
That's why we moved agent logic out of code and into skill documents, and the reason isn't only reuse or readability, though both are real. It's that a skill is a unit an optimizer can hold, and logic scattered across services is not. Making behavior addressable is the precondition for making it trainable.
SkillOpt shows what becomes possible once behavior is addressable. Treating the skill as the trainable external state of a frozen agent, it reports lifting a six-benchmark average from 58.8 to 82.3, ahead of prior text-space optimizers including GEPA and TextGrad. Two results matter more than the headline. (1) Skills transfer: one trained under a coding agent moved to a different harness and carried performance from 22.1 to 81.8, nearly matching a skill trained there directly. And (2) a small model with a trained skill outscored a larger model without one. In a domain where inference costs compound across every invoice and every reconciliation, that's an economic argument, not just a benchmark curiosity.
But the part I'd point to is the governance, because most of SkillOpt's design is governor rather than engine. As SkillOpt’s authors observed, skills "grow longer and drift with each rewrite," and a revision that looks perfectly reasonable "can quietly degrade real task performance." So edits are bounded add/delete/replace operations, a textual learning rate caps how much can change per step, a validation gate accepts a candidate only when it strictly improves a held-out score, and a buffer of rejected edits stops the loop re-proposing what already failed. Median optimized skill length lands around 920 tokens, with only one to four accepted edits in their case studies.
We built a skill optimization loop against our own stack, governor included, and it earned its keep on the first run. The target was the style guide our agent follows when drafting accounts-payable replies. Style has no exact-match answer, so the scorer was an LLM judge anchored to 64 held-out replies in the evaluation set. We started from a weak 74-character seed skill. The whole run:
Step one worked: the optimizer model read the judge's per-dimension critiques, proposed a single edit backed by six failing cases, and the score moved. Step two is the interesting one. It proposed two more well-argued edits, grew the document by another 56%, and scored worse. The gate reverted it and the run stopped. A revision that looked perfectly reasonable quietly degraded performance, in the second step we ran, and got caught. Optimization done properly yields an artifact a human can still read and sign.
A loop that can improve itself can also degrade itself, and that shapes where the rules live. Nothing the loop can rewrite is allowed to enforce a rule. In the Numos harness the model decides what it wants to do and a separate deterministic layer decides what it's allowed to do; the agent drafting a journal entry is not the agent validating it, and every action passes verification before it touches a system of record.
Conclusion: the human moves up a rung
The progression is clearer than it was a year ago. Optimize a prompt and you've tuned one instruction. Optimize a skill and you've tuned a workflow. Optimize the harness (the scaffolding, the tool wiring, the orchestration) and you're at the rung where "self-improving" stops being a stretch.
Every rung rests on the same foundation, and it isn't the optimizer. Standing this up was fast precisely because the three hard prerequisites already existed in our stack: (1) tapes, (2) real eval datasets, and (3) scoring functions that explain their reasoning instead of returning a bare number. Those took years. Wiring the optimizer took weeks.
Which brings the ladder back to where this started. Anthropic puts the last human-held rung at research taste, judging which problems matter. Ours puts it at grader authorship, defining what correct means when correctness is scattered across an ERP, a bank statement PDF, and a Slack thread from the close lead. Those are the same claim arriving from opposite ends of the ladder. Every rung a system climbs takes over more of the how and leaves the human holding more of the what.
Anyone can rent the same frontier model you do. Nobody can clone your verifier.
For the finance teams running Numos, that rung has a concrete shape. They tell the system what correct looks like for their close, in examples and exceptions, and the system does the climbing. The controller's job moves from checking the agent's work to defining what the work is.
So if your agents could rewrite themselves tonight using nothing but today's tapes, the constraint wouldn't be the optimizer, or the model, or the compute. It would be whether your definition of "correct" is good enough to be worth converging on.
Sources: When AI Builds Itself (Anthropic Institute) · SkillOpt: Executive Strategy for Self-Evolving Agent Skills (arXiv 2605.23904) · SkillOpt: Agent skills as trainable parameters (Microsoft Research) · GEPA (arXiv 2507.19457) · TapeAgents (arXiv 2412.08445) · TapeAgents talk (YouTube)
