Executive reading · ~60 seconds
Retry recovers a try; Learning changes the probability of recurrence. A material failure closes when it produces mechanism, regression, owner, validity and evidence of effectiveness.
An agent fails, receives a new prompt, and gets it right on the second try. The incident is marked as resolved. Days later, another session encounters the same situation, repeats the error and again consumes time, tokens and human review.
The system corrected an execution. He didn't learn.
Executive summary
Retry restores an attempt. Learning changes the probability of recurrence. A loop closes when failure produces at least one durable mechanism:
- case of regression;
- eval or versioned rubric;
- new or reinforced control;
- change of context, tool or authority;
- addressable operational documentation;
- hypothesis discarded;
- monitoring capable of detecting repetition.
Not every failure deserves a major retrospective. Every material failure deserves an explicit decision about recurrence, impact and destination of learning.
Five answers that feel like learning
Try again
A retry is correct for transient failure. It becomes noise when it hides deterministic error, missing authority, or invalid input.
Increase the prompt
Adding instructions may resolve the current case and degrade others. Without eval, we don't know if the change generalizes or just memorized the example.
Change the model
A more capable model can reduce a certain class of error. It also changes cost, latency, behavior and safeguards. Upgrade does not replace failed property analysis.
Record a note
Documentation without recovery path, owner and trigger depends on accidental discovery. Learning exists, but it is not connected to execution.
Blame the agent
“The model hallucinated” describes a broad symptom. It does not answer why the system accepted the output, what check was missed or what authority allowed the effect.
From failure to mechanism
Use a simple string:
evento -> impacto -> classe -> causa controlável -> mecanismo -> regressão -> owner -> validade
Event
Register the subject: session, input, model, tool, version, environment and output. Do not copy sensitive data that is unnecessary for learning.
Impact
Separate reversible internal failure from external effect, altered data, information exposure, financial decision or damage to the user. The depth of the response follows the risk.
Class
A small taxonomy helps you find recurrence:
| Class | Example | Candidate Durable Response |
|---|---|---|
| intention | ambiguous task | spec and acceptance criteria |
| context | missing or old information | versioned source and validity |
| policy | action outside authority | deny-by-default and gate |
| execution | tool failed | limited retry and fallback |
| verification | checker did not cover the class | negative test/eval |
| evidence | result without binding | receipt linked to subject |
| feedback | correction did not propagate | learning item and owner |
Controllable cause
It is not necessary to discover a metaphysical cause of the model. Look for the boundary that the system can change: input, tool, contract, isolation, verification, authority or operation.
Mechanism
Transform the lesson into an executable or operational artifact. A recommendation without owner and trigger remains a good intention.
Regression
Reproduce the event in a sanitized way. A deterministic test is preferable when it exists; eval with initialing comes in when the property requires judgment. The quality claim needs explicit criteria and observable evidence; trace or subjective inspection alone are incomplete. [R3-C1]
The learning item as a unit of change
A useful log contains:
- stable identifier;
- sanitized event and evidence;
- class and risk;
- decision made;
- mechanism or justification for not changing;
- affected repositories, products or controls;
- owner and term;
- regression testing;
- status:
candidate,accepted,implemented,verifiedorretired.
This unity prevents two extremes: forgetting the failure or transforming any detail into a global rule.
When not to generalize
Learning can also be about keeping the system as it is. An isolated, low-impact and highly unlikely case may not justify permanent complexity.
Record the no-change with:
- estimated impact and frequency;
- cost of control;
- signal that reopens the decision;
- validity of the conclusion.
“No change” is a decision. “We didn’t do anything” is an absence of process.
The danger of the learning backlog
Extensive catalogs can become an archive. Three guardrails keep the loop alive:
- applicability: each item points to the execution point that must change;
- priority by risk and recurrence: not on a first-come, first-served basis;
- verification: implemented is not synonymous with effective.
Useful metrics include recurrence by class, time to engine, expired items, and proportion of material failures regressing. Raw number of lessons encourages documentation, not learning.
A fifteen-minute ritual
For a material failure, respond:
- what happened and which subject was active?
- What effect occurred or almost occurred?
- where could the system have detected it earlier?
- What changes in a durable way?
- How will we prove that the change works?
- Who responds and when will we revisit?
If there is no answer to 4 and 5, the stream probably fixed the episode, not the system.
What this article proves — and what it doesn't prove
Incident response, evals and continuous improvement practices support the separation between restoring and learning. FORGE materializes this separation in records, gates and regressions. The article does not prove that every error can be eliminated, nor that more rules always increase reliability.
Sociotechnical systems change; models, people and processes share flaws. The goal is to reduce preventable recurrence and make residual risk visible.
Conclusion
Trying again may be the correct operational response. It just shouldn't be confused with learning.
The system learns when an experience changes its future behavior in a way that is addressable, testable, and proportionate to the risk. Without this, each new session pays again for the same lesson.
Previous reading: From checklist to executable control. Next suggested reading: The agent cannot be its own auditor.
Editorial and responsibility note
- Research cutoff
- Last review
- Recorded corrections
- No corrections recorded.
The cutoff above applies to the canonical claims. Additional sources and their access dates are identified in the article body.
This article combines cited sources, analysis, and the author's professional experience. Verifiable data and factual statements are linked to their respective sources. Interpretations, hypotheses, projections, recommendations, and opinions represent the author's professional point of view at the time of publication; they do not constitute proven facts, a promise of results, or legal, financial, or technical advice applicable to a specific case. Consult the original sources and qualified professionals before making decisions.
Claims and sources
R3-C1
An AI quality claim requires explicit eval criteria and observable execution evidence; traces alone and subjective inspection alone are incomplete.
Limit: Neither source defines FORGE-specific pass thresholds; baselines must be calibrated per use case. This is a research-synthesis design input; it does not prove product adoption, operational maturity, independent attestation or outcome.