Article · AI translation
Let every task become the starting point for the next improvement.
Yunzhong reporting from Aofei Temple
QbitAI | WeChat public account QbitAI
Yunzhong reporting from Aofei Temple
QbitAI | WeChat public account QbitAI
Your colleague uses an Agent to organize reports, and everything goes smoothly. You use the same model, tools, and workflow with your own files, but the Agent starts repeatedly trying to read them and frequently redoing its work.
The model has not changed, and the Harness has not changed. Why does it stop working when the environment changes?
Different file locations, mixed data formats, and changing template requirements can all make previously effective methods fail. The model provides reasoning capabilities, and the Harness organizes the execution workflow, but the Agent still needs to determine: What is this environment like, and does past experience still apply?
This also reveals an easily overlooked issue in an Agent's ongoing self-improvement: the Agent must not only learn “how to do it” from tasks, but also determine under what environmental conditions that experience remains valid.
During the continuous optimization of RSI (Recursive Self-Improvement), an Agent can use task feedback to improve its execution methods and accumulate reusable experience. But if its understanding of the environment is not updated at the same time, formerly useful experience may become a new obstacle: interface conditions have changed, but it keeps retrying old calls; data formats have changed, but it keeps applying old rules.
When a failure occurs, should the method be changed, or should the Agent's understanding of the environment be corrected?
Research on knowledge structures in cognitive science offers insight into distinguishing these two kinds of problems:
In 1983, cognitive scientist John R. Anderson systematically discussed two important forms of human knowledge in research related to ACT-R theory:
Declarative Knowledge: describes “what the world is like.” For example, a closed container obscures the objects inside it; they can only be observed after it is opened.
Procedural Knowledge: describes “how to act.” For example, when looking for multiple objects of the same kind, after finding the first one, you should continue checking other possible locations.
Applying this perspective to Agents shows that understanding an environment and knowing how to act are related, but require different forms of validation and updating. Judgments about the environment must be checked against the facts on the ground, while methods of action must be tested for effectiveness under the relevant conditions. Confusing the two can lead to treating an operation that succeeded by chance as a reliable rule, or continuing to use a method that no longer works after the environment changes.
Following this line of thought, openJiuwen has further proposed TTSE (Two-Track Self-Evolution) in its ongoing exploration of RSI.
In its previously published introduction to a two-dimensional RSI framework, openJiuwen used two-dimensional Harness+Artifacts optimization to enable Agents to continuously improve their work methods and deliverables based on feedback from real tasks. TTSE focuses further on the reliability and reusability of experience itself: it enables Agents to accumulate effective execution methods while continually correcting their understanding of the environment, so that experience does not become invalid when the environment changes.
To this end, TTSE draws on the distinction between declarative and procedural knowledge in cognitive science and divides experience gained from tasks into two tracks:
FACT (environmental facts): continually checks and updates “what this environment is like.”
TIP (conditional techniques): continually improves “what to do under these conditions.”
Distinguishing the types of experience is only the first step. How can an Agent keep learning from task feedback and turn effective experience into improvements for the next round of execution?
This is also the core idea of RSI: use feedback from real tasks to drive improvements, then continually validate and refine them in subsequent execution. TTSE applies this idea to experience management by executing tasks in batches, continually testing existing experience against new evidence, determining what is worth retaining and what needs revision or retirement, and then using the updated experience for the next round of tasks.
The framework consists of four parts—two-track knowledge representation, cold start, online incremental evolution, and final output and evaluation—as shown in the figure below:
① Two-track knowledge representation: giving environmental facts and execution methods their own evidentiary basis
FACT describes the attributes and patterns of the environment itself, emphasizing direct verifiability and stability across tasks in applicable settings; TIP describes execution methods under specific task conditions, expressed in the form “under what conditions, perform what action.”
The two types of knowledge each have separate identifiers, presentation formats, and evidence-tracking mechanisms. This allows each piece of experience to be cited, checked, and updated individually, and makes it easier to trace which specific factual judgment or operational recommendation influenced an execution problem.
For classification, the framework makes a judgment based on what the experience describes and what it does (expand on how classification is performed according to the mechanism); content that is hard to classify defaults to TIP, reducing the risk of mistaking operational experience that has not been adequately validated for an environmental fact.
② Cold start: deriving initial experience from the first batch of tasks
TTSE starts with an empty rule base. The Agent executes the first batch of tasks (chunk 0), then derives FACTs and TIPs separately from its execution traces to establish an initial two-track experience base.
This stage focuses on induction. Because no historical rules have yet been used in execution, it does not assign blame to or retire old rules; instead, it establishes the first set of experiences backed by task evidence for subsequent evolution.
③ Online incremental evolution: testing and correcting existing experience through new tasks
Subsequent tasks proceed in batches. In each round, the Agent loads the FACTs and TIPs from the previous round, uses that experience to execute the current batch of tasks, and then completes four steps in sequence based on execution traces and evaluation results:
Blame: analyzes failure traces and identifies rules that may have misled execution, pinpointing what needs to be corrected.
Retire: moves rules assigned blame into a retirement pool and keeps records of the reasons, batch, time, and other details so the adjustment process remains traceable.
Contradict / Synthesize: examines contradictions among retired rules, attempts to identify their implicit conditions of applicability, and synthesizes seemingly conflicting experiences into TIPs with clearer conditions.
Induce: compares success and failure traces, or extracts effective experience from success traces, and updates the two-track experience base by adding, revising, confirming, or removing entries.
Blame→Retire→Contradict/Synthesize→Induce runs cyclically across multiple task batches, continually testing experience against new evidence: it updates both the Agent's understanding of the environment and its execution methods under the relevant conditions.
④ Final output and evaluation: testing the practical benefits of evolving experience
After multiple rounds of incremental evolution, the framework outputs the final FACT and TIP experience bases and evaluates their impact on accuracy or task performance in test tasks.
At the core of this mechanism is the separate accumulation and traceability of environmental facts and execution methods, with both tested and corrected through continued execution. Each time an Agent completes a batch of tasks, it can turn new evidence from its interactions into experience it can use in the next round, gradually forming a loop from interaction with the environment to knowledge updates and improved action.
“But one key question remains: is evolving environmental facts and action techniques separately really more effective than traditional single-track experience management?”
TTSE's methodology and experimental results are examined in the paper A Two-Track Online Self-Evolution Framework for LLM Agents.
To test whether the two-track experience mechanism is genuinely effective, the research team evaluated it on five benchmarks—ALFWorld, ScienceWorld, GDPevo, SOPBench, and PinchBench—with three independent repetitions of each experiment to examine both its practical benefits and operational stability.
ALFWorld: a success rate as high as 98.51%. On GLM-5, TTSE achieved a 98.51% success rate; on DeepSeek-V3.2, it achieved 97.8%, an improvement of 14.2 percentage points over the previous best method, GraSP, at 83.6%, while also reducing the number of task execution steps.
ScienceWorld: significant improvement in task performance. MACRO Reward rose from 0.7204 to 0.8534, and Native Success rose from 53.08% to 73.46%, an increase of 20.38 percentage points.
GDPevo: two tracks outperform one. The full FACT+TIP approach achieved an average score of 0.2616, about 2.8 times the zero-shot baseline of 0.0947. Neither TIP nor FACT alone matched the performance of the full two-track approach, validating the value of managing environmental facts and action techniques together.
SOPBench: further improvement when integrated with Bayesian skill management. The overall success rate rose from the baseline of 79.15% to 91.59% with the single-track approach, then to 94.58% with the integrated two-track approach, an additional increase of 2.99 percentage points over the single-track approach.
PinchBench: consistent gains on real-world tasks. TTSE was integrated non-invasively with OpenClaw and run independently three times on 147 real end-to-end tasks. The two-track approach increased the average score from 0.7439 to 0.7822, a gain of 3.83 percentage points; gains were observed in all three runs, and the range across runs fell from 3.33 percentage points to 1.61 percentage points.
These results span embodied interaction, scientific reasoning, enterprise operations, industrial processes, and real-world end-to-end tasks, indicating that TTSE's value is not limited to a particular model or single setting.
The GDPevo ablation experiment provides more direct evidence: the full two-track approach outperforms single-track approaches that record only facts or only techniques.
The reason is that understanding the environment and choosing an action strategy serve different purposes. With FACT alone, an Agent may know the environment's constraints but lack a suitable way to act; with TIP alone, it may have operational techniques but be unable to identify accurately the conditions under which those techniques work. Only by combining the two can it answer both “what is the environment like?” and “what should be done under the current conditions?”
Thus, the two-track design does more than simply split the experience base in two: it lets different types of knowledge be derived, validated, and reused according to their meanings and conditions of applicability.
Across models, tasks, and real-world evaluations, the experimental results validate the effectiveness of TTSE's two-track experience mechanism and further demonstrate the potential of RSI for learning from experience: without changing model parameters, an Agent can gradually improve its performance on subsequent tasks by continually accumulating, validating, and updating task experience.
This also provides empirical support for bringing TTSE from a research framework into practical office Agent products.
TTSE has not only been validated in paper experiments; it has also been formally integrated into openJiuwen's WorkSwarm office Agent swarm.
Want to try TTSE yourself? Simply install and launch WorkSwarm version 0.2.7 or later and enable the TTSE self-evolution switch in Settings to experience the efficiency gains from accumulating and reusing experience.
Next, let's look at two real office tasks to see how TTSE makes an Agent faster and more efficient in practice.
In everyday office work, Agents are already widely used for tasks such as data cleaning and analysis, document editing and organization, report generation, information extraction, and format conversion. Yet when faced with different file structures, data formats, and business requirements, Agents often need to explore the environment repeatedly, try tool calls, and work out the task execution path anew.
To illustrate TTSE's practical effects, the team selected two typical office scenarios—Excel data cleaning and document and spreadsheet organization—and compared the Agent's performance on its first run with its performance after experience was injected.
The results show efficiency gains of varying degrees in both scenarios:
These gains did not come from changes in the model’s capabilities. Rather, in the comparison tasks shown, the Agent could use experience from past executions to identify environmental constraints more quickly and choose appropriate ways to handle them.
So what experience has TTSE accumulated, and how does it help the Agent reduce trial and error?
Case 1: Excel data cleaning—reducing trial and error in data identification and processing
The challenge in this type of task is not just cleaning the data. It also involves locating the input file, identifying hidden data format issues, and determining deduplication rules that meet business requirements.
On the first run, the Agent had no reusable task experience. It had to explore where the input file was located, determine the data format, and work out rules for handling duplicate records. The process involved 57 tool calls, 3 of which failed, and took a total of 6 minutes and 22 seconds.
After the task ended, TTSE extracted reusable task experience from the actual execution trace. This experience was no longer merely a record of past operations; it could directly inform decisions on the next run:
Locate input materials quickly. When the user has not specified a file location, check workspace/input first and look directly for upstream files. This avoids repeated searches through unrelated task directories and unnecessary follow-up questions to the user.
Identify data issues in depth. For dates, amounts, and text in Excel that appear normal, use openpyxl to inspect the raw stored values directly. This reveals hidden issues such as dates stored as text, currency symbols, and trailing spaces, reducing omissions caused by inappropriate reading methods.
Reuse validated deduplication rules. Group records by business fields, retain the record with the earliest date, and record how many were deleted. There is no need to work out the deduplication criteria from scratch, reducing repeated judgments and trial and error.
When a similar task was run again, this experience directly informed tool selection and processing decisions. Tool calls fell to 20, failed calls fell to 1, and the total time dropped to 1 minute and 48 seconds.
Case 2: Document and spreadsheet organization—reducing trial and error in file exploration and formatting
Unlike Excel data cleaning, the main challenges in this type of task are unknown file locations, unclear CSV encoding and delimiters, and complex Word template structures. On the first run, the Agent had to explore the files step by step, identify their formats, and determine a suitable way to handle the template.
In the document and spreadsheet organization task, the Agent’s first run took 1 minute and 49 seconds, involved 27 tool calls, and had 2 failed tool calls. After the task was completed, TTSE distilled a set of environmental facts and operational techniques from the actual execution trace.
On the next run, the Agent no longer had to explore the file directories from scratch or test ways to read the CSV, nor did it have to wait until after generating the document to discover template replacement problems. It could directly reuse validated experience about the environment and choose more suitable tools and processing steps in advance.
The second run took 1 minute and 26 seconds, with tool calls falling to 18 and failed calls to 1. Overall time decreased by about 21.1%, tool calls by 33.3%, and failed calls by 50%.
TTSE goes beyond simply remembering which operations were performed in the past: it lets the Agent accumulate both knowledge of the environment and methods that work under specific conditions.
The key to turning accumulated experience into improved capability is reusing it correctly.
The two office task examples show TTSE’s value in practical tasks: rather than exploring from scratch each time, the Agent can use experience validated in previous tasks to guide later runs.
This is the kind of continuous improvement RSI aims to achieve: each task run produces a deliverable and offers a chance to accumulate useful experience for the next run. Through its two-track FACT and TIP mechanism, TTSE makes that accumulated experience more verifiable, updatable, and reusable.
But are these benefits limited to specific tasks, or can they also be seen across a wider range of office tasks?
To further assess TTSE’s performance in broader office workflows, the team tested the effect of integrating TTSE into WorkSwarm on WorkBuddy Office Bench, an open-source industry benchmark.
It improved task performance while also effectively reducing token consumption and tool-call overhead: the evaluation results showed that, compared with the version without TTSE, task performance improved by 3.81%, input token consumption fell by 1.70%, output token consumption fell by 30.09%, and the average number of tool calls decreased by 2.
We are at an inflection point where Agents are moving from “one-off tools” to “partners that continually evolve.”
The core value of RSI is not just enabling the Agent to complete more tasks, but enabling it to gain feedback from every real execution, accumulate experience, and turn that experience into an ability to improve continuously.
From optimizing both Harness and Artifacts to TTSE’s two-track evolution of environmental facts and execution techniques, openJiuwen is exploring different paths toward continual self-improvement for agents: improving how they work and what they deliver, while continually correcting their understanding of the environment and accumulating reliable experience.
TTSE provides a verifiable, traceable mechanism for the self-evolution of experience. This enables the Agent not only to learn “how to do it,” but also to understand “under what environmental conditions this approach works,” and to keep improving through repeated task execution.
Experience should not simply fade away. Teaching the Agent both “how to do it” and “what the world is like” is its first lesson on the path to long-term autonomy.
Experience should not simply fade away. Teaching the Agent both “how to do it” and “what the world is like” is its first lesson on the path to long-term autonomy.
Academic paper: TTSE: A Two-Track Online Self-Evolution Framework for LLM Agents. Full paper:
https://arxiv.org/pdf/2609.24289
TTSE’s two-track evolution Rail is now available in WorkSwarm version 0.2.8. Interested users can visit the official website to download it with one click, enable the self-evolution of experience, and try it now:
Try WorkSwarm now:
https://openjiuwen.com/workswarm GitHub code repository:
https://github.com/openJiuwen-ai/ AtomGit code repository:
https://atomgit.com/openJiuwen/
If you are also working on Agent/LLM self-evolution or memory and skill systems, you are welcome to discuss this in the comments:
What do you think an Agent should most remember: a technique, or a rule about its environment?
*This article is republished by QbitAI with authorization. The views expressed are solely those of the original author.