SUN: Persistent Programs For Language-Grounded Control-to-Learning-to-Real Policies

SUN: Persistent Programs For Language-Grounded Control-to-Learning-to-Real Policies Weiqi Wang1,∗ , Zhi Li1,∗ , Yudong Lei2 , David Martinez1 , Xiaofeng Gao3,‡ , Yuxin Jiang1 , Chenfanfu Jiang1 , Yingnian Wu1,† , Demetri Terzopoulos1,† , Ran Gong4,†

arXiv:2608.31167v1 [cs.RO] 31 Aug 2026

1 University of California, Los Angeles

2 University of California, Santa Barbara

Abstract— Bridging model-based control and learned policies in long-horizon manipulation has harbored a silent disagreement: control executes specified objectives, learning amortizes that behavior into a reactive policy, yet existing protocols discard task semantics, leaving rewards hand-crafted and behavior drifting from what control verified. We introduce Semantically UNified (SUN) Programs, typed executables where geometric and contact relations are defined once and compiled into aligned Model Predictive Control (MPC) costs, satisfaction predicates, RL rewards, transition guards, and diagnostics. Our system, Kuafu, driven by large vision language systems, automatically synthesizes SUN Programs from language and scene semantics, screens feasibility via MPC, and retains semantics while training stage-conditioned policies. Across nine tasks, Kuafu achieves 82.03% macro-success, outperforming sparse-reward (35.67%) and Stage-BC (24.75%) baselines. At 8192-way scale, it generates 10.57× the successful trajectory time per hour of human teleoperation. With 500 trajectories per task, Kuafu data trains DP3 policies to 46.0% simulation success (vs. 22.4% for alternatives) and 34.7% on physical Franka and Kinova robots. These results establish that simulation-screened task semantics can effectively amortize control into robust policies, without demonstrations or manual dense rewards, unifying symbolic planning and data-driven execution.

Index Terms—Integrated Planning and Learning, Manipulation Planning, Optimization and Optimal Control, Imitation Learning, Datasets for Robot Learning I. I NTRODUCTION Reinforcement learning (RL) is an attractive method for multi-stage manipulation because it can learn highdimensional, contact-rich behaviors directly from interaction without requiring an accurate model of the underlying dynamics [1]. However, RL requires dense, structured rewards to expose intermediate progress, yet manually crafting these signals is laborious and brittle. Minor adjustments to reward terms or transition thresholds often yield drastically different behaviors, demanding extensive trial-and-error tuning. While recent methods automate reward generation [2], they suffer from a critical latency: the impact of a candidate reward can only be evaluated after full policy training and rollout. Consequently, each iterative revision incurs the full cost of a training cycle, rendering the search over reward structures inefficient. Model-based planning and control explicitly encode task structures, defining geometric objectives and transition logic ∗ Equal contribution. † Corresponding author. ‡ This work does not relate to the author’s position at Amazon.

3 Amazon

4 Robotics and AI Institute

Fig. 1. Kuafu preserves the SUN Program across language grounding, MPC feasibility screening, policy learning, and monitored data generation. Downstream visual-policy training and zero-shot deployment rely exclusively on the generated observations and actions, unaware of the original program.

to enable testing and revision under closed-loop dynamics [3, 4]. However, this approach is highly sensitive to model fidelity; successful execution relies on accurate state reconstruction, precise interaction models, and strict adherence to task definitions. In practice, even minor discrepancies, such as model mismatch, observation noise, or specification errors, can cause controller failure. Furthermore, constructing these robust formulations demands substantial manual engineering for every new task [5]. These complementary properties have motivated efforts to bridge model-based control and learning. Most prominently, model-based controllers are used to generate successful trajectories for policy training [5, 6]. While this transfers physically grounded behavior, it discards the objectives, constraints, stage transitions, or satisfaction boundaries that defined why those actions were correct. Although recent methods expose richer task structure to control and learning [7, 8], the representations are often recast as distinct learning signals, forcing stage or completion logic to be reimplemented. As a result, the rewards guiding the policy may drift from the specification validated by control, allowing inconsistencies to surface only after deployment. We argue that a persistent task program that survives closed-loop screening in simulated physics can significantly enhance multi-stage robot learning. To this end, we introduce the Semantically UNified (SUN) Program, a languagegrounded, typed executable artifact. Given a language in-

Fig. 2. Kuafu grounds language instruction into a typed SUN Program over scene frames, screens feasibility with simulated physics via MPC, and retains it across policy learning and data production. Validated MPC rollouts and program annotations supervise behavioral cloning, while the retained program provides transitions, residual rewards, calibration, and diagnostics to the controller, which then generates sensor-action data for visual policies.

struction, an LLM agent constructs each SUN Program from system-provided scene frames and typed operators, binding their arguments and organizing the geometric relations into ordered stages. Each declared relation is coupled with a satisfaction boundary, and compiled into transition guards, diagnostic view, and aligned objectives for both Model Predictive Control (MPC) and Residual RL. Its end-to-end implementation, Kuafu 1 (Figure 1), retains the accepted artifact alongside a stage-conditioned whole-task policy. Optimization thus serves not merely as a trajectory generator, but as an efficient screener of task knowledge, rejecting faulty objectives and transitions before they incur repeated policy-training cycles. Kuafu then uses the screened SUN supervision to learn contact-rich execution through Residual RL, retaining control’s ability to validate task structure while replacing model-dependent online optimization with a deployable policy. Across nine manipulation tasks, we demonstrate that the value of the persistent program scales with task horizon and cannot be reduced to mere behavior transfer. Compared with reward-learning and online-planning baselines, Kuafu achieves higher success rates with fewer language-model interactions and at a fraction of the computational cost. Furthermore, the persistent program guides the efficient collection of simulation data for sim-to-real learning, yielding significantly higher downstream policy success than comparable data-generation methods. Finally, we validate these results through sim-to-real transfer on physical Franka and Kinova robots, confirming the robustness of our approach in real-world settings. Our contributions are threefold: 1 Kuafu is a figure in Chinese mythology who chased the sun; our system likewise pursues a persistent SUN Program across control, learning, and deployment.

We introduce Semantically UNified (SUN) Programs, a typed executable artifact that preserves task semantics across control, learning, and deployment, eliminating the semantic drift common in current handoffs. • We present Kuafu, a system that synthesizes SUN Programs from natural language, screens them via MPC, and trains stage-conditioned policies without manual reward engineering or seed demonstrations. • We demonstrate that Kuafu achieves higher success rates with fewer language-model interactions and at a fraction of the computational cost of competing methods, while enabling robust zero-shot sim-to-real transfer on physical robots. •

II. R ELATED W ORK Executable Task Representations. Long-horizon manipulation requires representations connecting discrete task progression with continuous motion. Symbolic planning and program generation couple action sequences to geometric feasibility or scene grounding [3, 9]. Language-grounded optimization converts task descriptions into composable 3D value maps or reward parameters evaluated by MPC, exposing spatial objectives to closed-loop control [10, 11]. Formal approaches encode temporal progress as finite-state rewards or signal-temporal-logic constraints for learning and predictive control [12, 13]. While these representations constrain search and operationalize task progress, they generally remain confined to a single planning, control, or learning stage, breaking semantic continuity across the pipeline. Consequently, a state accepted by one component may still produce infeasible refinements, premature handoffs, or misleading learning signals that compound over longhorizon execution. Control-to-Policy Transfer. Task structure enters policy learning through generated objectives or controller-derived

supervision. Language models reduce manual design by generating executable rewards and, for sim-to-real transfer, domain-randomization configurations [2, 14, 15]. Since iterated objectives can overfit observed behavior or encode invalid task specifications, a high return therefor may not translate into strong performance on the true task metric [16]. Complementarily, optimized trajectories provide supervised policy targets, while residual learning preserves nominal control and learns corrective actions [17, 18]. Iterative rewards generation are flexible, but each revision costs a full reinforcement-learning trial and another round of language model query. We instead restrict language models to executable geometric relations that control method can screen under close loop physics, rejecting invalid specifications at a fraction of the interaction cost. Autonomous Robot Data Generation. Data generators scale robot experience by retargeting object-relative demonstration segments across scene configurations and decomposing behavior into reusable skills connected by planned motions [19, 20]. These convert sparse demonstrations into diverse imitation data, yet task semantics remain bound to the originals, lacking an independent, grounded monitor to validate learned-policy rollouts. The closest precedents span three complementary interfaces. ReKep [8] expresses multi-stage tasks as sequences of grounded keypoint cost functions optimized in closed loop, while OmniManip [21] maps object-centric interaction primitives to 3D spatial constraints with high-level VLM checking and low-level pose tracking. LATO [6] adapts an MPC teacher to the state distribution induced by the learner, whereas MPC-Net [5] trains a policy with a Hamiltonian loss encoding the original control objective and constraints. RoboGen [22] and GenSim2 [23] automate simulation task and scene construction together with skill or demonstration generation through foundation models, optimization, planning, and RL. Kuafu unifies these interfaces by enforcing a persistent, MPC-screened representation where each relation’s identity, grounding, and satisfaction boundary remain fixed from controller validation through policy acquisition, learned execution, and monitored data production. III. M ETHODS We consider a long-horizon manipulation task described by a user-provided natural-language instruction I. Kuafu uses an LLM to construct a SUN Program Γ from registered scene frames F and the typed operator library L. After closed-loop screening in simulated physics, the accepted program provides compiled supervision for learning a stageconditioned whole-task controller π. At time t, π maps the current observation ot and active stage zt to a robot action ut , such that ut = π(ot , zt ). Closed-loop execution of π completes the task stages and generates sensor–action trajectories for downstream visual-policy learning. We assume access to registered scene semantics F , containing the identities and spatial and geometric information of the objects, and a typed operator library L that defines reusable relations over this information. Kuafu obtains π by constructing a SUN

Program Γ from (I, F, L), screening its execution through MPC, and using the successful rollouts to train the StageBC policy πθ . The retained program then supplies stage context and program-derived learning signals to train the residual policy πψ ; the bounded composition of πθ and πψ defines the whole-task policy π. We define a lineage as the coherent chain linking an accepted SUN Program, its MPC-generated trajectories, the Stage-BC prior, the bounded residual controller, and the final evaluated policy. Figure 2 outlines the life cycle of a SUN Program within Kuafu. A. Semantically UNified (SUN) Programs Central to Kuafu is the SUN Program Γ, a typed, executable representation of the task specified by I. Kuafu uses an LLM agent to construct Γ by selecting operators from the system-provided library L, binding them to registered frames in F, and organizing the grounded relations into stages according to I. The LLM therefore composes existing typed components rather than inventing new frames, relations, or implementations. Each relation is defined once and compiled into aligned interfaces for MPC, stage monitoring, and reinforcement learning. The resulting program is retained across screening, policy learning, and data generation. Consider the instruction “open the drawer.” The scene registry F provides the location of the drawer’s base, handle and individual drawer, as well as the orientation and operation direction in terms of axis (handle bar along xaxis, +y for pushing in). L provides typed relations such as ”align axis”, ”direction move”, ”fix orientations”. We use the drawer-opening portion of this task as a running example below. Semantic formation via LLM. For long-horizon tasks, we define the augmented state as x̃t = (xt , zt , ξzt ), where zt is the active stage and ξk records physical snapshots (e.g., frame poses, gripper state) captured upon entry to stage k. The robot state, observation, and action are denoted by xt , ot , and ut , respectively. Each operator schema in the library L declares typed arguments, a canonical residual function, units, stage-entry snapshot behavior, a satisfaction tolerance, and a default reward weight. Kuafu forms the SUN Program by selecting schemas from L and binding their arguments to registered frames in F and task-specific scalar parameters: m

Γ = {Γk }K k=1 ,

k Γk = {(ek,j (x, ξk ), ϵk,j , wk,j )}j=1 . (1) Here, K is the number of stages and mk is the number of grounded relations in stage k. Each term contains a grounded physical relation represented by ek,j , its satisfaction tolerance ϵk,j , and its relative learning weight wk,j . For each stage, the compiler maps every grounded relation to three executable interfaces: n omk rl Compile(Γk ) = ϕmpc . (2) k,j , Pk,j , ck,j

j=1

The MPC term ϕmpc k,j contributes to the stage objective, the predicate Pk,j is evaluated by the stage monitor, and the learning term crl k,j contributes to the reinforcement-learning signal. Their exact numerical forms are operator-specific, but

all three retain the same relation identity, grounding, and stage membership. For example, to pull the drawer open, the relation ”directional move” can measure the drawer displacement relative to the pose stored in ξk when the stage begins. Its compiled MPC term penalizes the remaining displacement, Pk,j tests the same residual against ϵk,j , and crl k,j supplies the corresponding learning cost. An additional ”fix orientation” will measure the tool’s angular displacement relative to the snapshot, keeping the grip’s direction steady. This typing reduces synthesis complexity to verified operator selection and typed argument binding. We refer to this correspondence across control, monitoring, and learning as Semantic Unification. Stage progression. At runtime, Γ also defines the stage monitor. Each Γk contains the relations evaluated during stage k, and all of their predicates participate in stage completion. After applying ut , the monitor evaluates the post-action augmented state x̂t+1 = (xt+1 , zt , ξzt ) before updating the active stage. We use Dk to denote that every predicate Pk,j in stage k is satisfied. The program advances from zt to zt + 1 when Dzt is true; otherwise the current stage remains active. When a new stage begins, its entryscoped snapshot is recorded and its best-progress score is reset. Upon entering a new stage, the program initializes the entry-scoped fields in ξzt+1 from xt+1 . Stage transitions are irreversible, and final-stage completion terminates the rollout. This same stage structure persists through control and learning, zt conditions the Stage-BC and residual controllers, while the stored snapshot ξzt preserve the references required by stage-relative relations. For drawer opening, the monitor remains in the draweropening stage until all associated predicates, such as ”direction move” and ”fix orientation”, are satisfied. Transitioning to the subsequent stage records a new entry snapshot, including the handle and the grippers position and orientation. B. Program Validation and Acceptance Independent task success. Because the same stage monitor is retained during learning and data generation, its completion logic must be checked against an independent criterion. For each task, we introduce an external evaluator gext , manually authored independently of SUN Program formation and frozen prior to synthesis. Crucially, gext is never exposed to the LLM, VLM, SUN Program, or the learned policy. Its binary outcome provides the sparse terminal signal used for residual learning, acceptance, calibration, and final evaluation. In parallel, the SUN Program independently declares completion through its internal predicate DK . A false completion occurs when the SUN monitor declares the final stage complete while gext reports that the required physical outcome has not been achieved. This separation supplies the independent acceptance criterion used during rollouts. For the same drawer opening task, The independent evaluator gext only declares success when the drawer joint reaches 12 cm displacement. If SUN monitor declares its

final geometric predicates satisfied without gext , the rollout will be treated as a false completion. MPC screening. Following initial formulation, task-space nonlinear MPC is the first controller to consume the compiled interfaces of Γ, and screens each SUN Program in closedloop simulation (Step 1 in Figure 2). For stage Pmk,k thempccompiled MPC terms form the stage cost ℓk (x̃) = j=1 ϕk,j (x̃). At state xt , we solve via direct multiple shooting min

x1:H , u0:H−1

s.t.

H−1 X

ℓzt (x̃h )

h=0

x0 = xt ,

(3)

xh+1 = f (xh , uh ).

Here, H is the planning horizon, f is the prediction model. Within the prediction horizon, x̃h = (xh , zt , ξzt ), so the active stage and its entry snapshot remain fixed while the problem is solved. Only the first optimized action is executed before re-observation and replanning. After observing the resulting state, Kuafu evaluates the current-stage predicates and gext before updating the stage index; in the final stage, the predicates determine DK . The MPC problem is then resolved from the updated runtime state. Acceptance and repair. Each candidate SUN Program is screened under 10 task-sampled initial conditions in closedloop simulation. The program is accepted when at least one rollout achieves the required physical outcome and no rollout produces a false completion. Failures may trigger up to five whole-program repair attempts based on unsatisfied terms and their measured violations. After acceptance, Kuafu fixes the grounded relations, satisfaction tolerances, predicates, term definitions, and stage order of Γ. The accepted program and its successful MPC trajectories form the handoff to policy learning: the trajectories supervise the Stage-BC policy πθ , while Γ is retained as the task-semantic core. C. SUN-Guided Policy Amortization The accepted SUN Program and successful MPC trajectories provide complementary supervision for policy learning. Stage-conditioned behavioral cloning. For each task, we collect 1,000 physically successful MPC trajectories and use them to train a deterministic Stage-BC policy πθ (Step 2 of Figure 2). The policy maps the current observation and active stage to a 10-step action chunk, and we optimize it using normalized mean-squared error. During learned execution, a reactive policy πθ replaces the MPC optimization loop at each control step, thereby amortizing the planning computation [24]. The observation ot contains the end-effector, gripper, and object/frame poses. Additionally, the policy is conditioned on the active stage zt . The policy predicts 10step chunks of absolute actions. Training chunks are strictly stage-bound: padding repeats the final action, and only the first predicted action is executed by the residual controller before re-evaluation. Consequently, in the drawer opening example, while securing the grip on the drawer handle, stageBC chunks cannot cross into the drawer opening manipulation. (1) We use πθ (ot , zt ) to denote the first action in the predicted chunk. Only this action enters the combined policy

before the SUN monitor reevaluates the active stage. The retained SUN Program continues to supply zt and carries the task semantics into residual learning. Bounded residual reinforcement learning. We freeze πθ and use it as the behavior anchor for Step 3, training a bounded additive residual policy πψ via PPO with generalized advantage estimation. Following RRL [18], the combined whole-task policy is   (1) π(ot , zt ) = clip πθ (ot , zt ) + αρ πψ (ot , zt ), umin , umax , ut = π(ot , zt ),

∥πψ (ot , zt )∥∞ ≤ ρmax .

(4) Here, θ and ψ are the parameters of the Stage-BC and residual policies, respectively. The scalar αρ controls the residual contribution, ρmax bounds it coordinate-wise, and umin and umax are the final environment action limits. Program-derived residual reward. For stage k, the same learning costs and weights attached to the terms in Γk define the normalized stage score Pmk rl j=1 wk,j ck,j (x̃) Pmk , (5) Sk (x̃) = 1 − j=1 wk,j Pmk where wk,j ≥ 0, j=1 wk,j > 0, and a larger score indicates greater progress within the stage. Kuafu rewards monotonic progress:   rt = λprog Szt (x̂t+1 ) − Szbest + λstage 1[zt+1 > zt ] t ,t−1 + (6) + λtrue 1[TSt ] − λfalse 1[FCt ] − λact ∥ut ∥22 . The coefficients λprog , λstage , λtrue , λfalse , and λact weight the corresponding reward terms. S best initializes best at zero at episode/stage entry, updates as Sk,t = best max(Sk,t−1 , Sk (x̂t+1 )), and resets after a stage transition. TSt and FCt denote the physical success and falsecompletion events, respectively. An episode terminates at the first such event or at timeout. The remaining terms provide program-derived supervision while the frozen prior and residual bound constrain adaptation. Learning parameters calibration. Inspired by iterative reward refinement [2, 15], Kuafu actively monitors learning trends and constraint satisfaction via SUN Program performance during Step 3 (Figure 2). A liberal residual policy may perform well on short tasks but risks deviation on prolonged tasks demanding precision. These statistics are exposed to an LLM through a constrained schema. We introduce an LLM agent to tune the relative reward weights declared in the accepted program, the residual strength αρ , and the residual bound ρmax . Crucially, predicate tolerances, logic, grounded relations, stage order, and term definitions remain fixed at their MPC-accepted values. These updates calibrate the learning signal and the permitted deviation from the StageBC policy while preserving the task semantics validated by MPC. For example, while pulling the drawer opening, calibration may realize the ”fix orientation” term is easier to reach, and lower its weight relative to ”direction move”.

Fig. 3.

Nine-task evaluation suite in Isaac Lab.

D. Data Production The same semantically fixed program remains active after residual learning. In Step 4, Kuafu supplies zt to the state-based BC and residual controllers, providing stage progression, completion monitoring, and false-completion detection during data generation. For the downstream visualpolicy study, trajectories are retained using the common physical-success criterion gext , ensuring consistent outcome evaluation across all data generators. At the final handoff, the retained trajectories supervise a visual policy using only their recorded observations and actions. The visual policy receives neither zt nor any other output from the SUN monitor during training or deployment. Thus, the accepted SUN Program preserves the correspondence among control, monitoring, and learning throughout data production, while the resulting visual policy executes without the program or privileged state information. IV. E XPERIMENTS We address four central questions in our experiments: (1) Formulation Reliability: How reliably can SUN Programs be synthesized from language and scene semantics? (2) Performance Elevation: Does the persistent support of SUN Programs significantly elevate task success compared to non-persistent baselines? (3) Data Efficiency: Can Kuafu outperform existing data-generation methods in terms of trajectory quality and sample efficiency? (4) Sim-to-Real Transfer: Does the dataset produced by Kuafu enable visual policies and robust zero-shot transfer to physical robots? As shown in Figure 3, we designed nine multi-stage tasks to evaluate physical capabilities that recur throughout everyday manipulation. The suite spans constrained translational and rotational motion—including pushing, pulling, twisting, and reorienting—together with precise picking and placing, orientation-sensitive grasping, and high-precision manipulation. These capabilities are instantiated through: E-stop interactions (T0: push, T3: twist), stacking (T1: two cubes, T6: three cubes), drawer manipulation (T2: pull open, T7: open and place object), object reorientation (T4: right a bottle), insertion (T5: marker into cup), and dexterous hanging (T8: hang cup by handle). The tasks isolate different forms of compositional scalability for objects and interactions. T0/T3 vary the complexity of contact-constrained motion on the same mechanism. T6 extends T1 by repeating its precision pick-and-place and alignment sequence. T7 combines T2’s constrained drawer motion with the pick-and-place skill

TABLE I F ORMATION , VALIDATION , AND LANGUAGE INTERACTION COST Metric / method

T0

T1

T2

T3

T4

T5

T6

T7

T8

Mean

A. SUN Program formation and refinement: canonical accepted programs and their formation cost Stages K Constraints Formation iterations Refinement iterations LLM calls/task

3 12 1 0 11.2

8 25 1 0 19.8

6 19 1 0 15.2

6 19 1 0 16.2

7 27 2 1 24.6

9 31 2 1 29.2

15 53 2 0 37.0

14 44 1 0 32.8

10 31 1 2 28.8

8.67 29.00 1.33 0.44 23.87

5/5 0/5 5/5 0

4/5 1/5 5/5 2

0/5 5/5 5/5 5

1/5 4/5 5/5 9

5/5 0/5 5/5 0

2/5 3/5 3/5 11

2/5 3/5 5/5 3

29/45 16/45 43/45 30

144.1 243.0 21.8 38.0

183.7 208.9 18.8 31.4

152.5 248.8 19.9 39.7

163.1 261.0 23.5 49.8

110.50 234.40 16.99 31.60

B. Five repeated formation-and-MPC-validation runs per task Success w/o recovery Recovery entry Final success Recovery cycles

5/5 0/5 5/5 0

5/5 0/5 5/5 0

C. Comparator LLM interaction cost (Kuafu/Eureka: one-time per task; VoxPoser: per rollout) Kuafu tokens/task (k) Eureka tokens/task (k) VoxPoser calls/run VoxPoser tokens/run (k)

37.6 224.3 12.9 25.1

84.0 246.2 13.9 24.6

57.1 241.1 13.0 24.9

62.6 227.2 13.0 23.6

exercised in T1.T 4/T5 both require object reorientation, but T5 adds an inter-object geometric constraint by aligning the reoriented marker with the cup opening. Lastly, T5 and T8 compare high-precision manipulation across distinct object morphologies and contact geometries. We report overall performance as task-macro externaloutcome success: the unweighted mean of the gext success rate across all tasks, ensuring each task contributes equally regardless of rollout count. All LLM calls (Kuafu, Eureka, VoxPoser) use GPT-5.5, and simulations run in Isaac Lab 5.1. State-controller evaluations use 8192 parallel environments with rollouts capped at 180 seconds. All methods share the same task-specific initial-condition sampler. A. Formulation Reliability Rows A and B of Table I characterize accepted programs and the formation-validation outcomes. Programs span 3–15 stages and 12–53 constraints, with each canonical program obtained within one or two formation rounds. A formation round regenerates the program, whereas recovery repairs individual terms flagged by failed MPC screening. Of 45 independent runs, 29 pass initially; diagnostic repair accepted 14 of the remaining 16, raising the yield from 64.4% to 95.6%. Recovery was most critical for T4 and T5, improving acceptance from 1/10 to 10/10. T7 was the only task with unrecovered runs (3/5), accounting for 11 of the 30 total recovery cycles. Since the larger 15-stage T6 program succeeded in all five runs without recovery, raw program size alone does not explain formation difficulty. Instead, the T7 result suggests that combining heterogeneous manipulation phases poses a greater challenge. Within the tested scope, these results establish diagnostic-guided repair as an effective mechanism for recovering initially invalid programs. Rows C report the language interaction. Kuafu uses 110.5k tokens per task, compared with 234.4k for Eureka’s onetime reward search. VoxPoser instead uses 16.99 calls and 31.6k tokens per rollout. At the reported mean rates, its cumulative interaction exceeds Kuafu’s after just two executions (calls) and four rollouts (tokens). Thus, forming a persistent program converts language interaction from a recurring execution cost into a task-level cost.

110.1 208.7 16.1 27.2

B. Performance and Ablation Analysis Table II reports structural handoff controls that remove program-derived progress supervision, the behavioral prior, or the residual constraint. For scalar refinement, Kuafu permits at most 5 residual-RL iterations per task. An iteration is deemed collapsed if its final success falls below 50% of its peak. Refinement halts at the first non-collapsed configuration or upon exhausting the 5-iteration budget, freezing the controller for evaluation. The closest matched ablation isolates program-derived supervision. Starting from the same Stage-BC prior and bounded residual learner, the sparse-reward variant achieves 35.67% task-macro success, versus 82.03% for Kuafu’s L42 mean, a 46.36-point gain. Separately, the 3-run robustness aggregate reaches 79.43% success. Because the accepted program, MPC mentor set, and residual training vary jointly across these runs, this result confirms the complete pipeline’s effectiveness beyond a single lineage. False-completion rates remain low posthandoff, at 0.60% across the three lineages. The remaining partial handoffs establish complementarity. Stage-BC alone achieves 24.75%. With the SUN Program reward, residual architecture, and action bound held fixed, removing the accepted Stage-BC prior drops performance to 0% (Pure RL, SUN Program reward). Applying residual RL to a stage-free BC reaches only 18.56%. Similarly, adding a behavioral prior raises TD-MPC2 from 12.63% to 31.95%, though success remains concentrated on T0, T2, and T3. Removing or broadening the residual bound collapses all tasks to 0% success; we have omitted the resulting all-zero row from Table II. Neither accepted behavior nor program supervision alone reproduces the complete handoff. Conversely, baselines that rely on iterative reward search or online planning without persistent task semantics struggle to achieve robust, multi-stage success. Eureka exhibits nonmonotonic instability: its best candidate within 5 search rounds reaches 18.24% success, but the final policy degrades to 12.85% (e.g., Task T0 drops from 45.14% to 0.21%), indicating reward overfitting. VoxPoser achieves only 10.00% success, failing completely on five tasks. These results confirm that Kuafu’s gains stem not from increased

TABLE II C ONTROL - TO - LEARNING HANDOFF AND BASELINES Variant / method

T0

T1

T2

T3

T4

T5

T6

T7

T8

Mean

A. Accepted-program transfer success (%): MPC, Stage-BC, and bounded residual learning (with/without program supervision) MPC Stage-BC Bounded RRL + Sparse Rewards Kuafu (L42 ) Kuafu (L42 , L43 , L44 )

100.00 36.01 58.67 98.48 98.37±0.81

96.15 10.79 87.20 95.78 94.75±0.62

100.00 25.11 49.56 98.01 97.58±1.04

89.29 29.80 58.67 84.62 82.09±0.65

62.50 22.47 58.96 58.80 56.24±1.81

58.14 31.82 0.00 69.70 62.04±5.34

94.34 13.81 0.00 87.25 84.72±1.32

70.92 38.80 7.95 95.21 94.06±1.06

82.50 14.16 0.00 50.43 44.99±1.84

83.76 24.75 35.67 82.03 79.43

0.05 0.03±0.02

3.26 3.21±0.27

0.45 0.48±0.07

0.00 0.00±0.00

0.00 0.00±0.00

0.00 0.00±0.00

1.05 1.07±0.50

0.58 0.60±1.05

0.00 0.00 0.00 17.49 0.00 0.00

0.00 0.00 0.00 3.83 0.00 0.00

0.00 0.00 0.00 0.16 0.00 0.00

0.77 0.00 0.00 16.69 0.00 0.00

9.44 0.07 0.00 18.56 12.63 31.95

0.01 0.00 0.01 0.01 0.00

0.00 0.00 0.00 0.00 10.00

0.00 0.00 0.01 0.00 0.00

0.01 0.00 0.01 0.01 0.00

7.41 10.33 18.24 12.85 10.00

B. False completion rates (%) for Kuafu final controllers Kuafu (L42 ) Kuafu (L42 , L43 , L44 )

0.37 0.59±0.57

0.00 0.01±0.00

C. Kuafu component-ablation success (%): partial handoffs and alternative state-policy learners Stage-free-BC Pure RL, naive reward Pure RL, SUN Program reward Stage-free-BC + RRL TD-MPC2 [25], naive reward TD-MPC2 + prior, naive reward

52.20 0.04 0.00 1.20 79.64 77.58

12.16 0.00 0.00 67.30 27.00 13.00

3.38 0.13 0.00 0.03 0.00 97.47

11.17 0.00 0.00 0.00 5.99 99.27

5.29 0.43 0.00 60.31 1.07 0.20

D. Baseline performance success (%): iterative reward search (Eureka) and online planning (VoxPoser) Eureka: Train@1 Eureka: Train@5 Eureka: Best Train ≤ 5 Eureka: Final Eval. VoxPoser: Direct

45.14 0.21 60.53 25.85 50.00

0.02 0.01 0.02 0.01 10.00

20.25 78.96 78.96 71.75 0.00

0.19 8.53 18.17 15.16 20.00

1.03 5.29 6.47 2.83 0.00

TABLE III E ND - TO - END CONTROLLER COMPUTE (GPU- HRS ) Kuafu

Sum

TABLE IV K UAFU DATA GENERATION

Eureka

A. Downstream DP3 success (%) (500 successful trajectories per task)

Form.

MPC

BC

RRL

Total

Search

Final

Total

1.55

69.03

3.21

48.86

122.67

412.24

67.36

479.60

LLM search budget, but from the synergistic integration of a verified prior, program-derived supervision, and bounded correction, which eliminate semantic drift. The paired tasks provide a structured test of horizon robustness. Kuafu drops only 8.53 points as stacking grows from 8 stages (T1, 95.78%) to 15 (T6, 87.25%), 2.80 points when drawer opening (T2, 98.01%) is extended to the 14stage open-and-place task (T7, 95.21%), and 13.86 points from the close-E-stop task (T0, 98.48%) to the more difficult open-E-stop task (T3, 84.62%). Across the three pairs, Kuafu retains 91.4% of its shorter-task performance; the matched sparse-reward handoff instead falls from 65.14% to 22.21%. In a boarder view, no non-Kuafu learned handoff or baseline reaches 50% on either T6 or T7, with taskwise maxima of only 13.81% and 38.80%, whereas Kuafu reaches 87.25% and 95.21%. This shows that the retained SUN Program preserves competence as stage sequences lengthen rather than merely improving isolated tasks. Table III details the end-to-end compute costs, encompassing program formation, MPC validation, mentor trajectory collection (1000 per task), Stage-BC, and bounded residual learning. In constrast, the Eureka baseline accounts for 5 rounds of 4-candidate reward-searches plus finalization. Under one-GPU worker-hour accounting, Eureka uses 479.60 GPU-hours, compared with 122.67 for Kuafu (a 3.91× efficiency gain). Notably, while MPC validation and mentor collection dominate Kuafu’s cost, program formation is negligible (1.55 GPU-hours). Thus, Kuafu’s superior performance is achieved at a fraction of the computational cost, ruling out compute budget as a confounding factor.

Task MimicGen SkillMimic

Isaac Lab Isaac Skill Mimic Mimic

Kuafu

T0 T1 T2 T3 T4 T5 T6 T7 T8

94.52 4.95 36.12 33.33 8.27 0.00 0.00 0.00 0.00

97.07 3.91 30.65 62.00 8.05 0.00 0.00 0.00 0.26

90.78 14.21 40.92 36.54 9.35 0.00 0.00 0.00 0.00

72.82 1.04 22.54 15.25 5.47 0.00 0.00 0.00 0.26

95.70 58.70 55.50 85.30 0.78 15.80 12.70 69.00 20.70

Mean

19.69

22.44

21.31

13.04

46.02

B. MPC-relative Q95 yield and successful-trajectory diagnostics Method

Q95 Time/ EEF path/ Action path/ Action ∆2 / (%↑) MPC50 ↓ MPC50 ↓ MPC50 ↓ MPC50 ↓

MimicGen SkillMimic Isaac Lab Mimic Isaac SkillMimic Kuafu (ours)

20.4 6.9 12.1 6.2 57.3

1.11 1.66 1.12 1.61 1.04

1.14 1.15 1.38 1.46 0.67

1.94 3.01 1.96 2.89 0.52

1.27 1.28 1.25 1.21 0.28

Human reference

0.0

3.06

1.59

2.81

1.25

C. Successful trajectory-time throughput (min/active production hr, ↑) Task

Human MimicGen SkillMimic

Task macro vs. Human

23.34 1.00

29.36 1.26

Isaac Lab Isaac Skill Kuafu Mimic Mimic

26.48 1.13

32.19 1.38

24.79 246.79 1.06 10.57

D. Final-controller readiness and successful-demonstration break-even Task Task macro Across-task range

Controller Human demos Break-even Post-ready ready (GPU h) at ready demos collect (min) 13.63 4.84–25.52

494 222–805

513 225–854

29.10 4.35–76.19

C. Data Efficiency During data collection, the learned controller replaces online MPC, while the retained SUN Program continues to supply stage annotations, completion monitoring, diagnos-

TABLE V V ISUAL POLICY PERFORMANCE AND ZERO - SHOT TRANSFER A. Single-task success (%) by visual-policy architecture (same Kuafu data) RGB policy

Mean

Point-cloud policy

Mean

ACT [26] Diffusion Policy smolVLA [28] ManiFlow-RGB [29]

35.40 38.52 26.73 34.09

DP3 FlowPolicy [27] ManiFlow-PC [29] MP1 [30]

46.02 21.01 46.46 12.94

B. Per-task modality maxima (RGB vs. point-cloud) and state-controller success (%) T8

Mean

RGB max 84.53 30.54 73.98 60.70 25.50 10.10 12.10 73.00 23.00 Point-cloud max 98.38 65.53 86.28 85.30 10.05 15.80 12.70 69.00 20.70 State controller 98.48 95.78 98.01 84.62 58.80 69.70 87.25 95.21 50.43

T0

T1

T2

T3

T4

T5

T6

T7

43.72 51.53 82.03

D. Visual Learning and Zero-Shot Transfer

C. Multi-task VLA success (%) (single policy across T0–T8) Policy

T0

T1

T2

T3

T4

T5

T6

T7

T8 Mean

SmolVLA [28] 5.87 0.26 12.94 2.08 0.00 0.00 0.00 0.00 0.00 2.35 WALL-X [31] 22.22 1.56 60.07 11.78 0.00 0.00 0.00 10.51 0.00 11.79 π0.5 [32] 26.61 0.0 81.87 0.00 4.17 4.17 0.00 69.17 0.00 20.67 D. Zero-shot Sim2Real (DP3) transfer to physical robots Franka FR3 Franka Hand

advantage stems from downstream utility, not data volume. Rows B of Table IV characterize trajectory fidelity relative to the accepted MPC reference. Kuafu attains a 57.3% Q95 yield (vs. 20.4% for MimicGen, a 2.81× gain). Among successful rollouts, it produces the shortest paths and smoothest actions (metrics: 1.04 completion time, 0.67 end-effector path, 0.52 action path, 0.28 action second difference). Rows C and Rows D of Table IV separate steady-state data-generation throughput from the one-time generator cost. With the final state controller, Kuafu generates 246.79 successful trajectory minutes per GPU-hour, a 10.57× increase over human teleoperation (23.34 min/hr).

Franka FR3 Robotiq 2F-85

Kinova Gen3 Robotiq 2F-85

Task

T1

T2

T7

T8

T4

T5

Task Macro

Trials Success Rate (%)

20 3 15.0

20 11 55.0

20 7 35.0

30 9 30.0

10 4 40.0

6 2 33.3

34.72

tics, and false-completion filtering. Table IV evaluates downstream utility under a matched successful-data budget, while also reporting MPC-relative trajectory diagnostics, steadystate production throughput, and controller amortization. To assess efficiency and smoothness beyond physical success, we define a rollout as Q95-qualified only if it satisfies gext and its completion time, end-effector path length, action path length, and action second difference are each no greater than the corresponding task-specific 95th percentile among successful MPC trajectories. Q95 yield is the fraction of evaluated rollouts that satisfy all of these conditions. As an MPC-relative metric, Q95 measures mentor-aligned efficiency and smoothness rather than universal quality; thus, the fixed-learner test reported in Rows A of Table IV serves as the primary utility measure. For reporting, metrics (time, paths, action derivatives) are normalized by task-specific successful-MPC medians (MPC50). Human baselines were collected via simulation teleoperation using an AgileX PIKA gripper, with measured poses defining the simulated endeffector targets. The simulated manipulator is still subjected to dynamics constraints and collisions. Rows A of Table IV provide the primary data-utility comparison. Under a matched budget of 500 successful trajectories per task and a fixed DP3 learner, Kuafu data reach 46.02% task-macro success, compared with 22.44% for SkillMimic, the strongest alternative, a 2.05× gap. The seed-42 source leads on seven tasks and alone gives nonzero downstream success on all nine. Since L42 is used exclusively for this comparison (excluding L43 and L44 ), the result isolates data quality from selection bias. On T5–T7, all competing sources achieve 0%, whereas Kuafu reaches 15.80%, 12.70%, and 69.00%, respectively. Thus, Kuafu’s

Table V presents the utility of Kuafu’s trajectories after removing privileged state and stage inputs. Across four RGB and four point-cloud architectures, every single-task policy family achieves nonzero task-macro success. Notably, ManiFlow-PC reaches 46.46% and DP3 reaches 46.02%, outperforming the strongest RGB policy 38.52%. Rows B report per-task maxima within each modality family; these represent optimal architectures for each task, not a single deployable oracle. While maxima nearly match the state controller on T0 (98.38% vs. 99.61%) and exceed it on T3 (85.30% vs. 84.95%), the 15-stage T6 shows the largest gap (12.70% vs. 87.51%). These gaps, along with the multi-task results, reflect losses from observation conditioning, partial observability, and architectural constraints rather than statebased execution alone. For physical deployment, each simulated setup reproduces the calibrated two or three RGBD-cameras configuration of the coresponding workspace, with constrained randomization. We double the dataset to 1,000 trajectories per task and deploy the resulting policies without real-world fine-tuning. DP3 achieves 36 successes in 106 trials across six tasks and three robot-gripper configurations without real-world finetuning. This yields a 34.72% task-macro and nonzero success on every tested task. The pooled rate is 33.96% (95% Wilson interval: 25.6–43.4%, with the smallest task showing 2/6 successes (9.7%–70.0%). Inspection of failed trials identifies precise grasp acquisition as the main remaining challenge: T1 has the lowest success rate at 15%, and grasping is the most frequent observed failure mode on T7. These results establish zero-shot sim-to-real feasibility using dataset produced under persistent SUN programs. V. C ONCLUSIONS We presented Kuafu, a control-to-learning framework that preserves an MPC-screened Semantically UNified (SUN) Program as a persistent, executable task artifact alongside a learned policy. Across three independent lineages, Kuafu achieves an average 79.43% task-macro success rate, demonstrating end-to-end robustness on multi-stage manipulation tasks. In our evaluation, the complete handoff reaches 82.03% success, significantly outperforming the sparsereward baseline (35.67%) and alternative learners; ablation studies further isolate the distinct contributions of the verified

behavioral prior and bounded residual correction. Leveraging the SUN Program’s supervision in data generation, Kuafu trajectories yield 46.02% success in fixed-volume DP3 training, support eight single-task visual architectures and three multi-task VLAs, and enable zero-shot transfer across 106 physical trials. Results establish a key principle: task semantics validated through control remain effective even after behavior is amortized into a learned policy. Scope and limitations. SUN Programs rely on a registered scene interface with informative object frames and axes, as well as a finite, typed-operator library. Consequently, Kuafu cannot handle deformable objects or fluids without defining new operators. Furthermore, our evaluation does not cover held-out instructions, operators, scene semantics, or task compositions, limiting strong claims of openworld generalization. Methodologically, the stage monitor advances monotonically and cannot rollback after a physical regression. Additionally, the Q95 metric is MPC-relative, measuring efficiency against a specific planner rather than absolute trajectory quality. Finally, the validation (6 task, 106 trials) establishes zero-shot feasibility across the tested robot-gripper configurations but does not guarantee broad sim-to-real reliability in unstructured environments. R EFERENCES [1] H. Zhu, A. Gupta, A. Rajeswaran, S. Levine, and V. Kumar, “Dexterous manipulation with deep reinforcement learning: Efficient, general, and low-cost,” in 2019 International Conference on Robotics and Automation (ICRA). IEEE, 2019, pp. 3651–3657. [2] Y. J. Ma, W. Liang, G. Wang et al., “Eureka: Human-level reward design via coding large language models,” in Proc. Int. Conf. Learn. Representations (ICLR), 2024. [3] M. Toussaint, “Logic-geometric programming: An optimization-based approach to combined task and motion planning,” in Proc. Int. Joint Conf. Artif. Intell. (IJCAI), 2015, pp. 1930–1936. [4] D. Q. Mayne, J. B. Rawlings, C. V. Rao, and P. O. M. Scokaert, “Constrained model predictive control: Stability and optimality,” Automatica, vol. 36, no. 6, pp. 789–814, 2000. [5] J. Carius, F. Farshidian, and M. Hutter, “MPC-Net: A first principles guided policy search,” IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 2897– 2904, 2020. [6] G. Kahn, T. Zhang, S. Levine, and P. Abbeel, “PLATO: Policy learning using adaptive trajectory optimization,” in 2017 IEEE International Conference on Robotics and Automation. IEEE, 2017, pp. 3342–3349. [7] T. Mu, M. Liu, and H. Su, “DrS: Learning reusable dense rewards for multi-stage tasks,” in Proc. Int. Conf. Learn. Representations (ICLR), 2024. [8] W. Huang, C. Wang, Y. Li, R. Zhang, and L. Fei-Fei, “ReKep: Spatio-temporal reasoning of relational keypoint constraints for robotic manipulation,” in Proc. Conf. Robot Learn. (CoRL), vol. 270. PMLR, 2025, pp. 4573–4602. [9] I. Singh, V. Blukis, A. Mousavian et al., “ProgPrompt: Generating situated robot task plans using large language models,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2023, pp. 11 523–11 530.

[10] W. Huang, C. Wang, R. Zhang, Y. Li, J. Wu, and L. Fei-Fei, “VoxPoser: Composable 3D value maps for robotic manipulation with language models,” in Proc. Conf. Robot Learn. (CoRL), vol. 229. PMLR, 2023, pp. 540–562. [11] W. Yu, N. Gileadi, C. Fu et al., “Language to rewards for robotic skill synthesis,” in Proc. Conf. Robot Learn. (CoRL), vol. 229. PMLR, 2023, pp. 374–404. [12] R. Toro Icarte, T. Q. Klassen, R. Valenzano, and S. A. McIlraith, “Using reward machines for high-level task specification and decomposition in reinforcement learning,” in Proc. Int. Conf. Mach. Learn. (ICML), vol. 80. PMLR, 2018, pp. 2107–2116. [13] Y. Meng and C. Fan, “Signal temporal logic neural predictive control,” IEEE Robotics and Automation Letters, vol. 8, no. 11, pp. 7719–7726, 2023. [14] T. Xie, S. Zhao, C. H. Wu et al., “Text2Reward: Reward shaping with language models for reinforcement learning,” in Proc. Int. Conf. Learn. Representations (ICLR), 2024, spotlight. [15] Y. J. Ma, W. Liang, H.-J. Wang et al., “DrEureka: Language model guided simto-real transfer,” in Robotics: Science and Systems, 2024. [16] S. Booth, W. B. Knox, J. Shah, S. Niekum, P. Stone, and A. Allievi, “The perils of trial-and-error reward design: Misdesign through overfitting and invalid task specifications,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 5, 2023, pp. 5920–5929. [17] S. Levine and V. Koltun, “Guided policy search,” in Proc. Int. Conf. Mach. Learn. (ICML), vol. 28. PMLR, 2013, pp. 1–9. [18] T. Johannink, S. Bahl, A. Nair et al., “Residual reinforcement learning for robot control,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2019, pp. 6023–6029. [19] A. Mandlekar, S. Nasiriany, B. Wen et al., “MimicGen: A data generation system for scalable robot learning using human demonstrations,” in Proc. Conf. Robot Learn. (CoRL), 2023. [20] C. R. Garrett, A. Mandlekar, B. Wen, and D. Fox, “SkillMimicGen: Automated demonstration generation for efficient skill learning and deployment,” in Proc. Conf. Robot Learn. (CoRL), vol. 270. PMLR, 2025, pp. 2750–2790. [21] M. Pan, J. Zhang, T. Wu, Y. Zhao, W. Gao, and H. Dong, “OmniManip: Towards general robotic manipulation via object-centric interaction primitives as spatial constraints,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2025, pp. 17 359–17 369. [22] Y. Wang, Z. Xian, F. Chen et al., “RoboGen: Towards unleashing infinite data for automated robot learning via generative simulation,” in Proc. Int. Conf. Mach. Learn. (ICML), vol. 235. PMLR, 2024, pp. 51 936–51 983. [23] P. Hua, M. Liu, A. Macaluso et al., “GenSim2: Scaling robot data generation with multi-modal and reasoning llms,” in Proc. Conf. Robot Learn. (CoRL), vol. 270. PMLR, 2025, pp. 5030–5066. [24] A. Byravan et al., “Evaluating model-based planning and planner amortization for continuous control,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2022. [25] N. Hansen, H. Su, and X. Wang, “TD-MPC2: Scalable, robust world models for continuous control,” in Proc. Int. Conf. Learn. Representations (ICLR), 2024. [26] T. Z. Zhao, V. Kumar, S. Levine, and C. Finn, “Learning fine-grained bimanual manipulation with low-cost hardware,” in Proc. Robot.: Sci. Syst. (RSS), Daegu, Republic of Korea, 2023. [27] Q. Zhang, Z. Liu, H. Fan, G. Liu, B. Zeng, and S. Liu, “FlowPolicy: Enabling fast and robust 3D flow-based policy via consistency flow matching for robot manipulation,” in Proc. AAAI, vol. 39, no. 14, 2025, pp. 14 754–14 762. [28] M. Shukor, D. Aubakirova, F. Capuano et al., “SmolVLA: A vision-languageaction model for affordable and efficient robotics,” arXiv:2506.01844, 2025. [29] G. Yan, J. Zhu, Y. Deng et al., “ManiFlow: A general robot manipulation policy via consistency flow training,” in Proc. Conf. Robot Learn. (CoRL), vol. 305. PMLR, 2025, pp. 2268–2293. [30] J. Sheng, Z. Wang, P. Li, and M. Liu, “MP1: MeanFlow tames policy learning in 1-step for robotic manipulation,” in Proc. AAAI, vol. 40, no. 22, 2026. [31] A. Zhai, B. Liu, B. Fang et al., “Igniting VLMs toward the embodied space,” arXiv preprint arXiv:2509.11766, 2025. [32] K. Black, N. Brown, J. Darpinian et al., “π0.5 : A vision-language-action model with open-world generalization,” in Proc. Conf. Robot Learn. (CoRL), vol. 305. PMLR, 2025, pp. 17–40.

Related documents

• SUN: Persistent Programs For Language-Grounded Control-to-Learning-to-Real Policies#819182
• SUN: Persistent Programs For Language-Grounded Control-to-Learning-to-Real Policies#809190
• Preference Goal Tuning: Post-Training as Latent Control for Frozen Policies#189228
• End-to-end QP-based policies: A unified perspective on robust control and robot learning#1115861
• Coding Agent Memory Post-training: Unlocking the Memory Potential of Pre-trained File Operations for Long-Horizon Tasks via Reinforcement Learning#1128693