IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS
1
Time-Lag-Aware Deep Reinforcement Learning for Flexible Job-Shop Scheduling in PPVC Module Factories
arXiv:2607.11725v1 [cs.LG] 13 Jul 2026
Ziheng Zhang, Student Member, IEEE, and Wei Zhang, Senior Member, IEEE
Abstract—Prefabricated prefinished volumetric construction moves most building work into module factories, whose production floor operates as a flexible job shop. One complication is decisive: long post-operation time-lags caused by concrete curing, watertightness ponding tests, and paint drying, during which a module is blocked while its workstation stays free. On benchmark instances grounded in an official national prefabrication guidebook, these lags inflate even the optimal reference makespan by about 67% on average, and ignoring them at decision time, then repairing to feasibility, is worse than every dispatching rule. We adapt a state-of-the-art dual-attention deep reinforcement learning solver through three minimally invasive, individually ablatable extensions: lag-aware dynamics with an admissible reward bound, two anticipatory lag feature channels, and livenessmasked operation- and station-type embeddings. With every extension disabled the implementation reproduces the original solver exactly, so all gains are attributable to the adaptations. We release a public, guidebook-grounded benchmark generator. On held-out instances the learned policy is the strongest solverfree scheduler: it reaches within about 4% of a constraintprogramming reference and beats every dispatching rule and a genetic-algorithm metaheuristic, with its advantage widening under capacity contention, and a single size-mixed policy carries this lead across the trained range of factory sizes. It needs no solver, model, or license in the loop and re-plans within seconds of a disruption; where an exact solver can be deployed, that solver remains the quality ceiling, a boundary we map explicitly. Index Terms—Deep reinforcement learning, flexible job-shop scheduling, prefabricated prefinished volumetric construction (PPVC), reactive rescheduling, time-lags.
I. I NTRODUCTION
P
REFABRICATED Prefinished Volumetric Construction (PPVC) relocates structural work, mechanical, electrical, and plumbing (MEP) installation, and interior finishing from the construction site into a factory that mass-produces fully fitted three-dimensional modules [1], [2]. Singapore’s Building and Construction Authority (BCA), for instance, codifies PPVC production requirements in an official guidebook [3] and requires PPVC on selected government land-sale sites, making module-factory throughput a binding constraint on national housing programs. The factory floor is a flexible job shop: each module visits a sequence of specialized stations Z. Zhang (Research Fellow) and W. Zhang (Associate Professor) are with the Singapore Institute of Technology, Singapore (e-mail: [email protected]; [email protected]). (Corresponding author: Wei Zhang.) Manuscript submitted July 2026. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.
TABLE I M AKESPAN I NFLATION W HEN T IME -L AGS A RE M ODELED (5- MODULE MIXED INSTANCE , DEFAULT FACTORY; FIFO: FIRST- IN - FIRST- OUT ). Scheduler FIFO dispatching CP-SAT optimal
Lag-Free (h)
Lag-Aware (h)
Inflation
113.0 100.0
206.0 198.0
+82% +98%
(mold preparation, casting or welding, MEP fit-out, tiling, painting, quality gates), and each visit may be served by any station of a compatible type. What separates PPVC factories from the classical flexible job-shop scheduling problem (FJSP) is the prevalence of postoperation time-lags. After concrete is poured, the module must cure for 24–48 h; after waterproofing, wet modules undergo a ponding test of 24–48 h; each paint coat dries for 12– 24 h. During a lag the module is blocked, but the station that produced it is immediately free to serve other modules. In BCA-grounded instances the total lag volume amounts to 0.88–1.2× the total processing volume: lags are not a perturbation; they are the dominant temporal structure. Two observations motivate this article. First, lags reshape the problem even under perfect optimization: on one 5module instance, the optimal makespan computed by the CPSAT constraint-programming solver [4] rises from 100.0 h to 198.0 h (+98%) once lags are modeled (Table I; the 100instance test mean is 67.2%, Section VI). Second, the common industrial practice (planning without lags, then repairing the plan afterward) is costly. Right-shifting a lag-blind schedule to feasibility yields 261 h: 27% worse than the makespan the same dispatching rule attains when it instead observes the lags while it builds the schedule. The penalty is the price of deferring lag-awareness, not of the rule itself. Scheduling decisions must therefore be lag-aware at decision time. Lag-awareness must moreover be fast: breakdowns, rework, rush modules, and realized lag durations that deviate from plan all recur and invalidate the standing schedule, so replanning inside a manufacturing-execution loop must complete in seconds to be actionable [5] (Section VI-F). An exact solver returns strong feasible plans in seconds on capacity-rich instances and, given a stability objective or a heuristic warm start, re-plans with little churn; where one can be deployed it is hard to beat. The obstacle is operational, not algorithmic, and not mere licensing: hosting a solver in the control loop carries its runtime, its optimization model, and a re-tuned penalty
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS
on every disruption, a footprint that burdens the low-compute edge controllers and lightweight execution systems common in PPVC plants, even though the underlying OR-Tools solver is itself free. A learned policy is instead a fixed sub-megabyte network run in a few matrix multiplications, with no solver in the loop. Current practice right-shifts the lag-blind plan to feasibility: fast and solver-free, but far from optimal. The loop therefore needs a fast, solver-free re-planner that improves on right-shift, which a learned constructive policy provides. Deep reinforcement learning (DRL) has recently produced fast, high-quality constructive solvers for FJSP [5]–[9], learning dispatching policies over graph or attention-based state encodings that generalize across instance sizes. However, these methods uniformly assume an operation releases its successor the moment it completes, precisely the assumption PPVC violates. Three literatures bracket our problem but none covers it. DRL-for-FJSP learns fast policies yet ignores lags (above). The time-lag scheduling literature models lags exactly but offers only complexity results and exact or metaheuristic methods for small instances [10]–[13], with no learning-based constructive solver. PPVC-specific production scheduling, finally, captures the application but simplifies the shop to a (flexible) flowshop solved by metaheuristics [14], [15], discarding the routing flexibility and lag dynamics that define the real cell. We bridge this gap by adapting DANIEL [8], a state-of-theart dual-attention DRL solver for FJSP, to the PPVC setting. The obstacle is not representational but structural: under postoperation lags a lag-blind solver’s decision process is illposed, because its transition (when a job becomes ready) and its dense, telescoped makespan-bound reward are both wrong, so the policy optimizes a miscalibrated signal however expressive its network is. Correcting these (Adaptation 1) is therefore a prerequisite for any constructive DRL solver in this domain, not a tuning choice. Around that necessary correction our design principle is minimal invasiveness: the network backbone, the proximal-policy-optimization (PPO) training scheme, and the reward principle are left untouched, and every extension is gated by an independent flag. With all flags off, our implementation reproduces the published DANIEL evaluation bit-exactly (mean makespan 408.40 on the public SD2 benchmark [7], 100/100 instances identical). Every reported gain is therefore attributable to a named adaptation rather than to incidental reimplementation differences, and the adaptations port unchanged to other constructive FJSP solvers. To the best of our knowledge, this is the first learned constructive solver explicitly designed for flexible job shops with job-blocking, machine-free post-operation time-lags; concretely, the contributions are: 1) Problem formalization (C1). We formalize PPVC module-factory scheduling as FJSP with finish-start time-lags (FJSP-TL) under job-blocking, machine-free lag semantics (Section III), and quantify empirically that lags dominate both optimal and heuristic makespans (Sections III and VI). 2) Lag-aware DRL adaptations (C2). Three minimally invasive, individually ablatable extensions to DANIEL: lag-aware transition dynamics with an admissible lag-
2
augmented reward bound; anticipatory lag feature channels; and type embeddings injected after per-step feature normalization with a liveness mask that provably preserves the backbone’s deleted-node pooling semantics (Section IV). The lag-aware dynamics are the essential correction (a lag-blind decision process is ill-posed) and carry the makespan gain on capacity-rich shops; the anticipatory features and type embeddings are lightweight enhancements whose value we test rather than assume, isolating it under capacity contention and cross-shell transfer (Section VI). 3) BCA-grounded benchmark (C3). A public instance generator whose routing classes, station taxonomy, and lag ranges are extracted from the official BCA PPVC guidebook, with explicit provenance tags separating guidebook facts from estimated durations (Section V). 4) Systematic evaluation and a stated scope (C4). A four-arm ablation; priority-rule, genetic-algorithm, and cold- and warm-started CP-SAT baselines; statistical testing; and zero-shot transfer across capacities, shell systems, and module counts. The study positions the policy precisely: the strongest solver-free scheduler at the trained scales and under contention, matching a warm-started exact solver in the real-time window, with no quality win claimed over an exact solver where its infrastructure is available, a boundary we map rather than blur (Section VI). II. R ELATED W ORK A. DRL for Flexible Job-Shop Scheduling Learning-to-dispatch methods encode the scheduling state as a (disjunctive) graph and train a policy to select operationmachine pairs step by step. Zhang et al. [6] pioneered this for the job shop, and Song et al. [7] extended it to FJSP with heterogeneous graph neural networks (GNNs). Moving beyond message passing, DANIEL [8] encodes the same state with dual attention over operation and machine nodes, attaining state-of-the-art quality at millisecond inference (the property a reactive factory loop requires, and the reason we build on it). Subsequent end-to-end work scales these solvers (multi-action and large-scale variants [5], [9]) and explores transformerbased state encodings [16]; see [17] for a recent survey. Attention-based and end-to-end formulations also advance dynamic and distributed job-shop scheduling [18], [19]. Closest to our setting, real-time DRL has been applied to partialno-wait flexible job shops [20]; no-wait is the special case in which the inter-operation gap is fixed to zero, whereas PPVC lags are strictly positive, operation-dependent waiting windows during which the station is released. What none of these formulations represents, however, is a post-operation time-lag: every method assumes the successor of a finished operation is immediately ready. We retain the DANIEL backbone unchanged and isolate exactly the minimal modifications required once that assumption no longer holds. B. Scheduling With Time-Lags Minimal and maximal time-lags between operations were introduced in project scheduling [10]; even single-machine
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS
TABLE II M AIN N OTATION . Symbol
Description
Ji Mk Oij n, m ni Mij pijk ℓij sij cij kij Cmax ĉi,j ρi lag ηij rem ηij τij , τk Eop , Emch rt
module (job) i station (machine) k j-th operation of module i number of modules, stations number of operations of module i indices of stations eligible for Oij processing time of Oij on Mk post-operation time-lag of Oij start time of Oij completion time, cij = sij + pij,kij station assigned to Oij factory makespan, maxi ci,ni lag-augmented completion lower bound job ready time, cij + ℓij static lag channel, = ℓij remaining in-flight lag at t operation / station type index op / station type embeddings PPO reward, decrease of Ĉmax
variants are NP-hard [11], and exact or metaheuristic schemes address single-machine, job-shop, and flowshop variants with positive, negative, and maximal lags [12], [13], [21], [22] (maximal lags capture bounded post-operation windows, e.g., painting soon after curing, that also arise in PPVC). Across these the machine routing is fixed, the time-lag/routingflexibility interaction of an FJSP is unaddressed, and the methods learn no reusable policy, requiring a solver in the loop when applied reactively (Section VI-F). To our knowledge no learned constructive solver for lag-constrained flexible job shops has been reported. C. Precast and PPVC Production Scheduling Precast and PPVC production has traditionally been modeled as a flowshop with curing constraints and solved by heuristics, metaheuristics, or constraint programming [1], [2], [14], [15]. Reinforcement learning has only lately entered offsite construction [23]–[27], but these studies fix the routing (flowshop), treat curing as a fixed delay, or optimize a single project offline. We instead model the factory as a flexible job shop with job-blocking lags, matching how modern PPVC plants share stations across concurrently produced module types, with a learned solver fast enough (seconds per instance) for reactive rescheduling. III. P ROBLEM F ORMULATION A. FJSP With Post-Operation Time-Lags (FJSP-TL) An instance comprises n modules (jobs) J = {J1 , . . . , Jn } and m stations (machines) M = {M1 , . . . , Mm }. Module Ji is fabricated by an ordered route Oi,1 ≺ Oi,2 ≺ · · · ≺ Oi,ni . Operation Oij may be processed by any station in a compatible index set Mij ⊆ {1, . . . , m}, with processing time pijk on station Mk , k ∈ Mij , and carries a post-operation time-lag ℓij ≥ 0. Each operation further carries a type index τij (5 classes) and each station a type τk (9 classes); the concrete
3
BCA-derived taxonomy is given in Section V. Table II summarizes the main notation. A schedule assigns each operation a station kij ∈ Mij and a start time sij ≥ 0, with completion cij = sij +pij,kij , subject to: si,j+1 ≥ cij + ℓij ,
∀i, 1 ≤ j < ni ,
(1)
[sij , cij ) ∩ [si′ j ′ , ci′ j ′ ) = ∅ whenever kij = ki′ j ′ , (i, j) ̸= (i′ , j ′ ).
(2)
Constraint (1) is a finish-start precedence with a minimal timelag: the next operation on the module cannot start until the lag (curing, ponding, drying) has elapsed. Constraint (2) encodes the job-blocking, machine-free semantics central to PPVC: the station is occupied only during [sij , cij ); the lag detains the module in a buffer area, not the station. This distinguishes FJSP-TL from (i) inflated processing times p+ℓ, which would block the station and overestimate congestion (Fig. 1), and (ii) sequence-dependent setups, which attach to the machine rather than the job. The objective is the factory makespan Cmax = maxi ci,ni .1 FJSP-TL is strongly NP-hard: setting ℓ ≡ 0 and |Mij | = 1 recovers the classical job shop, whose makespan minimization is already strongly NP-hard [28]. In the disjunctive-graph view, once Oij is assigned to station kij , its outgoing conjunctive (job) arc carries weight pij,kij +ℓij while its outgoing disjunctive (machine) arcs carry weight pij,kij , a single asymmetry with consequences for every makespan estimator used during search or learning. B. A Lag-Augmented Admissible Lower Bound Constructive DRL solvers rely on an estimated completiontime lower bound both for dense rewards and as a state feature. For a partial schedule at decision time t, define recursively for each job i ĉi,j = max ĉi,j−1 + ℓi,j−1 , t + min pijk , (3) k∈Mij
for each unscheduled operation, with the convention ĉi,0 = ℓi,0 = 0 and seeded with realized completions ĉi,j = cij for scheduled operations; the aggregate bound is Ĉmax = maxi ĉi,ni . Proposition 1 (Admissibility). For any partial schedule and any feasible completion of it, Ĉmax ≤ Cmax , and the bound is tight at episode termination, where Ĉmax = Cmax . Proof. Equation (3) is the makespan of a relaxation of the residual problem that (i) drops the station-capacity constraint (2) and (ii) replaces each processing time by its minimum mink pijk over compatible stations, while retaining the exact lag precedence (1) and the elapsed time t. Both relaxations only remove constraints or lower durations, so the optimal makespan cannot increase: ĉi,j lower-bounds the completion of Oij in every feasible extension, and the maximum over terminal operations bounds Cmax . At termination every 1 The terminal lag ℓ i,ni delays module shipment but occupies no factory resource; we therefore report station-side makespan. In our benchmark every route ends with final quality-gate and wrap-and-ship operations with ℓi,ni = 0, so the two definitions coincide.
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS
4
(a) Processing-time inflation p11 +ℓ: station blocked (a) Cmax
M1
O11
A1 lag-aware dynamics
A2 +2 lag channels
A3 type embeddings
FJSP-TL environment
state features
per-step z-norm
O21
ℓ
M2
dual attention
nonzero pooling
policy πθ
DANIEL backbone (unchanged)
t 0
3
6
(b) Job-blocking, machine-free (ours): station free during lag M1 free at c11 M1
O11
(b) Cmax
(a) Cmax
we test rather than assume. Fig. 2 locates the three extensions on the backbone, and each is individually ablatable.
O12
A. Base Solver ℓ saving
0
Fig. 2. Lag-aware adaptation of the DANIEL dual-attention backbone. White boxes are the unmodified backbone (trained by PPO on the telescoping reward shown); the shaded callouts A1–A3 are this article’s flag-gated additions. With all flags off the path is bit-identical to the original solver.
O21
M2
module 1 (buffer)
scheduled (Oij , Mk ); PPO reward rt = Ĉmax (st ) − Ĉmax (st+1 ) (admissible, A1)
9
3 processing p
6
t 9 lag ℓ (buffer)
Fig. 1. Two ways of modeling a post-operation time-lag ℓ, on the same toy instance (2 modules, 2 stations). Module 1’s casting O11 on station M1 carries a curing lag; module 2 wants M1 next. (a) Inflating the processing time to p11 +ℓ blocks the station for the whole lag, so M1 cannot serve O21 until the lag elapses, overestimating congestion and the makespan. (b) The job-blocking, machine-free semantics of FJSP-TL, (2): M1 is occupied only until the physical completion c11 and is immediately free to serve O21 , while module 1 cures in a buffer (hatched, off-station) and its successor O12 starts only at c11 +ℓ. The station-side makespan is correspondingly shorter.
operation is scheduled and seeded with its realized completion cij , so the relaxation coincides with the realized schedule and the inequality becomes equality. The predecessor lag ℓi,j−1 enters before the successor’s processing time; with ℓ ≡ 0, (3) reduces exactly to the classical FJSP bound. Proposition 1 also makes precise why the lag term cannot be dropped: omitting it still yields a valid lower bound, but one that is systematically loose under lag-aware dynamics; each scheduling commitment reveals lag-induced delays the lag-free estimate never anticipated, so its per-step rewards form a systematically looser shaping signal, even though both bounds telescope to the same makespan objective (Section IV-B). IV. M ETHOD : L AG -AWARE A DAPTATIONS OF DANIEL We adapt DANIEL to FJSP-TL through three extensions, organized around a single principle: change the environment’s semantics where the lag genuinely alters the problem, and leave the learning machinery provably intact everywhere else. Two of the three changes therefore come with correctness guarantees rather than tuning: the lag-aware dynamics inherit an admissible makespan bound (Proposition 1) under which the telescoped PPO return remains objective-equivalent to the true makespan, and the type embeddings are gated so that they provably preserve the backbone’s pooling set (Proposition 2). The third, anticipatory lag observability, is the one component
DANIEL [8] encodes the scheduling state with operation features (10 channels) and machine features, z-normalized per channel at every decision step, processes them through stacked dual-attention blocks (operation- and machine-level self-attention with cross conditioning [29]), pools node embeddings by nonzero averaging (finished operations are deleted by zeroing their feature rows), and outputs a probability over eligible operation-machine pairs. Training is by PPO [30] with a dense reward rt = Ĉmax (st ) − Ĉmax (st+1 ), the decrease of the estimated makespan bound. Because these rewards telescope and the bound is tight at termination (Proposition 1), the undiscounted return (γ = 1) equals Ĉmax (s0 ) − Cmax , exactly the makespan objective. We change neither the architecture, nor the PPO scheme, nor the reward principle. Algorithm 1 summarizes one constructive episode, marking this section’s three adaptation points. B. Adaptation 1: Lag-Aware Dynamics and Admissible Reward Adaptation 1 installs the job-blocking, machine-free lag semantics of (1)–(2) directly in the environment, and pairs them with the admissible bound of (3) so that the reward stays faithful to the true makespan. The transition changes in one place: upon scheduling Oij on station k with completion cij , the station becomes free at cij (unchanged), while the job becomes ready at ρi = cij + ℓij , (4) implementing the semantics of (1)–(2). The reward bound is replaced by the lag-augmented estimate (3). This pairing is essential: with lag-aware dynamics but the lag-free bound, the episode return still telescopes to Ĉmax (s0 ) − Cmax , so the learning objective is unchanged; what degrades is the shaping, since each dense per-step reward is computed from an estimate that never anticipated the lag delays, yielding higher-variance, less informative advantages. Lag magnitudes are normalized by the same scale as processing times (zero-preserving, no shift), so a 24 h lag and a 24 h operation look equally long to the network, and the ℓ ≡ 0 case remains bit-identical to the original solver.
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS
5
Algorithm 1 Lag-aware constructive scheduling with typeembedded dual attention (one episode) Require: routes Oi,1 ≺ . . . ≺ Oi,ni ; eligible sets Mij ; times pijk ; lags ℓij ; type indices τij (op), τk (station); policy πθ Ensure: station kij and start sij for all Oij ; makespan Cmax 1: t ← 0; all Oij unscheduled; ρi ← 0, freek ← 0 ∀i, k 2: seed bound ĉi,j from (3) ▷ lag-augmented (A1) 3: while some operation is unscheduled do 4: build op features (10 ch.) and station features lag rem = ℓij , ηij (t) ▷ lag channels (A2) 5: append ηij 6: z-normalize every channel; zero deleted (finished) rows 7: eij ← Eop (τij ), ek ← Emch (τk ) 8: concat e after norm., masked by liveness 1[row ̸= 0] ▷ type emb. (A3) 9: dual attention → πθ over eligible (Oij , Mk ), k ∈ Mij 10: pick pair (Oij , Mk ): greedy arg max or sampled 11: sij ← max(ρi , freek ); cij ← sij +pijk ; set kij ← k 12: freek ← cij ; ρi ← cij +ℓij ▷ free station, block job (A1) 13: advance t; recompute ĉi,j and Ĉmax via (3) 14: if training then 15: rt ← Ĉmax (st ) − Ĉmax (st+1 ) ▷ admissible reward (A1) 16: end if 17: end while 18: return {kij , sij } and Cmax = maxi ci,ni
C. Adaptation 2: Anticipatory Lag Features Lag-aware dynamics alone let the policy experience lags only after committing to them. To let it anticipate them, we add two operation-feature channels, widening the operation features from 10 to 12: lag ηij = ℓij , rem ηij (t) = 1[Oij scheduled] · clip cij + ℓij − t, 0, ℓij
(5)
, (6)
through it would be corrupted. We therefore carry raw indices through the state and embed them inside the network, eij = Eop (τij ) ∈ R8 (and analogously ek = Emch (τk ) for stations), concatenating after normalization, before the first attention block. One subtlety matters: DANIEL deletes a finished operation by zeroing its entire feature row, and its nonzero-averaging pooling excludes exactly the all-zero rows. A naively concatenated type embedding eij ̸= 0 would make a deleted node’s row nonzero again and re-admit it into the pool. We therefore gate the embedding by the node-liveness indicator λij = 1[feature row of Oij ̸= 0] (the same criterion the pooling uses), appending λij eij . Proposition 2 (Pooling-set invariance). With liveness gating, the augmented network’s nonzero-averaging pool selects exactly the same set of nodes as the unmodified backbone, in every state. Proof. A deleted operation has both job-neighbor links masked (it completes after its predecessor, which is itself deleted), so in the local operation-block attention it attends only to its own zero row and stays exactly zero at every layer; nonzero-averaging, though it reads post-attention rows, therefore selects exactly the live nodes, as a mask fixed before attention would. The liveness gate λij eij is 0 precisely when the backbone row is 0 (the same zero-row criterion the pool uses, covering even a live node that per-step z-normalization maps to zero) and leaves the operation mask untouched, so this applies verbatim to the augmented network; the station pool is gated identically. Pooling membership is thus maskdetermined, independent of post-attention values. Empirically, across all 4400 decision states of 20 held-out M-class instances the maximum activation on a deleted row is 0 to machine precision with no deleted node entering either pool; the check ships with the released code. E. Backward Compatibility and Ablation Arms
the lag an operation will impose once scheduled (e.g., pouring early so that curing overlaps other work), and the portion of an in-flight lag still pending at the decision instant (equivalently, how soon blocked successors become available again). Both pass through DANIEL’s per-channel normalization; because the channels normalize independently, the original 10 channels remain numerically unchanged, which we confirmed bitexactly across all 111 decision states of a held-out instance. The channels are flag-gated, giving ablation arms that differ only in observability.
The three adaptations are designed for unambiguous attribution: each is gated by an independent flag with inert defaults, and the inputs it requires (time_lag, type indices) are optional. With every flag off, the code path reproduces the published DANIEL solver bit-exactly (mean makespan 408.40 on SD2, the full regression check of Section I), so any measured difference is caused by the adaptations alone rather than by an incidental reimplementation change. The flag matrix directly yields four ablation arms: A0 (lag-blind training with right-shift repair at evaluation, the current industrial practice), A1 (lag-aware dynamics only), A2 (+features), and A3 (full method).
D. Adaptation 3: Type Embeddings With Liveness Masking
F. Complexity
Operations and stations carry semantic class information that the pure-feature backbone cannot exploit, and injecting it naively breaks the backbone’s invariants. The PPVC classes are categorical (5 operation types: structural, MEP, finishing, assembly, quality gates; 9 station types), whereas DANIEL’s pipeline z-normalizes every channel per step and deletes finished nodes by zeroing, so integer codes pushed
The three adaptations leave the backbone’s asymptotics unchanged: with |O| = 22n operations and m stations, each decision step is dominated by the O(|O|2 d + m2 d + |O|m d) dual-attention cost (embedding dimension d), and an episode performs |O| steps. A1 adds O(1) per scheduled operation, A2 O(|O|) per step, and A3 a lookup and liveness mask, O(|O| + m) per step, all dominated by the attention terms,
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS
6
TABLE III B ENCHMARK P ROVENANCE : W HICH E LEMENTS A RE G ROUNDED IN THE BCA G UIDEBOOK V ERSUS E STIMATED OR PARAMETRIC . Element
Source
Status
Routing / operation sequences Station taxonomy (9 types) Shell × fit-out classes Lag categories and ranges Processing durations Factory station counts Buffer capacities
BCA guidebook BCA guidebook BCA guidebook Industry norms Literature estimate Generator preset Not modeled
Grounded [G] Grounded [G] Grounded [G] Norm-based [N] Estimated [E] Parametric Future work
giving a worst-case bound cubic in the module count n. In practice, attention parallelized on a graphics processing unit (GPU) and small constants keep the measured greedy runtime close to linear in the operation count (1.82, 3.61, and 7.52 s at 10, 20, and 40 modules; Section VI), so a re-plan stays within a real-time control loop across the scales we test.
V. BCA-G ROUNDED B ENCHMARK G ENERATOR No public FJSP benchmark carries PPVC-realistic lags, and classical suites (e.g., [31]) have none at all. We release a generator2 grounded in the official BCA PPVC guidebook [3] with explicit provenance tags (Table III): [G] facts extracted from the guidebook, [E] estimated durations (the guidebook prescribes sequences, not durations), [N] industry-norm lag ranges. Stating this split explicitly is deliberate: the generator is a reproducible, BCA-grounded testbed for lag-aware FJSPTL research, not a calibrated digital twin of any specific fabricator. Routing classes [G]. 4 classes, namely {reinforced concrete (RC), steel} shells × {wet, dry} fit-outs, each aggregated to exactly 22 station visits from the guidebook’s production sequences (mold/jig preparation through trial assembly, including pre-pour, weld, MEP, finishing and final quality gates). Equal route lengths across classes allow training on any module mix in fixed-shape batches without padding; this is an instance-design choice, not a solver limitation. Factory steel connections are welded (bolting is the on-site method), and wet:dry module ratios default to 3:1, both per the guidebook. Stations [G]. 9 station types, from casting through final quality gates (quality gates are modeled as stations; curingadjacent buffer areas are not). Factory presets: DEFAULT (25 stations), TIGHT (15), SMALL (9); station counts are generator parameters since layouts are fabricator-specific. Durations [E] and lags [N]. Processing times are literaturebased integer-hour estimates (±20% across compatible stations); lags are sampled from industry norms: curing 24– 48 h [14], ponding 24–48 h, and paint drying 12–24 h per coat. The resulting lag-to-processing volume ratio spans 0.88– 1.2, supporting the central claim that lags rival processing in temporal volume. 2 Generator, instance seeds, CP-SAT references, trained models, and evaluation code: https://github.com/NTUZZH/FJSP-DRL-PPVC.
TABLE IV T RAINING H YPERPARAMETERS (DANIEL DEFAULTS ; ADAPTATIONS ADD ONLY THE TYPE - EMBEDDING DIMENSION ). MLP: MULTILAYER PERCEPTRON ; GAE: GENERALIZED ADVANTAGE ESTIMATION . Parameter
Value
Dual-attn. layers Attention heads Embed. dim. (per layer) Actor/critic MLP Type-embed. dim. Learning rate Parallel envs
2 PPO epochs/update 4 4 Minibatch size 1024 32, 8 Clip ϵ 0.2 3 layers, 64 Discount γ 1.0 8 GAE λ 0.98 3 × 10−4 Entropy coef. 0.01 20 PPO updates 1000
Parameter
Value
VI. E XPERIMENTS A. Setup Instance classes. S (5 modules, SMALL factory), M (10 modules, DEFAULT; main training class, mixed shells), M-tight (10 modules, TIGHT; capacity contention), L (20 modules), and single-shell RC/steel project classes. Train, validation (100 instances), and test (100 instances) streams use disjoint seed namespaces; all evaluation schedules pass an independent feasibility validator (written against the formulation, sharing no code with the environment). Baselines. (i) CP-SAT [4] on the FJSP-TL model, (1)–(2) with a lag-widened horizon, at 300 s per instance, providing optimality references (86/100 proven optimal on the M-class test set); (ii) four lag-aware priority dispatching rules (PDRs) acting on the same environment: first-in-first-out (FIFO), most operations remaining (MOR), shortest processing time (SPT), and most work remaining (MWKR); (iii) a PDR-seeded genetic-algorithm (GA) metaheuristic on the FJSP-TL model, run at a 60 s per-instance budget, a search-based peer probed at 30× the learned policy’s runtime (CP-SAT, by contrast, serves purely as an optimality reference); and (iv) the four ablation arms (A0–A3 above), each trained for 1000 PPO updates (approximately 5.0 h wall-clock) with the hyperparameters of Table IV and 20 parallel environments. Hardware. All experiments ran on a single NVIDIA RTX PRO 5000 GPU and an Intel Core Ultra 9 285K central processing unit (CPU). Reported learned-policy and dispatchingrule times are for full constructive rollouts on the GPU-resident environment (not a single network forward pass); CP-SAT and the GA run on CPU, with CP-SAT using all available threads. Metrics. Mean makespan, gap to CP-SAT reference, paired win/tie/loss and Wilcoxon signed-rank tests, and wall-clock time per instance. B. Main Results (M Class) On the held-out M class (Table V), the full method (A3, greedy decoding) is the strongest solver-free method and approaches the exact solver’s quality. It reaches 210.9 h mean makespan, cutting the best dispatching rule’s gap to the CPSAT reference from 6.4% (SPT) to 3.8% and achieving a 73/11/16 win/tie/loss against SPT over the 100 instances (Wilcoxon signed-rank test, p = 1.8 × 10−10 ), in 1.97 s per instance and without an optimization model or solver in the loop. Sampling 100 rollouts and keeping the best
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS
C. Ablation Analysis Table V and Fig. 4 isolate the contribution of each adaptation, and we test every contrast with a paired Wilcoxon signedrank test over the 100 common test instances. Modeling lags at decision time (A0 → A1) is decisive. The lag-blind arm A0 reproduces current industrial practice: it plans without lags (reaching a deceptively low plan makespan of 129.0 h) and then right-shifts the plan to feasibility once the true lags are revealed. Repair inflates that plan by 137% to 306.4 h, leaving A0 not only 51.0% above the CP-SAT reference but worse than every lag-aware priority rule, including FIFO (231.2 h). Simply making the same backbone lag-aware (A1) cuts the makespan to 211.1 h, a 51.0 → 3.9% gap reduction that is highly significant (p = 3.9 × 10−18 ). Lag-awareness at decision time, not post-hoc repair, drives the gain.
220
CP-SAT lag-aware makespan (h)
(A3, sampling-100) closes the gap further to 2.1% (207.4 h, significantly below greedy, p < 10−15 ) at 40.3 s per instance. It also beats a PDR-seeded genetic algorithm at a 60 s budget (214.2 vs. 210.9 h, p = 2.7 × 10−7 ) despite the GA using about 30× its inference time, and remains ahead even at the GA’s full 300 s budget (211.4 h on a 20-instance subset, A3 better on 13/20), so the margin is not a search-budget artifact; it widens under contention (Section VI-D). The exact solver is itself fast on this capacity-rich class. Profiling CP-SAT’s anytime behavior, a strong feasible plan arrives quickly: 208.4 h at 1 s (already below greedy A3’s 210.9 h), and essentially the 203.2 h reference by 5 s (203.6 h). The remaining budget mainly buys certification, not better plans: the proven-optimal count rises from 36 to 67, 74, and 86 of 100 at 1, 5, 30, and 300 s (all instances feasible throughout). On instances of this size the exact solver is the quality ceiling, which the solver-free policy reaches to within 3.8% with no optimization model, solver, or license in the loop. Warmstarting CP-SAT with a dispatching schedule sharpens this ceiling at small and out-of-range scales (e.g., 386.1 h within a second at 40 modules, below both SPT’s 388.2 h and the policy’s 409.6 h); at medium scale, though, the solver-free policy holds its own in the real-time window, matching a warmstarted CP-SAT at 20 modules (259.7 versus 260.0 h at 5 s). Warm-starting, moreover, needs the solver, the optimization model, and a heuristic scheduler, the very infrastructure the policy is built to avoid. The policy delivers near-reference schedules from a single constructive rollout as the strongest solver-free method, and re-plans reactively in about a second, beating current right-shift practice (Section VI-F), the setting a manufacturing-execution loop actually runs in. Validation and test means agree (212.0 versus 210.9 h), indicating no overfitting to the training stream. Retraining A3 under three independent seeds yields a test makespan of 211.5 ± 0.6 h (mean ± standard deviation across seeds; range 1.2 h), confirming the headline is not a training-seed artifact. Finally, the CP-SAT references confirm the single-instance motivation of Table I across the whole test population: over the 100 test instances, modeling lags inflates the reference makespan by 67.2% on average (Fig. 3).
7
200
180
mean inflation +67.2%
160
140 y = x (no lag) OPTIMAL FEASIBLE
120
120
140
160
180
200
220
CP-SAT lag-free makespan (h) Fig. 3. Time-lags dominate CP-SAT reference schedules, including the proven-optimal subset. Each point is one M-class test instance: CP-SAT makespan with lags ignored (x) versus the lag-aware reference (y); filled markers are proven optimal, hollow markers feasible within the 300 s cap. Every instance lies far above the y = x line, with a mean per-instance inflation of 67.2%, confirming that the lag structure, not machine contention alone, governs the achievable makespan.
TABLE V H ELD -O UT T EST S ET, M C LASS (100 INSTANCES , 10 MODULES × 22 OPERATIONS , 25 STATIONS ). Mean (h)
Std.
Gap to ref.†
s/inst
SPT (best PDR) MWKR FIFO MOR GA metaheuristic (60 s)
216.0 224.7 231.2 233.8 214.2
8.6 9.6 9.4 11.8 8.4
6.4% 10.7% 13.9% 15.2% 5.5%
1.4 1.4 1.4 1.4 60
A0 lag-blind + repair A1 bare backbone A2 + lag features A3 full (greedy) A3 full (sampling-100)
306.4 211.1 210.7 210.9 207.4
21.9 9.5 9.6 9.4 8.7
51.0% 3.9% 3.7% 3.8% 2.1%
1.87 1.86 1.85 1.97 40.3
CP-SAT (300 s)
reference (86/100 proven optimal)
Method
† Per-instance gap to the CP-SAT lag-aware reference, averaged over
full coverage (100/100; 86 proven optimal); on the 14 instances not proven optimal within the budget, this gap is a lower bound on the true optimality gap. PDR inference times are rounded to a common 1.4 s (individual range 1.33–1.38 s). Bold marks the proposed method (A3, greedy decoding); the sampling-100 row is a higher-budget configuration of the same model.
On the capacity-rich M class the anticipatory features and type embeddings are statistically neutral (Fig. 4): A2 and A3 are indistinguishable from the bare lag-aware backbone A1 (p = 0.51 and p = 0.89). When capacity is generous the transition dynamics alone already let the policy place curingheavy operations early, so the extra observability has little left to exploit; its value surfaces where capacity binds, as Sec-
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS
mean makespan (h)
M (10x25, capacity-rich) M-tight (10x15, contention)
225 A0 lag-blind+repair: 306 h 280 A0 lag-blind+repair: 421 h A3 vs A1: p=2e-11 *** vs A1: n.s. (p=0.51) 220 A3 A3 vs A2: n.s. (p=0.89) 270 A3 vs A2: p=1e-3 *** 215 211.1 210.7 210.9 260 257.9 255.5 254.1 210 250 205
A1 A2 A3 lag-dyn +feats +types
A1 A2 A3 lag-dyn +feats +types
Fig. 4. Ablation across capacity regimes. Lag-aware dynamics (A1) carry the gain; the anticipatory features (A2) and type embeddings (A3) are statistically neutral on the capacity-rich M class but separate significantly under contention (M-tight). The lag-blind arm A0 (plan-then-repair, current practice) lies beyond the plotted range (catastrophic). Dashed line: best dispatching rule (SPT); error bars: cross-seed standard deviation; brackets: paired Wilcoxon p.
tion VI-D confirms (each adaptation significant, p ≤ 0.022). We therefore deploy A3: tied with A2 on the M class, but the arm that improves significantly once capacity binds, hence the safer choice across regimes at no asymptotic cost.
D. Capacity-Tight Contention Halving the factory to 15 stations (M-tight), a plausible high-load configuration for a busy plant running many modules over few lines rather than an artificial stress test, sharply raises machine contention. Here the learned policy’s advantage over dispatching widens: the full method reaches 254.1 h, narrowing the best rule’s gap to the CP-SAT reference from 12.2% (SPT) to 6.6% and achieving a 93/2/5 win/tie/loss against SPT over the 100 instances (p = 8.1 × 10−17 , Table VI), a wider margin than the 6.4% → 3.8% improvement on the capacityrich M class. Two effects compound. First, tighter capacity makes instances harder for the exact solver (only 14 of 100 proven optimal within the budget, so the M-tight gaps are largely lower bounds on the true optimality gap). Second, the mean lag inflation falls to 36.2% as machine contention, not lags alone, becomes co-dominant; this is precisely the regime where look-ahead over operation-machine assignment pays off. The genetic-algorithm metaheuristic, given 60 s, reaches only 264.2 h here (10.9% gap); greedy A3 beats it by over 10 h (p = 6×10−16 ), far wider than on the M class. Crucially, the per-arm ablation that was neutral on the M class separates under contention, and this is not a seed artifact (Fig. 4): across three independent seeds the A1→A2→A3 ordering holds in every seed, with cross-seed standard deviation (≤ 1.7 h) well below the inter-arm gaps, the full stack significantly beating both the bare backbone (pooled p = 2 × 10−11 ) and the features-only arm (pooled p = 1 × 10−3 ). The anticipatory features and type embeddings, statistically inert under generous capacity (p > 0.5 on the M class), thus earn their place exactly where the shop is hard. Lag-blindness is conversely catastrophic here: right-shifting the lag-blind plan to feasibility reaches 421.4 h (76.6% above the reference), worse than every dispatching rule.
8
TABLE VI H ELD -O UT T EST S ET, M- TIGHT C LASS (100 INSTANCES , 10 MODULES , 15 STATIONS ). Mean (h)
Std.
Gap to ref.†
W/T/L‡
A0 lag-blind + repair A1 bare backbone A2 + lag features A3 full (greedy)
421.4 257.9 255.5 254.1
43.5 13.1 13.0 13.0
76.6% 8.2% 7.2% 6.6%
0/0/100 82/7/11 90/1/9 93/2/5
SPT (best PDR) MWKR FIFO MOR GA metaheuristic (60 s)
267.3 299.4 306.3 319.8 264.2
12.7 18.3 14.1 22.9 12.9
12.2% 25.7% 28.6% 34.2% 10.9%
− 6/0/94 0/0/100 0/0/100 66/7/27
Method
CP-SAT (300 s)
reference (14/100 proven optimal)
† Per-instance
gap to the CP-SAT lag-aware reference (full 100/100 coverage; 14 proven optimal, the rest lower bounds).
‡ Win/tie/loss versus SPT over the 100 instances.
Bold marks the proposed method (A3, greedy decoding). Ablation arms (A0–A3) are shown for a representative seed; their crossseed means±std over three seeds are reported in the text and preserve the A1–A3 ordering.
E. Zero-Shot Generalization We apply the M-trained A3 policy, without any retraining, to four unseen classes (Table VII); transfer holds across shell systems but degrades across module count. Across shell systems at the trained factory size, namely single-shell RC and steel projects, the policy transfers cleanly, beating the best dispatching rule on both (231.5 vs. 235.8 h, p = 3.1×10−4 on RC; 221.7 vs. 235.9 h, p = 1.1 × 10−7 on steel). Because this axis varies the operation- and station-types the embeddings encode, it is the more pertinent test of the type-embedding adaptation, and the learned policy clearly helps. Across instance sizes, a policy trained at a single module count need not dominate at another, a known limitation of fixed-shape DRL training that size-mixed training removes (Fig. 5). The singlesize policy is overtaken by the scale-free SPT rule on the small (5-module) and large (20-module) classes (win/tie/loss 13/2/35 and 10/2/38 versus SPT); one policy trained on a {10,20,30}module mix instead beats SPT at 10 and 20 modules (211.4 and 259.7 h versus 216.0 and 265.1 h, over 100 and 50 held-out instances; sampling widens this to 253.9 h at 20) and matches it to within 0.4% at the largest trained size of 30 (326.2 versus 324.9 h over 20 instances, with no certified reference at this size), at an inference cost linear in the operation count (1.82– 7.52 s from 10 to 40 modules); beyond the trained range, at 40 modules, the real-time greedy policy trails SPT while higherbudget sampling brings it nearly level (390.1 versus SPT’s 388.2 h), marking the current exploration boundary of the learned policy and a natural opening for neural-metaheuristic hybrids. Size specialization is a deployment choice rather than a constraint in PPVC, where a factory’s layout and throughput are fixed over multi-year cycles, so a capacity-specific policy is the natural deployment unit and the binding transfer axis is the shell system, where the policy generalizes out of the box. F. Reactive Rescheduling Under Disruptions The premise that lag-awareness must be fast (Section I) is testable only under disruption. We execute the A3 plan
mean makespan − SPT (h); below 0 beats SPT
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS
20 15
single-size (M-only) size-mixed (proposed)
10
+8
5 0
−5
9
+1
SPT floor -5
-5 10
TABLE VIII R EACTIVE R ESCHEDULING A FTER A M ID -E XECUTION D ISRUPTION (100 M- CLASS INSTANCES ; DISRUPTION AT 40% COMPLETION ).
+17
-5 20
factory scale (modules)
30
Fig. 5. Size-mixed training closes the DRL size-generalization gap within the trained range: mean makespan relative to the SPT floor (below zero beats SPT; held-out 100/50/20 instances at 10/20/30 modules). The single-size policy falls behind SPT at 20 and 30 modules, whereas one policy trained on a {10,20,30}-module mix beats SPT at 10 and 20 and matches it to within 0.4% at 30. Beyond the trained range (40 modules) both trail SPT. Shaded band: trained range.
TABLE VII Z ERO -S HOT S HELL -S YSTEM T RANSFER OF THE M-T RAINED A3 P OLICY ( MEAN MAKESPAN , H ; LOWER IS BETTER ; 50 INSTANCES PER CLASS ). Shell system
A3 (ours)
SPT
W/T/L†
RC project Steel project
231.5 221.7
235.8 235.9
38/1/11 43/2/5
† A3 win/tie/loss versus SPT (bold = lower mean
makespan): the policy transfers cleanly across shell systems at the trained size. The size axis is shown in Fig. 5.
Disruption
Method
Mk (h)
Infl.%
Time (s)
Nerv.†
Lag pert.
DRL-reactive Right-shift Resid. CP-SAT (mk.)‡ Stab.-aware CP-SAT§
220.7 233.0 218.2 220.4
4.7 10.5 3.5 4.6
1.2 0.001 0.56 0.3
37.5 37.2 170.1 9.1
Breakdown
DRL-reactive Right-shift Resid. CP-SAT (mk.)‡ Stab.-aware CP-SAT§
211.0 220.1 207.9 210.7
0.04 4.4 -1.4 -0.1
1.3 0.001 0.73 0.2
15.0 59.9 160.8 0.3
† Schedule nervousness: remaining operations forced to change (machine
reassignment, or start-time shift > 1 h) versus the original plan, i.e., the downstream churn one disruption propagates. The 1 h threshold matches the benchmark’s integer-hour resolution. ‡ Makespan-only prefix-respecting residual CP-SAT (committed prefix fixed, breakdown encoded as a machine-unavailability interval): nearoptimal in makespan but indifferent to churn, so it re-routes most of the plan. § Stability-aware residual CP-SAT, objective C max + w·nervousness at w = 0.5: the strongest fair reactive baseline. It dominates DRLreactive on both axes (far lower nervousness at comparable makespan), in under 0.3 s with proven optimality on all 100 instances. Bold marks the proposed learned method: the strongest solver-free option, beating current right-shift practice.
edge hardware with no solver, model, or penalty in the loop, where it is the strongest solver-free option and the only one that improves on right-shift by re-routing. Its zero-shot churn grows with cell size, so at large scales it pairs with exact reoptimization where a solver can be hosted. G. Threats to Validity
to 40% completion, inject a mid-execution disruption (a lag perturbation, a curing/drying time realizing at 2× plan; or a machine breakdown), and re-solve the residual with three methods: DRL-reactive (the policy continues from the disrupted state, free to re-route); right-shift repair (assignments frozen, start times pushed to feasibility: current practice); and a prefix-respecting residual CP-SAT that pins the committed prefix and re-optimizes the rest. Because a makespan-only re-optimization ignores churn, we also give CP-SAT the fair objective Cmax + w · nervousness and sweep w to trace its stability-makespan frontier, the strongest fair reactive baseline (Table VIII, 100 M-class instances; all schedules feasibilityvalidated, the prefix and outage honored exactly). Versus current practice the learned policy wins clearly: it cuts the post-disruption makespan by 5.3% (lag) and 4.1% (breakdown) over right-shift by actively re-routing rather than only delaying, solver-free. Where an exact solver can be hosted, a stability-aware CP-SAT is the absolute quality and stability ceiling: at w = 0.5 it changes only 9.1 and 0.3 operations (lag and breakdown) versus DRL-reactive’s 37.5 and 15.0, at comparable makespan and under 0.3 s. This maps a ceiling, not a drop-in alternative: reaching it carries the full constraint-programming model and solver binaries in the realtime loop, re-tuned on every disruption. The learned policy is the complement: a static compute graph evaluated in a few matrix multiplications, re-planning in about a second on
Benchmark realism. Guidebook provenance covers sequences, station taxonomy, and lag ranges; durations are estimates ([E]) pending industry calibration, so conclusions rest on relative comparisons under identical instances, not absolute hours. We verify this robustness directly: when A3 and SPT are re-evaluated under multiplicative duration perturbations U [1 − δ, 1 + δ], the A3<SPT ranking holds at every δ up to ±30% (A3 winning 73–78% of instances, p ≤ 6 × 10−10 ), so the relative conclusions do not hinge on the specific duration estimates. Lags are modeled as machine-independent constants per operation; equal-length routes are a generator design choice enabling fixed-shape batching, and factory layouts are parametric presets rather than a specific fabricator’s floor (a reproducible testbed, not a calibrated digital twin, as Section V states). Modeling and evaluation. The formulation treats the offstation lag buffers (curing, ponding, drying) as nonbinding. As an a posteriori check, across the 100 M-class instances the policy never holds more than 10 modules in lag buffers at once (mean 8.9), the same as SPT; at 10 modules this peak equals the batch, so the off-station lag buffers are nonbinding at the deployed scale. Modeling finite buffers as explicit resources (an FJSP-TL with resource-constrained buffers), and the floor resources we omit (labor crews, material staging, crane and transport interfaces, and stochastic lag durations) are left to future work. For the 14 instances whose CP-SAT reference
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS
is not proven optimal, the reported gaps are lower bounds on the true optimality gap. Seed robustness holds on both axes (Sections VI and VI-D): the headline result replicates across three training seeds, the contention ordering holds in every seed, and seed variance affects no reported conclusion. VII. C ONCLUSION We formalized PPVC module-factory scheduling as FJSP with job-blocking, machine-free time-lags, and showed empirically that these lags dominate both optimal and heuristic makespans. To solve it, we adapted a state-of-the-art DRL solver through three individually ablatable extensions, two of which come with correctness guarantees: lag-aware dynamics with an admissible makespan bound under which the learning objective stays equivalent to the true makespan, and liveness-masked type embeddings that provably preserve the backbone’s pooling set. With all three extensions disabled the implementation reproduces the original solver bit-exactly, so every reported gain is attributable to the adaptations. On a BCA-grounded benchmark the full method outperforms both the strongest dispatching rule and a genetic-algorithm metaheuristic given far more compute, and reaches the CP-SAT references (86/100 proven optimal) to within 3.8% with a solverfree policy. This margin widens under capacity contention and holds across training seeds and duration perturbations, indicating practical potential pending calibration on factory data. Where an exact solver and optimization expertise are available they remain the stronger choice, including for stable reactive re-optimization; the learned policy’s niche is solverfree deployment. The learned policy transfers cleanly across shell systems, and a single size-mixed policy beats the best dispatching rule from 10 to 20 modules; larger sizes remain the frontier. Future work includes variable-length routes, maximal lags, stochastic and station-dependent lag durations, finite curing buffers, multi-factory coupling, and duration calibration from factory data. R EFERENCES [1] A. W. A. Hammad, H. Grzybowska, M. Sutrisna, A. Akbarnezhad, and A. Haddad, “A novel mathematical optimisation model for the scheduling of activities in modular construction factories,” Constr. Manage. Econ., vol. 38, no. 6, pp. 534–551, 2020. [2] A. Peiris, F. K. P. Hui, C. Duffield, and T. Ngo, “Production scheduling in modular construction: Metaheuristics and future directions,” Autom. Constr., vol. 150, 2023, Art. no. 104851. [3] Building and Construction Authority, “Design for manufacturing and assembly (DfMA): Prefabricated prefinished volumetric construction (PPVC) guidebook,” 2017, Singapore. Accessed: Jun. 13, 2026. [Online]. Available: https://www1.bca.gov.sg/growth-andtransformation/productivity/design-for-manufacturing-and-assemblydfma/prefabricated-prefinished-volumetric-construction-ppvc/ [4] L. Perron and F. Didier, “CP-SAT solver, Google OR-Tools v9.14,” 2025, Accessed: Jun. 13, 2026. [Online]. Available: https: //developers.google.com/optimization/cp/cp solver/ [5] K. Lei, P. Guo, Y. Wang, J. Zhang, X. Meng, and L. Qian, “Large-scale dynamic scheduling for flexible job-shop with random arrivals of new jobs by hierarchical reinforcement learning,” IEEE Trans. Ind. Informat., vol. 20, no. 1, pp. 1007–1018, 2024. [6] C. Zhang, W. Song, Z. Cao, J. Zhang, P. S. Tan, and C. Xu, “Learning to dispatch for job shop scheduling via deep reinforcement learning,” in Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 33, 2020, pp. 1621–1632. [7] W. Song, X. Chen, Q. Li, and Z. Cao, “Flexible job-shop scheduling via graph neural network and deep reinforcement learning,” IEEE Trans. Ind. Informat., vol. 19, no. 2, pp. 1600–1610, 2023.
10
[8] R. Wang, G. Wang, J. Sun, F. Deng, and J. Chen, “Flexible job shop scheduling via dual attention network-based reinforcement learning,” IEEE Trans. Neural Netw. Learn. Syst., vol. 35, no. 3, pp. 3091–3102, 2024. [9] K. Lei, P. Guo, W. Zhao, Y. Wang, L. Qian, X. Meng, and L. Tang, “A multi-action deep reinforcement learning framework for flexible jobshop scheduling problem,” Expert Syst. Appl., vol. 205, 2022, Art. no. 117796. [10] M. Bartusch, R. H. Möhring, and F. J. Radermacher, “Scheduling project networks with resource constraints and time windows,” Ann. Oper. Res., vol. 16, no. 1, pp. 201–240, 1988. [11] E. D. Wikum, D. C. Llewellyn, and G. L. Nemhauser, “One-machine generalized precedence constrained scheduling problems,” Oper. Res. Lett., vol. 16, no. 2, pp. 87–99, 1994. [12] P. Brucker, T. Hilbig, and J. Hurink, “A branch and bound algorithm for a single-machine scheduling problem with positive and negative timelags,” Discrete Appl. Math., vol. 94, no. 1–3, pp. 77–99, 1999. [13] A. Caumond, P. Lacomme, and N. Tchernev, “A memetic algorithm for the job-shop with time-lags,” Comput. Oper. Res., vol. 35, no. 7, pp. 2331–2356, 2008. [14] W. T. Chan and H. Hu, “Production scheduling for precast plants using a flow shop sequencing model,” J. Comput. Civ. Eng., vol. 16, no. 3, pp. 165–174, 2002. [15] S. Wang and X. Zhang, “Production scheduling of prefabricated components considering delivery methods,” Sci. Rep., vol. 13, 2023, Art. no. 15094. [16] S. Xu, Y. Li, and Q. Li, “A deep reinforcement learning method based on a transformer model for the flexible job shop scheduling problem,” Electronics, vol. 13, no. 18, 2024, Art. no. 3696. [17] I. G. Smit, J. Zhou, R. Reijnen, Y. Wu, J. Chen, C. Zhang, Z. Bukhsh, Y. Zhang, and W. Nuijten, “Graph neural networks for job shop scheduling problems: A survey,” Comput. Oper. Res., vol. 176, 2025, Art. no. 106914. [18] C.-L. Liu, C.-J. Tseng, and P.-H. Weng, “Dynamic job-shop scheduling via graph attention networks and deep reinforcement learning,” IEEE Trans. Ind. Informat., vol. 20, no. 6, pp. 8662–8672, 2024. [19] J.-P. Huang, L. Gao, and X.-Y. Li, “An end-to-end deep reinforcement learning method based on graph neural network for distributed job-shop scheduling problem,” Expert Syst. Appl., vol. 238, 2024, Art. no. 121756. [20] S. Luo, L. Zhang, and Y. Fan, “Real-time scheduling for dynamic partialno-wait multiobjective flexible job shop by deep reinforcement learning,” IEEE Trans. Autom. Sci. Eng., vol. 19, no. 4, pp. 3020–3038, 2022. [21] J. Fondrevelle, A. Oulamara, and M.-C. Portmann, “Permutation flowshop scheduling problems with maximal and minimal time lags,” Comput. Oper. Res., vol. 33, no. 6, pp. 1540–1556, 2006. [22] E. Dhouib, J. Teghem, and T. Loukil, “Non-permutation flowshop scheduling problem with minimal and maximal time lags: Theoretical study and heuristic,” Ann. Oper. Res., vol. 267, no. 1–2, pp. 101–134, 2018. [23] T. Kim, Y.-W. Kim, D. Lee, and M. Kim, “Reinforcement learning approach to scheduling of precast concrete production,” J. Clean. Prod., vol. 336, 2022, Art. no. 130419. [24] L. Zu and W. Liao, “Reinforcement learning–based multiobjective and multiconstraint production scheduling for precast concrete,” J. Constr. Eng. Manage., vol. 151, no. 8, 2025, Art. no. 04025089. [25] Y. Du and J.-Q. Li, “A deep reinforcement learning based algorithm for a distributed precast concrete production scheduling,” Int. J. Prod. Econ., vol. 268, 2024, Art. no. 109102. [26] H. Wang, L. Liao, W. Yi, and L. Zhen, “Transportation scheduling for modules used in modular integrated construction,” Int. J. Prod. Res., vol. 62, no. 11, pp. 3918–3931, 2024. [27] M. ElMenshawy, L. Wu, B. Gue, and S. AbouRizk, “Automating pipe spool fabrication shop scheduling for modularized industrial construction projects using reinforcement learning,” J. Comput. Civ. Eng., vol. 39, no. 3, 2025, Art. no. 04025013. [28] M. R. Garey, D. S. Johnson, and R. Sethi, “The complexity of flowshop and jobshop scheduling,” Math. Oper. Res., vol. 1, no. 2, pp. 117–129, 1976. [29] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 30, 2017, pp. 5998–6008. [30] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” 2017, arXiv:1707.06347. [31] P. Brandimarte, “Routing and scheduling in a flexible job shop by tabu search,” Ann. Oper. Res., vol. 41, no. 3, pp. 157–183, 1993.