Singular Curvature in ReLU Training:Differentiation and the Gradient-Flow Limit Need Not Commute

Preprint

S INGULAR C URVATURE IN R E LU T RAINING : D IFFERENTIATION AND THE G RADIENT-F LOW L IMIT N EED N OT C OMMUTE

arXiv:2608.30960v1 [cs.LG] 31 Aug 2026

Xiaoyang Li Runni Zhou College of Medicine and Biological Information Engineering Northeastern University, Shenyang, China 20246389@stu.neu.edu.cn 20246378@stu.neu.edu.cn ORCID: 0009-0002-4863-5761 ORCID: 0009-0006-8365-4852

A BSTRACT Gradient descent (GD) is explicit Euler for gradient flow, but a state-accurate continuous-time surrogate need not remain accurate after differentiation. At every fixed nonresonant step size, ordinary automatic differentiation exactly differentiates the executed hard-ReLU GD program. We prove that, over a fixed finite horizon, the GD states converge and these exact discrete derivatives approach an event-free regional propagator, whereas the derivative of the limiting flow also contains speednormalized activation-event transfers. A prepoint Stieltjes representation separates the absolutely continuous regional Hessian from atomic interface curvature; one nonzero gradient jump produces an exactly rank-one endpoint discrepancy, and global convexity prevents complete multi-event cancellation whenever an event is strict. Nevertheless, a standard family of globally 1-strongly convex residual-ReLU squared-loss risks realizes arbitrarily large reciprocal sensitivity ratios on open initialization sets, with a uniform transversality margin. The same discrete-versusflow decomposition extends to parameters and reverse-mode adjoints; resolved smoothing in the scalar or autonomous-normal regime and consistent event localization recover the flow sensitivity. The results concern deterministic full-batch, finite-horizon dynamics with a stable finite itinerary of separated same-direction transverse events; they are consistency theorems, not prevalence claims for largescale training.

1

I NTRODUCTION

Small-step gradient descent (GD) is often replaced by gradient flow, then differentiated to study initialization dependence, meta-parameters, data weights, differentiable optimization, or reversemode sensitivity. This second step needs more than state accuracy. If Θη,T is the GD endpoint map at fixed continuum time T and φT is the flow map, the required commuting relation is Θη,T (θ0 ) → φT (θ0 )

?

=⇒

DΘη,T (θ0 ) → DφT (θ0 ).

For a stable hard-ReLU event itinerary, it need not hold: Θη,T (θ0 ) −→ φT (θ0 ), AD DΘη,T (θ0 ) = Jη,T −→ Jreg = Φm · · · Φ0 ,

(1)

DφT (θ0 ) = Φm Ξm Φm−1 · · · Ξ1 Φ0 . At every fixed nonresonant step size, AD exactly differentiates the executed discrete program. What fails is exchanging differentiation with the vanishing-step limit. The missing term is event timing. Nearby initializations reach an activation boundary at slightly different times and therefore spend different durations under the two regional vector fields. This residence-time change produces the classical saltation transfer Ξj . A hard-branch numerical step has 1

Preprint

only an I + O(η) branchwise Jacobian, so no finite event transfer survives in its derivative limit. The states converge. Their tangent dynamics need not approach the same object. We express this mechanism through a speed-normalized pathwise curvature measure. Its continuous density is the regional Hessian; each activation event contributes an atom. The corresponding prepoint Stieltjes system gives the saltation-interleaved flow derivative, whereas deleting the atoms gives Jreg . One nonzero event jump produces an exact rank-one gap whose norm and outer-gradient visibility are explicit. For multiple events, transported products give the exact cancellation criterion and quantitative volume and interaction bounds. Convexity determines the sign of the event. For a globally convex loss, every event weakly attenuates its direct normal mode; consequently, one strict event rules out complete cancellation over a finite same-direction itinerary. Strong convexity does not by itself guarantee commutation: the standard residual-ReLU risk 2 2 1 1 2 (θ − m) + 2 (ReLU(θ) − c) has a globally 1-strongly convex family for which DφT = Jreg /R for any fixed R > 1, on an open initialization set and with a transverse-speed margin independent of R. This is an arbitrary multiplicative reciprocal ratio, not an unbounded additive claim. The same limit applies when a parameter enters the vector field, initialization, or activation surface: the flow derivative has a classical affine event jump that the event-free discrete limit omits. Reverse mode uses Ξ⊤ j and, for a moving surface, an event accumulator. This does not say that discrete hypergradients are wrong; it limits when a differentiated flow surrogate represents increasingly fine discrete training. Saltation, discontinuous-ODE sensitivity errors, parameter and adjoint jumps, and singular Hessian measures are prior art (Hiskens & Pai, 2000; Tolsma & Barton, 2002; Stewart & Anitescu, 2010; Corner et al., 2020; Kong et al., 2024). The learning-specific object here is the T /η-deep sequence of exact hard-ReLU GD derivatives and its comparison with the limiting training-flow derivative. Section 7 draws the claim-by-claim boundary. Our results concern deterministic full-batch dynamics over a fixed finite horizon, with finitely many separated stable same-direction transverse events. They exclude grazing, sliding, simultaneous events, chattering, and Zeno behavior, and do not establish prevalence in modern large networks. Our contributions are: 1. a learning-limit decomposition, expressed through speed-normalized atomic pathwise curvature, separating exact nonresonant program derivatives from the full flow derivative; 2. exact one- and multi-event geometry, including low-rank error, log-volume, event-budget, and transported-cancellation formulas; 3. no-cancellation under global convexity, a globally strongly convex residual-ReLU arbitrary reciprocal-ratio family, and a residual/downstream ReLU event law; and 4. parameter and reverse-mode extensions, verified on a controlled two-parameter residualReLU training map. Figure 1 summarizes the framework.

2

S ETUP AND THE NONCOMMUTING LIMIT

Let {Rα } be the activation regions met near a reference trajectory. The continuous loss L : Rd → R has a C 2 extension Lα on each region, with fα = −∇Lα ,

Aα = Dfα = −Hα .

The solution θ(t) = φt (θ0 ) meets interfaces Σj = {gj = 0} at 0 < t1 < · · · < tm < T . Write nj = ∇gj (zj ), νj = nj /∥nj ∥, and orient gj from the incoming to the outgoing region. We assume a stable, separated itinerary and same-direction transversality, − n⊤ j fj (zj ) ≥ c0 ,

+ n⊤ j fj (zj ) ≥ c0 .

2

(2)

Preprint

A Differentiate first or take the limit first?

B

Pathwise curvature has event atoms

C Interface curvature controls normal transfer same-direction transverse crossing

d = Hac (θ(t)) dt βj

η↓0

Θη, T

+ ∑ ⊤ − νj νj⊤ δtj ν f j j j

φT

β>0

β=0

β<0

positive atom

no atom

negative atom

0 < λ⟂ < 1 attenuation

λ⟂ = 1 invisible

λ⟂ > 1 amplification

speed-normalized event atom

D DΘη, T = Jη,AD T exact at fixed nonresonant η

D η↓0

Jreg

≠

regional density Hac (θ(t)) dt

DφT dZ = −d Z −

event-free limit vs. flow derivative with event transfers

⟹ Z(t tj j+ ) = Ξj Z(tj− )

t

λ ⟂ (Ξ) =

ν⊤f + β =1− ⊤ − ν⊤f − ν f

Figure 1: The discrete derivative limit omits atomic event curvature. (a) Fixed-step AD exactly differentiates each nonresonant GD program considered here; its limit is Jreg , while the flow derivative also contains event transfers. (b) Pathwise curvature consists of a regional density and speednormalized event atoms. (c) Under same-direction transversality, positive, zero, and negative atomic curvature respectively attenuate, preserve, and amplify the direct normal mode. This orientation is local to each event; it does not require all units to activate. Grazing, sliding, simultaneous hits, chattering, Zeno behavior, and terminal-time events are excluded. Appendix A gives the complete assumptions. For T = N η, hard-branch GD and its branchwise derivative are η θk+1 = θkη + ηfαk (θkη ),
AD,η Jk+1 = I + ηAαk (θkη ) JkAD,η ,

(3) J0AD,η = I.

(4)

η Let Θη,T (θ0 ) = θN and let Θη (t; θ0 ) be the piecewise-linear interpolation of the iterates. Set AD,η AD Jη,T = JN . If no evaluated iterate lies on an interface, its branch sequence is locally fixed and AD Jη,T = DΘη,T (θ0 ).

(5)

At an exact hit, software returns a selected product although the endpoint map need not be classically differentiable. − Let Φj propagate Ż = Aαj (θ(t))Z between t+ j and tj+1 , with t0 = 0, tm+1 = T , and define

Jreg = Φm Φm−1 · · · Φ0 .

(6)

Theorem 1 (Noncommuting hard-ReLU learning limit). Under the assumptions above, for all sufficiently small nonresonant step sizes, sup ∥Θη (t; θ0 ) − φt (θ0 )∥ ≤ Cη,

(7)

0≤t≤T

and, along such step sizes, AD DΘη,T = Jη,T −→ Jreg .

(8)

The limiting flow derivative is DφT = Φm Ξm Φm−1 · · · Φ1 Ξ1 Φ0 ,

Ξj = I +

(fj+ − fj− )n⊤ j − n⊤ j fj

.

(9)

Hence state convergence alone does not guarantee derivative convergence. Smooth Euler estimates apply between isolated event tubes. Transversality localizes each crossing to O(η); finitely many straddling factors remain I + O(η), while perturbing the flow event time AD produces Ξj . Appendix F proves the estimates. In particular, ∥Jη,T − Jreg ∥ = o(1), and it is O(η) for locally Lipschitz regional Hessians; the distinct gap ∥Jreg − DφT ∥ may remain O(1). Proposition 2 (Transported cancellation, volume, and event budget). For any submultiplicative e j = Q−1 Ξj Qj , Sej = Ξ e j − I, and matrix norm normalized by ∥I∥ = 1, set Qj = Φj−1 · · · Φ0 , Ξ j 3

Preprint

ET =

P

j ∥Sj ∥. Then

e

−1 em · · · Ξ e1, Jreg DφT = Ξ e e 1 = I, DφT = Jreg ⇐⇒ Ξm · · · Ξ X νj⊤ fj+ log | det DφT | − log | det Jreg | = log rj , rj = ⊤ − , νj fj j −1 ∥Jreg DφT − I∥ ≤ eET − 1, X −1 Jreg DφT = I + Sej + R2 ,

(10) (11) (12) (13) ET

∥R2 ∥ ≤ e

− 1 − ET .

(14)

j

Thus

Q

j rj ̸= 1 is sufficient for mismatch, but

Q

j rj = 1 is not sufficient for equality.

Appendix G proves the ordered factorization. Within the verified one-step class of Appendix K, the derivative-limit defect persists for event-blind schemes regardless of regional order. The result does not cover unverified or event-localizing schemes.

3

ATOMIC CURVATURE AND EXACT EVENT GEOMETRY

3.1

F ROM EVENT TIME TO ATOMIC CURVATURE

Consider one transverse crossing at time t∗ . If ζ − is an incoming perturbation evaluated at the nominal clock, then the perturbed trajectory reaches the interface at t∗ + δt∗ , where n⊤ ζ − . (15) n⊤ f − The two trajectories therefore spend different amounts of time under the incoming and outgoing vector fields. Returning both perturbations to the same clock gives the classical saltation update δt∗ = −

ζ + = Ξζ − ,

Ξ=I+

(f + − f − )n⊤ . n⊤ f −

(16)

For a continuous piecewise-smooth loss, tangential derivatives agree across the interface. With ν = n/∥n∥, there is a scalar β such that βνν ⊤ ν⊤f + , λ⊥ (Ξ) = ⊤ − . (17) ⊤ − ν f ν f Thus the direct event update is a rank-at-most-one perturbation of the identity (rank one when β ̸= 0) and acts only in the event-normal mode. [∇L] = βν,

Ξ=I−

The spatial distributional Hessian is D2 L = Hac dθ + βνν ⊤ Hd−1 Σ. (18) The first term is regional curvature; the second is curvature concentrated on the activation interface. The correct pathwise object must also encode how quickly the trajectory reaches that interface. Proposition 3 (Speed-normalized pathwise curvature). Along the reference itinerary, define the finite matrix-valued measure m X βj dKθ (t) = Hα(t) (θ(t)) dt + ν ν ⊤ δ (dt). (19) ⊤ f − j j tj ν j=1 j j Under the left/prepoint convention Z

dKθ (s) Z(s− ),

Z(t) = I −

(20)

(0,t]

the unique right-continuous solution satisfies − Z(t+ Z(T ) = DφT . (21) j ) = Ξj Z(tj ), Deleting the atoms from Kθ gives the terminal value Jreg , which is the limit of branchwise discrete AD in Theorem 1. 4

Preprint

The prepoint convention is essential. Equation (19) is an event-time-induced, speed-normalized pathwise measure, not a convention-free pullback of the spatial Radon measure in (18). Nor does one obtain Ξj by exponentiating the bare interface mass. The atom acts as the linear jump prescribed in (20). The spatial BV decomposition and saltation formula are classical (Demengel, 1984; Ambrosio et al., 2000; Hiskens & Pai, 2000; Kong et al., 2024); the learning-limit content is that the discrete derivative sequence retains only the absolutely continuous part. The complete proof is in Appendix E. 3.2

O NE EVENT: EXACT LOW- RANK DISCREPANCY

Proposition 4 (One-event rank-one geometry). For one transverse event, let ∆T = DφT − Jreg . Then β (22) ∆T = − ⊤ − (Φ1 ν)(ν ⊤ Φ0 ). ν f If β ̸= 0, then rank(∆T ) = 1 and ∥∆T ∥2 = ∥∆T ∥F =

|β| ∥Φ1 ν∥2 ∥Φ⊤ 0 ν∥2 . |ν ⊤ f − |

(23)

⊥ Its kernel is (Φ⊤ 0 ν) , and its image is span(Φ1 ν). For a scalar outer objective with gT = ∇q(φT (θ0 )), β ⊤ ⊤ ⊤ (24) ∆⊤ T gT = − ⊤ − (ν Φ1 gT )Φ0 ν. ν f Hence the outer-gradient discrepancy vanishes exactly when β = 0 or gT ⊥ Φ1 ν. If V is uniform on the unit sphere in Rd , then E∥∆T V ∥22 = ∥∆T ∥2F /d. (25)

The result separates event strength, inverse crossing speed, incoming alignment, and outgoing transport. It does not claim that random network trajectories typically realize a large value. Appendix E gives the rank-one and expectation calculations.

4

C ONVEXITY AND R E LU EVENT STRUCTURE

4.1

C ONVEXITY FIXES THE DIRECTION OF TRANSFER

Theorem 5 (Convex attenuation and no complete cancellation). Suppose, in addition to Theorem 1, that L is globally convex. At every traversed interface, [∇L]j = βj νj =⇒ βj ≥ 0,

rj = 1 −

βj ∈ (0, 1]. ⊤ νj fj−

(26)

Thus a convex interface only attenuates its direct normal first variation and is nontrivial exactly when βj > 0. Moreover, m Y det DφT = rj . (27) det Jreg j=1 If any event is strict, this product is below one, so transported events cannot cancel completely. This conclusion is volumetric, not directionwise, and uses same-direction transversality. Conversely, β < 0 gives r > 1: negative singular interface curvature is incompatible with global convexity and amplifies the normal mode. Hence the local phases are positive-curvature attenuation (β > 0), invisibility (β = 0), and nonconvex amplification (β < 0). Appendix E proves the theorem. 4.2

A R E LU EVENT LAW

Suppose one preactivation s(θ) switches for one sample and unit while all other masks remain fixed. Let q = ∇θ s ̸= 0, write Fact − Finact = a s + o(∥θ − z∥), let gy be the loss gradient at the common boundary output, and set χ = ⟨gy , a⟩. 5

Preprint

Corollary 6 (Residual/downstream ReLU event law). With empirical averaging factor γ and oriented normal n = σq, where σ = 1 for activation and −1 for deactivation, [∇θ L] = γχn,

β = γχ∥q∥.

(28)

γχ∥q∥2 . q⊤ f −

(29)

For inactive-to-active crossing with q ⊤ f − > 0, λ⊥ (Ξ) = 1 −

Under same-direction transversality, χ > 0, χ = 0, χ < 0 give attenuation, invisibility, and amplification. For squared loss, χ is the residual projected onto the switching unit’s downstream output direction. This is a single-simple-event statement; simultaneous switches and a genericity claim are excluded. Appendix E proves it by differentiating the adjacent network graphs. 4.3

A STANDARD RESIDUAL -R E LU FAMILY

For one constant input, use the standard residual-ReLU graph and half-squared risk   θ Fθ (1) = , Lm,c (θ) = 12 ∥Fθ (1) − (m, c)∥22 . ReLU(θ)

(30)

Theorem 7 (Residual-ReLU phase theorem and uniform ratios). Fix m > 0, c > −m, T > 0. For every θ0 ∈ UT = (m(1 − eT ), 0), the flow crosses zero once at   m − θ0 m+c − + − + t∗ = log ∈ (0, T ), (f , f , H , H , β, r) = m, m + c, 1, 2, −c, , (31) m m and

AD Jreg = e−t∗ e−2(T −t∗ ) , DφT = rJreg , DΘη,T = Jη,T → Jreg (32) along every nonresonant mesh sequence. For c ≤ 0, the objective is globally 1-strongly convex with unique minimizer (m + c)/2. Thus −m < c < 0 gives 0 < r < 1, c = 0 gives r = 1, and c > 0 gives r > 1, where the objective is globally nonconvex owing to the interface.

For prescribed a > 0, t∗ ∈ (0, T ), r > 0, choose  (a/r, (r − 1)a/r), 0 < r ≤ 1, (m, c) = (a, (r − 1)a), r ≥ 1,

θ0 = m(1 − et∗ ).

(33)

Then the event time is fixed and min{f − (0), f + (0)} = a. In particular, for every R > 1, the globally 1-strongly convex choice m = Ra, c = −(R − 1)a satisfies DφT = R−1 Jreg ,

|Jreg | = R, |DφT |

(34)

with a transverse-speed lower bound independent of R. This multiplicative reciprocal ratio holds on the open set UT ; it is not an unbounded additive claim. The limit is for each fixed R, then η ↓ 0, not a uniform joint limit. Appendix M proves global strong convexity, the recurrence, the uniform margin, and the countable exact-hit set. Corollary 8 (Outer-gradient consequence). Under Theorem 7, along any nonresonant mesh sequence, let q ∈ C 1 (R) satisfy q ′ (φT (θ0 )) ̸= 0, let Gη = q ◦ Θη,T , G = q ◦ φT . Then DGη → q ′ (φT )Jreg ,

DG = r q ′ (φT )Jreg .

5

H YPERPARAMETERS , REVERSE MODE , AND RESOLUTION

5.1

PARAMETERS INSIDE TRAINING

(35)

Let λ ∈ Rp enter regional fields fα (θ, λ), surfaces gj (θ, λ) = 0, or the initialization θ(0; λ) = h(λ). Set Z = Dλ θ, A = Dθ f , and B = Dλ f . 6

Preprint

Theorem 9 (Parameter derivatives and the vanishing-step limit). Assume (A1)–(A5), joint regional smoothness, and persistence of the stable, separated same-direction itinerary on a parameter neighborhood. Regionally, Ż = Aα Z + Bα , Z(0) = Dh(λ). (36) At event j, with gλ,j = ∇λ gj , Zj+ = Ξj Zj− + Cj ,

Cj =

⊤ (fj+ − fj− )gλ,j − n⊤ j fj

.

(37)

At each fixed nonresonant step size, AD exactly differentiates the discrete program. As η ↓ 0 along such nonresonant step sizes, those derivatives converge to (36) continued without event jumps, whereas the limiting flow derivative obeys (37). An initialization-only parameter has B = gλ = 0; a vector-field parameter adds regional forcing; a moving surface adds Cj . These effects can coexist. The step size is not differentiated. Appendix J proves the theorem by augmenting the state with λ. 5.2

R EVERSE MODE

Corollary 10 (Event-aware adjoint). For terminal scalar objective q(θ(T ), λ), let p(T ) = ∇θ q and solve −ṗ = A⊤ α p regionally. The fixed-clock event update and parameter gradient are ⊤ + p− j = Ξj pj ,

(38)

∇λ q = qλ + Dh⊤ p(0) +

Z T 0

(fj+ − fj− )⊤ p+ j B ⊤ p dt + gλ,j . ⊤f − n j j j=1 m X

(39)

A regional adjoint that omits event jumps and accumulators returns the event-free sensitivity limit, which need not equal the limiting flow derivative. The transpose follows from the fixed-clock pairing p+⊤ ΞZ − = (Ξ⊤ p+ )⊤ Z − , not an inverse transpose. Reverse-mode AD through each fixed nonresonant discrete program remains exact. 5.3

R ESTORATION AND SMOOTHING

An event-aware sensitivity that localizes the stable itinerary and inserts consistent one-sided transfers satisfies SC b mΞ bm · · · Ξ b1Φ b 0 −→ DφT . Jη,T =Φ (40) This is a consistency construction, not a scalable training algorithm: explicit tracking requires one-sided data for every detected switch. Smoothing is guaranteed to restore the same transfer when its shrinking curvature layer is resolved. For the scalar/autonomous normal layer fτ (x) = F (x/τ ), ρ = η/τ , Appendix N proves log Jρ,R − log

a+ ≤ CF ρ + δF (R) a−

if

ρ∥F ′ ∥∞ ≤ 21 .

(41)

Thus ρ → 0, R → ∞, τ R → 0 recovers saltation. The condition is sufficient, not necessary; the sharp-first limit is directed through nonresonant step sizes and retains profile, phase, and tail dependence. No general curved-interface smoothing theorem is claimed.

6

C ONTROLLED NUMERICAL VERIFICATION

These deterministic studies verify proved formulas, not prevalence in larger networks. Plotted values come from preserved CSV/JSON/NPZ output; PyTorch and finite differences independently check the executed programs. 7

Preprint

globally strongly convex

10

10

endpoint sensitivity outer hypergradient

1

0

−1

10

1

exact DφT /Jreg = r DφT /Jη,AD T , η = 0.0002

10

−1

10

0

10

Worst-case errors under step refinement

C

exact max(r, r −1 )

reciprocal gap factor

sensitivity ratio

10

B Reciprocal sensitivity and hypergradient gaps

regionwise convex, globally nonconvex

worst relative error over r

Event transfer across the convexity split

A

10

10

1

10

0

10

−1

10

−2

10

−3

0

1

10

event speed ratio r

−1

10

0

10

1

10

−2

event speed ratio r state

J AD → Jreg

J SC → DφT

10

−3

step size η ↓

event-aware outer gradient

J AD vs. DφT

Figure 2: Convexity controls the residual-ReLU event transfer. (a) The endpoint-sensitivity ratio follows r across globally strongly convex attenuation, the C 1 interface, and nonconvex amplification, with fixed speed margin. (b) Endpoint- and outer-gradient reciprocal sensitivity ratios equal max{r, r−1 } in the continuum limit. (c) State, regional-AD, and event-aware errors vanish under refinement, while ordinary AD remains separated from the flow derivative for r ̸= 1. Lines connect deterministic values. Residual-ReLU phases. m=

For the graph Fθ (1) = (θ, ReLU(θ)), set

1 , min{1, r}

c = (r − 1)m,

θ0 = m(1 − e0.3 ),

T = 0.8.

The event occurs at 0.3, its minimum one-sided speed is one for every r, and DφT = rJreg . We use r ∈ {1/40, 1/20, 1/10, 1/5, 1/2, 1, 2, 5, 10, 20, 40} and eight nonresonant step sizes. All 88 runs have one crossing. Figure 2 confirms attenuation, invisibility, and amplification, including the globally strongly convex r < 1 family. At η = 2 × 10−4 , the worst relative sensitivity-ratio and outer-gradient-ratio errors are 2.31 × 10−4 and 1.29 × 10−3 . PyTorch AD agrees with the analytic branch product within 3.82 × 10−14 ; finite-difference errors are at most 3.37 × 10−8 . Two parameters and a moving threshold.

We next use

F(u,v),λ (1) = (u, v, ReLU(u − λ) + 0.5v),

y = (1, 0.2, 1),

with T = 0.8, λ = 0, (u0 , v0 ) = (1 − e0.3 , 0). The prescribed trajectory has one verified event, with normal speeds 1 and 1.91244, and β = −0.912441. At η = 2 × 10−4 , the state error is 8.24 × 10−5 ; AD AD ∥Jη,T − Jreg ∥2 = 6.79 × 10−5 while ∥Jη,T − DφT ∥2 = 0.26865, and event correction reduces −4 the latter to 1.23 × 10 . The analogous moving-threshold errors are 4.09 × 10−5 , 0.36260, and 1.10 × 10−4 . All independent forward/adjoint, PyTorch, and finite-difference checks satisfy the reported tolerances; Appendix O gives every value and tolerance.

8

Preprint

Two-parameter residual-ReLU ERM: state convergence with persistent derivative gaps

−1

10

−2

endpoint state AD → Jreg AD → DφT

10

10

event-aware → DφT

−3

−4

Euclidean error

spectral / Euclidean error

10

(b) 10

−1

10

−2

Threshold sensitivity ∂λ θT

AD → Zreg AD → Zflow

10

−3

10

−4

event-aware → Zflow

(c) terminal-loss derivative

Initialization sensitivity

(a)

0.150 0.125 0.100 0.075 0.050 0.025 0.000

10

−3

10

−2

step size η

10

−3

10

−2

Forward = adjoint (η = 0.0002)

discrete reverse AD regional limit

∂ u0

∂v0

flow sensitivity event-aware

∂λ

step size η

Figure 3: Matrix and parameter limits omit the same event. States approach the flow endpoints, while branchwise derivatives approach regional limits and remain separated from the flow derivatives; event-aware forward and adjoint calculations restore both.

7

R ELATED WORK AND NOVELTY BOUNDARY

Hybrid and fixed-program sensitivity. Event-time derivatives, saltation, hybrid adjoints, and switch-aware schemes are classical (Hiskens & Pai, 2000; Tolsma & Barton, 2002; Taringoo & Caines, 2009; Corner et al., 2020; Stewart, 1990; Dieci & Lopez, 2012). Closest here, Stewart & Anitescu (2010) prove persistent mesh-refinement gradient errors for discontinuous ODE simulations. Conservative-field and generalized-derivative theories instead characterize selections for nonsmooth functions or fixed programs (Bolte & Pauwels, 2020; Bolte et al., 2022; Park et al., 2024). In our AD setting, Jη,T and DφT are classical derivatives of a nonresonant program and a stable-itinerary flow map; we identify neither with a particular generalized selection. Program depth grows as T /η. Learning boundary. GD–flow state comparison and ReLU boundary curvature are established prior art (Elkabetz & Cohen, 2021; Duersch et al., 2024). At the reference point, our pair is limη↓0, η nonresonant DΘη,T (θ0 ) = Jreg versus DφT (θ0 ) = Φm Ξm · · · Ξ1 Φ0 . We characterize when they differ, including transported cancellation, no-cancellation under global convexity, and an arbitrary reciprocal ratio in a globally strongly convex residual-ReLU risk. Appendix L gives an object-byobject comparison.

8

L IMITATIONS AND CONCLUSION

Limitations. We study deterministic full-batch, finite-horizon dynamics with continuous piecewiseC 2 objectives and a stable finite itinerary of separated same-direction transverse events. This excludes grazing, sliding, simultaneous or created events, chattering, and Zeno behavior. The complete smoothing theorem is scalar or assumes an autonomous normal reduction. Our controlled systems do not establish prevalence in large networks, and explicit event tracking is a diagnostic rather than a scalable training method. Conclusion. At every fixed nonresonant step size, automatic differentiation can exactly differentiate the executed hard-ReLU GD program. Yet differentiation and the vanishing-step flow limit need not commute: the discrete derivative limit is event-free, whereas the flow derivative contains finite, speed-normalized event transfers. One nonzero event jump gives a rank-one discrepancy. For a globally convex loss, any strict event prevents complete cancellation. A family of globally 1strongly convex residual-ReLU risks nevertheless realizes arbitrarily large reciprocal sensitivity ratios at a fixed transverse margin; the same ratio reaches the corresponding initialization outer gradient. More generally, the event calculus extends to parameter sensitivities and adjoints. Under the stated scalar/autonomous-normal and localization conditions, resolved smoothing or event-aware differentiation restores the flow sensitivity. Open questions include stochastic updates, changing itineraries, cancellation statistics, and scalable aggregate estimation of event contributions. 9

Preprint

R EPRODUCIBILITY STATEMENT The appendix states every assumption used by the theorems, gives complete proofs, and records the numerical definitions. Numerical marks are read from preserved JSON or NPZ arrays; analytic curves and schematics are regenerated from the stated formulas. No plotted value is digitized from a raster image.

R EFERENCES Mark Ainsworth and Yeonjong Shin. Plateau phenomenon in gradient descent training of ReLU networks: Explanation, quantification, and avoidance. SIAM Journal on Scientific Computing, 43(5):A3438–A3468, 2021. doi: 10.1137/20M1353010. URL https://epubs.siam.org/ doi/10.1137/20M1353010. Luigi Ambrosio, Nicola Fusco, and Diego Pallara. Functions of Bounded Variation and Free Discontinuity Problems. Oxford Mathematical Monographs. Oxford University Press, Oxford, 2000. doi: 10.1093/oso/9780198502456.001.0001. URL https://academic.oup.com/ book/53762. Jérôme Bolte and Edouard Pauwels. A mathematical model for automatic differentiation in machine learning. In Advances in Neural Information Processing Systems, volume 33, pp. 10809–10819. Curran Associates, Inc., 2020. URL https://arxiv.org/abs/2006.02080. Jérôme Bolte, Edouard Pauwels, and Samuel Vaiter. Automatic differentiation of nonsmooth iterative algorithms. In Advances in Neural Information Processing Systems, volume 35, pp. 26404–26417, 2022. doi: 10.52202/068431-1915. URL https://proceedings.neurips.cc/paper_files/paper/2022/hash/ a9077da44185792cb63599cc9e0357bc-Abstract-Conference.html. Samuel A. Burden, S. Shankar Sastry, Daniel E. Koditschek, and Shai Revzen. Event-selected vector field discontinuities yield piecewise-differentiable flows. SIAM Journal on Applied Dynamical Systems, 15(2):1227–1267, 2016. doi: 10.1137/15M1016588. URL https://epubs.siam. org/doi/10.1137/15M1016588. Ricky T. Q. Chen, Brandon Amos, and Maximilian Nickel. Learning neural event functions for ordinary differential equations. In The Ninth International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=kW_zpEmMLdP. Sebastien Corner, Adrian Sandu, and Corina Sandu. Adjoint sensitivity analysis of hybrid multibody dynamical systems. Multibody System Dynamics, 49:395–420, 2020. doi: 10. 1007/s11044-020-09726-0. URL https://link.springer.com/article/10.1007/ s11044-020-09726-0. Françoise Demengel. Fonctions à hessien borné. Annales de l’Institut Fourier, 34(2):155–190, 1984. doi: 10.5802/aif.969. URL https://www.numdam.org/item/AIF_1984__34_ 2_155_0/. Mario di Bernardo, Christopher J. Budd, Alan R. Champneys, and Piotr Kowalczyk. Piecewisesmooth Dynamical Systems: Theory and Applications, volume 163 of Applied Mathematical Sciences. Springer London, 2008. doi: 10.1007/978-1-84628-708-4. URL https://link. springer.com/book/10.1007/978-1-84628-708-4. Luca Dieci and Luciano Lopez. A survey of numerical methods for IVPs of ODEs with discontinuous right-hand side. Journal of Computational and Applied Mathematics, 236(16):3967–3991, 2012. doi: 10.1016/j.cam.2012.02.011. URL https://www.sciencedirect.com/science/ article/pii/S0377042712000684. Jed A. Duersch, Tommie A. Catanach, Alexander Safonov, and Jeremy Wendt. Curvature in the looking-glass: Optimal methods to exploit curvature of expectation in the loss landscape, 2024. URL https://arxiv.org/abs/2411.16914. arXiv preprint, v1. 10

Preprint

Omer Elkabetz and Nadav Cohen. Continuous vs. discrete optimization of deep neural networks. In Advances in Neural Information Processing Systems, volume 34, pp. 4947–4960, 2021. URL https://papers.nips.cc/paper/2021/hash/ 274ad4786c3abca69fa097b85867d9a4-Abstract.html. A. F. Filippov. Differential Equations with Discontinuous Righthand Sides, volume 18 of Mathematics and Its Applications. Kluwer Academic Publishers, Dordrecht, 1988. doi: 10.1007/978-94-015-7793-9. URL https://link.springer.com/book/10.1007/ 978-94-015-7793-9. Santos Galán, William F. Feehery, and Paul I. Barton. Parametric sensitivity functions for hybrid discrete/continuous systems. Applied Numerical Mathematics, 31(1):17–47, 1999. doi: 10.1016/S0168-9274(98)00125-1. URL https://doi.org/10.1016/S0168-9274(98) 00125-1. William W. Hager. Runge–kutta methods in optimal control and the transformed adjoint system. Numerische Mathematik, 87:247–282, 2000. doi: 10.1007/s002110000178. URL https:// link.springer.com/article/10.1007/s002110000178. Ian A. Hiskens and M. A. Pai. Trajectory sensitivity analysis of hybrid systems. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, 47(2):204–220, 2000. doi: 10.1109/81.828574. URL https://ieeexplore.ieee.org/document/828574. Arnulf Jentzen and Adrian Riekert. Convergence analysis for gradient flows in the training of artificial neural networks with ReLU activation. Journal of Mathematical Analysis and Applications, 517 (2):126601, 2023. doi: 10.1016/j.jmaa.2022.126601. URL https://www.sciencedirect. com/science/article/pii/S0022247X22006151. Sham M. Kakade and Jason D. Lee. Provably correct automatic sub-differentiation for qualified programs. In Advances in Neural Information Processing Systems, volume 31, pp. 7125–7135. Curran Associates, Inc., 2018. URL https://arxiv.org/abs/1809.08530. Nathan J. Kong, J. Joe Payne, James Zhu, and Aaron M. Johnson. Saltation matrices: The essential tool for linearizing hybrid dynamical systems. Proceedings of the IEEE, 112(6):585– 608, 2024. doi: 10.1109/JPROC.2024.3440211. URL https://ieeexplore.ieee.org/ document/10638633. Wonyeol Lee, Hangyeol Yu, Xavier Rival, and Hongseok Yang. On correctness of automatic differentiation for non-differentiable functions. In Advances in Neural Information Processing Systems, volume 33, pp. 6719–6730. Curran Associates, Inc., 2020. URL https://arxiv. org/abs/2006.06903. Yingbo Ma, Vaibhav Dixit, Michael J. Innes, Xingjian Guo, and Christopher Rackauckas. A comparison of automatic differentiation and continuous sensitivity analysis for derivatives of differential equation solutions. In 2021 IEEE High Performance Extreme Computing Conference (HPEC), pp. 1–9. IEEE, 2021. doi: 10.1109/HPEC49654.2021.9622796. URL https:// ieeexplore.ieee.org/document/9622796. Armin Nurkanović, Mario Sperl, Sebastian Albrecht, and Moritz Diehl. Finite elements with switch detection for direct optimal control of nonsmooth systems. Numerische Mathematik, 156(3):1115– 1162, 2024. doi: 10.1007/s00211-024-01412-z. URL https://link.springer.com/ article/10.1007/s00211-024-01412-z. Sejun Park, Sanghyuk Chun, and Wonyeol Lee. What does automatic differentiation compute for neural networks? In The Twelfth International Conference on Learning Representations, pp. 52559– 52585, 2024. URL https://proceedings.iclr.cc/paper_files/paper/2024/ hash/e8711daef520be07cb9852390c673de8-Abstract-Conference.html. J. M. Sanz-Serna. Symplectic runge–kutta schemes for adjoint equations, automatic differentiation, optimal control, and more. SIAM Review, 58(1):3–33, 2016. doi: 10.1137/151002769. URL https://epubs.siam.org/doi/10.1137/151002769. 11

Preprint

Radu Serban and Antonio Recuero. Sensitivity analysis for hybrid systems and systems with memory. Journal of Computational and Nonlinear Dynamics, 14(9):091006, 2019. doi: 10.1115/1.4044028. URL https://doi.org/10.1115/1.4044028. David Stewart. A high accuracy method for solving ODEs with discontinuous right-hand side. Numerische Mathematik, 58:299–328, 1990. doi: 10.1007/BF01385627. URL https://link. springer.com/article/10.1007/BF01385627. David E. Stewart and Mihai Anitescu. Optimal control of systems with discontinuous differential equations. Numerische Mathematik, 114(4):653–695, 2010. doi: 10.1007/s00211-009-0262-2. URL https://link.springer.com/article/10.1007/s00211-009-0262-2. Farzin Taringoo and Peter E. Caines. The sensitivity of hybrid systems optimal cost functions with respect to switching manifold parameters. In Hybrid Systems: Computation and Control, pp. 475–479. Springer, 2009. doi: 10.1007/978-3-642-00602-9 38. URL https://doi.org/10. 1007/978-3-642-00602-9_38. John E. Tolsma and Paul I. Barton. Hidden discontinuities and parametric sensitivity calculations. SIAM Journal on Scientific Computing, 23(6):1861–1874, 2002. doi: 10.1137/S106482750037281X. URL https://epubs.siam.org/doi/10.1137/ S106482750037281X. Hong Zhang, Shrirang Abhyankar, Emil M. Constantinescu, and Mihai Anitescu. Discrete adjoint sensitivity analysis of hybrid dynamical systems with switching. IEEE Transactions on Circuits and Systems I: Regular Papers, 64(5):1247–1259, 2017. doi: 10.1109/TCSI.2017.2651683. URL https://doi.org/10.1109/TCSI.2017.2651683.

12

Preprint

A

A SSUMPTIONS AND NOTATION

Let θ ∈ Rd . In a neighborhood of a compact tube K around the reference trajectory, a finite family of open regions {Rα }α forms a piecewise-smooth partition. On Rα , write L(θ) = Lα (θ),

fα (θ) = −∇Lα (θ),

Aα (θ) = Dfα (θ) = −Hα (θ).

The following are the common base assumptions for the sharp-interface results; individual theorems state any additions. (A1) Each Lα has a C 2 extension to a neighborhood of Rα ∩ K. The one-sided traces agree on common interfaces, so L is continuous although its gradient may jump. (A2) Every interface met by the reference trajectory is locally a regular C 2 hypersurface Σj = {gj = 0}, with nj (θ) := ∇gj (θ) ̸= 0. The defining function is oriented so that the trajectory moves from gj < 0 to gj > 0. At the event state zj , the associated unit normal is νj = nj (zj )/∥nj (zj )∥. We use nj in formulas invariant to rescaling of gj , and νj whenever a coefficient or spectral statement requires a unit normal. (A3) The patched Carathéodory solution θ(t) = φt (θ0 ) exists on [0, T ], remains in K, and has exactly m < ∞ events 0 < t1 < · · · < tm < T,

zj = θ(tj ).

Consecutive events and the endpoints of the time interval are separated by at least ∆t > 0. Each event has an isolating neighborhood containing no other interface met by the trajectory. On a neighborhood of θ0 , this event count, order, separation, and itinerary persist. (A4) The crossings are uniformly same-direction transverse. For some c0 > 0, after shrinking the isolating neighborhoods if necessary, nj (θ)⊤ fj− (θ) ≥ c0 ,

nj (θ)⊤ fj+ (θ) ≥ c0

on the respective one-sided neighborhoods. Here “same-direction” is local to the oriented event: the normal points from the incoming region to the outgoing region. It does not require all ReLU units to switch in one common global direction; it excludes grazing, sliding, and opposing one-sided normal velocities at that crossing. (A5) There is no grazing, sliding, chattering, simultaneous multi-surface hit, event creation, Zeno accumulation, or event at the terminal time T . These assumptions give a unique piecewise-classical trajectory and a locally stable event order. At an exact discrete boundary landing, the state update selects one of the adjacent one-sided fields whose oriented normal component remains at least c0 . This condition is needed for the state estimate. Whenever ordinary AD is identified with a classical derivative of the discrete endpoint map, we additionally require that no iterate land exactly on an interface. At an exact hit, software AD still returns a selected branch product, but that product need not be a classical derivative. For T = N η, let θkη denote the GD iterates and define the endpoint map and its piecewise-linear interpolation by η Θη,T (θ0 ) := θN ,

Θη (t; θ0 ) := the interpolation of {θkη }N k=0 .

AD We use Jη,T for ordinary executed-branch autodiff, Jreg for the event-free regional continuum product, SC DφT for the true flow derivative, and Jη,T for the event-aware corrected sensitivity. The matrix Φj − propagates the regional variational equation between t+ j and tj+1 , with t0 = 0 and tm+1 = T .

Rate assumption. Under (A1), the regional Hessians are uniformly continuous on the relevant compact sets. We write  ωH (r) = max sup ∥Hα (x) − Hα (y)∥ : x, y ∈ K ∩ Rα , ∥x − y∥ ≤ r . α

Thus ωH (r) → 0 as r ↓ 0. An O(η) sensitivity rate additionally requires locally Lipschitz regional Hessians (Lα ∈ C 2,1 ) and first-order event localization. 13

Preprint

B

P IECEWISE - SMOOTH R E LU OBJECTIVES

Proposition 11 (Finite-sample ReLU objectives). Consider a finite ReLU network on a finite data set, with a loss smooth in the network outputs. On every fixed activation-pattern cell, the empirical risk is smooth (and polynomial for squared loss). The risk is continuous across activation boundaries. At a point where exactly one preactivation vanishes and its parameter gradient is nonzero, the boundary is locally a regular hypersurface and the one-sided parameter gradients have well-defined traces. Proof. Fixing every ReLU mask replaces each activation by either zero or its preactivation. The network output is then a finite composition of affine and multilinear operations in the parameters. It is polynomial on that mask cell, and so is a squared empirical risk. ReLU itself is continuous, hence the network and risk traces agree across a mask boundary. If a single preactivation s(θ) vanishes with ∇s ̸= 0, the implicit-function theorem makes {s = 0} a regular hypersurface. Smooth regional formulas provide the gradient traces. The output is generally not piecewise affine in all network parameters. For a two-layer unit a ReLU(w⊤ x + b), it is bilinear on an active cell, and the squared risk may have degree four. Only piecewise smoothness is used here. For a shallow squared-loss sample, orient s(θ) = q ⊤ θ + b from inactive to active. At s = 0, with output coefficient c and residual r = Fθ (x) − y, direct differentiation gives [∇L] = rcq up to the empirical averaging factor. The jump is normal to the activation surface. A simultaneous change of several samples or units is outside the single-interface result.

C

E VENT- TIME DERIVATIVE AND SALTATION

For one event, let z be the nominal impact at time t∗ , let n = ∇g(z), and let ζ − be the incoming perturbation evaluated at the nominal time. The perturbed event time t∗ + δt obeys 0 = n⊤ (ζ − + f − δt),

δt = −

n⊤ ζ − . n⊤ f −

The perturbed impact point changes by ζ − + f − δt, while the outgoing segment loses time δt. Returning to the nominal clock gives   (f + − f − )n⊤ + − − + ζ = ζ + (f − f )δt = I + ζ −. n⊤ f − This is the saltation matrix without a state reset. The event time is C 1 by the implicit-function theorem because n⊤ f − ̸= 0. Iterating this calculation over the stable itinerary gives the hybrid product in Theorem 13.

D

D ISTRIBUTIONAL H ESSIAN AND SINGULAR INTERFACE CURVATURE

Proposition 12 (Distributional Hessian). Let a C 2 hypersurface Σ split a neighborhood into minus and plus regions. Suppose L is continuous and has C 2 extensions L± with one-sided gradients. For the unit normal ν from minus to plus, there is a scalar trace β such that [∇L] = βν, and, as a matrix-valued distribution, D2 L = H − 1− dx + H + 1+ dx + βνν ⊤ Hd−1 ↾Σ . Proof. The traces of L+ and L− agree on Σ. Their tangential derivatives therefore agree, so their gradient difference is normal: [∇L] = βν. Integrating each component of ∇L by parts on the two regions gives the regional Hessians as volume terms. The outward normal of the minus region is ν, while that of the plus region is −ν; the boundary terms combine as [∇L] ⊗ ν = βνν ⊤ , which is the stated measure. 14

Preprint

The decomposition is standard BV calculus. Its role here is to locate the curvature omitted by regional Hessian products. Pulling the interface term through a transverse trajectory must account for the changing normal speed. In the resolved layer of Appendix N, the scalar identity d log Z = d log f converts the concentrated curvature into f + /f − , exactly the normal saltation factor. A naive exponential of the bare hypersurface mass would be incorrect.

E

ATOMIC CURVATURE AND EVENT GEOMETRY

This section proves the pathwise-curvature, one-event, convexity, and ReLU-event statements used in Sections 3 and 4. The Stieltjes convention below is part of the definition; it is important because the vector field itself jumps at an event. E.1

P REPOINT S TIELTJES FORMULATION

Along the reference trajectory, write H(t) = Hα(t) (θ(t)),

aj =

βj , ⊤ νj fj−

and define the finite matrix-valued measure Z X Kθ (B) = H(t) dt + aj νj νj⊤ . B

(42)

j:tj ∈B

Here and below, event data are evaluated at the nominal event state. A right-continuous matrix function Z, with left limits, solves the prepoint Stieltjes system when Z Z(t) = I − dKθ (s) Z(s− ). (43) (0,t]

Thus an atom acts linearly on the left limit. Equation (43) is not an exponential convention for matrix atoms. Proof of Proposition 3.

Off the event times, the absolutely continuous part of (43) gives Ż(t) = −H(t)Z(t) = Aα(t) (θ(t))Z(t).

At tj , subtracting the left limit from the right value gives − − Z(t+ j ) − Z(tj ) = −Kθ ({tj })Z(tj )

βj = − ⊤ − νj νj⊤ Z(t− j ). νj fj Hence Z(t+ j )=

I−

βj νj νj⊤ νj⊤ fj−

! − Z(t− j ) = Ξj Z(tj ).

Solving the regional linear equations and applying these finitely many jumps in chronological order yields Z(T ) = Φm Ξm Φm−1 · · · Φ1 Ξ1 Φ0 = DφT . Existence and uniqueness follow from the same finite construction. If the atoms are removed from Kθ , the solution remains continuous at each event and its terminal value is Φm · · · Φ0 = Jreg . Theorem 1 identifies this atom-free solution with the vanishing-step branchwise-AD limit. The spatial measure in Proposition 12 and the pathwise measure in (42) are related but not identical objects. Since the trajectory velocity has two one-sided values at the interface, a bare distributional pullback does not select a crossing speed. The incoming-speed normalization and the prepoint convention in (43) encode the fixed-clock event derivative. In particular, exp(−aj νj νj⊤ ) is generally different from I − aj νj νj⊤ = Ξj . 15

Preprint

E.2

O NE - EVENT RANK - ONE GEOMETRY

Proof of Proposition 4.

For one event, Theorem 1 and (17) give ∆T := DφT − Jreg = Φ1 (Ξ − I)Φ0 β = − ⊤ − (Φ1 ν)(ν ⊤ Φ0 ). ν f

(44)

Both regional fundamental matrices are invertible, so Φ1 ν ̸= 0 and Φ⊤ 0 ν ̸= 0. If β ̸= 0, (44) is therefore a nonzero outer product and has rank one. A rank-one matrix uv ⊤ has one nonzero singular value ∥u∥2 ∥v∥2 . This proves |β| ∥∆T ∥2 = ∥∆T ∥F = ⊤ − ∥Φ1 ν∥2 ∥Φ⊤ 0 ν∥2 . |ν f | The same outer-product formula gives ⊥ ker ∆T = (Φ⊤ 0 ν) , and, for gT = ∇q(φT (θ0 )),

im ∆T = span(Φ1 ν),

β ⊤ ⊤ ⊤ ∆⊤ T gT = − ⊤ − (ν Φ1 gT )Φ0 ν. ν f Since the last vector factor is nonzero, the hypergradient discrepancy vanishes exactly when β = 0 or gT ⊥ Φ1 ν. Finally, if V is uniform on the unit sphere in Rd , then E[V V ⊤ ] = I/d, and hence
∥∆T ∥2F ⊤ E∥∆T V ∥22 = tr ∆⊤ . T ∆T E[V V ] = d This is a directional statement at a fixed event, not a prevalence claim for network trajectories. E.3

C ONVEXITY AND THE SIGN OF AN EVENT

Proof of Theorem 5. Let z be an event point and let ν point from the minus to the plus region. For sufficiently small ε > 0, the points xε = z − εν and yε = z + εν lie in the adjacent smooth regions. Monotonicity of the gradient of a differentiable convex function gives
⊤ ∇L(yε ) − ∇L(xε ) (yε − xε ) ≥ 0. Divide by 2ε and pass to the one-sided traces. Since [∇L] = βν, the limit is β ≥ 0. Gradient structure gives ν ⊤ f + = ν ⊤ f − − β. Same-direction transversality makes both sides positive. Therefore ν⊤f + β = 1 − ⊤ − ∈ (0, 1], ⊤ − ν f ν f with r < 1 exactly when β > 0. This attenuates the direct normal mode; tangential modes remain eigenvectors with eigenvalue one. r=

Every regional fundamental matrix is invertible and has positive determinant by Liouville’s formula. The matrix determinant lemma gives det Ξj = rj . Taking determinants in the saltation-interleaved product and in Jreg yields m m Y X det DφT = rj , log | det DφT | − log | det Jreg | = log rj . (45) det Jreg j=1 j=1 If any convex event is strict, one rj < 1 while all other factors are at most one and positive. The product is then less than one, so DφT ̸= Jreg ; exact transported cancellation is impossible. If β < 0, the same speed identity gives r > 1 under a same-direction crossing. The trace monotonicity argument above also shows that such a downward normal-gradient jump is incompatible with global convexity. This is negative singular interface curvature. The converse statement is deliberately local: β > 0 fixes the sign of the atomic part but does not, without convex regional Hessians, prove global convexity. 16

Preprint

E.4

A R E LU EVENT IN RESIDUAL / DOWNSTREAM COORDINATES

Proof of Corollary 6. Let s(θ) be the changing preactivation and q = ∇θ s(z) ̸= 0. With every other mask fixed, write the active and inactive network outputs near z as Fact (θ) − Finact (θ) = a s(θ) + o(∥θ − z∥). The two outputs agree at z, while differentiation gives DFact (z) − DFinact (z) = aq ⊤ .

(46)

Let gy be the loss gradient at the common output and let γ be the empirical averaging factor. The parameter-gradient jump from inactive to active is
⊤ γ DFact − DFinact gy = γ⟨gy , a⟩q. Set χ = ⟨gy , a⟩. For activation, the oriented normal is n = q. For deactivation, incoming and outgoing sides are exchanged and the oriented normal is n = −q; both the gradient jump and n change sign. Thus, in either orientation, [∇θ L] = γχn,

β = γχ∥q∥.

For an inactive-to-active event, n = q, and substitution into (17) yields λ⊥ (Ξ) = 1 −

γχ∥q∥2 . q⊤ f −

Under same-direction transversality, the signs of χ and β therefore give attenuation, invisibility, and amplification as stated. For squared loss, gy is the residual, so χ is its inner product with the downstream output direction a.

F

G ENERAL STATE AND FIRST- VARIATION THEOREM

For clarity, first take T = N η; a final partial step changes none of the conclusions. The executedbranch Euler/GD recursion is η θk+1 = θkη + ηfαk (θkη ) = θkη − η∇Lαk (θkη ),

θ0η = θ0 ,

(47)

where αk is the executed region and exact hits follow the convention in Appendix A. Ordinary executed-branch AD satisfies
AD AD AD Jη,0 = I, Jη,(k+1)η = I + ηAαk (θkη ) Jη,kη . (48) On the jth smooth segment, let Φj (t, s) solve ∂t Φj (t, s) = Aαj (θ(t))Φj (t, s),

Φj (s, s) = I,

(49)

+ and abbreviate Φj = Φj (t− j+1 , tj ), with t0 = 0 and tm+1 = T .

Theorem 13 (State consistency and first-variation limits). Under (A1)–(A5), there are constants C < ∞ and η0 > 0 such that, for 0 < η < η0 , sup ∥Θη (t; θ0 ) − φt (θ0 )∥ ≤ Cη.

(50)

0≤t≤T

Moreover, AD Jη,T −→ Jreg := Φm Φm−1 · · · Φ0 ,

(51)

AD ∥Jη,T − Jreg ∥ ≤ C η + ωH (Cη) = o(1).

(52)

with the quantitative estimate

Under the rate assumption in Appendix A, the right-hand side is O(η). The terminal flow map φT is C 1 on a neighborhood of θ0 with the same event itinerary, and DφT (θ0 ) = Φm Ξm Φm−1 Ξm−1 · · · Φ1 Ξ1 Φ0 ,

17

Ξj = I +

(fj+ − fj− )n⊤ j − n⊤ j fj

,

(53)

Preprint

where all event data are evaluated at zj and nj = ∇gj (zj ). If [∇L]j = βj νj for the unit normal νj , then νj⊤ fj+ βj νj νj⊤ rj = ⊤ − . (54) Ξj = I − ⊤ − , νj fj νj fj Thus Ξj − I is symmetric with rank at most one, tangent directions have eigenvalue one, and the normal eigenvalue is rj . For d ≥ 2, ∥Ξj ∥2 = max{1, |rj |};

(55)

for d = 1, ∥Ξj ∥2 = |rj |. For fixed η, if no iterate lies on an interface, the endpoint map is C 1 near θ0 and AD Dθ0 Θη,T (θ0 ) = Jη,T .

(56)

AD At an exact hit, Jη,T remains a selected branch product but need not be a classical derivative.

Proof. On a compact regional subset, Dfα and fα are bounded. Before the first event tube, the usual smooth Euler recursion gives ek = ∥θkη − θ(kη)∥.

(57)

Near event j, Taylor expansion of the defining function yields
gj x + ηfα (x) = gj (x) + η∇gj (x)⊤ fα (x) + O(η 2 ).

(58)

ek+1 ≤ (1 + Cη)ek + Cη 2 ,

The transversality bound and the exact-hit convention make this increment at least ηc0 /2 for sufficiently small η. Smooth Euler accuracy up to entry into the isolating tube places the numerical and exact entries within O(η). Monotone progress of gj then gives one numerical crossing within O(η) of tj . A straddling step may have O(η), rather than O(η 2 ), local error, but only a uniformly bounded number of steps lies in the O(η) timing-mismatch window. Restart the smooth recursion on the outgoing side. Induction over the finite, ∆t -separated itinerary and discrete Grönwall prove (50). For the matrix limit, let Aη (t) be the step function equal to Aαk (θkη ) on [kη, (k + 1)η). Outside event windows of total length O(mη), the numerical and exact paths use the same branch, and the state estimate gives ∥Aη (t) − A(t)∥ ≤ ωH (Cη). (59) The matrices are uniformly bounded inside the event windows. Hence
∥Aη − A∥L1 (0,T ) ≤ C η + ωH (Cη) .

(60)

Continuous dependence of fundamental matrices on their coefficients in L1 , followed by the standard O(η) comparison of I + ηAk with exp(ηAk ), proves (52). Appendix C proves the single-event saltation update. The isolating neighborhoods, event separation, and transversality permit its implicit-function argument to be iterated over the stable itinerary, yielding (53). Proposition 12 gives [∇L]j = βj νj ; substitution proves (54). Tangent vectors are fixed by Ξj , while direct application to νj gives the eigenvalue rj . Symmetry then yields (55) and its one-dimensional variant. Finally, if every iterate has a strict regional sign, those signs persist on a neighborhood of θ0 . The endpoint is then a composition of C 1 regional Euler maps, and the chain rule gives (56). Exact landing removes this argument and can make the endpoint map nondifferentiable.

G

M ULTI - EVENT CANCELLATION CRITERION

Several nontrivial saltation matrices do not by themselves imply a matrix mismatch: transported event effects may cancel. Proposition 14 (Transported saltation criterion). For j = 1, . . . , m, define Qj = Φj−1 Φj−2 · · · Φ0 , 18

e j = Q−1 Ξj Qj . Ξ j

(61)

Preprint

Then −1 emΞ e m−1 · · · Ξ e1. Jreg DφT = Ξ

(62)

Consequently, ⇐⇒

DφT = Jreg

em · · · Ξ e 1 = I. Ξ

(63)

A sufficient, but not necessary, condition for mismatch is m Y νj⊤ fj+ j=1

νj⊤ fj−

̸= 1.

(64)

Equality of the scalar product to one only makes the determinant of the transported matrix product equal to one; it does not imply that the matrix product is I. Regional propagation can rotate and couple the affected directions before subsequent events. For any submultiplicative matrix norm normalized by ∥I∥ = 1, set Sej = Q−1 j Sj Qj ,

Sj = Ξj − I,

ET =

m X

∥Sej ∥.

(65)

j=1

Then the exact log-volume correction is log | det DφT | − log | det Jreg | =

m X

log rj ,

rj =

j=1

νj⊤ fj+ νj⊤ fj−

> 0,

(66)

and −1 ∥Jreg DφT − I∥ ≤

m Y

(1 + ∥Sej ∥) − 1 ≤ eET − 1,

(67)

j=1 −1 Jreg DφT = I +

m X

Sej + R2 ,

(68)

j=1

∥R2 ∥ ≤ eET − 1 − ET ≤ 12 eET ET2 .

(69)

The first-order sum contains the transported event contributions; R2 contains their ordered interactions. These are upper bounds only. No lower bound follows from ET , because transported factors may cancel. For one event, DφT − Jreg = Φ1 (Ξ1 − I)Φ0 , (70) so equality holds if and only if Ξ1 = I. In the continuous-loss gradient setting this is equivalent to equality of the one-sided gradients. A single genuine gradient jump therefore forces a matrix mismatch, although perturbation directions satisfying ν1⊤ Φ0 v = 0 are unaffected by that event. −1 Proof. Every regional fundamental matrix is invertible. Expanding Jreg DφT and cancelling regional factors from the outside inward gives −1 −1 Jreg DφT = (Q−1 m Ξm Qm ) · · · (Q1 Ξ1 Q1 ),

which proves the factorization and exact cancellation criterion. Moreover, det Ξj = 1 +

νj⊤ (fj+ − fj− ) νj⊤ fj−

=

νj⊤ fj+ νj⊤ fj−

,

so the determinant condition follows. Under (A4), every ratio is positive; taking logarithms proves (66). Expanding the ordered product in (62) gives (I + Sem ) · · · (I + Se1 ) = I +

X j

19

Sej +

m X

X

k=2 jk >···>j1

Sejk · · · Sej1 .

Preprint

Submultiplicativity therefore yields (I + Sem ) · · · (I + Se1 ) − I ≤

Y (1 + ∥Sej ∥) − 1, j

Y X ∥R2 ∥ ≤ (1 + ∥Sej ∥) − 1 − ∥Sej ∥. j

j

Using 1 + x ≤ ex proves (67) and the first inequality in (69). The final inequality follows from ex − 1 − x ≤ 12 ex x2 for x ≥ 0. For one event, invertibility of Φ0 and Φ1 makes (70) zero exactly when Ξ1 − I = 0. Since the denominator is nonzero, the latter is equivalent to f1+ = f1− , or β1 = 0.

H

E VENT- AWARE FIRST- VARIATION CONSISTENCY

Let b tηj and zbjη be numerical event times and states with the correct event count and order, and suppose
max |b tηj − tj | + ∥b zjη − zj ∥ ≤ εη . (71) j

Evaluate one-sided data at the localized event and define (fb+ − fbj− )(b nj )⊤ bη = I + j Ξ . j b− n b⊤ j fj

(72)

Let δη bound additional errors in these fields or normals. Split each straddling step, or otherwise b η denote the resulting regional AD allocate its pre- and post-event portions consistently, and let Φ j product. Define SC b ηm Ξ b ηm Φ bη Ξ bη b η bη b η Jη,T =Φ (73) m−1 m−1 · · · Φ1 Ξ1 Φ0 . Theorem 15 (Consistency of the event-aware first variation). Under (A1)–(A5), if the procedure detects the correct stable itinerary, εη → 0, and δη → 0, then SC Jη,T −→ DφT (θ0 ).

(74)

SC ∥Jη,T − DφT (θ0 )∥ ≤ C [η + ωH (Cη) + εη + δη ] .

(75)

More quantitatively,

Under the regional Hessian rate assumption and εη + δη = O(η), the error is O(η). The corrected matrix is not, in general, the derivative of unmodified hard-branch GD. It is the derivative of an event-aware solver only when that solver differentiates the localized event time and split steps. Proof. The transversality margin keeps the denominator in (72) bounded below for small η. Regional C 1 regularity of fj± and of the normal field gives b η − Ξj ∥ ≤ C(εη + δη ). ∥Ξ j The proof of Theorem 13, applied to consistently split regional intervals, gives
b η − Φj ∥ ≤ C η + ωH (Cη) + εη . ∥Φ j

(76)

(77)

All exact and numerical factors are uniformly bounded, and their number is fixed. A telescoping perturbation bound for the finite product, together with (53), proves (75) and the stated conclusions.

I

A RBITRARY- GAP CONSTRUCTION

Corollary 16 (Arbitrarily large discrete-to-flow derivative gap). Let 0 < a− < a+ and, for θ = (x, y) ∈ R × Rd−1 , define ( 1 (x − a− )2 , x ≤ 0, 1 2 La− ,a+ (x, y) = ∥y∥ + 21 (78) 1 2 2 2 2 2 (x − a+ ) + 2 (a− − a+ ), x ≥ 0. 20

Preprint

For every T > 0, the open strip  UT = (x0 , y0 ) : a− (1 − eT ) < x0 < 0

(79)

has a stable one-event itinerary before time T . On UT , both regional Hessians equal Id , and Jreg = e−T Id , DφT = e−T diag

(80) 



a+ , Id−1 . a−

(81)

Consequently, −T



∥DφT − Jreg ∥2 = e

 a+ −1 , a−

a+ ∥DφT ∥2 = . ∥Jreg ∥2 a−

(82) (83)

Thus any multiplicative gap R > 1 is obtained by taking a+ /a− = R, and any additive gap G > 0 by taking a+ /a− = 1 + GeT . If a+ /a− > eT , the true flow is expansive although ordinary branchwise AD converges to the strict contraction e−T Id . For η = T /N and any step size with no exact interface landing, AD DΘη,T = Jη,T = (1 − η)N Id −→ e−T Id .

(84)

Inserting the exact event matrix once gives SC Jη,T = (1 − η)N diag



a+ , Id−1 a−

 = DφT + O(η).

(85)

Here the inserted saltation matrix is exact. Since T = N η, (1 − η)N = e−T + O(η), so the entire O(η) difference in (85) is the ordinary Euler error of the regional dynamics. There is no eventlocalization or one-sided-data error in this construction; those additional errors are represented by εη and δη in Theorem 15. The loss is regionwise strongly convex but not globally convex. Proof. The additive constant makes the two traces agree at x = 0. On the left, ẋ = a− − x, so a point with x0 < 0 reaches the interface at t∗ (x0 ) = log

a− − x0 . a−

(86)

The strict strip inequalities are exactly 0 < t∗ (x0 ) < T . Both one-sided speeds at impact are positive, namely a− and a+ , so the crossing is stable and same-direction transverse. The regional fundamental matrices are scalar contractions whose total duration is T , while   a+ Ξ = diag , Id−1 . a− This proves the regional and true derivatives and both gap formulas. Since both regional Hessians are Id , every executed Euler Jacobian is (1 − η)Id , irrespective of the branch count. The regional factors commute with Ξ, proving the corrected formula. Finally, the normal derivative of L jumps downward from −a− to −a+ , so the continuous loss has negative singular interface curvature and is not globally convex.

J

H YPERPARAMETER SENSITIVITY AND REVERSE MODE

This section proves Theorem 9 and Corollary 10. The formulas are classical hybrid sensitivity identities; their role here is to compare the derivative of a hard-ReLU training flow with the limit of exact derivatives of its nonresonant discrete programs. 21

Preprint

J.1

PARAMETER - DEPENDENT ASSUMPTIONS

Let λ ∈ Rp . In addition to (A1)–(A5), suppose that each regional field fα (θ, λ) is C 1 jointly in state and parameter on the relevant compact tube. Write Aα = Dθ fα ∈ Rd×d ,

Bα = Dλ fα ∈ Rd×p .

The matrices Aα and Bα are uniformly continuous on that tube. Each event function gj (θ, λ) is C 2 , and the initialization is h(λ) with h ∈ C 1 . The event count, order, separation, and same-direction transversality persist on a parameter neighborhood of the reference value. The mesh, horizon, state dimension, and program depth are held fixed while differentiating with respect to λ. Let Z(t) = Dλ θ(t; λ) ∈ Rd×p , J.2

nj = ∇θ gj ∈ Rd ,

gλ,j = ∇λ gj ∈ Rp .

F ORWARD JUMP AND THE DISCRETE LEARNING LIMIT

Proof of Theorem 9.

Inside a smooth region, ordinary differentiation gives Ż = Aα Z + Bα ,

Z(0) = Dh(λ).

(87)

At event j, perturb the parameter by δλ and denote the resulting event-time change by δtj . Differentiating gj (θ(tj + δtj ; λ + δλ), λ + δλ) = 0 on the incoming side gives δtj = −

− ⊤ (n⊤ j Zj + gλ,j )δλ − n⊤ j fj

.

Returning the outgoing perturbation to the nominal clock yields Zj+ δλ = Zj− δλ + (fj− − fj+ )δtj ! ⊤ (fj+ − fj− )gλ,j (fj+ − fj− )n⊤ j − Z δλ + δλ. = I+ j − − n⊤ n⊤ j fj j fj Since this holds for every parameter perturbation, Zj+ = Ξj Zj− + Cj ,

Cj =

⊤ (fj+ − fj− )gλ,j − n⊤ j fj

.

(88)

The dimensions are Zj± , Cj ∈ Rd×p . Equivalently, augment the state by the constant parameter: y = (θ, λ),

ẏ = (fα (θ, λ), 0),

Gj (y) = gj (θ, λ).

The ordinary saltation matrix of the augmented system is   Ξ Cj Sj = j , 0 Ip

(89)

which gives a second derivation of both the sign and the block dimensions. At a fixed nonresonant Euler/GD program, local branch constancy gives the exact derivative recursion
η Zk+1 = I + ηAαk (θkη , λ) Zkη + ηBαk (θkη , λ), Z0η = Dh(λ). (90) Introduce the augmented regional coefficient and sensitivity     A(t) B(t) Z(t) A(t) = , W (t) = . 0 0 Ip The discrete factor corresponding to (90) is  I + ηAk I + ηAk = 0 22

 ηBk . Ip

Preprint

Outside event windows, the state estimate and uniform continuity imply that both Ak and Bk approach their reference regional coefficients. Inside event windows their norms are bounded and the total window length is O(η). Consequently,   ∥Aη − A∥L1 (0,T ) ≤ C η + ωA (Cη) + ωB (Cη) . The same fundamental-matrix comparison used in Theorem 13 gives   AD ∥Zη,T − Zreg (T )∥ ≤ C η + ωA (Cη) + ωB (Cη) ,

(91)

where Zreg solves (87) and is continued without a jump at each nominal event time. If Aα , Bα are locally Lipschitz, the right-hand side is O(η). The true flow derivative instead obeys (88). For one event, subtracting the event-free continuation and propagating to T gives the useful identity Zflow (T ) − Zreg (T ) = Φ1

(f + − f − )(n⊤ Z − + gλ⊤ ) . n⊤ f −

(92)

If f + ̸= f − , a parameter direction has zero event-induced gap exactly when its first-order event-time change vanishes. If f + = f − , the gap vanishes identically. The formula separates three cases. If a parameter enters only the initialization, then B = 0, gλ = 0, and events act through ΞZ − . If it enters the vector field but not the interface, then regional forcing B is accumulated and subsequently transferred by Ξ, while C = 0. If it enters the interface explicitly, the affine term C is present. These effects coexist when a parameter enters all three objects. J.3

R EVERSE MODE

Proof of Corollary 10.

For a terminal scalar objective q(θ(T ), λ), set −ṗ = A⊤ αp

p(T ) = ∇θ q(θ(T ), λ),

on each regional segment. The fixed-clock forward event map in (88) pulls covectors back by ⊤ + p− j = Ξj pj .

(93)

Indeed, + ⊤ − ⊤ + (p+ j ) Zj = (pj ) (Ξj Zj + Cj ) + ⊤ − + ⊤ = (Ξ⊤ j pj ) Zj + (pj ) Cj .

This proves that the transpose, rather than the inverse transpose, is the correct jump for the fixed-clock derivative map. Between events, d ⊤ (Z p) = B ⊤ p. dt At an event, (93) gives − ⊤ − ⊤ + (Zj+ )⊤ p+ j − (Zj ) pj = Cj pj . Integrating segment by segment and adding the jumps yields Z T m X ∇λ q = qλ + Dh(λ)⊤ p(0) + B(t)⊤ p(t) dt + Cj⊤ p+ j , 0

where Cj⊤ p+ j = gλ,j

(94)

j=1

(fj+ − fj− )⊤ p+ j − n⊤ j fj

.

(95)

If gλ,j = 0, the direct event accumulator vanishes, but the event can still change the gradient through ⊤ + the jump p− j = Ξj pj . Omitting all event terms gives the regional continuous adjoint and hence the transpose of the event-free sensitivity limit. Reverse-mode AD through every fixed nonresonant discrete program remains its exact classical derivative. 23

Preprint

K

A RESTRICTED EVENT- BLIND ONE - STEP EXTENSION

The Euler calculation is not the only way to obtain the event-free derivative limit. Regional consistency alone, however, is insufficient: it places no restriction on the derivative of a step that samples more than one region. The following result states the additional hypotheses needed for a safe extension. Theorem 17 (Event-blind one-step derivative limit). Assume (A1)–(A5). Consider a fixed-step method whose executed update is a piecewise-C 1 map xk+1 = Ψη,σk (xk ), where σk records a locally fixed finite pattern of regional evaluations. Suppose the following bounds hold uniformly on the relevant compact tube. 1. On a step whose evaluations remain in one region α, Ψη,α (x) = x + ηfα (x) + O(η 2 ), 2

DΨη,α (x) = I + ηAα (x) + O(η ).

(96) (97)

2. Every mixed step in an isolating event tube satisfies ∥Ψη,σ (x) − x∥ ≤ Cη,

∥DΨη,σ (x) − I∥ ≤ Cη.

(98)

3. The method makes oriented interface progress at least cη in each event tube and executes at most M mixed steps per event, with c > 0 and M < ∞ independent of η. 4. No evaluation used by the executed program lies exactly on an interface. Then the numerical state converges uniformly on [0, T ], and the exact Jacobian of the executed program satisfies DΘΨ (99) η,T −→ Jreg . The derivative of the limiting flow remains the saltation-interleaved product in Theorem 1. Thus the two derivatives differ whenever the transported event product is nonidentity. Proof. On pure regional steps, (96) gives the standard first-order consistency and stability recursion. In each isolated event tube, positive progress and the mixed-step bound imply an O(η) timing error and only O(1) local defects of size O(η). Restarting the regional recursion after every separated event gives uniform state convergence by the same finite induction as Theorem 13. For the derivative, pure regional factors have the expansion I + ηAα + O(η 2 ). Their ordered products converge to the corresponding regional fundamental matrices. Every mixed factor is I + O(η), and only finitely many such factors occur, so their combined product converges to I, not to a nonidentity saltation matrix. A finite product perturbation argument across the separated itinerary proves (99). Strict branch evaluations make this product the classical derivative of the executed finite program. The theorem includes Euler/GD. It also includes any event-blind regional one-step construction that satisfies the mixed-step hypotheses, irrespective of its higher regional order. It does not cover an event-localizing method, whose mixed-step derivative is designed to retain the event-time transfer, or an adaptive, implicit, or stage-switching scheme unless those hypotheses are verified separately. Smooth preconditioned Euler is covered relative to its own limiting field f = −P (θ)∇L(θ), provided the same event and regularity assumptions hold; this is not a statement about the unpreconditioned gradient flow. Fixed-coefficient momentum and heavy-ball limits are not covered.

L

D ETAILED PRIOR - ART COMPARISON

Table 1 compares limiting objects rather than claiming that the cited sources exhaust their fields. The phrase learning consequence refers to no-cancellation under global convexity and residual-ReLU results proved here, not to saltation itself. The classical-flow row is anchored by Filippov (1988); di Bernardo et al. (2008); Burden et al. (2016); Hiskens & Pai (2000); Tolsma & Barton (2002); Galán et al. (1999); Taringoo & Caines (2009); Corner et al. (2020); Kong et al. (2024); Zhang et al. (2017); Serban & Recuero (2019). The 24

Preprint

numerical and event-localization rows use Stewart & Anitescu (2010); Stewart (1990); Dieci & Lopez (2012); Nurkanović et al. (2024); Ma et al. (2021); Chen et al. (2021). Fixed-program nonsmooth differentiation is represented by Kakade & Lee (2018); Lee et al. (2020); Bolte & Pauwels (2020); Bolte et al. (2022); Park et al. (2024); smooth discretize–differentiate comparators include Hager (2000); Sanz-Serna (2016). The learning-state and ReLU-curvature context uses Elkabetz & Cohen (2021); Ainsworth & Shin (2021); Jentzen & Riekert (2023); Duersch et al. (2024); Demengel (1984); Ambrosio et al. (2000).

Line of work

Table 1: Closest objects and limits. limiting object event sensitivity

relation to this paper

Classical hybrid piecewise-smooth sensitivity flow map

event time, saltation, supplies the flow-side calparameter, and adjoint culus jumps Discontinuous nu- fixed-horizon mesh re- persistent gradient error, closest numerical anmerical sensitivity finement Dirac layers, and multi- tecedent; not specialized to ple schemes hard-ReLU GD or the convex learning consequences Event-localizing refined switch-aware retain event time and re- supplies the principle beschemes discretization cover sensitivity conver- hind the diagnostic correcgence tion fixed programs or program-dependent or characterizes programNonsmooth AD iteration-to-fixedgeneralized derivative dependent selections; point limits selections here classical correctness follows from local branch smoothness, while the T /η-deep limit differs GD versus gradi- fixed-time state trajec- primarily state approxi- motivates the stronger ent flow tory mation derivative-consistency question This work fixed T , hard-ReLU exact program deriva- adds the learning-limit GD depth T /η → ∞ tives versus the limiting decomposition, notraining-flow derivative cancellation under global convexity, and the globally strongly convex residualReLU reciprocal-ratio family

M

F INITE - SAMPLE R E LU EMPIRICAL - RISK REALIZATION

Proposition 18 (Exact residual-ReLU flow and discrete derivatives). Fix m > 0, c > −m, and T > 0, and consider (100) Lm,c (θ) = 12 (θ − m)2 + 12 (ReLU(θ) − c)2 . For θ0 ∈ (m(1 − eT ), 0), let t∗ = log[(m − θ0 )/m] and r = (m + c)/m > 0. The flow has exactly one stable transverse crossing before T , and Jreg = e−t∗ e−2(T −t∗ ) ,

DφT = rJreg .

The regional Hessians are H − = 1 and H + = 2, while the interface coefficient is β = −c. If c ≤ 0, the objective is globally 1-strongly convex; in the crossing range its unique minimizer is (m + c)/2 > 0. For η = T /N < 1/2, define t∗ . − log(1 − η) If κη ∈ / N, the step size is nonresonant, the first positive iterate is Kη = ⌈κη ⌉, and κη =

AD DΘη,T = Jη,T = (1 − η)Kη (1 − 2η)N −Kη −→ Jreg .

25

(101)

Preprint

Moreover, the piecewise-linear GD interpolation converges uniformly to the flow at O(η) for each fixed pair (m, c). Proof. For the constant input x = 1, define the standard two-output residual-ReLU network and target     θ m zθ (1) = θ, Fθ (1) = , y= . ReLU(θ) c Then 12 ∥Fθ (1) − y∥22 is exactly (100). Its regional vector fields and Hessians are f − (θ) = m − θ,

H − = 1,

f + (θ) = m + c − 2θ,

H + = 2.

Before the event, θ(t) = m + (θ0 − m)e−t , which reaches zero at t∗ . The initialization interval is exactly the condition 0 < t∗ < T . The one-sided speeds at zero are m and m + c, so the crossing is stable, same-direction transverse, and has saltation factor r = (m + c)/m. After the event, θ(t) =

m+c 1 − e−2(t−t∗ ) > 0, 2

so there is no recrossing. Multiplication of the regional propagators and the event factor gives the stated formulas for Jreg and DφT . The normal derivative jumps from −m to −(m + c), hence β = −c. If c ≤ 0, the function h(θ) = 12 (ReLU(θ) − c)2 is convex: its left derivative at zero is 0, its right derivative has trace −c ≥ 0, and the latter then increases with slope one. Since 12 (θ − m)2 is 1-strongly convex, their sum is globally 1-strongly convex. For c > −m, its positive-region critical point (m + c)/2 is positive and is therefore the unique minimizer. For c > 0, the downward derivative jump −c < 0 instead proves global nonconvexity. For the discrete dynamics, while θkη ≤ 0, θkη = m + (θ0 − m)(1 − η)k . Thus an exact landing occurs precisely when κη is an integer, and otherwise the first positive iterate η is Kη = ⌈κη ⌉. The incoming update gives 0 < θK < ηm. Since 0 < 1 − 2η < 1, the outgoing η recurrence   m+c m+c η θkη = + θK − (1 − 2η)k−Kη η 2 2 is a convex combination of positive quantities at each step and therefore stays positive. Hence the discrete itinerary also has one crossing. Its branch sequence is locally fixed at nonresonant sizes, and differentiating the two affine recurrences gives (101). Because − log(1 − η) = η + O(η 2 ), Kη η = t∗ + O(η). Taking logarithms in (101) therefore gives the claimed Jacobian limit. The standard uniform comparisons (1 − η)k = e−kη [1 + O(η)] and (1 − 2η)k = e−2kη [1 + O(η)], together with the O(η) crossing-time and overshoot errors, prove the uniform state bound. Its constant may depend on (m, c). Corollary 19 (Uniform-margin amplification and convex attenuation). Fix a > 0, t∗ ∈ (0, T ), and a prescribed event ratio r > 0. Choose  (a/r, (r − 1)a/r), 0 < r ≤ 1, (m, c) = θ0 = m(1 − et∗ ). (a, (r − 1)a), r ≥ 1, Then the event occurs at t∗ , min{f − (0), f + (0)} = a, and DφT = rJreg . If r ≤ 1, the objective is globally 1-strongly convex. In particular, for R > 1, the choice m = Ra, c = −(R − 1)a gives DφT = R−1 Jreg ,

|Jreg | = R, |DφT |

(102)

while a common interface tube |θ| < a/4 has normal-speed lower bound a/2 for every R. The limit is taken for each fixed R and then η ↓ 0; the Euler state-error constant is not asserted to be uniform in R. 26

Preprint

Proof. The parameter choice gives m + c = rm and makes the smaller of m and m + c equal to a. The event-time formula and Proposition 18 prove the ratio. When r ≤ 1, c ≤ 0, so global 1-strong convexity follows from that proposition. In the reciprocal family, the incoming speed is Ra. On the incoming side of |θ| < a/4 it is at least Ra, while on the outgoing side the speed is a − 2θ ≥ a/2. This proves the uniform transversality claim. Proposition 20 (Exact-hit set for the residual-ReLU family). For the pre-event recurrence in Proposition 18, the positive step sizes that place an evaluated iterate exactly at the interface are ηk = 1 − e−t∗ /k ,

k ∈ N.

(103)

This set is countable, has Lebesgue measure zero, and accumulates at zero. Requiring T = N η selects only its intersection with the mesh family {T /N : N ∈ N}. Proof. An exact hit at iterate k means (1 − η)k = e−t∗ , which is equivalent to (103). The set is countable and 1 − e−t∗ /k ∼ t∗ /k, proving accumulation at zero. The final claim is set intersection and makes no assertion that the intersection itself accumulates. Proof of Corollary 8.

At every nonresonant step size, the ordinary chain rule gives DGη (θ0 ) = q ′ (Θη,T (θ0 ))DΘη,T (θ0 ).

Proposition 18, continuity of q ′ , and state convergence therefore yield DGη (θ0 ) → q ′ (φT (θ0 ))Jreg . The stable-itinerary flow map is classically differentiable, so DG(θ0 ) = q ′ (φT (θ0 ))DφT (θ0 ) = rq ′ (φT (θ0 ))Jreg . Both limits are nonzero because q ′ (φT (θ0 )) ̸= 0 and Jreg > 0. This proves the stated factor-r relation. At resonance, an evaluated iterate equals zero. If another step remains, the incoming and outgoing updates from that state differ by ηc, which is propagated through the remaining outgoing recurrence. The endpoint map therefore need not have a classical derivative; software AD returns a selected branch product. The state estimate still holds under the admissible positive-speed selection. This is why every derivative statement above, including Theorem 7 and Corollary 8, follows a nonresonant sequence.

N

S MOOTHING AND DISCRETIZATION THEORY

This section gives a self-contained scalar result that separates three effects: truncating a smoothing profile, discretizing its inner dynamics, and taking the sharp-interface limit. It also records the phase qualification needed for the sharp-first limit. Throughout, the update is written with a plus sign because the vector field already includes the negative gradient. N.1

L AYER ASSUMPTIONS , RESCALING , AND ENTRY– EXIT CONVENTION

Let F : R → R satisfy the following standing assumptions: (S1) F ∈ C 2 (R) and there are constants 0 < m ≤ M < ∞ such that m ≤ F (u) ≤ M for every u; (S2) the positive traces a− := lim F (u), u→−∞

a+ := lim F (u) u→+∞

exist; (S3) LF := ∥F ′ ∥∞ < ∞ and F ′′ ∈ L1 (R). For the sharp-first result we additionally assume |uF ′ (u)| −→ 0 27

as u → ±∞.

(104)

Preprint

Assumption (S1) is the scalar same-direction transversality condition. In particular, neither grazing nor reversal of the continuous trajectory occurs inside the layer. The scalar saltation factor is a+ Ξ := > 0. (105) a− For a smoothing width τ > 0, define fτ (x) := F (x/τ ),

xk+1 = xk + ηfτ (xk ).

(106)

The inner variables uk := xk /τ, give the exact map and fixed-itinerary cocycle

ρ := η/τ

(107)

δuk+1 = 1 + ρF ′ (uk ) δuk .

uk+1 = uk + ρF (uk ),

(108)

We use the following entry–exit convention for the canonical layer. Given a normalized half-window R > 0, set u0 = −R and Nρ,R −1

Nρ,R := min{n ≥ 1 : un ≥ R},

Jρ,R :=

Y

1 + ρF ′ (uk ) .

(109)

k=0

The index is finite because every increment is at least ρm. The product in (109) is the derivative of the Nρ,R -fold iterate on a phase cell on which Nρ,R is locally constant. At a phase where the stopping index changes, the stopped map need not be differentiable; no derivative across such a phase boundary is claimed. The physical tube is [−τ R, τ R]. A localized interface limit requires both R → ∞ and τ R → 0. N.2

E XACT FLOW TRANSFER AND RESOLVED E ULER TRANSFER

Lemma 21 (Exact scalar flow transfer). For the inner flow and its variational equation, v̇ = F ′ (u)v,

u̇ = F (u), the transfer from u = −R to u = R is flow JR =

F (R) . F (−R)

(110)

flow → Ξ as R → ∞. Consequently, JR

Proof. Along the trajectory, d d log v = F ′ (u) = log F (u). dt dt Integrating between the entry and exit points proves (110). The trace limit follows from (S2). Define the logarithmic tail truncation δF (R) := log

F (R) F (−R) + log . a+ a−

(111)

Theorem 22 (Resolved discrete layer). Under (S1)–(S3), suppose that ρLF ≤

1 . 2

(112)

Then |log Jρ,R − log Ξ| ≤ CF ρ + δF (R), where one admissible constant is M LF ∥F ′′ ∥L1 CF := + . min{1/2, m/M } m In particular, if ρj → 0 and Rj → ∞, then Jρj ,Rj → Ξ. 28

(113)

(114)

Preprint

Proof. Write N = Nρ,R and, for 0 ≤ k < N , set Ak := 1 + ρF ′ (uk ),

Bk :=

F (uk+1 ) . F (uk )

Because uk+1 − uk = ρF (uk ), Taylor’s formula with integral remainder gives Z uk+1 1 |Bk − Ak | ≤ (uk+1 − s)|F ′′ (s)| ds F (uk ) uk Z uk+1 ≤ρ |F ′′ (s)| ds. uk

Condition (112) gives Ak ≥ 1/2, whereas (S1) gives Bk ≥ m/M . With c := min{1/2, m/M }, the mean-value theorem for the logarithm yields | log Ak − log Bk | ≤ c−1 |Ak − Bk |. The intervals [uk , uk+1 ] are disjoint. Summing and using the telescoping product of the Bk therefore gives F (uN ) ρ∥F ′′ ∥L1 log Jρ,R − log ≤ . (115) F (−R) c The exit overshoot obeys 0 ≤ uN − R ≤ ρM. ′ Since |(log F ) | ≤ LF /m, | log F (uN ) − log F (R)| ≤

M LF ρ. m

(116)

Combining (115), (116), and (111) proves the claim. Remark 23 (Rate, entry phase, and physical localization). If Eρ,R := CF ρ + δF (R), then |Jρ,R − Ξ| ≤ ΞeEρ,R Eρ,R . An entry displacement |u0 + R| = O(ρ) adds only another O(ρ) endpoint term to the proof. If δF (R) ≤ Ce−γR , the choice R(ρ) ≥ γ −1 log(1/ρ) gives an O(ρ) bound, provided an embedded interface also satisfies τ R(ρ) → 0. For fixed R, however, the resolved limit is F (R)/F (−R), not exactly Ξ; a fixed-window error must not be reported as Euler discretization error. N.3

T HE TWO ORDERS OF LIMITS

Iterated limits are meaningful only after fixing a common state map. Fix x0 < 0 and T > −x0 /a− , take η = T /n, and define η,τ η,τ xη,τ xη,τ = x0 , 0 k+1 = xk + ηF (xk /τ ), n−1   Y η Jη,τ = 1 + F ′ (xη,τ k /τ ) . τ

(117) (118)

k=0

The horizon assumption places the limiting sharp trajectory strictly on the positive side at time T . Theorem 24 (Smooth-first iterated limit). Under (S1)–(S3), h i a + lim lim JT /n,τ = = Ξ. (119) τ ↓0 n→∞ a− Proof. For fixed τ , the vector field fτ is smooth and globally Lipschitz. Standard Euler convergence applied jointly to the state and variational equations gives lim JT /n,τ = DφτT (x0 ),

n→∞

where φτt is the flow of ẋ = F (x/τ ). The scalar autonomous identity from Lemma 21 gives DφτT (x0 ) =

F (φτT (x0 )/τ ) . F (x0 /τ ) 29

(120)

Preprint

For any reached point x, elapsed time is characterized by Z x dz t= . F (z/τ ) x0 The integrand is bounded by 1/m and converges pointwise away from zero to a−1 − on the negative side and a−1 on the positive side. Dominated convergence, followed by monotonic inversion of the + time-of-flight map, shows that φτT (x0 ) converges to the sharp trajectory endpoint   x0 a+ T + > 0. a− Thus the denominator and numerator in (120) converge to a− and a+ , respectively. For the opposite order, define the sharp Euler orbit  yk+1 = yk + ηf0 (yk ),

f0 (y) =

a− , y < 0, a+ , y > 0,

y0 = x0 .

(121)

The value of f0 (0) is irrelevant when the orbit is nonresonant. Theorem 25 (Nonresonant sharp-first limit). Assume (S1)–(S3) and (104). Fix η = T /n and suppose that yk ̸= 0, 0 ≤ k ≤ n − 1. (122) Then lim Jη,τ = 1. τ ↓0

Consequently, for the set Enr of step sizes satisfying (122),   lim lim Jη,τ = 1. η↓0 η∈Enr

τ ↓0

(123)

(124)

Before the first crossing, condition (122) is equivalent to −x0 /(ηa− ) ∈ / N. Proof. For fixed η, the sharp orbit has finitely many states and dη := min |yk | > 0. 0≤k<n

Induction in (117), using the trace convergence in (S2) at each state away from zero, gives xη,τ k → yk for every k. Hence |xη,τ | ≥ d /2 for all sufficiently small τ . For each of the finitely many Jacobian η k factors, xη,τ η ′ η,τ η k F (xk /τ ) = η,τ F ′ (xη,τ k /τ ) τ |xk | τ −→ 0 by (104). Their product therefore converges to one. The outer limit in (124) is then the limit of the constant inner-limit value one over nonresonant step sizes. Remark 26 (Why the phase qualification is necessary). For the tanh profile below, F ′ (0) ̸= 0 and the tails are exponential. If a sharp iterate satisfies yj = 0, all preceding smooth-versus-sharp state errors are exponentially small in 1/τ , so xη,τ j /τ → 0. The corresponding factor behaves as 1+

η ′ F (0), τ

whose magnitude diverges. Resonant step sizes can accumulate at zero. Thus an unrestricted statement limη→0 limτ →0 Jη,τ = 1 is generally false. The proved replacement is the directed nonresonant limit (124). 30

Preprint

N.4

F INITE - RATIO TRANSFER DEPENDS ON PHASE AND PROFILE

At fixed ρ, equation (108), rather than the trace ratio alone, is the limiting inner problem. Proposition 27 (Phase and profile are genuine data). At finite ρ, the fixed-itinerary cocycle need not be determined by (a− , a+ , ρ). Proof. Phase dependence follows already from the explicit tanh calculations in Subsection N.5: the same profile and ρ give transfers near 46 and 1 for two different incoming phases. For profile dependence, fix a finite itinerary u0 , . . . , uN away from a stopping-index boundary. Choose a smooth bump h supported in a small neighborhood of u0 , disjoint from all later sampled states, such that h(u0 ) = 0 and h′ (u0 ) ̸= 0. For sufficiently small ε, the function Fe = F + εh remains positive and has the same traces. Every sampled state is unchanged because Fe(uk ) = F (uk ), but the first cocycle factor changes because Fe′ (u0 ) ̸= F ′ (u0 ). Hence the transfer changes while the traces do not. N.5

TANH - PROFILE CONSTRUCTIONS

Consider 1 + tanh(u/2) a+ − a− , F ′ (u) = sech2 (u/2). (125) 2 4 This profile satisfies all assumptions above. In the expanding example a− = 0.2 and a+ = 2, so Ξ = 10 and ∥F ′ ∥∞ = 0.45. F (u) = a− + (a+ − a− )

Skip.

With u0 = −20, exit threshold 20, and ρ = 205, u1 = −20 + 205F (−20) = 21.00000076056569 > 20.

There is one factor and J = 1 + 205F ′ (−20) = 1.0000007605656835.

(126)

This is a constructive skip phase, not a universal large-ρ limit. Tuned hit.

Choose

20 = 99.99999814496176. F (−20) Then u1 = 0 and u2 = ρ∗ F (0) = 109.99999795946 > 20. Thus

J = 1 + ρ∗ F ′ (−20) 1 + ρ∗ F ′ (0) = 46.0000162315841. ρ∗ :=

(127)

(128)

More generally, taking u0 = −R and ρR = R/F (−R) lands exactly at the maximizer u = 0. For the increasing profile the following step exits +R for all sufficiently large R, and JρR ,R ∼ ρR F ′ (0) −→ ∞. The phase is retuned with R (and hence with ρR ); this is not a fixed-phase divergence statement. Same ratio, different phases. u0 = −20 :

At ρ = 100 with exit threshold 20, direct iteration gives

J = 46.00001706635,

u0 = −10 :

J = 1.01634222248.

(129)

This supplies the phase comparison used in Proposition 27. Orientation reversal of the discrete cocycle. For the contracting twin a− = 2, a+ = 0.2, take u0 = −20, R = 20, and ρ = 5. The orbit contains u1 = −10.00000001855,

u2 = −0.00040859936,

u3 = 5.50051074919.

At the near-central sample, 1 + 5F ′ (u2 ) = −1.24999990609 < 0. 31

(130)

Preprint

The two preceding factors are positive. For u ≥ u3 , the magnitude of the negative derivative in (125) decreases, and all remaining factors are at least 0.9635. Therefore exactly one sampled factor is negative and J = −1.17941975827. (131) The continuous same-direction saltation factor is a+ /a− = 0.1 > 0; the sign reversal is a finite-step critical-regime effect. The inequality ρ max(−F ′ ) > 1 merely makes a negative factor possible. It does not force a given phase to sample that factor, nor does it determine the parity of all negative factors in a longer itinerary. N.6

E FFECTIVE RESOLUTION AND SCOPE BEYOND THE SCALAR FLAT LAYER

The local dimensionless stiffness is ρeff := η∥fτ′ ∥∞ = ρ∥F ′ ∥∞ .

(132)

For (125) with traces 0.2 and 2, ρeff = 0.45ρ. The condition ρeff ≤ 1/2 is a sufficient condition for Theorem 22; it is not a necessary condition for an accidental finite-ρ value to lie near Ξ. Across a family of profiles, ρeff also does not replace the tail error, positivity bounds, or ∥F ′′ ∥L1 appearing in (113). The contrast quantity ρ|a+ − a− |/a− measures net trace gain, not local spike resolution. The preceding proofs are scalar. They extend without new ideas only when a multidimensional system genuinely reduces to an autonomous normal coordinate, for example at a flat interface with fixed normal, a rank-one normal jump, and separately controlled tangential dynamics. In that setting the scalar normal factor is embedded in the usual matrix I+

(f + − f − )ν ⊤ . ν⊤f −

The scalar proof does not establish a general matrix theorem: matrix cocycles do not telescope through a scalar speed ratio, normal–tangential blocks need not commute, and tangential drift can change both interface location and incoming phase. For a curved interface, flattening changes the normal across the physical tube of radius ε = τ R and introduces curvature terms. Any claimed O(ε) + O(η/τ ) bound therefore requires a separate geometric argument with bounded curvature, transverse speed, tangential regularity, and ε → 0. Grazing, sliding, chattering, and simultaneous interface hits are outside the scope of this section. Logical scope. The results above do not assert an unrestricted sharp-first iterated limit, an exact saltation value at fixed R, an O(ρ) error without a tail term, or universality of η/τ over arbitrary smoothing families. They also do not assert that every unresolved phase skips the layer, that the tuned-hit divergence holds at fixed phase, or that ρ max(−F ′ ) > 1 forces orientation reversal. Finally, the scalar argument does not by itself prove a curved-interface or general multidimensional extension.

O

N UMERICAL METHODS AND REPRODUCIBILITY

All numerical plots use deterministic resolution sweeps. Analytic curves are recomputed from the parameters below, and plotted numerical values are read from preserved JSON or NPZ arrays rather than digitized from raster figures. Residual-ReLU curvature-phase sweep. The main scalar study uses the standard one-sample graph Fθ (1) = (θ, ReLU(θ)), y = (m, c), L = 12 ∥Fθ (1) − y∥22 . For each r ∈ {1/40, 1/20, 1/10, 1/5, 1/2, 1, 2, 5, 10, 20, 40}, we choose m = 1/ min{1, r}, c = (r − 1)m, and θ0 = m(1 − e0.3 ), with T = 0.8. Hence the event time is 0.3, the minimum one-sided speed is exactly one, and DφT = rJreg . The eight step sizes are 0.04, 0.02, 0.01, 0.005, 0.002, 0.001, 0.0005, 0.0002. All 88 rows are nonresonant and have one crossing. The outer objective is q(z) = z 2 /2, and the diagnostic event-aware derivative inserts the exact scalar event factor into the discrete regional product. 32

Preprint

Fixed-profile resolution map; dashed: sufficient contour ρeff = 1/2 Layer transfer J (fixed tanh, Ξ = 10)

10

−1

Error to saltation |J − Ξ|

B 10

−1

10

−2

10

−3

10

−4

10

−5

10

−2

10

−3

10

−4

10

−5

10

1

transfer J

10

smoothing width τ

smoothing width τ

30

10−3

absolute error

A

10−6 1

10

−5

10

−4

10

−3

10

−2

10

−1

10

GD step size η

−5

10

−4

10

−3

10

−2

10

−1

GD step size η

Figure 4: A fixed-profile smoothing resolution study. For the tanh profile and normalized window stated above, (a) shows the discrete tangent transfer and (b) its error to the saltation factor. The lightly marked contour is only the sufficient fixed-profile condition ρeff = 1/2, not a necessary threshold. The theorem retains tail truncation and entry-phase dependence and makes no universality claim across smoothing families.

The script stores the branch-stability radius and exact-hit distance for every row. The minimum normal speed is one, the minimum resonance distance is 0.150005 of a step, and the minimum branch-stability radius is 4.05 × 10−5 . Maximum discrepancies are 3.82 × 10−14 for PyTorch endpoint AD, 4.11 × 10−13 for its outer derivative, 3.66 × 10−9 for the centered finite-difference endpoint derivative, and 3.37 × 10−8 for the finite-difference outer derivative. Two-parameter residual-ReLU problem.

The second study uses
F(u,v),λ (1) = u, v, ReLU(u − λ) + 0.5v , y = (1, 0.2, 1),

with T = 0.8, λ = 0, and (u0 , v0 ) = (1 − e0.3 , 0). This analytically prescribed trajectory has one verified event. The exact event time is 0.3, the event state is (0, 0.1751180039), the one-sided normal speeds are 1 and 1.9124409981, and β = −0.9124409981. We use the same eight nonresonant step sizes as above. Each row stores the state, initialization Jacobian, moving-threshold sensitivity, eventaware counterparts, PyTorch reverse derivatives, centered finite differences, and forward–adjoint pairings. All independent checks meet the reported tolerances; the largest PyTorch discrepancy is 4.87 × 10−14 , the largest discrete finite-difference discrepancy is 2.59 × 10−8 , and the largest flow finite-difference discrepancy is 1.62 × 10−9 . Continuous forward–adjoint agreement is within 1.39 × 10−17 , while the independently implemented discrete event-aware reverse recurrence agrees with its forward counterpart within 1.43 × 10−14 . Identity-Hessian control. The two-bowl construction of Appendix I is retained as a mechanismisolating control: H − = H + = I, so all persistent discrepancy comes from the interface transfer rather than regional curvature. This exact construction is not used as the main learning realization. Smoothing grid. The profile is F (u) = 0.2 + 1.8[1 + tanh(u/2)]/2, with normalized window [−20, 20]. Both η and τ use 25 logarithmically spaced values from 10−5 to 10−1 , giving 625 finite float64 cells and ρ ∈ [10−4 , 104 ]. Transfer is the product in (109), and error is |J − 10|. The R = 20 truncation floor is approximately 2.04 × 10−7 . Figure 4 is the fixed-profile heatmap associated with this scalar theorem. It illustrates the sufficient resolution condition; it does not define a universal phase boundary. Artifact structure. The two main experiment directories contain complete CSV/JSON/NPZ arrays, configuration, seed, software versions, validation summary, editable SVG, and vector PDF. Their generating scripts copy the validated vector exports into the manuscript figure directory. No numerical value is recovered from a raster image. 33

Preprint

P

A DDITIONAL SCOPE AND EXCLUDED REGIMES

Multidimensional scope. The sharp-interface results are fully multidimensional. Tangential modes pass through an interface of a continuous piecewise-smooth loss unchanged, while the normal mode is multiplied by (ν ⊤ f + )/(ν ⊤ f − ). Regional dynamics before and after an event may rotate and couple these modes, which is why the exact multiple-event criterion uses transported factors. The smoothing theorem is narrower: it extends directly only when the system admits an autonomous scalar normal coordinate with separately controlled tangential propagation. A general curved layer introduces tangential drift, a varying normal, noncommuting matrix cocycles, and flattening-curvature terms. No general curved-interface O(η/τ ) theorem is claimed. Excluded regimes and rate qualifications. The sharp-interface and smoothing results retain the following restrictions. 1. Piecewise C 2 regularity gives the modulus estimate for ordinary AD; an O(η) sensitivity rate needs Lipschitz regional Hessians or an equivalent time-regularity assumption. 2. At an exact discrete interface landing, DΘη,T may not exist even though software AD returns a selected branch product. 3. Constants are nonuniform near grazing because event-time and saltation formulas contain (n⊤ f − )−1 . 4. Opposite-sign normal speeds can produce sliding or nonunique patched solutions. Simultaneous hits require a separate multi-surface analysis and cannot be assigned an arbitrary ordered saltation product. 5. Infinite or nonseparated event sequences invalidate the finite induction and finite-product perturbation arguments. 6. If T is an event time, the fixed-time derivative may be one-sided or convention-dependent. 7. Multiple nonidentity event factors can cancel after transport; a switch count alone is not a mismatch criterion. 8. The identity ∥Ξ∥2 = max{1, |r|} requires d ≥ 2; in one dimension the norm is |r|. 9. Positive-definite regional Hessians do not imply global convexity of a continuous nonsmooth loss. 10. The resolved smoothing theorem includes a tail term and does not make η/τ universal across profile families. Its sharp-first identity limit is directed through nonresonant step sizes.

34

Related documents

• Mildly Overparameterized ReLU Networks on Orthogonal Data: Incremental Learning and Implicit Bias#229507
• Building Transformation Layers for Riemannian Neural Networks#1131694
• Reciprocity Separates Gradient Flow from Rotation in Conservative Physical Learning#622546
• On Parameter Symmetries and Conservation Laws in Gradient Flow#1128814
• RW-Flow: One-Step Generation on Compact Manifolds via Wasserstein Gradient Flows#1144238