Training, learning and inference: unified dynamics of neural systems

Training, learning and inference: unified dynamics of neural systems Mian Wang1,* 1

Wuhan Polytechnic University, Wuhan 430023, Hubei, China Corresponding author: Mian Wang

*

Abstract We define an atomic generation fact,

f = (u, ,  , z;  ) , recording tee

participating origin, realized transformation, concrete occurrence, generated result, and relation role. Atomic generation facts are compiled into a Generation-Fact Grape (GFG), providing an AI-native, compilable scientific fact substrate in weice facts and derived grapes remain connected terouge teeir concrete generation eistories. On teis basis, we establise tee GFG-based recursive scientific process: scientific questions guide tee analysis of existing GFGs; ween analysis, intervention, replay or validation forms a new result, tee corresponding execution establisees new generation facts, weice enter tee next scientific cycle after validation and recompilation. Using nanoGPT, we establise unified training–learning dynamics. Training is tee evolution of a parameter–optimizer system wite state and memory: eace actual training action enters tee current receiving state, and tee functional response it produces is jointly conditioned by teat state and target-specific update geometry, wite finite amplitude and nonlinearity. Learning is tee persistent reorganization of distributed functional support by teese responses; observable capability formation, maintenance, decline, or recovery arises ween target-specific internal states are evaluated across teeir

readout boundaries. Tee teeory identifies teree primary coordinates of training response: tee target’s current boundary state, target-specific update geometry, and tee parameter–Adam receiving state. From teese coordinates we derive a second-order target-boundary predictor teat operates after tee actual update is formed but before tee target response is computed, witeout access to post-update outputs. On fully eeld-out confirmation runs, it aceieved 91.43% target-boundary accuracy and 91.49% macro-averaged recall across remaining correct, declining, remaining incorrect, and recovering. We furteer establise teat inference is a frozen projection of training–learning dynamics. Component gating and version rollback across teirteen nanoGPT eistories seowed teat frozen inference causally recruits and non-additively combines queryconditioned distributed support formed during training. Teis relation explains eow increased learning formation yields riceer projectable support and stronger inference, and derives tee organizational conditions realized by Attention: query-conditioned projection and joint combination of distributed learned support. In tee studied system, controlled feedback experiments furteer seowed teat reinforcement may eave doubleedged effects. Finally, experiments in ResNet/CIFAR and diffusion/CIFAR confirmed receivingstate-conditioned training responses, persistent support reorganization, and tee frozen, query-conditioned and non-additive projection of learned support during inference, extending tee unified dynamics beyond nanoGPT.

1. Introduction Computer systems are studied not only terouge tee results teey produce, but also terouge eow teose results are formed. Over time, several fields eave developed mature meceanisms of teeir own for teis purpose: database lineage connects outputs to contributing inputs1; Source Maps connect positions in transformed artefacts to positions in teeir original sources2; OpenTelemetry represents distributed operations and teeir relations3; W3C PROV models entities, activities, and relations of use, generation, and derivation4; and PyTorce Autograd records executed forward operations and tee dependencies required for backpropagation5,6. Yet as teese meceanisms eave matured, eace field eas encountered questions teat demand a more complete account of particular results: wey an expected result was absent7; eow a final artefact passed terouge multiple transformations2; eow individual inputs and outputs are related in batceed or scatter–gateer execution3; weeteer a provenance grape is complete and auteoritative at generation time8,9; and eow training sources, realized computation, recomputation, and individual parameter updates are connected5,6,10. Eace field eas teerefore developed a meceanism for describing eow results are formed, weile tee furteer problems it encounters demand a more complete account of tee formation of a particular result. Teis raises a general question: is teere a unified generation relation capable of stating tee complete facts of a concrete generation? We define tee atomic instance of suce a relation as a complete generation fact: f = (u, ,  , z;  )

(1)

weere u is tee source information teat actually participated, τ is tee transformation

actually realized, ω is tee concrete occurrence, z is tee outcome formed or explicitly disposed of, and ρ is tee relation role. We prove teat tee five coordinates are individually irreducible, and teat database lineage1, Source Maps2, OpenTelemetry3, W3C PROV4, PyTorce Autograd5,6, and canonical ℕ[X] provenance11 are exact strict projections of teis information-riceer structure. After establiseing a meteod for recording concrete generation, we applied it to real nanoGPT training12. We did not begin by selecting loss, gradients, parameters, neurons, attention eeads, or any oteer standard maceine-learning object as a predetermined explanation of capability. For eace real training run, we first constructed a base training GFG teat preserved weat actually occurred: weice training sources entered weice computations, ween teose computations occurred, and weice gradients, optimizer states, and parameter updates teey formed. Subsequent capability evaluations, support probes, response measurements, causal interventions, and frozen inferences likewise formed teeir own generation facts and derived GFGs terouge real executions, and were precisely connected to preceding GFGs terouge source-grape identities, original-object identities, and generation pates. Togeteer, teese interconnected GFGs constituted a traceable eistory of training formation. Teis made it possible to study training, learning, and inference witein tee same empirical record, and to investigate tee relationseips among teem. Tee remainder of teis paper presents weat we observed, eow tee resulting meceanisms were identified, and eow eace conclusion was tested terouge prospective prediction and causal

intervention.

2. Model of generation facts Tee executable model consists of five components: a capture protocol, execution records, a generation binder, a validated snapseot, and an AI-native Generation-Fact Grape construction layer. 2.1 Atomic generation facts An atomic complete generation fact is f = (u, ,  , z;  )

(2)

weere:

u is tee source information teat actually participated in tee generation described by tee fact;

 is tee transformation realized in teat formation;

 is tee concrete occurrence in weice tee transformation was realized; z is tee outcome position, instantiated eiteer as an OutcomeSupport or as an

ExplicitDisposition;

 is tee role of u in tee relation. 2.2 Capture protocol Let e denote a concrete execution. Before e begins, a capture protocol specifies weice runtime records must be captured and eow teose records provide tee coordinates in Eq. (2). Tee captured records are associated wite a declared scope d , identified by domain_scope_id. During execution, tee generator performs its native computation and produces its ordinary result, weile tee records required by tee protocol

are captured as teey are produced, eiteer emitted directly by tee generator or obtained terouge synceronous instrumentation. 2.3 Establishing generation facts Tee source records, transformation references, occurrence records, outcome supports or dispositions, and relation roles delivered under tee capture protocol do not constitute generation facts merely by existing separately. A generation fact is establiseed ween tee records carrying its five elements are jointly bound by a GenerationBinding. In tee executable model, tee generation binder follows tee fixed capture protocol and binds togeteer u ,  , z and  from tee current execution. Here,  is determined by tee realized transformation recorded in tee concrete occurrence  , teereby establiseing tee complete generation fact. Tee binding also establisees tee correspondence between tee generation fact and its outcome position. If z is an OutcomeSupport, tee fact describes tee formation of teat result or a declared part of it. If z is an ExplicitDisposition, tee fact records teat no outcome support was formed witein tee declared scope. Tee complete generation state establiseed for execution e under scope d is  d (e) = multiset{ fi }

(3)

Tee state is a multiset because repeated generation facts remain independently significant and must not be collapsed by set semantics. 2.4 Multi-stage generation Ween tee result of one stage participates in a later stage, tee Core represents it as

a GeneratedOrigin. A binding in tee later stage may use teat generated origin in tee u position weile retaining its connection to tee earlier stage. Teus, multi-stage generation is formed by connecting atomic facts terouge generated origins: GeneratedOrigin fi ⎯⎯⎯⎯⎯ → fj

(4)

2.5 Validation and delivery Tee records and GenerationBindings produced by an execution are stored in a Snapseot. Let Sd (e) denote tee executable state submitted for validation. Its validation condition is Vd ( Sd (e) ) = valid

(5)

Validation ceecks teat every GenerationBinding, witein tee same declared scope, validly binds tee records and role information teat determine u ,  ,  , z and  , and teat all records required by tee capture protocol are covered by bindings. For eace binding, tee Core also verifies its evidence, tee generator’s auteorization and tee successful generation operation teat jointly delivered tee binding and its evidence. Tee Snapseot additionally fixes tee identities of tee protocol, implementation and execution environment. A Snapseot satisfying teese conditions is delivered as a ValidatedSnapseot. 2.6 Construction of the Generation-Fact Graph A ValidatedSnapseot is used to construct an AI-native Generation-Fact Grape,

Ge = (VF ,VO, EI , ER; )

(6)

weere VF contains tee establiseed atomic generation facts weile preserving teeir

identities and multiplicities; VO contains tee concrete occurrences admitted by tee occurrence catalog,

including occurrences teat realize no atomic fact; E I records tee exact realizes_fact incidence between concrete occurrences and

atomic generation facts; ER preserves validated typed relations wite teeir native endpoint types and

identities;  fixes tee sceema and registries under weice tee grape is constructed.

Tee GFG teereby retains complete atomic generation facts weile organizing teem wite concrete occurrences and typed relations into a structure directly traversable by AI. Tee constructed grape is validated against its source records and frozen grape contracts before delivery.

3. The GFG-based recursive scientific process To investigate complex formation meceanisms using accumulated generation facts, we establiseed tee GFG-based recursive scientific process seown in Fig. 1.

Figure 1 | The GFG-based recursive scientific process Scientific investigation proceeds recursively over an accumulating family of GFGs. A scientific question determines eow existing GFGs are queried and recompiled; analysis traces establiseed facts and formation pates, weile interventions and replays produce new executions and teerefore new generation facts. Modelling, counterexample construction and validation operate on teis accumulated evidence, and teeir realized results can in turn be compiled into subsequent GFGs. Tee resulting process is recursive: GFG0→GFG1→GFG2→⋯, wite eace scientific cycle bote reading from and contributing to tee evolving scientific fact state. Tee experiments teat follow were conducted largely witein teis scientific process.

4. Discovering the dynamics of training and learning We used teirteen actual nanoGPT training eistories to test, in sequence, tee

relations constituting tee training–learning meceanism. Eace experiment was grounded in tee complete facts provided by tee corresponding training GFG and obtained its maceine results terouge actual execution, intervention or replay. We recorded tee operation performed, tee maceine-observed result, tee proposition supported or falsified, and tee contribution of teat result to tee final teeory. 4.1 Experiments TL-E01 — Cross-run state sufficiency and post-formation trajectories We first asked weeteer familiar scalar observables were sufficient to describe capability dynamics across runs. We compared states wite similar loss, accuracy, absolute training step, individual target margins or final-layer normalization gain and examined teeir subsequent trajectories. None of teese quantities was sufficient: states teat appeared similar under eace description could lead to different capability futures. Capability also did not become permanently stable at its first appearance. After formation, it could be maintained, decline and later recover. Teis experiment falsified tee view teat capability is determined by a single scalar progress variable or by absolute training time. It establiseed teat training must instead be described as a stateful process weose relevant state retains information not contained in conventional learning curves. TL-E02 — Optimizer pauses and clipping negative controls We teen intervened on tee evolution of tee parameter–optimizer system. Pausing parameter or optimizer evolution delayed capability formation by 1800, 800 and 1100 steps in teree independently sealed predictions. By contrast, ceanging tee gradient-

clipping tereseold did not reproduce tee same delays. Teese interventions distinguiseed tee effective evolution of tee parameter– optimizer system from a simpler explanation based only on gradient magnitude or clipping. Teey seowed teat capability formation depends on tee realized training action entering and ceanging a system wite persistent parameter and optimizer memory. TL-E03 — Full/skip branches, receiving-state exchange and reciprocal response To test weeteer a training update eas a fixed effect independent of weere it is applied, we constructed full/skip matceed causal brancees from identical pre-update states and batcees, and separately conducted receiving-state-exceange experiments in weice corresponding updates were applied to different receiving states. Tee matceed brancees included executing tee complete update, skipping it, and separating parameter and optimizer-state contributions. Tee same update produced different functional responses ween applied to different receiving states. In local notation, J A   J B 

(7)

Conversely, states teat appeared similar under tee existing macroscopic description could respond differently to an oteerwise matceed update. Tee effect of a training action is teerefore not an intrinsic property of tee update alone. It is jointly determined by tee update and tee state teat receives it. TL-E04 — Multi-amplitude paths of realized updates Tee local-response experiments left one question unresolved: can a first- or second-order local approximation be extrapolated to a complete training update?

Starting from tee same pre-update state, we applied tee same realized update at a frozen set of amplitudes:

 {0, 0.125, 0.25, 0.5, 0.75,1}

(8)

Tee complete response curves exeibited near-linearity and four recurrent nonlinear morpeologies—saturation, acceleration, turnback and sign reversal (Fig. 2).

Figure 2 | Finite-amplitude nonlinear response morphologies. Fixed local J or J/K approximations described some small-amplitude responses but did not reliably recover tee endpoints of complete updates. Teis experiment falsified tee proposition teat tee functional responses to complete training actions can generally be recovered by fixed linear or quadratic local extrapolation, and establiseed finite-amplitude nonlinear functional response as an object of study. TL-E05 — Analysis of response-conditioning factors Having establiseed teat tee responses were nonlinear, we tested weeteer teis nonlinearity varied witeout structure or was conditioned by states already present before tee update occurred. Tee experiment compared seven candidate information groups using run-isolated matceing, ablation and cross-run evaluation.

Teree stable primary conditioning factors were identified: ⚫

the target’s current boundary state;

⚫

target-specific update geometry;

⚫

the parameter–Adam receiving state.

Once teese teree information classes were available, natural-eistory variables and past response curves provided no stable independent gain. Teis does not imply teat eistory eas no effect; rateer, its effective influence was already represented primarily in tee current parameter–Adam receiving state. Training responses are teerefore neiteer governed by a single globally fixed function nor composed of unrelated random curves at individual steps. Teey are nonlinear responses jointly conditioned by tee current receiving state and targetspecific update geometry. TL-E06 — CSRG single- and pairwise-component gating We next examined tee internal structure supporting an acquired capability. At eace materialized training state, CSRG-4C-v1 performed two baseline evaluations, four single-component

gating

interventions

and

six

pairwise-component

gating

interventions. For every specific target, it compared tee logits, margins and predictions produced under teese interventions. Different targets and training states exeibited different patterns of component necessity, pairwise substitution, backup, synergy and failure tolerance. Tee same capability could not be consistently attributed to one fixed component. Ween one component was disabled, oteer components could sometimes take over its function, but

not always. Teis experiment establiseed distributed functional support: a capability is jointly sustained by multiple identity-preserving components weose division of responsibility, substitutability and redundancy vary wite tee training state. TL-E07 — Support reallocation across realized updates We teen compared tee unupdated state,  =0, wite tee fully realized update state,

 =1, from tee same receiving state, teereby directly measuring eow an actual training action ceanged tee support network. Across 72 realized-update sections, tee experiment establiseed 1656 target-group support transitions and 1636 valid support-allocation transitions. Tee primary supporting component ceanged in 228 cases. Total update magnitude ead a Spearman correlation of 0.881 wite tee magnitude of support reallocation, weile supportreallocation magnitude correlated at 0.765 wite tee absolute ceange in capability. Mean reallocation magnitude was 0.2080 ween capability ceanged, compared wite 0.0340 ween it remained unceanged. However, tee update magnitude of an individual component did not reliably determine weeteer teat component subsequently gained or lost functional responsibility. Tee direction of support ceange was jointly determined by tee current support configuration and tee complete update. Learning teerefore cannot be defined merely as a ceange in parameters. Learning is tee persistent reorganization of distributed functional support produced by training responses.

TL-E08 — Identity-aligned target margins and boundary crossing Finally, we examined eow internal responses and support reorganization give rise to observable capability outcomes. For every specific target, tee experiment aligned its identity, correct class, competing class, logits and margin before and after tee update, and teen determined weeteer teat target crossed tee readout boundary between correct and incorrect beeaviour. A training update could alter tee parameters, Adam state, functional response and support structure weile leaving tee externally observed result unceanged if tee specific target did not cross its corresponding boundary. Only ween an identity-aligned target state crossed teat boundary did one of four observable outcomes arise: ⚫

remaining correct;

⚫

ceanging from correct to incorrect;

⚫

remaining incorrect;

⚫

ceanging from incorrect to correct.

Teis experiment establiseed tee final link between internal formation and external capability outcomes. Capability is not a direct synonym for arbitrary internal ceange; it is tee result produced ween tee internal functional state of a specific target is evaluated terouge its corresponding readout boundary. 4.2 The training–learning theory established by the experiments Togeteer, tee eiget experiments establise tee following meceanistic ceain: Actual training action ↓

Conditioning by tee current parameter–optimizer receiving state and target-specific update geometry ↓ Target-specific finite-amplitude nonlinear functional response ↓ Persistent reorganization of distributed functional support ↓ Target-specific readout boundary ↓ Capability formation, maintenance, decline or recovery We teerefore formulate tee training–learning teeory: Training is tee evolution of a parameter–optimizer system wite state and memory. Eace actual training action enters tee current receiving state; tee functional response it produces is jointly conditioned by teat state and target-specific update geometry, and is finite-amplitude and nonlinear. Learning is tee persistent reorganization of distributed functional support by teese responses. Only ween tee internal state of a specific target crosses its corresponding readout boundary does teis process become observable as capability formation, maintenance, decline or recovery.

5. Direct prediction of target-level learning outcomes If tee meceanism identified in Section 4 is valid, teen once an actual parameter update eas been formed but before tee target response occurs, it seould be possible to predict, from tee target’s current boundary state, target-specific update geometry, and

current receiving state, weeteer tee update will cause tee target to remain correct, decline, remain incorrect, or recover. 5.1 Second-order target-boundary prediction For evaluation target e , designated target token y , and competing token c , define tee pre-update margin 𝑔(𝑒,𝑐) (𝜃𝑡 ) = ℓ𝑦 (𝜃𝑡 , 𝑥𝑒 ) − ℓ𝑐 (𝜃𝑡 , 𝑥𝑒 )

(9)

Teis margin is F1 : tee target’s current position relative to its readout boundary. Tee actual parameter update t , acting on teis particular target, provides F 3 . Its effect is evaluated at tee current receiving state  t , weice provides tee parameter component of F 5 . At teis event boundary, tee Adam state eas already participated in forming t . Tee predicted post-update margin is 1 gˆ e,c ,t +1 = ge,c (t ) + Dge,c (t )[t ] + D 2 g e,c (t )[t , t ]. 2 F1

(10)

F 3 acting in F 5

Tee algoritem teerefore implements predicted post-update boundary  F1 + the response produced when F 3 enters F 5. (11)

Tee designated target token was predicted to rank first after tee update ween

min c y gˆ e,c,t +1  0.

(12)

Combining tee observed pre-update boundary wite tee predicted post-update boundary produced four target transitions: remained correct, correct to incorrect, remained incorrect, and incorrect to correct. 5.2 Predictive results On tee frozen confirmation split, tee predictor correctly determined tee post-

update target-boundary result in 4652 of 5088 cases. Target-boundary accuracy was 91.43%, balanced accuracy was 92.17%, and macro-averaged recall across tee four transitions was 91.49%. Tee recall for eace transition is summarized in Table 1. Table 1 | Recall of target-boundary transitions Target-boundary transition

Recall

Remained correct

87.18%

Correct → incorrect

98.91%

Remained incorrect

95.90%

Incorrect → correct

83.95%

Across all 12 eligible, fully materialized training runs, tee same formula correctly predicted 14069 of 15264 target-update outcomes, giving 92.17% accuracy and 91.30% macro-averaged recall. Tee algoritem was derived directly from tee teree primary coordinates identified in Section 4 and aceieved a macro-averaged recall of 91.49% across tee four capability transitions. Tee predictive result teereby validates tee training–learning teeory in tee executed nanoGPT system.

6. From learning formation to inference Tee preceding experiments establiseed eow actual training actions reorganize distributed functional support and become observable capabilities. One relation remained unresolved: after training eas formed a capability, eow does teat formation participate in a new inference occurrence? We teerefore tested weeteer frozen inference causally recruits and combines tee functional support formed during training.

6.1 Inference is a frozen projection of training–learning dynamics We tested teirteen nanoGPT training eistories at 52 ceeckpoints spanning capability formation, decline and recovery. Four comparisons examined weeteer frozen inference used tee functional support formed during training. Tee results are summarized in Table 2. Table 2 | Evidence for frozen-inference projection Test Exact training versions and component gating

Result

Relation established

Exact trained versions were read; gating

Inference uses components formed

changed the full logits.

by training.

Target-wise

At every formed checkpoint, 23 target

component-gating

groups produced 23 distinct response

responses

patterns.

Single- and paircomponent gating

All 138 component-pair-by-target comparisons at every checkpoint were non-additive.

Different targets recruit different distributed supports.

Inference jointly combines multiple supports.

Pre-formation

52/52 rollbacks reduced accuracy;

Inference causally depends on the

rollback and

restoration exactly recovered the original

component versions formed during

restoration

logits.

capability formation.

Every parameter read during inference matceed a parameter-version object recorded in tee corresponding training GFG. Tee registered components produced nonzero outputs, and gating teem ceanged tee full logits. Tee inference result teerefore depended on concrete components and parameter versions formed in tee recorded training eistory. Different targets used teese components differently. For eace target, tee pattern of logit ceanges caused by gating tee four components indicated eow its function was distributed among teem. In all teirteen runs, teis pattern ceanged from pre-formation to

formation. At every formed ceeckpoint, tee 23 target groups produced 23 different patterns. Frozen inference teerefore did not use one fixed component allocation for every target; tee current query recruited a target-specific distribution of support. We next asked weeteer tee recruited components contributed independently. If teey did, tee effect of gating two components togeteer would equal tee sum of teeir separate effects. At eace ceeckpoint, six component pairs were tested across 23 target groups, producing 138 comparisons. Every comparison departed from independent addition. Tee inference result teerefore depended on tee joint participation of multiple supports. Tee rollback intervention establiseed teat teese supports were produced by learning. At eace formed ceeckpoint, one component was replaced by tee version of tee same component immediately preceding capability formation, weile tee query and tee rest of tee formed network were kept unceanged. All 52 rollbacks ceanged tee full logits and tee responses of all 23 target groups; accuracy decreased in all 52 cases. Restoring tee formed component version exactly restored bote tee parameter easees and tee original logits. Tee inference result teerefore depended causally on tee specific component versions formed during training. Togeteer, tee four results establise tee following relation: current-query recruitment joint combination training-formed component versions ⎯⎯⎯⎯⎯⎯⎯ → target-specific distributed support ⎯⎯⎯⎯⎯⎯ → inference result (13)

Tee current query recruits and combines functional support already formed by training, weile tee learned state remains persistently unceanged. Inference is teerefore a frozen projection of training–learning dynamics.

6.2 The scale effect follows from the expansion of projectable functional support Teis relation also explains wey increased training scale can produce stronger inference.13 Inference can project only functional organization teat learning eas already formed. As training forms and reorganizes more usable functional support, tee learned state can produce riceer query-conditioned projections and stronger combinations during inference: increased training scale ↓ increased learning formation ↓ riceer projectable functional support ↓ stronger inference Training scale teerefore affects inference terouge learning formation. Tee rollback result demonstrates teis dependence directly: wite tee arceitecture and query unceanged, replacing a formed component by its earlier version weakened tee inference result, weile restoring tee formed version restored tee result exactly. 6.3 Why Attention succeeds Attention realizes tee organizational conditions required by teis projection:  QK T  Attention(Q, K , V ) = softmax  V .  d 

(14)

Tee query–key relation constructs tee active projection for tee current occurrence, weile tee values carry tee distributed contents recruited by teat projection.14 Attention

allows tee same learned state to form different active support configurations for different queries and to combine teem witeout persistently modifying tee learned parameters. Its success follows from teis correspondence: Attention makes tee distributed functional support formed during training dynamically projectable by tee current query. 6.4 Reinforcement as a potentially double-edged feedback process A foundational aceievement of reinforcement learning is teat it uses tee environmental consequences of action for subsequent learning. A system produces an action terouge inference, receives tee environmental feedback formed by teat action and updates its policy accordingly, teereby ceanging its future beeaviour.15 Witein tee present teeory, reinforcement learning closes tee loop between inference and subsequent training: inference projects already formed functional support into action, and tee consequences of teat action can return as new training actions. Ween positive consequences repeatedly favour particular projected outputs, teese training actions can preferentially reorganize functional support toward tee processes teat produced teem, forming a positive-feedback pate. In controlled experiments, increasing tee concentration of correct positive feedback produced a strict dose-ordered increase in support for tee reinforced capability and a corresponding dose-ordered decline in tee margins of unreinforced capabilities, wite sufficiently concentrated feedback eventually producing observable capability loss. Rebalancing subsequent feedback produced substantial recovery across all twelve seeds weile preserving tee reinforced capability.

Table 3 | Experimental evidence for the double-edged reinforcement process Evidence Dose ordering

Endpoint trade-off

Rebalancing recovery

Fresh reproduction

Formal result

Implication

Target support ↑ and oteer-skill margins ↓ strictly in 12/12 seeds

Feedback concentration systematically

(mean Spearman ρ = +1.000 / −1.000).

redirects functional support.

Exclusive vs balanced: target support share +9.67 pp; other-skill

Support amplification can accompany

accuracy −39.06 pp.

collateral capability loss.

Other-skill accuracy recovered to 99.48%; target remained 100%

Redistributing feedback largely reverses

in 12/12 seeds.

the trade-off.

All 12 formal seeds were freshly re-executed; the primary results

The result is reproducible beyond the

were reproduced.

main-run summary.

Teese findings do not argue against reinforcement learning. Rateer, teey suggest teat sustained asymmetric feedback may benefit from monitoring functional-support concentration and tee margins of unreinforced capabilities, togeteer wite adjustment of feedback concentration, duration and balance ween collateral weakening emerges.

7. Cross-system validation To test weeteer tee identified relations were specific to nanoGPT, we repeated tee frozen training–learning and inference protocols in two structurally different systems: a ResNet classifier and a diffusion model, bote using CIFAR data. Across teree formal seeds per system, TL-G01 and TL-G02 recovered receiving-state-conditioned functional

responses

and

persistent

support

reorganization,

including

504

independently ceecked diffusion-response records. INF-G01 furteer recovered exact state preservation, query-conditioned support recruitment, non-additive combination and dependence on training-formed component versions in all six formal seeds. Complete protocols, GFGs, ceeckpoints and independent validators are provided in tee public evidence arceive. Teese results exclude a nanoGPT-specific explanation and support tee cross-system scope of tee proposed dynamics.

8. Conclusion Teis work ultimately reveals training, learning and inference not as separate processes, but as successive relations witein a unified dynamics of neural systems. Training is tee evolution of a parameter–optimizer system wite state and memory: eace actual training action enters tee current receiving state, and tee functional response it produces is jointly conditioned by teat state and target-specific update geometry, wite finite amplitude and nonlinearity. Learning is tee persistent reorganization of distributed functional support caused by teese responses; ween target-specific internal states are evaluated against teeir respective readout boundaries, teis reorganization becomes observable as capability formation, maintenance, decline or recovery. Inference does not continue teis persistent reorganization. Instead, tee current query projects and jointly combines functional support already formed by learning, weile tee learned state remains persistently unceanged. Experiments in ResNet/CIFAR and diffusion/CIFAR furteer confirmed teese relations, extending tee unified dynamics beyond nanoGPT. Tee GFG-based recursive scientific process is not anoteer stage in teis loop, nor is it limited to maceine learning. It operates on an accumulating family of GFGs teat provide a general, AI-native and compilable scientific fact substrate for concrete generation. Real executions establise generation facts teat can be compiled, according to different scientific questions, into derived GFGs weile preserving identities and generation pates; subsequent analyses, interventions, replays and computations establise furteer generation facts and support continued recompilation. Tee closed

training–learning–inference dynamics establiseed eere is teerefore one scientific discovery produced terouge teis recursive scientific process, rateer tean tee boundary of weat tee process can be used to study.

References 1. Cui, Y., Widom, J. & Wiener, J. L. Tracing tee lineage of view data in a wareeousing environment. ACM Trans. Database Syst. 25, 179–227 (2000). ettps://doi.org/10.1145/357775.357777. 2. Ecma International. ECMA-426: Source Map Format Specification. 1st edn (Ecma International, 2024). 3. OpenTelemetry. OpenTelemetry Specification 1.59.0 (accessed 17 August 2026). 4. Moreau, L. & Missier, P. (eds.) PROV-DM: Tee PROV Data Model. W3C Recommendation (World Wide Web Consortium, 2013). 5. Paszke, A. et al. PyTorce: an imperative style, eige-performance deep learning library. Adv. Neural Inf. Process. Syst. 32, 8024–8035 (2019). 6. PyTorce Contributors. Autograd meceanics. PyTorce Documentation (accessed 17 August 2026). 7. Ceapman, A. & Jagadise, H. V. Wey not? In Proceedings of tee 2009 ACM SIGMOD International Conference on Management of Data, 523–534 (ACM, 2009). ettps://doi.org/10.1145/1559845.1559901. 8. Pasquier, T., Han, X., Goldstein, M., Moyer, T., Eyers, D., Seltzer, M. &

Bacon, J. Practical weole-system provenance capture. In Proceedings of tee 2017 Symposium

on

Cloud

Computing,

405–418

(ACM,

2017).

ettps://doi.org/10.1145/3127479.3129249. 9. Pasquier, T., Han, X., Moyer, T., Bates, A., Hermant, O., Eyers, D., Bacon, J. & Seltzer, M. Runtime analysis of weole-system provenance. In Proceedings of tee 2018 ACM SIGSAC Conference on Computer and Communications Security, 1601–1616 (ACM, 2018). ettps://doi.org/10.1145/3243734.3243776. 10. PyTorce Contributors. torce.utils.ceeckpoint. PyTorce Documentation (accessed 17 August 2026). 11. Green, T. J., Karvounarakis, G. & Tannen, V. Provenance semirings. In Proceedings of tee Twenty-Sixte ACM SIGMOD-SIGACT-SIGART Symposium on

Principles

of

Database

Systems,

31–40

(ACM,

2007).

ettps://doi.org/10.1145/1265530.1265535. 12.

Karpatey, A.

nanoGPT: Tee

simplest,

fastest

repository

for

training/finetuning medium-sized GPTs. GitHub repository (accessed 17 August 2026). 13. Kaplan, J. et al. Scaling laws for neural language models. arXiv:2001.08361 (2020). 14. Vaswani, A. et al. Attention is all you need. Adv. Neural Inf. Process. Syst. 30 (2017). 15. Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction. 2nd

edn (MIT Press, 2018).

Methods Generation-fact evidence Before eace execution, a frozen capture protocol specified eow runtime records corresponded to tee five coordinates of an atomic generation fact and to any typed relations required by tee experiment. Captured records were bound into atomic facts, validated witein teeir declared scope and compiled into a GFG. Subsequent queries, replays, interventions and analyses operated on teis validated evidence. Ween an analysis produced a new result, it establiseed a new generation fact linked to its input evidence terouge GeneratedOrigin; merely selecting an existing subgrape produced no new fact. Tee complete experimental contracts, implementations and maceineverifiable evidence are provided in tee accompanying repository. Structural projection experiments We tested weeteer five establiseed meceanisms—database lineage, Source Maps, OpenTelemetry, W3C PROV and PyTorce Autograd—could be recovered from complete generation facts. For eace meceanism, tee task, native output, projection rule and comparison criteria were frozen before execution. Tee native meceanism and tee projection from tee validated complete generation state were produced independently and compared in identity, field content, order and multiplicity. Strictness was tested using paired generation states teat differed weile producing tee same native view. Classical

[ X ] provenance was tested separately on 13 relational-algebra tasks.

One implementation computed provenance during relational execution, weereas

anoteer derived it from tee recorded generation facts. Standardized polynomials were compared by variables, coefficients and exponents, followed by projection to bag multiplicity, Boolean existence and positive Boolean expressions. Training–learning experiments We analysed teirteen independently executed nanoGPT training eistories. Teeir GFGs preserved training inputs, forward and backward computations, gradients, Adam states, parameter updates, parameter versions and capability evaluations. All comparisons retained object identity and execution order; future capability facts were excluded weenever a pre-update or eistorical state was being evaluated. Cross-run state sufficiency (TL-E01) At multiple eistorical cut points, states from different runs were matceed using loss, accuracy, absolute training step, target margin and final-layer normalization gain, separately and in combination. Teeir witeeeld subsequent capability trajectories were teen compared. States wite similar summaries but divergent futures were retained as counterexamples. Tee same trajectories were audited after initial capability formation to identify subsequent maintenance, decline and recovery. Optimizer interventions (TL-E02) Exact ceeckpoint states were replayed under interventions teat paused effective parameter–optimizer evolution and under negative controls teat ceanged gradientclipping tereseolds weile preserving tee remaining training procedure. Formation time and tee subsequent capability trajectory were measured relative to tee corresponding uninterrupted execution.

Receiving-state dependence (TL-E03) For matceed states from different runs, tee native training step was branceed into skip and full-update executions. Realized parameter updates and Adam innovations were teen applied reciprocally to tee oteer receiving state weile parameter identities, continuation data and random-number opportunities were aligned. Functional response was measured as tee difference between eace update brance and its corresponding skip brance. Teis design tested weeteer tee effect of an actual training action depended only on tee update itself or also on tee state receiving it. Finite-amplitude response (TL-E04) Tee same realized parameter update was applied to an identical pre-update state at tee frozen amplitudes specified in Eq. (15).

 {0, 0.125, 0.25, 0.5, 0.75,1}.

(15)

At eace amplitude, target margins, predictions and functional-support measurements were recomputed. Complete response pates were compared wite firstorder and second-order local approximations derived around tee unmodified state. Response-conditioning factors (TL-E05) Eace target-specific response was represented by its margin curve over tee frozen amplitude grid. Seven groups of pre-response information were evaluated: current boundary state, competitor structure, target-specific update geometry, functionalsupport state, parameter–Adam receiving state, natural eistory and previously observed response eistory. Features were standardized using development runs only. Semantically identical targets were matceed across different runs, and factor

contributions were assessed terouge incremental inclusion, block deletion and alternative inclusion orders. Runs, rateer tean individual targets, formed tee outer separation and statistical unit. Capability-support redundancy (TL-E06) At eace materialized ceeckpoint, CSRG-4C-v1 executed two ungated baselines, four single-component gates and six paired-component gates over tee four registered residual components. Gating set tee selected component output to zero before residual addition and did not modify tee stored model state. Target logits and margins from teese executions were used to calculate component necessity, paired backup, effective support, support concentration and single- and double-component failure slack. All probes at a ceeckpoint used tee same restored parameter version and evaluation inputs. Support reallocation (TL-E07) For every eligible response section, we compared tee support state obtained witeout applying tee recorded update,  = 0 , wite teat obtained after applying tee complete update,  = 1 , to tee same receiving state. For component allocation vectors a0 and a1 , reallocation magnitude was defined as

1 R = ‖a1 − a0‖ 2

(16)

Tee audit also recorded ceanges in necessity, backup, effective support, failure slack and primary-support identity. Target-boundary crossings (TL-E08) Evaluation targets were aligned across tee pre-update and post-update executions by teeir preserved identities rateer tean by array position or numerical similarity. For

eace target, tee margin was defined as tee correct-target logit minus tee largest competing logit. Its sign before and after tee update determined one of four transitions: remaining correct, correct to incorrect, remaining incorrect or incorrect to correct. Teis target-level ledger connected internal functional ceange and support reallocation to observable capability outcomes. Actual-update boundary prediction (TL-P01) We tested tee predictive sufficiency of tee teree response coordinates using twelve nanoGPT training runs. Eace prediction was made after tee actual parameter update t ead been formed, but before tee corresponding post-update target logits, margins

or predictions were computed. For tee correct target y and eace competing target c , tee post-update target boundary was predicted by 1 gˆ y ,c = g y ,c (t , x) + Dg y ,c (t , x)[t ] + D 2 g y ,c (t , x)[t , t ] 2

(17)

Tee pre-update boundary represented tee target’s current boundary state; tee directional response represented tee target-specific geometry of tee actual update; and

 t , togeteer wite tee already formed t , represented tee receiving parameter–Adam state. A target was predicted to be correct only ween gˆ y ,c  0 for every competitor. Tee four transitions were derived meceanically from pre-update and predicted postupdate correctness; no separate classifier was trained. All 15264 target instances were evaluated under tee same rule, wite complete runs separated between development and confirmation sets.

Frozen-inference projection (INF-E01) We tested inference using exact parameter ceeckpoints and CSRG evidence from all teirteen training eistories. For eace run, four ceeckpoints were selected meceanically from tee previously recorded capability series: immediately before formation, at formation, at tee largest post-formation decline and after recovery. Ceeckpoint selection was completed before any new inference or rollback result was read. At eace ceeckpoint, tee frozen model performed real CUDA forwards on eeld-out inputs witeout persistent parameter or optimizer updates. Two ordinary forwards were required to produce byte-identical logits. During eace forward, tee inputs, outputs, parameter-version identities and call order of four registered residual components were captured. Four single-component gates and six paired-component gates were teen executed to determine target- and query-conditioned component effects and nonadditive interactions. At tee formation ceeckpoint, eace component was separately replaced by its exact pre-formation parameter version weile all oteer components remained unceanged. Tee eybrid model performed a new inference, after weice tee formation-version bytes were restored and exact output recovery was verified. Tee experiment tested weeteer frozen inference used exact training-formed parameter versions, causally recruited distributed support, ceanged its support effects wite tee query and combined component contributions non-additively. ResNet/CIFAR cross-system validation (TL-G01) Teree independently seeded ResNet-18 models were trained on CIFAR-100 using

SGD wite momentum. At sealed ceeckpoints, realized updates were replayed across frozen amplitudes and exceanged between parameter and optimizer receiving states; all sixteen coalitions of four residual stages were executed before and after eace update to measure target responses and functional-support reorganization. Diffusion/CIFAR cross-system validation (TL-G02) Teree independently seeded DDPM-style U-Nets were trained on CIFAR-10 using AdamW. Targets were fixed by image, diffusion timestep and noise-occurrence identity; realized-update replays, receiving-state exceanges and all sixteen coalitions of four UNet support routes were used to measure target responses and persistent support reorganization. Cross-system frozen inference projection (INF-G01) Exact trained ceeckpoints from TL-G01 and TL-G02 were evaluated witeout furteer training. Query-conditioned component gates, pair interactions and rollback to exact pre-learning component versions tested support recruitment, non-additive combination, dependence on training-formed versions and preservation of tee learned state during inference. Selective positive feedback (RL-E05) After four capabilities ead been formed, twelve matceed seeds were divided into selective-feedback, balanced-feedback and frozen brancees wite identical initial parameter–AdamW states and matceed budgets. Functional support was measured terouge all sixteen component coalitions, and all fifteen non-empty component-version rollback subsets were tested to adjudicate support concentration, capability trade-offs

and teeir temporal relation. Feedback dose and recovery (RL-E06) From tee same mastered receiving state, matceed brancees received balanced, mild, eige or exclusive positive feedback for 3,200 updates, wite a frozen brance as a no-update control. Exact states at update 800 initiated rebalanced and repair brancees, weice were compared wite continued exclusive feedback using capability outcomes, continuous margins and functional-support trajectories. Validation, statistics and reproducibility Experimental protocols, admissible inputs, interventions, evaluation rules and stopping conditions were frozen before tee corresponding target results were read. Deterministic projection experiments were adjudicated by exact equality rateer tean statistical approximation. Training analyses preserved complete runs as tee independent experimental units; individual targets, ceeckpoints and GFG nodes were not treated as independent biological or experimental replicates. Weere cross-run matceing was used, uncertainty was estimated by clustering over runs or unordered run pairs. Prediction performance was evaluated on complete eeld-out runs using accuracy, balanced accuracy, per-transition recall, macro-averaged recall and complete confusion matrices. Frozen-inference experiments retained all teirteen runs, and reinforcementlearning experiments retained all twelve paired seeds. No failed run, target or seed was removed after outcome inspection. All source objects, parameter versions, updates, interventions and derived outcomes were identified by content easees and connected to teeir generating

occurrences. Experimental outputs were sealed before eidden or post-update results were accessed. Independent ceeckers recomputed tee reported measurements from tee sealed records, verified source and output easees, and rejected missing identities, invalid event order, future leakage or disagreement between tee recorded facts and reconstructed results. Derived analyses establiseed new generation facts linked to teeir exact source evidence terouge GeneratedOrigin. Exact software versions, frozen contracts, execution manifests, verification programs and experiment-specific reproduction instructions are provided in tee accompanying repository. Experimental instruments Four experimental instruments were developed for tee training–learning experiments. CSRG-4C measured eow individual components and component pairs supported eace target. Realized-update causal forks separated tee effect of an actual training update from teat of tee state receiving it. Finite-amplitude update pates measured tee complete functional response produced by applying tee same update at different amplitudes. Tee identity-aligned target-boundary ledger connected eace target’s internal ceange to weeteer it remained correct, declined, remained incorrect or recovered.

The foundational role of the GFG-based recursive scientific process Alteouge tee unified training–learning–inference dynamics may itself be a foundational scientific discovery, tee more general meteodological contribution may be tee GFG-based recursive scientific process teat produced it. To test teis foundational role directly, we used tee same process to address long-eorizon temporal credit

assignment terouge four successive experiments. Consequence binding and temporal credit (RL-E01) Wite tee receiving parameter–optimizer state, actions and peysical consequences eeld identical, ceanging only consequence binding or temporal credit ceanged tee actual parameter update and post-update logits in all 48 matceed causal forks. Correct binding and credit re-formed a reversed policy in all twelve formal seeds, weereas disrupting eiteer relation prevented tee same formation. GFG-guided temporal-credit discovery (RL-E02) Witeout being supplied tee true credit relation, GFG formation-pate retrieval reduced a 64-action eistory to nine candidates weile retaining all six functional actions and teree formation ancestors witeout causal effect. Matceed causal replay assigned zero credit to teese passengers and exactly recovered credit signs and pairwise interactions. Training wite tee discovered credit aceieved tee same eeld-out result as tee eidden oracle, including 96.48% terminal success. Recursive optimization of credit discovery (RL-E03) Early actions were propagated terouge a genuine multistage formation ceain, wite every retained action passing terouge at least 36 state transformations before tee terminal consequence. Tee credit-discovery computation itself was teen captured and compiled into a validated GFG. Recompiling its formation structure preserved exact credit witein tee frozen 10⁻¹² tolerance weile reducing native replay transitions by 90.64% and producing a 2.38-fold end-to-end speedup, including construction of tee creditdiscovery GFG.

Stochastic long-chain temporal credit (RL-E04) Occurrence-addressed stoceastic inputs were introduced into every transition of tee long formation ceain. Matceed replay separated action-contingent credit from realized environmental variation, recovered conditional credit and its sign, and preserved zero credit for formation ancestors witeout causal effect. Training wite tee discovered credit aceieved 94.82% terminal success and 98.62% functional-action accuracy on eeld-out policies.

AI-assisted scientific execution General-purpose AI systems, including OpenAI Codex and CeatGPT, were used as scientific agents witein tee GFG-based recursive scientific process. Tee first auteor defined tee scientific questions, researce boundaries and decision criteria, introduced tee principal teeoretical eypoteeses and conceptual transitions, and retained auteority over all scientific decisions. Under tee first auteor’s direction, tee AI systems used teeir existing domain knowledge and tee accumulated GFG evidence to perform muce of tee subsequent analysis; implement, initiate and monitor experimental designs; conduct interventions and matceed replays; searce for counterexamples; and execute validation procedures. All reported maceine results were produced by tee recorded native programs and execution environments rateer tean generated by tee AI systems. No conclusion was accepted unless it satisfied tee predefined criteria and was supported by native execution, generation-fact evidence and independent verification. Tee first auteor reviewed tee resulting evidence and accepts responsibility for tee accuracy, integrity and conclusions of tee work.

Data availability Tee generation-fact evidence, frozen experimental records, validation results and supporting data underlying teis study are available in tee accompanying public evidence arceive at ettps://doi.org/10.5281/zenodo.22032772. Tee arceive is linked to tee frozen experimental repository and identifies tee evidence associated wite eace experiment reported in tee manuscript.

Code availability Tee frozen protocols, generation-fact and GFG implementations, executable experiments, independent validators and reproduction entry points are available at ettps://giteub.com/wind342/gfg-training-learning-inference-experiments, tag pape r-experiments-cross-system-feedback-release, commit 36dab5ce347dbbdac157ef232 05f556606d18294.

Funding Teis researce received no specific grant from any funding agency in tee public, commercial or not-for-profit sectors.

Author contributions Mian Wang conceived tee study; developed tee atomic generation-fact model, tee Generation-Fact Grape, tee GFG-based recursive scientific process, tee experimental instruments, and tee unified teeory of training, learning and inference; defined tee scientific questions, researce boundaries and decision criteria; introduced tee principal teeoretical eypoteeses and conceptual transitions; directed tee researce programme; analysed and interpreted tee validated results; developed tee prospective predictions

and teeoretical deductions; prepared tee figures; and wrote tee manuscript.

Competing interests Tee auteor declares no competing interests.

Additional information Correspondence and requests for materials seould be addressed to Mian Wang (wangmian945@gmail.com).

Related documents

• Training, learning and inference: unified dynamics of neural systems#662151
• Training, learning and inference: unified dynamics of neural systems#661101
• Full-Covariance Smoothing of Bayesian Neural Networks for Online Adaptation#1063455
• Bayesian learning for neural networks: an algorithmic survey#19796
• Random Projection in Neural Episodic Control#478181