F ROM P REIMAGE S EARCH TO S OURCE -G ROUNDED F EATURE I NVERSION* A P REPRINT Kaixiang Shu Independent Researcher [email protected]
July 15, 2026
arXiv:2607.12526v1 [cs.LG] 14 Jul 2026
A BSTRACT Interpreting a neural network requires understanding what its internal features extract from a particular input. Feature inversion seeks to express a selected feature in the input domain, but canonical iterative methods search for an input whose re-encoded representation matches the target. Because many inputs can satisfy this constraint, target matching alone does not specify the inverse associated with the sample that generated the feature. We formulate source-grounded feature inversion by conditioning the inverse on the source-local network geometry at the target-generating input. At each boundary of the computational DAG, backpropagation provides the correct reverse dependencies but transports an adjoint signal rather than an upstream-state estimate. We locally repair this signal with a closed-form matrix Wiener map from a mean-seed VJP to an upstream-state estimate. A second Wiener map corrects the JVP forward-consistency residual, and the repaired states are composed through the same DAG in one finite reverse pass. Across diverse architectures, tensor components, and visual distributions, the same formulation yields calibrated zero-intercept map families that support new inputs, depths, channels, and channel groups without query-specific optimisation. Matched target and source controls verify that each inverse depends on the selected feature and the local operators of the sample being explained, rather than a target-independent image template. Prediction-conditioned feature atlases align these visualisations with independent interventions on the corresponding internal features. Together, source-grounded feature inversion opens the model’s hidden feature hierarchy to inspection at the level of individual layers and channels. It links what the network extracts from an input to the internal evidence that shapes its decision. Keywords Feature inversion · model interpretability · matrix Wiener filtering · computational DAGs · closed-form repair
1
Introduction
Modern neural networks form predictions through a hierarchy of internal representations, yet these representations remain difficult to inspect. A tensor activation records a state in feature space, but it does not directly show which structures that state extracts from a particular input. Output explanations and sensitivity maps can identify influential input regions, but they do not provide an input-domain view of a selected internal state at an arbitrary depth or channel. A representation-level interpretation should therefore answer a sample-specific question: what does this feature, at this layer and channel, extract from this input? Expressing internal features in the input domain would place heterogeneous layers and architectures in a common coordinate system and make their progression through the network observable. Feature inversion provides a natural interface for this purpose. Given a frozen model f , an input x, a target node T , and a selected channel set S, we seek an input-domain state associated with the target feature HT,S (x). This state is intended to reveal the structure identified by the selected feature in the computation generated by x. It is not posed as natural-image reconstruction from a compressed code, and similarity to x is not the definition of correctness. The ∗
Code: https://github.com/k-Jayus/source-grounded-feature-inversion
Source-Grounded Feature Inversion
A P REPRINT
source image is a reference for input-instance alignment, not a unique inverse label. The desired inverse is therefore sample-specific: it should explain the selected feature as it occurs in the forward computation of the input being interpreted. Canonical iterative inversion instead poses a target-preimage problem of the form [1, 2] z ⋆ = arg min d HT (z), HT (x) + R(z). z
The optimiser searches for an input whose re-encoded representation matches the target. When the representation is non-injective, many inputs can satisfy this constraint, and the returned preimage depends on the initialisation, regularisation, and image prior used to select among them. Although the frozen network participates in this search, its differential operators are evaluated along the candidate trajectory rather than used to condition the inverse on the forward point that generated HT (x). Learned decoders and gradient visualisations address related objectives [3–8], but they respectively introduce a separate learned mapping or expose an adjoint signal rather than a sample-specific upstream state. The missing condition is therefore not another image prior, but the local computation that produced the selected feature. An internal feature does not exist independently of that computation. The input x determines the forward activations, nonlinear operating regions, branch states, attention routes, and local Jacobians along the computational DAG that produces HT (x). We therefore treat a feature-inversion query as (f, x, T, S) rather than as a target tensor alone. We call the resulting object source-grounded feature inversion: the selected feature is traced through the source-local network geometry at the target-generating forward point. This input conditioning does not provide the answer through an image shortcut. The input is used only to instantiate the frozen forward activations and local differential operators, while the reverse recursion is seeded by the selected terminal state. The output is an estimated sequence of upstream states along this sample-specific local inverse branch. This definition turns the problem into a sequence of local upstream-state estimation tasks on the same DAG that generated the feature. Computing these local inverses requires more than raw backpropagation. A VJP follows the correct reverse dependencies, but it transports an adjoint signal J ∗ q rather than an upstream-state estimate. Under parent and child metrics Mv and Mu , the corresponding natural adjoint is J ♯ = Mv−1 J ∗ Mu . Under the centred local linear-Gaussian reference model, covariance-induced metrics make this operator coincide with local Wiener inversion. In a nonlinear network, however, the relevant metrics and local Jacobians depend on the boundary and source operating point. Explicit online estimation of Mv and Mu is therefore impractical. We instead use a mean-seed VJP as an observable and calibrate a zero-intercept, full-channel, frequency-specific Wiener map Gv from this observable to the upstream activation. This first projection amortises the missing metric correction over the calibration distribution while retaining the source-local Jacobian in its VJP input. The first projection can capture the dominant correction when the local operator is fixed or weakly sample dependent. Normalisation, attention, and data-dependent gating or routing can induce stronger sample dependence, for which a shared Gv can leave an operating-point-specific mismatch. A JVP evaluates the first-stage estimate at the actual source forward point, and a second Wiener map Dv predicts the remaining state error from the pulled-back child-consistency residual. The repaired states are composed through the selected boundary DAG in one finite reverse pass, as summarised in Figure 1. The theoretical development connects the MSE-optimal conditional mean to the covariance-induced natural adjoint, the deployed distributional projection, and finite-DAG error propagation. Once calibrated, the same local map family supports new inputs, depths, channels, and channel groups without query-specific optimisation. Across diverse CNN and Transformer families, tensor components, and visual distributions, the resulting inverses reveal architecture-specific spatial and frequency structure. The outputs do not collapse to an image template: zero targets yield zero inverses, channel selections produce distinct structures, and operating-point swaps redirect the inverse to the new source-local branch. These controls show that each visualisation is jointly determined by the selected feature and the local geometry that generated it. Layer-wise atlases reveal architecture-dependent progressions of extracted structure, while channel-wise queries separate distinct internal evidence within the same representation. For prediction-conditioned groups, top-positive feature groups produce larger prediction-margin drops than matched random controls in all 48 combinations of model, layer, and group size under both calibration-mean and zero replacement. Terminal-layer top-negative groups show the opposite signed effect. Independent intervention therefore grounds the visualised feature groups in the current decision without treating the inverse image as a causal stimulus. Together, these results expose what individual layers and channels extract from an input and which internal feature groups shape the model’s decision. The contributions of this work are threefold: • We define source-grounded feature inversion as upstream-state estimation along the target-generating sample’s local computational geometry, making that computation part of the inverse object rather than relying on target matching alone. 2
Source-Grounded Feature Inversion
A P REPRINT
• We derive a local, closed-form matrix Wiener repair that converts VJP adjoints into upstream-state estimates, corrects predictable operating-point-specific residuals, and composes the repaired states through the computational DAG in one finite reverse pass. • We demonstrate a reusable layer-wise and channel-wise interpretation interface across diverse model computations, and connect its visualised feature groups to independent interventions on the corresponding internal representations. Together, these contributions open the model’s hidden feature hierarchy to direct inspection at individual layers and channels, linking the input structures selected by internal features to the evidence that shapes the model’s prediction.
2
Related Work
2.1
Iterative Preimage Optimisation
The feature-inversion literature has conventionally formulated inversion as a preimage search. Given a target representation, iterative methods optimise an input whose re-encoded feature matches that target, typically together with hand-designed natural-image regularisers [1, 2]. This formulation is useful for probing which inputs are compatible with a representation, but the matching constraint generally defines an equivalence class rather than a unique inverse. Initialisation, regularisation, and the optimisation trajectory therefore select one member of that class. Moreover, the network derivatives used by the optimiser are evaluated along the evolving candidate input. We distinguish this target-consistent preimage search from source-grounded inversion, whose query also includes the forward operating point that generated the selected feature and whose reverse computation follows that source-local branch. 2.2
Learned Feature Decoders
Another line of work learns a mapping from internal representations to images. Convolutional decoders can be trained to invert fixed representations, while autoregressive density models can represent a conditional distribution of inputs given supervised features [3, 4]. These approaches can produce recognisable images, but their outputs are jointly determined by the target feature and an independently trained decoder with its associated architecture and data distribution. They also commonly require a separate training problem for a particular representation family, layer, or visual domain. The conditional mean used in our theory is instead a statistical target for local upstream-state estimation, not a separately trained image generator. Our deployed maps repair source-local VJP and JVP observables at computational boundaries and are composed through the frozen model’s actual DAG. 2.3
Gradient and Reverse Visualisation
Gradient and reverse-visualisation methods answer a different family of questions. Input gradients expose local sensitivity [5]; deconvolutional visualisation and Guided Backpropagation modify the reverse rules used to display activation patterns [6, 7]; and Grad-CAM aggregates class gradients into a spatial localisation map [8]. These methods do evaluate the model at the sample being explained, but the propagated object is a sensitivity, attribution, or localisation signal. In differential terms, ordinary backpropagation transports a cotangent through the adjoint of the local Jacobian; it does not by itself estimate the corresponding upstream forward state. Sanity checks have also shown that some saliency procedures can remain visually similar after model or data randomisation, motivating method-specific tests of model dependence rather than treating visual sharpness as sufficient evidence [9]. Source-grounded feature inversion retains the true VJP dependencies, but statistically repairs the adjoint signal into an upstream-state estimate before continuing through the graph. 2.4
Representation Interpretation and Decision Grounding
Representation-level interpretability has also been pursued by assigning semantic meaning to units and feature directions. Network Dissection measures the alignment between individual units and a labelled concept vocabulary [10]; TCAV tests the directional sensitivity of predictions to user-defined concepts [11]; and ACE discovers concepts automatically before measuring their importance [12]. These methods identify semantic units, concept directions, or influential feature groups, but they do not construct a source-grounded input-domain inverse for an arbitrary layer, channel, or channel group. Our prediction-conditioned atlases use the inverse to visualise the input-domain structure selected by a feature group, while an independent intervention on the original internal representation measures that group’s effect on the current decision. The inverse image is not re-input as a causal stimulus. Prior work therefore searches for target-consistent inputs, learns separate feature-to-image mappings, or propagates sensitivity and relevance signals. Source-grounded feature inversion 3
Source-Grounded Feature Inversion
A P REPRINT
instead treats the selected feature and the target-generating sample’s local computational geometry as a joint inversion object, then traces it through the actual DAG using reusable closed-form repairs.
3
Method: Closed-Form Matrix Wiener Repair
We use feature inversion as a representation-level interface for model interpretation: the goal is to expose which input-domain structures are identified by a selected internal feature at a particular layer or channel. The output is therefore not posed as conventional natural-image reconstruction from a compressed code. It is the input-domain state obtained by tracing the selected feature along the target-generating sample’s own local inverse branch. The known input instantiates that branch through the frozen model’s actual forward activations and differential operators; image-space similarity to the input is a descriptive alignment diagnostic rather than the definition of a correct feature inverse. We formulate feature inversion as local cotangent repair on the computational directed acyclic graph (DAG) of a frozen model. Raw backpropagation already follows the correct target-to-input topology, but its vector-Jacobian products (VJPs) are adjoint transport signals rather than inverse representations [13, 14]. Our method calibrates closed-form local repair maps at selected tensor boundaries and composes them from the target toward the input. The final estimator has two closed-form stages. First, a local matrix Wiener map Gv transforms a mean-seed VJP into an upstream-state estimate and amortises the missing covariance correction over the calibration distribution [15]. Because the VJP is evaluated at the current forward point, this stage already retains source-local differential geometry and is often strong when the local operator is fixed or weakly sample dependent. Second, a Jacobian-vector product (JVP) at the same forward point exposes the mismatch left by the shared first-stage map, and a second Wiener map Dv predicts the remaining local state error. This correction applies to any predictable first-stage residual. Evaluating the JVP at the actual source forward point is especially relevant for sample-dependent local operators. The setting is input-conditioned: the input x remains available to instantiate the frozen forward point and its JVP/VJP operators. The reverse recursion is seeded by the selected target state and does not use the true intermediate activations as online inverse targets. There is no direct additive pathway from x to the visualisation; x enters only through the selected target state and the local operators evaluated at its forward point. Figure 1 contrasts this reusable source-grounded pass with per-query target-preimage optimisation. The section first defines the graph-local observable, then derives its closed-form repair class and calibration. It next introduces JVP-FC, composes both stages through the boundary DAG, and closes with deployment conventions. 3.1
Feature Inversion as Local DAG Cotangent Repair
Let f be a frozen differentiable model and let G = (V, E) denote the tensor-level computational DAG exposed by autograd. For a selected boundary node v ∈ V, its forward activation on input x is denoted Hv (x). Let Cv be the selected non-overlapping child frontier of v on paths to the target. If two candidate children are nested, the upstream candidate is removed so the same downstream computation is not counted twice. The precise frontier construction is given in section A.1. For each child u ∈ Cv , fix the input x and hold any external subgraph inputs that are not descendants of v at their forward values. The intervening subgraph then defines a local partial map Hu (x) = fu←v,x (Hv (x)),
Ju←v (x) =
∂fu←v,x . ∂Hv Hv =Hv (x)
This Jacobian includes all directed paths from v to u inside the local subgraph and is the derivative applied by autograd. The local source used to calibrate node v is the VJP induced by a channel-averaged child energy. If child u has Cu channels, define 1 su (x) = Hu (x). (1) Cu Equivalently, su is the gradient of the scalar local functional Ψu (Hu ) = ∥Hu ∥2F /(2Cu ). Since the VJP is linear in its seed, the resulting VJP source is the average of the channel-energy cotangents of the child tensor. The factor 1/Cu is part of the seed definition and is applied at the child where the seed is created; coordinate positions are not averaged in this gauge. For a node with multiple selected children, autograd sums the child VJPs in its native multi-output gauge; no additional division by |Cv | is used. The local mean-seed VJP source is therefore X X Hu (x) Rv (x) = Ju←v (x)∗ su (x) = Ju←v (x)∗ . (2) Cu u∈Cv
u∈Cv
The calibration target is the forward activation at the same node: Gv (Rv (x)) ≈ Hv (x). 4
(3)
Source-Grounded Feature Inversion
a
A P REPRINT
b PER-QUERY SEARCH
CALIBRATE ONCE
Optimize a new preimage for every target
Follow the sample's source-local inverse branch actual forward DAG
noise z0 ↓ optimize through f
target hT
target match 0.958 input alignment 0.035
h2 h1
2,000 gradient steps same target · different seeds · different preimages
OFFLINE fit {Gv, Dv} once
hT
ONLINE one reverse pass 0 optimization steps
h3
source x finite source-local reverse pass
images depths channels groups
source reference
c
seed 101
seed 202
seed 303
1 fit → N feature-inverse queries
One calibration supports depth and channel queries without refitting depth queries middle h9
shallow h1
Source
channel / subset queries
deep h17
ch. 624
8-channel group
ch. 1501
image 1
image 2
Target match does not imply
e source identity
cosine
cosine (descriptive)
f
1.0
0.95
0.91
input alignment
mean ± 95% CI n = 1,024 high-frequency structure
0.87 1
5
9
17
target depth
0.5
target-feature match input alignment
0.0
cumulative online time (s)
d Fixed-map behavior across depth
three initializations mean and range
0
5
10
15
20
25
wall time per query (s)
30
Post-calibration online cost 0.13 s vs 23.01 s/query 180.7× lower online cost
104 103 102 101 100 10
iterative optimisation ours
−1
10
0
10
1
102
103
number of mixed queries
Figure 1: From target-preimage search to source-grounded feature inversion. a, Canonical iterative inversion starts from noise and repeats target-matching optimisation for every query; three seeds reach similar target-feature match but weak input-instance alignment. b, The proposed regime calibrates reusable local repairs and applies one finite reverse pass through the target-generating sample’s source-local boundary DAG. c, One ResNet50 map family serves image, depth, channel, and channel-group queries without refitting. d, Input-instance alignment and high-frequency structure across target depths (mean and 95% confidence interval, n = 1,024). e, Optimisation target-feature match and input-instance alignment versus wall time (mean and range over three initialisations). f, Post-calibration cumulative online time for repeated mixed queries; calibration time is excluded to isolate deployment-time scaling. Source images are denormalised for display, and inverse tensors use one global per-image 1st–99th percentile scaling. This fit is strictly graph-local: Rv depends only on the current boundary node, its selected child boundaries, the local Jacobians between them, and the child activations on the same input. Here local refers to adjacency in the selected boundary DAG, not necessarily to spatial locality; an adjacent subgraph may contain global attention or pooling. 3.2
Local Matrix Wiener Repair
After identifying the batch and channel axes, write each tensor, up to a notational axis permutation, as Hv ∈ RB×C×d1 ×···×dk , 5
(4)
Source-Grounded Feature Inversion
A P REPRINT
Table 1: One tensor rule for different node types. Node type
Tensor shape
Channel axis
Coordinate transform
CNN feature ViT token stream Vector / GAP / MLP Attention tensor
(B, C, H, W ) (B, N, D) (B, D) (B, h, N, N )
C D D h
2D Fourier over (H, W ) 1D Fourier over N None; one matrix 2D Fourier over query-key axes
where B is the batch dimension, C is the channel or feature dimension, and d1 , . . . , dk are coordinate dimensions. The coordinate dimensions may be spatial positions, token positions, query-key coordinates, or absent. The method uses one tensor rule: Fourier-transform the coordinate axes and use a full matrix over channels at each coordinate frequency. Let Fk be the k-dimensional Fourier transform over coordinate axes, with F0 defined as the identity. For the mathematical derivation we use the unitary convention for Fk , so that Parseval’s identity holds exactly. The implementation uses a real fast Fourier transform (FFT) and stores only the non-redundant spectrum, which represents the same real frequency-diagonal operator family. The relative ridge is defined over these stored bins; a change in FFT normalisation rescales the stored second moments and effective ridge consistently. We denote Hv′ = Fk Hv ,
Rv′ = Fk Rv .
At each coordinate frequency ω, the repair applies a full C × C matrix: Gv (Rv ) = Fk−1 [Gv (ω)(Fk Rv )(ω)] ,
Gv (ω) ∈ CC×C .
(5)
Thus channels may mix arbitrarily at a fixed coordinate frequency, while different coordinate frequencies are not coupled by the repair operator. This is the only cross-coordinate coupling restriction on the local map. It is introduced for statistical tractability and computational efficiency, not as a claim that the original network is translation equivariant. The estimator additionally remains graph-local, linear, and zero-intercept. Table 1 summarises the resulting axis rule for the tensor families considered in this paper. 3.3
Closed-Form Local Calibration
For each node v and frequency ω, the map Gv (ω) is obtained from a ridge-regularised empirical least-squares problem on a calibration set Dcal : 2
G⋆v (ω) = arg min Ex∼Dcal ∥GRv′ (x, ω) − Hv′ (x, ω)∥2 + λ∥G∥2F . G
(6)
Define the cross- and auto-spectral matrices SvHR (ω) = E[Hv′ (ω)Rv′ (ω)∗ ] ,
(7)
SvRR (ω) = E[Rv′ (ω)Rv′ (ω)∗ ] .
(8)
All expectations are empirical averages over calibration images. For each image and stored frequency, Hv′ (ω) and Rv′ (ω) are C-dimensional channel vectors; the batch dimension therefore contributes independent calibration samples and is not a matrix axis of Gv (ω). This convention assumes evaluation-mode models whose forward operators do not couple different batch elements, as in all experiments reported here. For λ > 0, the unique Tikhonov-regularised solution is the matrix Wiener map G⋆v (ω) = SvHR (ω) SvRR (ω) + λI
−1
.
(9)
No gradient-based parameter training is used for these maps. They are determined entirely by second-order calibration statistics and a scalar ridge parameter. In experiments we set the effective ridge by the relative rule RR λeff = ρ max meanω,c Sv,cc (ω), ϵridge , ρ = 0.01, ϵridge = 10−30 , (10) and use λ = λeff in eq. (9). The diagonal entries of SvRR (ω) are real and nonnegative because SvRR (ω) is Hermitian positive semidefinite; numerically, the real diagonal is used for this average. The numerical floor only protects a degenerate zero-source case and is negligible at ordinary source scales. The calibration of Gv is independent across selected nodes once the boundary DAG is fixed. Each node uses its own local child set Cv and the local source in eq. (2). The maps may therefore be solved in reverse order, shallow-to-deep order, or in parallel, provided that each node is paired with the same local source and target variables. 6
Source-Grounded Feature Inversion
3.4
A P REPRINT
JVP Forward-Consistency Correction
The first-stage source Rv (x) already contains the local Jacobians at the current source operating point through its VJP construction. However, the map Gv is shared across calibration and deployment samples and therefore represents a distribution of local source-to-state relations. This distributional projection can be accurate when the local operator is fixed or varies weakly across samples. When normalisation, attention, or data-dependent gating or routing makes the local operator strongly sample dependent, the ideal correction also changes with the operating point. JVP-FC constructs a second observable by testing the first-stage estimate under the actual differential at the source forward point. Its formal regression can correct any predictable first-stage residual, while its principal motivation is the operating-point-specific mismatch left by a shared first-stage map. The first-stage estimate
yv0 (x) = Gv (Rv (x)) (11) is a zero-intercept state estimate in the activation coordinates of node v. The remaining error is measured in the same local linear gauge used by autograd. Although yv0 is stored in activation coordinates, it is supplied to the JVP as a tangent vector at the fixed forward point. We ask whether the differential of the current local subgraph maps this vector to the child state observed at the adjacent boundary. We then repair the induced residual by a second closed-form map. For each child u ∈ Cv , compute the JVP at the actual forward point: b u|v (x) = Ju←v (x) yv0 (x), H
b u|v (x). eu|v (x) = Hu (x) − H
(12)
This residual is well defined in the child coordinate space, but it is not assumed to vanish when yv0 = Hv . Writing ∆v = Hv − yv0 gives the exact decomposition eu|v = Hu − Ju←v Hv +Ju←v ∆v . | {z }
(13)
bu|v
The base term bu|v is generally nonzero for maps with offsets or non-homogeneous nonlinearities. Equality Hu = Ju←v Hv does hold when the local map is positively one-homogeneous at the forward point, including the bias-free linear case, but it does not hold in general. The second term Ju←v (x)∆v transports the first-stage state error through the current sample’s local differential and therefore exposes the source-specific response of that error. JVP-FC therefore learns a distributional residual-to-error relation rather than enforcing an exact nonlinear forward equation. The correction source is the VJP of this child residual: X Cv (x) = Ju←v (x)∗ eu|v (x). (14) u∈Cv
Holding the forward point and child states fixed, this source is the negative gradient of the local linear child-consistency energy 1 X 2 Φv (y) = ∥Hu − Ju←v y∥F , Cv = −∇y Φv (y) y=y0 . v 2 u∈Cv
The first-stage seed is the differential of channel-averaged child energy, whereas the JVP-FC source pulls back a Euclidean child-state residual. No additional channel averaging is applied to eu|v ; the same residual gauge is used during calibration and online inversion. We fit a second matrix Wiener map Dv in the same frequency-diagonal, full-channel class: 2
Dv⋆ = arg min Ex∼Dcal D(Cv (x)) − (Hv (x) − yv0 (x)) 2 + λD ∥D∥2F . D
(15)
Equivalently, with ∆v (x) = Hv (x) − yv0 (x), Cv′ = Fk Cv , and ∆′v = Fk ∆v , the correction is solved frequency by frequency as −1 Dv⋆ (ω) = Sv∆C (ω) SvCC (ω) + λD I , Sv∆C (ω) = E[∆′v (ω)Cv′ (ω)∗ ], SvCC (ω) = E[Cv′ (ω)Cv′ (ω)∗ ]. (16) The correction ridge λD is set by the same relative rule as eq. (10), but using the diagonal mean of the correction-source second moment SvCC . The repaired local state is yv (x) = yv0 (x) + Dv⋆ (Cv (x)).
(17)
This stage is still closed-form: Dv is solved by the same spectral second-order estimator as eq. (9), with Cv as source and Hv − yv0 as target. The JVP is the differential of the current local subgraph at the actual forward activation. It defines the child response in the zero-intercept linear gauge of the repair problem; the method does not run the nonlinear subgraph on yv0 as if it were a new forward activation, nor does it require that Ju←v (x)yv0 equal fu←v (yv0 ). 7
Source-Grounded Feature Inversion
3.5
A P REPRINT
Topology-Respecting Feature Inversion
At inversion time, the solved maps are composed from the target toward the input. Choose a nonempty terminal channel set S ⊆ {1, . . . , CT }, let PS mask all channels outside S, and define CS = |S|. The terminal state and its online cotangent seed are yT PS H T yT = PS HT , son = . (18) T = CS CS The full target uses PS = I and CS = CT . A single-channel target uses CS = 1. For a shallower node v, assume that online child states {yu : u ∈ Cv } have already been inverted to that boundary. The seed attached to each online child is yT /CS , u = T, son = (19) u yu /Cu , u ̸= T. Thus only the terminal child uses the selected-channel count CS . Every inverted nonterminal child uses its physical channel count Cu . The online local source is X Rvon (x) = Ju←v (x)∗ son (20) u (x). u∈Cv
The first-stage online estimate is yv0 = Gv (Rvon ). The online child residual uses the repaired child states, not the true child activations or their normalised seeds: X Cvon (x) = Ju←v (x)∗ yu (x) − Ju←v (x)yv0 (x) . (21) u∈Cv
The final online state is yv (x) = yv0 (x) + Dv⋆ (Cvon (x)).
(22)
When v is the input node, yv is the reported input-domain feature inverse. Changing S changes only the terminal initialisation and the states induced by the ensuing recursion; the calibrated local maps remain fixed. Branches and multi-use tensors are handled by native autograd semantics. Residual additions distribute cotangents to branches, concatenations slice them back to their inputs, and one-to-many uses sum incoming cotangents at the shared tensor. Our method repairs the cotangent or state at selected tensor boundaries; it does not introduce special component-specific inverse rules. The estimator is component-agnostic once architecture-aware boundary and axis metadata have been specified. 3.6
Boundary Modes and Architecture-Agnostic Axes
The theoretical maximal form assigns a local matrix Wiener repair to every captured differentiable floating tensor node whose channel and coordinate axes are specified. Scalar nodes, non-floating tensors, and bookkeeping operations without a meaningful channel axis are traversed by autograd but are not instantiated as repair nodes. The implementation supports three boundary modes: • Full-DAG: instantiate repairs at all captured differentiable floating tensor nodes with a specified channel axis. • Coarse/stage block: instantiate repairs at the input, stem or patch embedding, and architecture block or stage edges; this is the default efficient mode for cross-model evaluation. • Custom list: capture the full DAG but instantiate only user-specified nodes. These are instantiation choices, not different mathematical methods. All modes use the same local source, Wiener repair, and JVP-FC correction in eqs. (2), (9) and (15). The axis rule is architecture-agnostic. The batch axis is fixed to dimension zero. The channel axis is either inferred or specified by the user. All remaining axes are coordinate axes and are transformed by Fk . Singleton coordinate axes that only record global pooling output are treated as trivial and may be collapsed before this rule is applied. For ViT token streams, this gives a 1D Fourier transform over token positions and a full matrix over hidden features. For attention weights or logits, the repaired object is a cotangent or local state estimate, not a probability matrix that is fed through the forward model. Softmax simplex constraints are therefore not explicitly enforced by the reverse estimator. The same closed-form rule is algebraically applicable, but the adequacy of this relaxation for internal attention tensors remains an empirical question. 8
Source-Grounded Feature Inversion
3.7
A P REPRINT
Practical Details
We use zero-intercept matrix maps throughout. There is no additive bias term in either Gv or Dv . Consequently, the fitted maps cannot inject a fixed target-independent template; all inverted structure must enter through the selected target state, the model Jacobians, and the calibrated linear repairs. Full matrix calibration requires sufficient effective samples for each C × C second-moment estimate. In high-dimensional token nodes, calibration size is a dominant practical factor; our ViT token-stream runs use the larger 4096-image calibration setting for this reason. Unless stated otherwise, the evaluated configuration uses mean-full seeds, meaning that all child channels are retained and divided by their channel count as in eq. (1). It also uses coarse/stage boundaries, no source normalisation, full-channel per-frequency solves, and unit JVP-FC residual and correction weights. These definitions complete the deployed algorithm. The next section derives its statistical and geometric motivation and separates the exact closed-form statements from the empirical proxy assumptions.
4
Theory: From Optimal Inversion to Closed-Form DAG Repair
We derive the deployed estimator through a sequence of explicit relaxations. Throughout this section, inversion means upstream-state estimation along the source sample’s local network geometry. The estimated objects are tensor states on this inverse branch, not generic natural-image reconstructions of the source input. The starting point is the minimummean-square statistical inverse, namely the conditional mean of an upstream state given a downstream observation. Under a centred local linear-Gaussian reference model, the conditional mean of the local deviations reduces to a covariance-weighted Wiener operator. The same operator is a natural adjoint under covariance-induced metrics. Directly constructing these boundary- and operating-point-dependent metrics is impractical. We therefore construct a graph-local Euclidean VJP observable and estimate its ridge-regularised closed-form distributional projection to the forward state. The VJP observable retains the current source-local Jacobian, but one fitted projection must interpret these observables across the calibration distribution. It can capture the dominant correction for fixed or weakly sample-dependent local operators, while strongly sample-dependent operators can leave an operating-point-specific error. A second Wiener projection uses a pulled-back JVP residual at the actual forward point to repair the predictable part of this remaining error. The resulting maps are then composed in one finite target-to-input DAG pass. 4.1
Optimal Statistical Inversion under Mean-Squared Error
Let Hv be an upstream state and Hu a downstream observation generated by a frozen model on the data distribution. Among all measurable estimators T , the MSE-optimal statistical inverse satisfies 2
T ⋆ ∈ arg min E ∥T (Hu ) − Hv ∥2 , T
T ⋆ (hu ) = E[Hv | Hu = hu ].
(23)
Thus the conditional mean is the ideal MSE estimator of the upstream state from the downstream feature alone. It is generally unavailable because it requires the full conditional law of Hv given Hu . Our setting is additionally input-conditioned. The input x being explained remains available to instantiate the actual forward activations and local derivatives of the frozen model. We do not replace eq. (23) by E[Hv | Hu , x]: in a deterministic frozen model, conditioning on the full input makes Hv known and no longer describes inversion from the downstream feature. Instead, x selects the forward point and local operators used to replace one global nonlinear inverse by a sequence of local inverse problems. 4.2
Local Linearisation and the Adjoint–Inverse Gap
Let a local downstream map y = f (h) be linearised at the current activation h0 as δy = J(h0 ) δh. Backpropagation of a seed q through this mapping gives r = J(h0 )⊤ q.
(24)
This is the Euclidean adjoint of the local Jacobian. We use ⊤ for real-domain Jacobians and ∗ for conjugate transpose in complex or Fourier coordinates. In general, J ⊤ ̸= J † , (25) except in special partial-isometry cases, where every nonzero singular value of J equals one. Raw VJPs therefore follow the correct computational dependencies but carry the geometry of adjoint transport rather than that of an upstream-state inverse. 9
Source-Grounded Feature Inversion
A P REPRINT
To express the missing geometry, equip the parent and child feature spaces with Hermitian positive-definite metric operators Mv and Mu . We use the weighted inner products ⟨a, b⟩Mv := a∗ Mv b,
⟨c, d⟩Mu := c∗ Mu d.
(26)
The unique Riesz adjoint of Ju←v under these metrics is ♯ ∗ Ju←v = Mv−1 Ju←v Mu .
(27)
Indeed, equality ♯ ⟨Ju←v δv , su ⟩Mu = ⟨δv , Ju←v su ⟩Mv ♯ ∗ for all δv and su implies Mv Ju←v = Ju←v Mu . The operators Mu and Mv are the metrics themselves. The factor Mv−1 appears when the adjoint is represented in parent coordinates.
4.3
Wiener Inversion as a Covariance-Induced Natural Adjoint
The statistical and geometric views coincide in a local linear-Gaussian model. Fix a forward point and hold Ju←v fixed in the following idealised local ensemble. Vectorise the local tensors for this unrestricted reference calculation, and e v and H e u for centred parent and child deviations. Assume write H e v∗ ] = 0. E[η H
e u = Ju←v H e v + η, H
(28)
For multiple children, stack their states and local Jacobians; the same formulas then apply to the resulting block operator. ev H e v∗ ], the residual covariance Ση = E[ηη ∗ ], and the child covariance Define the parent covariance Σv = E[H ∗ euH e u∗ ] = Ju←v Σv Ju←v Σu = E[H + Ση .
(29)
Assuming Σu is nonsingular, the linear minimum-mean-square-error (LMMSE) estimator is LMMSE ∗ Tu→v = Σv Ju←v Σ−1 u .
(30)
ev | H e u ], the zero-mean specialisation of For jointly Gaussian centred deviations, this linear estimator equals E[H eq. (23). If Σv is also nonsingular, choose covariance-induced precision metrics Mv = Σ−1 v ,
Mu = Σ−1 u .
(31)
Substitution into eq. (27) gives ♯ ∗ LMMSE = Σv Ju←v Σ−1 . (32) Ju←v u = Tu→v Hence, after vectors and covectors are identified through the stated metrics, the local Wiener inverse and the natural adjoint are the statistical and geometric forms of the same covariance-corrected local operator. Centring is used only for this ideal reference identity. The deployed maps do not subtract means or fit additive biases; their guarantees refer to the zero-intercept objectives stated in the Method.
This identity is an ideal reference rather than our deployment formula. In a nonlinear network, the Jacobian and relevant local geometry depend on the input and boundary. Direct deployment would require estimating and applying nontrivial parent and child metrics at every boundary for every online sample. A projected local-oracle experiment validates the stated identity under its controlled assumptions (section C.6). The next step replaces explicit metric estimation by a fixed, observable Euclidean cotangent and its ridge-regularised distributional repair. 4.4
Observable Mean-Seed Cotangent and the Proxy Bridge
For a child boundary u with Cu channels, define Hu su = = ∇Hu Cu
1 ∥Hu ∥2F 2Cu
.
(33)
The graph-local Euclidean VJP observable is Rv =
X
∗ Ju←v
u∈Cv
Hu . Cu
(34)
It is exactly the adjoint pullback of the summed channel-averaged child energies. For multiple selected children, native multi-output VJP semantics sum the child cotangents; no additional division by |Cv | is introduced. 10
Source-Grounded Feature Inversion
A P REPRINT
The observable in eq. (34) retains the actual local Jacobian and graph topology, but it does not explicitly apply the covariance metrics in eq. (32). The ideal child-side metric would act before the pullback, while the ideal parent-side metric would act after it. Estimating both online is precisely the deployment obstacle identified above. We therefore use Rv as a graph-local proxy source and Hv as its desired forward-state target. This substitution creates the main empirical bridge in the derivation. Jacobian sparsity guarantees that the VJP follows computation dependency, but it does not imply pointwise equality between cotangent support and activation support. The weaker assumption is that the organisation of Rv remains predictive of the spatial or semantic organisation carried by Hv . No-refit transfer across target depths and channel masks tests whether this relation is node-local, while cross-architecture and component experiments test the same estimator class and public recipe (sections C.2, C.3, 5.2 and 5.3). 4.5
Closed-Form Distributional Wiener Repair
Once (Rv , Hv ) is chosen as the observable source-target pair, the MSE-optimal nonlinear predictor would be E[Hv | Rv ]. We instead restrict the repair to the frequency-diagonal, full-channel linear operator class in eq. (5). Within this class, calibration solves G⋆v = arg min E∥G(Rv ) − Hv ∥22 + λ∥G∥2F . (35) G
For λ > 0, frequency-wise ridge regression gives the unique ridge-regularised matrix Wiener solution −1 G⋆v (ω) = SvHR (ω) SvRR (ω) + λI .
(36)
This is the same estimator defined in eqs. (6) and (9). It requires only the joint second moments of the observable and target. Gaussianity is not required for optimality of the stated ridge objective within the chosen linear class. The role of Gv is now precise. It is the unique ridge-regularised distributional projection from the Euclidean VJP observable to the upstream forward state in the selected operator family. It does not identify Ju←v , Mu , or Mv . Nor is it generally algebraically equal to the ideal natural adjoint. Rather, it amortises the missing covariance correction over the calibration distribution in a tractable spectral class. The source Rv (x) retains the actual Ju←v (x) through the VJP, but the same fitted Gv is applied to every sample at node v. When the local operator and covariance geometry are fixed or vary weakly, this shared projection can account for most of the required correction. When they vary strongly with the source operating point, Gv can leave a sample-specific state error even though its input remains source grounded. 4.6
JVP Residual Correction as a Second Wiener Projection
The second projection is designed to expose this operating-point-specific mismatch, while remaining applicable to any predictable first-stage error. Let yv0 = Gv Rv . At the actual forward point, the local JVP maps this estimate to the child differential coordinates. Here yv0 is used as a tangent vector in the parent coordinate space, not as a replacement forward activation. For each child, define eu|v = Hu − Ju←v yv0 .
(37)
eu|v = Hu − Ju←v Hv +Ju←v (Hv − yv0 ). | {z }
(38)
The residual has the exact decomposition bu|v
The base term bu|v records offsets and non-homogeneous components absent from the zero-intercept differential. Consequently, the residual need not vanish even when yv0 = Hv . It does vanish at that point for a positively onehomogeneous local map, as follows from Euler’s homogeneous-function identity. The second term Ju←v (x)(Hv − yv0 ) evaluates the first-stage state error under the current sample’s differential. It therefore carries the response of the shared first-stage error under the present operating point. Pulling the child residuals back gives a second observable X ∗ Cv = Ju←v eu|v .
(39)
u∈Cv
P Equivalently, with Φv (y) = 12 u∈Cv ∥Hu − Ju←v y∥2F , one has Cv = −∇Φv (yv0 ). Thus the second observable is a local child-consistency descent cotangent, not an arbitrarily chosen residual summary. The correction map is the second closed-form projection 2 Dv⋆ = arg min E D(Cv ) − (Hv − yv0 ) 2 + λD ∥D∥2F . (40) D
11
Source-Grounded Feature Inversion
A P REPRINT
Thus Dv has the same ridge-regularised Wiener optimality as Gv , but for a residual-to-error regression problem. This is the objective in eq. (15). The guarantee is conditional on the observable Cv . Child-residual components annihilated by the summed pullback do not enter Cv , and parent-error components that are not statistically predictable from Cv cannot be estimated by this stage. The child prediction Ju←v yv0 is a JVP response at the current forward point. It is not a new nonlinear evaluation of fu←v on yv0 . JVP-FC is therefore a second distributional projection in a specified local linear gauge, not an assertion that the nonlinear subgraph has no offsets or higher-order terms. Matched cross-architecture ablations test both the aggregate residual reduction and the non-monotonic boundary-level behaviour predicted by this distributional, rather than pointwise, guarantee (sections C.5 and 5.5). 4.7
Finite-DAG Composition and Stability
At deployment, the calibrated maps are composed from the selected terminal state toward the input. The input x fixes all forward activations and Jacobians, so each local online update is linear in its supplied inverted child states. For the concrete forward-state error statement in this subsection, consider full-target inversion, where yT = HT and the terminal error is zero. Define δv = yv − Hv . Equip every selected node space with a fixed norm and every inter-node block with its induced operator norm. The supplementary derivation constructs an effective two-stage reverse operator Kv and gives the exact recursion δv = ϵv + Kv δCv ,
(41)
where ϵv is the local defect obtained when the true child states are supplied. This identity separates local approximation error from error propagated by imperfect online children. If every local defect is zero, reverse topological induction gives exact input state inversion on the selected boundary DAG. More generally, writing κvu for the operator norm of the block mapping child u to parent v, the sufficient condition X sup κvu ≤ q < 1 v̸=T u∈C
v
implies maxv̸=T ∥ϵv ∥ . 1−q The pathwise bound remains informative when this uniform condition is not satisfied. Because the algorithm is one finite reverse pass, this is a stability result rather than convergence of an iterative fixed-point method. The target-conditioned general form and proof are given in section B.6. The empirical path-gain study in section C.6 does not provide a stable global error ranking, so it is used only as a conservative diagnostic and not as evidence of contraction or global convergence. max ∥δv ∥ ≤ v
4.8
Guarantees, Assumptions, and Boundaries
Both calibrated maps are zero-intercept: yv0 = Gv Rv ,
yv = yv0 + Dv Cv .
If the selected terminal state is zero, induction through the online recursion gives zero mean seeds, zero VJP sources, zero residual sources, and zero inverted states. The estimator therefore has no additive target-independent template. This is not a support non-expansion theorem because spectral multiplication can spread energy in coordinate space. The main structural restriction is coordinate-frequency diagonality. Channels may mix through a full matrix at each coordinate frequency, but different frequencies do not mix inside a repair map. This restriction makes full-channel calibration statistically and computationally tractable. Its adequacy for a given tensor family remains an empirical modelling question. The derivation therefore separates three levels of claim. The conditional-mean, natural-adjoint, Wiener, and finite-DAG statements are mathematical results under their stated conditions. The replacement of explicit sample-dependent metrics by a fixed distributional projection is a modelling decision. JVP-FC adds a second observable evaluated at the actual operating point, but it does not identify the online metrics or guarantee exact removal of every sample-dependent mismatch. Its approximation quality, the observability of the two regression sources, and the adequacy of the frequency-diagonal restriction are established empirically rather than by algebraic identity. These distinctions determine the experimental sequence: reuse and structural scope test the proxy bridge, matched ablations test both projections, and information-flow controls test the meaning and limits of the composed inverse. 12
Source-Grounded Feature Inversion
5
A P REPRINT
Experiments
The experiments test a single claim–evidence chain. We first ask whether one calibrated map family supports many feature-inverse queries without per-target optimisation. We then test whether the same public recipe applies across architectures, tensor components, and visual distributions. Matched ablations isolate the roles of local DAG composition, the two closed-form stages, frequency dependence, and cross-channel coupling. Finally, target–operator controls define what determines a feature inverse, and representation interventions connect the same visualised feature groups to the model decision. Calibration, boundary, runtime, and memory audits then delimit the practical deployment regime. 5.1
Experimental Protocol and Evaluation Axes
Final method. Unless stated otherwise, every experiment used the deployed configuration in section 3: local mean seeds, zero-intercept full-channel per-frequency Wiener maps for both Gv and Dv , model-native coarse or blockendpoint boundaries, and repaired online child states. No source, activation, or cotangent pre-normalisation was applied. The relative ridge was fixed to ρ = 0.01 for both stages. Raw VJP, Gv -only, global-map, and restricted-operator variants were used only as explicitly labelled controls. Data and evaluation. The ImageNet protocol used 4,096 randomly sampled training images for calibration and 1,024 disjoint validation images for evaluation [16, 17]. Calibration and evaluation seeds were fixed to 123 and 456, respectively. The representative distribution study used dataset-trained checkpoints on ImageNet, Oxford-IIIT Pets, and CUB-200-2011 [18, 19]. Pets used all 3,680 available training images for calibration, while the other dataset-specific runs used 4,096 images. The decision-conditioned study used 16 fixed correctly classified CUB test images per model and CUB-trained ResNet18, ConvNeXt-B, and ViT-B/32 checkpoints. Exact headline model, terminal-tensor, boundary, and axis records are reported in section C.1 and table 6; calibration/evaluation indices and preprocessing identifiers are retained in the run artifacts. We abbreviate a run with N calibration images and M evaluation images as cN /eM . Metrics and interpretation. Pixel cosine, low-pass luminance cosine, high-pass luminance cosine, chroma cosine, SSIM, and LPIPS describe alignment between a feature inverse and the known target-generating input [20, 21]. They are input-instance alignment diagnostics, not ground-truth correctness measures for an inverse. This distinction is essential for channel and channel-subset targets, whose theoretical target is PS HT rather than the complete input. Re-encoding cosine is reported separately as a representation-consistency diagnostic. All main ImageNet means use 1,024 evaluation images; per-image distributions and uncertainty summaries are provided in the supplement. 5.2
Feature Inversion at Scale: One Calibration, Many Queries
We first tested whether feature inversion could be used as a reusable interpretation interface rather than a separate optimisation problem for each query. A single ResNet50 calibration was reused across 1,024 images, four target depths, four individual terminal channels, and one eight-channel terminal subset. Changing the image, target depth, or channel mask did not refit Gv , Dv , the boundary DAG, or the ridge. Across the four full-state depths, pixel cosine remained between 0.928 and 0.945. The selected-channel targets produced distinct outputs under the same map family, confirming that the public terminal-subset gauge supports no-refit channel queries. Figure 2 therefore emphasises scale and reuse before any favourable case study. The sample indices, ordering, target nodes, and display transform were fixed before final rendering. The complete fixed list, additional target depths, channel queries, and per-image measurements are provided in section C.2. Comparison with iterative preimage search. We next compared the operational regimes on the same 16 ResNet18 images and terminal target. The optimisation reference used three predeclared initialisations and 2,000 Adam steps per image, with learning rate 0.05, total-variation weight 0.1, ℓ2 weight 0.001, and four-pixel jitter [22, 23]. It reached a target re-encoding cosine of 0.958 but had an input-instance pixel cosine of 0.035; mean pairwise output cosine across initialisations was 0.101. After calibration, the closed-form method reached an input-instance pixel cosine of 0.939 on the same images without online optimisation. These values should not be read as a strictly matched estimator comparison: the optimiser searches for a target-consistent preimage from noise, whereas our estimator additionally uses the target-generating sample’s local network geometry. The result instead demonstrates that the two procedures instantiate different inverse objects and information sets. Table 2 reports the matched operational comparison. On sequential batch-one image queries, post-calibration application required 0.139 s per query versus 24.379 s for iterative optimisation, a 176× reduction in online time. Across image, depth, channel, and mixed workloads, the closed-form path required 0.110–0.141 s per query, whereas the 2,000-step reference required 21.728–23.578 s. These 13
Source-Grounded Feature Inversion
a
A P REPRINT
one calibration · 24 images · no refit Source
Deep inverse
Source
Deep inverse
Source
Deep inverse
Source
Deep inverse
b Source
Early
Middle
Deep
four fixed depth trajectories under the same maps
Figure 2: Feature inversion at scale under one calibration. a, The dominant plate contains 24 predeclared ImageNet source references paired with deep ResNet50 feature inverses. b, The lower strip shows four fixed examples across early, middle, and deep targets. All outputs use one calibrated map family, one public inversion path, and no query-specific optimisation or refitting. The source images are references for input-instance alignment, not unique inverse labels. Table 2: Canonical feature-inversion regimes on a matched 16-image ResNet18 target. Pixel cosine is the mean descriptive input-instance alignment on the same fixed 16 images. Online time uses sequential batch-one image queries: the optimisation value is the mean over eight queries, while ours is the median over three isolated repeats. The optimisation reference uses 2,000 steps per image. Method Iterative preimage optimisation Ours, final JVP-FC
Source-local geometry
Per-query search
Online/query (s)
Pixel cosine
No Yes
2,000 steps 0 steps
24.379 0.139
0.035 0.939
measurements isolate deployment-time scaling: calibration is performed once and is characterised separately by sample demand, fitted-node count, map storage, and peak memory. 5.3
One Recipe Across Backbones and Tensor Components
We evaluated the same tensor-axis rule and public inversion path on residual, dense-connectivity, mobile, squeeze-andexcitation, modern convolutional, and Transformer families [24–31]. All nine ImageNet rows used the same 4,096/1,024 calibration/evaluation protocol. The final correction improved pixel alignment over Gv alone for every architecture, 14
Source-Grounded Feature Inversion
a
Source
ResNet50
DenseNet121
MobileNet V2
EfficientNet B0
RegNetY
A P REPRINT
ConvNeXt-B
Swin-T
ViT-B/32
same held-out image · Gv only (top) · final JVP-FC (bottom)
b
c Alignment / structure
JVP-FC improves every backbone family
ResNet50
Gv only
DenseNet121
Final JVP-FC
MobileNetV2 EfficientNet-B0 RegNetY ConvNeXt-B Swin-T ViT-B/32 0.0
0.4
0.6
0.8
Pixel cosine (descriptive input alignment)
ResNet50
0.931
0.957
0.940
ConvNeXt-B
0.845
0.288
0.651
ViT-B/32
0.702
0.759
0.773
final pixel cosine
ImageNet
Pets
0.6 0.4 0.2 0.0
1.0
ImageNet source
inverse
ViT-B/32
Pixel Pets source
Y-low
inverse
Y-high CUB source
SSIM
inverse
example 1
Dataset-trained distribution ladder
Swin-T
example 2
d
0.2
Transformer geometry is architecture-dependent
0.8
CUB
Image metrics describe alignment to the known target-generating input; they are not inverse ground truth.
Figure 3: Structural and distributional generality. a, One fixed held-out image across eight backbone families under one recipe, with Gv -only outputs above the final JVP-FC outputs. b, Compact quantitative summary of input-instance alignment and second-stage gain. c, Swin-T and ViT-B/32 on identical images, showing architecture-dependent spatial and frequency structure. d, Representative ImageNet, Pets, and CUB results. Full per-image distributions and component-level results are reported in the supplement.
with gains from 0.037 on ResNet18 to 0.391 on ViT-B/32 (table 3). The effect was therefore not restricted to one CNN topology or one normalisation regime. Figure 3 combines the cross-architecture, Transformer, and distribution evidence under the shared recipe. For MobileNetV2 and EfficientNet-B0, residual blocks remain single units ending at their residual adds, whereas each stage-entry non-residual block is factorised at its architecture-native expansion and depthwise tensors before the final projection. MobileNetV2 uses pre-activation convolution tensors; EfficientNet-B0 uses its post-activation spatial tensors while retaining squeeze-and-excitation gating and multiplication inside the same local edge. Both chains terminate at the final 320-channel feature block. This architecture-aware boundary onboarding changes neither the estimator, ridge, calibration size, nor online API, but raises final pixel cosine from the incomplete block-only capture to 0.833 and 0.796, respectively. The Swin–ViT contrast is particularly informative. Swin-T had lower pixel cosine than ViT-B/32 (0.504 versus 0.702) but substantially stronger SSIM and high-frequency alignment (0.519 and 0.614 versus 0.314 and 0.248). ViT-B/32 retained stronger low-frequency alignment (0.789). Strong spatial structure on Swin therefore rules out a blanket failure of the closed-form method on Transformers. The contrast instead shows that the revealed inverse geometry remains architecture dependent. It does not isolate patch size, hierarchy, or windowing as the sole cause. 15
Source-Grounded Feature Inversion
A P REPRINT
Table 3: One recipe across ImageNet backbone families. All values are means over 1,024 validation images. ∆ is the final-minus-Gv pixel-cosine gain. LPIPS is lower for stronger perceptual alignment. Backbone ResNet18 ResNet50 DenseNet121 MobileNetV2 EfficientNet-B0 RegNetY-1.6GF ConvNeXt-B Swin-T ViT-B/32
Gv pixel
Final pixel
∆
SSIM
LPIPS
0.901 0.889 0.751 0.781 0.659 0.872 0.734 0.178 0.312
0.938 0.931 0.824 0.833 0.796 0.914 0.845 0.504 0.702
0.037 0.042 0.074 0.052 0.137 0.043 0.111 0.325 0.391
0.684 0.726 0.588 0.710 0.722 0.597 0.617 0.519 0.314
0.314 0.277 0.419 0.260 0.272 0.423 0.399 0.520 0.757
Tensor-component coverage. The same public path was also evaluated on 11 component families, including DenseNet concatenations, depthwise and MBConv/SE states, ConvNeXt normalisation-adjacent tensors, Swin window and patch-merging states, ViT token streams, and pooled vectors. All 12 formal runs passed the component validator without a manual component-specific inverse. For the five ConvNeXt normalisation-adjacent targets, input-instance pixel cosine ranged from 0.943 to 0.957 and target re-encoding cosine from 0.935 to 0.981. The full target-node, shape, axis, and map-family audit is provided in section C.3. 5.4
Behaviour Across Visual Distributions
We next separated recipe generality from fixed-map transfer. Dataset-trained ResNet50, ConvNeXt-B, and ViT-B/32 checkpoints were calibrated and evaluated independently on ImageNet, Pets, and CUB under the same final recipe. ResNet50 remained strongly input aligned across the three datasets, with final pixel cosines of 0.931, 0.957, and 0.940. ViT-B/32 increased from 0.702 on ImageNet to 0.759 on Pets and 0.773 on CUB. ConvNeXt-B varied nonmonotonically, from 0.845 on ImageNet to 0.288 on Pets and 0.651 on CUB. The data therefore support distributionand model-instance-dependent behaviour, not a universal scalar ordering of dataset difficulty. The supplement expands this result in two ways. First, an ImageNet-weight breadth study evaluates all eight families on Pets and CUB without changing the public recipe. Second, a 3 × 3 calibration-source transfer matrix measures distribution transfer, while repeated calibration subsets quantify calibration-estimation variation separately. ResNet18 showed a small matched-versus-cross mean gap (0.937 versus 0.932), whereas ViT-B/32 showed a larger gap (0.721 versus 0.662). These experiments establish architecture-dependent transfer sensitivity without invoking a causal scalar dataset-complexity explanation. 5.5
Topology-Respecting Two-Stage Repair Explains the Gain
We used matched controls to isolate the method components. A single global closed-form map did not replace local DAG composition: pixel cosine was 0.072 for global G and 0.067 after a global correction, compared with 0.901 for local Gv and 0.938 for the final local method. Raw VJP was near zero (0.0002). The input-instance alignment gain therefore came from topology-respecting local composition, not merely from fitting one large linear map at the terminal target. The second stage improved all three representative architectures. On ResNet18, Gv alone reached a pixel cosine of 0.901 and the second stage added 0.037. The corresponding gain increased to 0.111 on ConvNeXt-B and 0.391 on ViT-B/32. Mean child-consistency NMSE fell by 23.0%, 15.4%, and 16.2%, respectively. Intermediate-state relative ℓ2 decreased by 1.5%, 0.4%, and 5.9%. This architecture-level ordering is consistent with JVP-FC becoming more important when normalisation and attention induce strongly sample-dependent local operators. Because architecture and component type were not independently controlled, the comparison does not attribute the gain to any single module. These measurements support JVP-FC as a residual repair stage, but they do not imply monotonic improvement at every boundary. Figure 4 aligns the quantitative ablations with fixed-image outputs and boundary-level residual diagnostics. The operator ablation was decisive. Gamma-only maps had near-zero global input-instance alignment on all three architectures. Per-frequency diagonal maps reached 0.253, 0.096, and 0.062, while retaining some visible low-frequency or spatial structure in selected cases. A rank-8 channel correction was useful on ResNet18 (0.834) but not on ViT-B/32 (0.050). Full channel rank with one shared-frequency map also remained far below the final operator. Thus crosschannel and frequency-specific second-order structure are both substantive, while low-rank compression is architecture dependent. 16
Source-Grounded Feature Inversion
a
Example 1 Gv only Raw VJP
Final JVP-FC
Example 2 Gv only Raw VJP
Source
Final JVP-FC
b
Mean child-consistency NMSE
ViT-B/32
ConvNeXt-B
ResNet18
Source
A P REPRINT
Local DAG composition beats a global repair
Raw VJP Global G Global G+D Local Gv Local final 0.0
0.2
0.4
0.6
0.8
1.0
c 1.5
JVP-FC reduces child residual ■ Before Dv
−23.0%
1.0
■ After Dv
−16.2% −15.4%
0.5
0.0 ResNet18
ConvNeXt-B
ViT-B/32
Pixel cosine
d
e
Channel coupling × frequency specificity
Source
ResNet18
0.006
0.253
0.834
0.271
0.938
ConvNeXt-B
0.003
0.096
0.476
0.209
0.845
-0.004
ViT-B/32
a
mm
Ga
0.062
q
fre
g./
Dia
0.050
r8
g.+
Dia
0.337
d
are
/sh
l Ful
0.702
Diag./freq
Diag.+r8
Full/shared
Full/freq
Visual outputs retain some coarse structure.
q
/fre
l Ful
Figure 4: Why topology-respecting two-stage repair works. a, Raw VJP, distributional Gv -only repair, and final JVP-FC outputs on two predeclared examples per architecture. b, Global single-boundary repair versus local DAG composition. c, Child-consistency residuals before and after JVP-FC. d, Restricted operator families across three architectures. e, Matched visual examples showing why scalar image metrics must be interpreted together with spatial and spectral structure.
5.6
Target Features and Source-Local Geometry Jointly Determine the Inverse
The method is intentionally conditioned on the forward point of the input being explained. We therefore tested target identity and operating-point geometry separately. For two images A and B, we crossed terminal states HA , HB with local operator families JA , JB . On ResNet18, both mismatched conditions followed the operating-point identity: HA with JB had pixel cosine 0.939 to B and 0.034 to A, while HB with JA had cosine 0.939 to A and 0.032 to B. Swin-T showed the same direction with weaker invariance. ViT-B/32 showed stronger terminal-state modulation, but source 17
Source-Grounded Feature Inversion
A P REPRINT
Table 4: Matched topology, stage, and operator ablations. a, The topology control is a matched ResNet18 experiment. b,c, Stage and operator comparisons use matched c4096/e1024 runs across the three representative architectures. Values are pixel cosine unless marked otherwise. Restricted operators measure which input-aligned structures their operator class retains; low values do not define feature-inversion invalidity. a Topology control (ResNet18) b Two-stage correction Variant Raw VJP Global G Global G+D Local Gv Local final
Pixel 0.0002 0.0722 0.0671 0.9009 0.9380
Computation
Model
adjoint transport one terminal map one terminal map boundary DAG boundary DAG
ResNet18 ConvNeXt-B ViT-B/32
c Operator Gamma-only Diagonal/per-frequency Diagonal+rank-8 Full/shared-frequency Full/per-frequency
Gv
Final
∆ pixel
Child NMSE red.
0.9009 0.7343 0.3116
0.9375 0.8450 0.7021
0.0367 0.1107 0.3905
23.0% 15.4% 16.2%
Operator family ResNet18
ConvNeXt-B
ViT-B/32
0.0062 0.2526 0.8337 0.2709 0.9375
0.0034 0.0961 0.4763 0.2090 0.8450
−0.0044 0.0620 0.0501 0.3366 0.7021
shuffling still followed the new operating point (0.692) rather than the old input (0.016). Figure 5 visualises these matched, swapped, interpolated, and zero-target conditions. Several controls exclude simpler explanations. Every zero target produced exactly zero output, as required by the zero-intercept maps. Changing the source geometry moved the inverse to the new operating point rather than preserving a fixed image template. Channel-specific outputs did not collapse to a single result: mean pairwise pixel cosine was 0.497 on ResNet18, 0.230 on ViT-B/32, and 0.099 on Swin-T. A continuous ResNet18 interpolation experiment gave a monotone identity transfer with crossover at 0.495, whereas same-image noise up to 0.1 changed input-instance pixel cosine by at most 0.00066. Together, these results identify feature inversion as a target-modulated, source-local branch computation. They do not make similarity to the original image the ground truth of the inverse. 5.7
Prediction-Conditioned Atlases Link Feature Structure to Decisions
The previous experiments establish a reusable representation-level interface. We next tested whether the same visualised feature groups were related to the current prediction. For each of 16 fixed correctly classified CUB images per model, channels were ranked by the activation–gradient contribution to the predicted margin [32]. At four predeclared depths, we selected nested groups of k ∈ {1, 4, 8, 16} top-positive, top-negative, activation-matched, and 20-repeat randommatched channels. Each group was inverted with the same c4096 map family and no refitting. The original internal representation was then intervened on independently, using calibration-mean replacement as the primary protocol and zero replacement as a robustness check. Figure 6 presents the inversion and intervention branches as parallel evidence rather than a single causal-image pipeline. The strict cross-architecture audit covered 36,864 intervention rows, 3,072 group inversions, and 1,536 deletion curves. Under calibration-mean replacement, the top-positive margin drop exceeded the within-image mean of the randommatched controls in all 48 model–layer–k settings, and every paired bootstrap interval excluded zero. Zero replacement reproduced the same 48/48 direction. All 12 terminal-layer top-negative controls had the expected opposite signed effect. The evidence chain was therefore consistent across residual CNN, normalisation-heavy CNN, and global-token Transformer families, although their visible feature-inverse geometries differed substantially. This experiment supports a bounded decision-grounded interpretation: the visualised feature groups have an independently measured effect on the current decision. It does not imply that the feature inverse itself causes the decision change, or that every visible structure has a unique named semantic label. Expanded image plates, deletion curves, and signed controls are reported in section C.10. 5.8
Calibration and Deployment Define the Practical Boundary
Calibration demand was strongly architecture dependent. For ViT-B/32, final pixel cosine increased from 0.129, 0.363, and 0.648 at 512, 1,024, and 2,048 calibration images to 0.702 at 4,096 and 0.724 at 8,192. Swin-T followed a similar diminishing-return pattern: 0.047, 0.290, 0.457, 0.504, and 0.526. Both token models showed their largest observed transition between 1,024 and 2,048 images, while doubling 4,096 to 8,192 added only about 0.022. We therefore use 18
Source-Grounded Feature Inversion
A P REPRINT
a Input A
Input B
H A , JA
H A , JB
H B , JA
H B , JB
target state H × operating-point operator family J
b
Source-local geometry selects the inverse branch
Identity transfers continuously with the operating point
c
Identity A
Identity B
Current operating point
1.0
ResNet18
Swin-T
Target-state identity Operating-point identity
0.0
d
0.2
0.4
0.6
0.8
0.6 0.4 0.2 0.0
1.0
0.0
Pixel cosine under mismatched H, J Source
Matched
Target shuffled
Source geometry shuffled
0.2
0.4
0.6
0.8
1.0
Operating-point interpolation A → B
e
Mean pairwise channel-inverse cosine
ViT-B/32
crossover 0.495
Pixel cosine
0.8
Channel outputs do not collapse to one template
0.5
0.497 Zero target → exactly zero output
0.4 0.3
0.230
0.2 0.099
0.1 0.0
t18
-T
Swin
e ResN
/32 ViT-B
Figure 5: What determines a feature inverse. a, Target–operator swap for one fixed pair. b, Similarity to target-state and operating-point identities under mismatched conditions. c, Continuous operating-point interpolation with a fixed target. d, Matched, target-shuffled, and source-geometry-shuffled controls. e, Channel diversity and the exact zero-target result. The controls show that source-local geometry selects the inverse branch, while the terminal state modulates that branch in an architecture-dependent manner.
4,096 as a representative diminishing-return operating point for these token architectures, not as a universal optimum. Table 5 reports post-calibration online application, throughput, map storage, and evaluation memory for representative architectures. Table 5: Representative post-calibration online deployment costs. All online values are measured after calibration on the c4096/e1024 quality runs. Online time is the measured 1,024-image application total divided by 1,024 and is therefore not isolated batch-one latency. Throughput is its reciprocal. Map storage includes both Gv and Dv ; evaluation peak is the measured peak allocated memory. These values describe the current public implementation rather than an optimised-kernel lower bound. Model ResNet18 ConvNeXt-B Swin-T ViT-B/32
Fitted nodes
Online (ms/image)
Throughput (images/s)
Map storage (GiB)
Eval peak (GiB)
9 37 26 14
58.4 369.7 129.0 105.1
17.1 2.70 7.75 9.52
0.83 15.51 5.98 2.99
1.71 16.78 6.56 3.61
19
Source-Grounded Feature Inversion
Example 1 Top-positive Top-negative Random matched
Source
Source
Example 2 Top-positive Top-negative Random matched
ViT-B/32
ConvNeXt-B
ResNet18
a
A P REPRINT
Positive groups exceed matched random in all 48 settings ResNet18 ConvNeXt-B
ViT-B/32 k=1
k=4 k=8
k=16
Decision-margin deletion AUC
Normalised positive − random effect
b 101
100
10−1 Early
Settings with required signed CI
d
Mid-1
Mid-2
Terminal
Zero replacement
5
solid: top-positive dashed: random matched
ResNet18 ConvNeXt-B ViT-B/32
4 3 2 1 0 Early
Cross-architecture signed-effect gate Mean replacement
c The same groups alter the original decision
Mid-1
Mid-2
Terminal
e Parallel evidence branches
Terminal negative (4/4)
16
Selected feature group
12
Hl, S
8
Closed-form inverse visual structure Internal intervention decision effect
4
The feature inverse is not re-input as a causal stimulus.
0 ResNet18
ConvNeXt-B
ViT-B/32
Figure 6: Decision-grounded feature atlases across architectures. a, Two predeclared source examples with toppositive, top-negative, and random-matched feature inverses for ResNet18, ConvNeXt-B, and ViT-B/32. b, Normalised positive-minus-random effects over all model–depth–group-size settings. c, Decision-margin deletion AUC across depth. d, Signed cross-architecture validation gate. e, Parallel inversion and internal-intervention evidence branches. The feature inverse is not re-input as a causal stimulus; causal evidence comes from intervention on the original internal representation.
Post-calibration application ranged from 58.4 ms/image for ResNet18 to 369.7 ms/image for ConvNeXt-B in the current implementation. The larger ConvNeXt-B deployment cost accompanies 37 fitted nodes and a 15.51-GiB map family, compared with 9 nodes and 0.83 GiB for ResNet18. This comparison therefore measures the deployed reverse pass actually used by each architecture, while leaving hardware-specific calibration engineering outside the cross-architecture timing claim. The default boundary policy also reflects a quality–cost trade-off. At matched c512/e1024 on ResNet18, a full-DAG fit increased pixel cosine from 0.929 to 0.963 and expanded the fitted nodes from 9 to 49. The corresponding postcalibration online application increased from 53.9 to 74.7 ms/image, exposing a direct quality–deployment trade-off for 20
Source-Grounded Feature Inversion
A P REPRINT
the chosen boundary factorisation. Ridge sensitivity was mild for ResNet18 over ρ ∈ {0.003, 0.01, 0.03}, but much stronger for ViT-B/32 at low calibration. The complete ridge, boundary, storage, peak-memory, repeated-subset, and true-child diagnostics are provided in the supplement. These resource results define the practical boundary of the paradigm change. The method removes per-target image optimisation and enables repeated layer/channel queries after calibration, but it does not make calibration or map storage negligible. The main practical limits are high-dimensional channel covariance estimation, large full-channel map storage, and the quality–cost trade-off of boundary factorisation.
6
Discussion and Limitations
The central implication of the target–operator controls is that source conditioning is part of the feature-inversion object rather than an auxiliary image cue. A target tensor produced by a non-injective nonlinear network does not by itself identify the local branch from which it arose. The forward operating point supplies that missing condition through its activations, nonlinear regimes, routing states, and local Jacobians. Accordingly, swapping the operating-point geometry redirects the inverse toward the new source-local branch, while changing the terminal state modulates the structure transported through that branch in an architecture-dependent manner. This behaviour is not explained by a source-copy shortcut: zero targets give exactly zero outputs, channel selections remain distinct under the same operating point, and the maps fail across mismatched checkpoints. Taken together, these controls support the interpretation of a feature inverse as a target-modulated state traced through the local computation that generated it, rather than as an image determined by the target tensor alone. The architecture-dependent outputs are therefore an interpretability result, not evidence of non-uniform reconstruction quality. Convolutional models typically expose coherent spatial and high-frequency structure, whereas ViT-B/32 retains a stronger patch-grid, low-frequency, and globally mixed inverse geometry. Swin-T shows substantially stronger spatial and high-frequency organisation than ViT-B/32 despite lower pixel cosine, ruling out a blanket failure of the closed-form estimator on Transformers. This contrast is consistent with the different image biases introduced by global patch-token mixing and hierarchical windowed computation [30, 31], although the current experiments do not isolate patch size, hierarchy, or windowing as a unique cause. More generally, similarity to the target-generating image remains an input-instance alignment diagnostic rather than the definition of correctness. A weak or visually sparse single-channel inverse may indicate that the selected channel carries little independently visible structure, or that its evidence is distributed across a channel combination; it does not by itself constitute an inversion failure. The prediction-conditioned atlases give the resulting inverse a specific role in model interpretation. The inversion branch visualises the input-domain structure selected by a layer, channel, or channel group, while the independent intervention branch measures how the same internal feature group affects the current prediction. Top-positive groups exceeded matched random controls in all tested model–layer–group-size settings under both replacement protocols, and terminal top-negative groups produced the opposite signed effect. The agreement connects selected feature structure to decision-relevant internal evidence across residual CNN, normalisation-heavy CNN, and Transformer models. The two branches nevertheless answer different questions. The inverse image is not re-input as a causal stimulus, so causality is attributed only to the intervention on the original representation. Nor does the method imply that every visible structure has a unique human-semantic name. It provides a representation-level view of what the model extracts, which can then be paired with independently grounded tests of how that representation participates in the decision. The main limitations arise from the deployed approximation to the ideal local inverse. The metric-aware operator Mv−1 J ∗ Mu is sample and boundary dependent, whereas the practical estimator replaces explicit online metric estimation with fixed zero-intercept Wiener projections fitted over a calibration distribution. JVP-FC exposes and repairs predictable residuals at the actual operating point, especially for sample-dependent components, but it does not guarantee exact recovery of the online metrics or removal of every local mismatch. The larger online-child oracle gap observed for ViT-B/32 identifies efficient sample-specific metric or state estimation as a natural next direction. The repair maps are also frequency-diagonal over coordinate axes and full-matrix over channels. This makes calibration tractable while excluding direct cross-frequency mixing inside a repair map, and it creates architecture-dependent calibration, storage, and online-application costs. Because the maps encode checkpoint-specific local geometry, a changed checkpoint requires recalibration; this specificity is also what distinguishes the estimator from an architecture-level image decoder. More efficient metric-aware updates and validated structured operator families could reduce these costs without discarding source-local geometry. Finally, the present empirical evidence concerns visual CNN and Transformer computations; internal attention tensors in broader architectures and language-model settings remain important validation domains for the same formal rule. 21
Source-Grounded Feature Inversion
7
A P REPRINT
Conclusion
This work addresses feature inversion as a representation-level interpretation problem: revealing what a selected internal feature extracts from the particular input that generated it. Target consistency alone leaves this inverse underdetermined because multiple inputs can share a representation and do not identify the source-local branch that produced the target. We therefore formulate source-grounded feature inversion by treating the selected feature and the target-generating sample’s local computational geometry as a joint inversion object. Along the actual computational DAG, closed-form matrix Wiener repairs convert mean-seed VJP adjoints into upstream-state estimates, while a second repair uses the JVP forward-consistency residual to correct predictable operating-point-specific mismatch, especially around sampledependent components. Composing the repaired states in one finite reverse pass yields a local inverse interface that, after calibration, supports new inputs, depths, channels, and channel groups without query-specific optimisation. Across diverse CNN and Transformer computations, tensor components, and visual distributions, the same formulation yields calibrated map families that support repeated feature-inverse queries. Matched target and operating-point controls show that each inverse is jointly determined by the selected feature and the source-local operators of the sample being explained, rather than by source copying or a target-independent image template. The resulting architecturedependent inverse geometries expose different spatial and frequency organisations of internal representations. Predictionconditioned atlases further connect these visualised feature structures to independently measured effects on the model decision through interventions on the original internal representation. Source-grounded feature inversion therefore establishes a reusable interface for inspecting what individual layers and channels extract from an input and how the corresponding internal evidence participates in a decision. More broadly, the results show that a model’s hidden feature hierarchy can be opened by tracing features back through the local computation that produced them, rather than by searching for an image that merely reproduces their values.
A
Supplementary Method Details
This appendix expands the method definition in section 3 and fixes the axis, seed, topology, and closed-form solve conventions needed for reproducibility. It proceeds from notation and mean-seed construction to the two calibration solves, the online recursion, topology and boundary instantiation, restricted operator families, complexity, and the final reproducibility checklist. A.1
Notation and Axis Convention
For each selected tensor node v in the computational DAG, we use the following objects: • Hv (x): the forward activation at node v. • Cv : the non-overlapping selected child frontier of v on paths to the target. • Rv (x): the local mean-seed VJP source in eq. (2). • Gv : the first-stage local matrix Wiener repair. • yv0 : the first-stage state estimate Gv Rv . • Cv (x): the pulled-back JVP residual source. • Dv : the JVP forward-consistency correction map. • SvHR (ω) and SvRR (ω): first-stage cross- and auto-spectral matrices. • Sv∆C (ω) and SvCC (ω): correction cross- and auto-spectral matrices. The implementation uses a deepest non-overlapping fitting frontier for Cv . If two candidate selected children u1 and u2 satisfy u1 ⇝ u2 in the selected boundary DAG, then u1 is removed from the fitting frontier. This prevents the same downstream computation from contributing through both a branch-internal child and its later merge or output child. The unpruned structural edges are retained for ancestor coverage and boundary placement; only the local source frontier is pruned. For an edge v → u in this fitting frontier, the local Jacobian is the partial derivative of the intervening subgraph with respect to Hv at the actual forward point. External inputs to that subgraph that are not descendants of v are held fixed at their forward values. All directed paths from v to u remain active, so residual branches and multi-use tensors contribute through native autograd semantics. The scalar Cu denotes the channel count at node u. The calligraphic set Cv denotes child boundaries, whereas Cv denotes the JVP-FC residual source. 22
Source-Grounded Feature Inversion
A P REPRINT
The batch dimension is always dimension zero. All reported models are frozen in evaluation mode and do not couple examples across this dimension, so each image contributes one channel vector per stored frequency to the empirical statistics. The channel axis is the feature axis over which the full matrix Wiener map is applied. For CNN features this is usually dimension one, while for token streams such as (B, N, D) it is the hidden dimension D. The implementation may infer this axis from tensor shape and module type, and the user may override it explicitly. All remaining non-batch, non-channel axes are coordinate axes. If there are no coordinate axes, then k = 0 and F0 is the identity transform. For global-pooled nodes, singleton coordinate axes that only record the pooled spatial extent are trivial and may be collapsed before applying this rule. Thus (B, C, 1, 1) is algebraically equivalent to the coordinate-free vector (B, C). For notation, tensors may be written as (B, C, d1 , . . . , dk ) after this axis identification; this does not require the original model tensor to store the channel axis in dimension one. This convention is intentionally axis-based rather than module-name-based. The same estimator applies to convolutional features, token features, vector nodes, and attention tensors once architecture-aware boundary and axis metadata are specified. A.2
Mean-Seed Local Source
For a child boundary u with channel count Cu , the seed is su =
Hu . Cu
This is the differential of the channel-averaged child energy Ψu (Hu ) =
1 ∥Hu ∥2F . 2Cu
∗ For a single child, the local source is Ju←v su . For multiple selected children, the source is the native multi-output VJP X Hu ∗ Rv = Ju←v . Cu u∈Cv
The division by Cu is applied once, at the child seed. Coordinate positions are not divided out, and there is no second division by the number of children. This convention preserves the standard autograd gauge: multi-output backpropagation sums cotangents contributed by each output. At online inversion time, let son u denote the seed attached to child state yu . For a nonterminal child, the same channel convention is used. For the terminal child, a selected nonempty channel set S uses yT = PS HT and CS = |S|: X yT /CS , u = T, on ∗ su = Rvon = Ju←v son u . yu /Cu , u ̸= T. u∈Cv
The full target has CS = CT . Thus channel normalisation is always part of the child seed, never part of the fitted Wiener map. A.3
Full Matrix Wiener Derivation
The repair family in eq. (5) is frequency diagonal along coordinate axes. For a fixed node v, let Hv′ = Fk Hv and Rv′ = Fk Rv . We take Fk to be the full unitary transform in the derivation. Parseval’s identity gives X 2 E∥Gv (Rv ) − Hv ∥2 = E ∥Gv (ω)Rv′ (ω) − Hv′ (ω)∥ . (42) ω
Since the unknown matrix at frequency ω appears only in the ω term, the objective decomposes across coordinate frequencies. For one frequency, write R = Rv′ (ω) ∈ CC , H = Hv′ (ω) ∈ CC , and G = Gv (ω), with all vectors treated as columns. The frequency-domain repair uses ordinary complex matrix multiplication GR; it is linear, not conjugate-linear, in R. The ridge objective is J(G) = E∥GR − H∥22 + λ∥G∥2F . (43) The problem decomposes over output channels. Let wi be the i-th row of G and Hi the i-th component of H. For that row, Ji (wi ) = E|wi R − Hi |2 + λ∥wi ∥22 . 23
Source-Grounded Feature Inversion
A P REPRINT
The complex least-squares stationarity condition is wi (E[RR∗ ] + λI) = E[Hi R∗ ]. It follows by treating wi and its complex conjugate as independent variables and setting the derivative with respect to the conjugate row to zero. Stacking the C row equations yields G (E[RR∗ ] + λI) = E[HR∗ ]. With
SvRR (ω) = E[RR∗ ],
SvHR (ω) = E[HR∗ ],
we obtain the closed form in eq. (9). Since SvRR (ω) is Hermitian positive semidefinite, SvRR (ω) + λI is strictly positive definite for λ > 0, so the solution is unique. If λ = 0, uniqueness requires SvRR (ω) to be nonsingular. In finite calibration, expectations are replaced by empirical averages. For real tensors, the Fourier spectrum is conjugate symmetric. Implementations may therefore use a real FFT and store only the non-redundant spectrum. This does not change the represented real operator family or the per-frequency normal equations, provided the forward and inverse transforms use a consistent convention and the inverse restores the corresponding real signal. The relative-ridge average below is defined over the stored spectrum, so results must report and retain that storage convention. The evaluated implementation uses orthonormal real transforms (norm=ortho). It converts channel-first tensors to float32 and stores spectral statistics and fitted gains in complex64. A nonunitary FFT normalisation rescales both stored second moments; the effective relative ridge then rescales by the same factor. A.4
Ridge and Zero-Intercept Solve
Both Gv and Dv are solved as zero-intercept maps. The zero-frequency coefficient is treated like every other coordinate frequency: it has a full channel matrix and no additive term. The ridge is implemented as a relative ridge with respect to the source auto-spectral second moment of the current solve: XX λeff = ρ max meanω,c Sv,cc (ω), ϵridge , ρ = 0.01, ϵridge = 10−30 , where X = R for the first-stage map Gv and X = C for the correction map Dv . This keeps the same nominal relative coefficient ρ meaningful across nodes with different source scales. In the closed-form solve, λeff is the value added to the diagonal of the current source second moment. Because SvXX (ω) is Hermitian positive semidefinite, its diagonal entries are real and nonnegative; implementations use the real diagonal when computing the relative ridge. The average is taken over the same stored frequency convention used by the solve, for example over the non-redundant spectrum when a real FFT implementation is used. The numerical floor guarantees a positive ridge for a degenerate zero source when ρ > 0; it has no practical effect at ordinary source scales. With N calibration examples, an empirical source second moment has rank at most N at each frequency. A positive ridge therefore also guarantees an invertible solve when N < C. A.5
Local Calibration Algorithm for Gv
The first-stage map Gv is calibrated from local source-target pairs (Rv , Hv ). It does not use repaired downstream maps as part of its source construction. Given frozen model f , target T , selected boundary nodes Vb , and calibration set Dcal : 1. For each selected node v ̸= T on a path to T : (a) find the deepest non-overlapping child frontier Cv on paths to T ; (b) initialise spectral accumulators SvHR (ω) and SvRR (ω); (c) for each calibration batch: i. run the forward pass and cache Hv and {Hu : u ∈ Cv }; ii. form child seedsPHu /Cu ; ∗ iii. compute Rv = u∈Cv Ju←v (Hu /Cu ) by autograd; iv. transform Hv and Rv over coordinate axes; v. accumulate Hv′ (ω)Rv′ (ω)∗ and Rv′ (ω)Rv′ (ω)∗ . (d) average the accumulators and solve eq. (9). Since each Gv uses only local child activations and local VJPs, the Gv fits are independent once the boundary DAG is fixed. The implementation may still traverse nodes in a particular order for memory management, but the estimator itself is local. 24
Source-Grounded Feature Inversion
A.6
A P REPRINT
Calibration Algorithm for JVP-FC Correction
After all first-stage maps Gv are available, the correction map Dv is calibrated from residual pairs. Each first-stage source already contains the local Jacobian of its calibration sample, but the fitted Gv is shared across the calibration distribution. For fixed or weakly sample-dependent local operators, this first projection can already account for most of the local correction. JVP-FC is principally motivated by operators whose differential changes more strongly with the source operating point, including normalisation, attention, and data-dependent gating or routing. It remains a general residual regression and can correct any first-stage error that is statistically observable from its correction source. For each selected node v ̸= T : 1. For each calibration batch, compute the local source Rv exactly as in the Gv calibration. 2. Apply the first-stage map yv0 = Gv Rv . In the following JVP, yv0 is supplied as a tangent vector at the cached forward point rather than as a replacement activation. 3. For every child u ∈ Cv , compute the local JVP b u|v = Ju←v yv0 . H b u|v . This residual contains both the zero-intercept base term Hu −Ju←v Hv 4. Form child residuals eu|v = Hu − H and the pushed-forward first-stage error Ju←v (Hv − yv0 ); it is not assumed to vanish at yv0 = Hv . The base term vanishes for a positively one-homogeneous local map but is generally nonzero in the presence of offsets or non-homogeneous nonlinearities. 5. Pull the residuals back: X ∗ Cv = Ju←v eu|v . u∈Cv
This is the negative gradient at yv0 of 12 6.
2 u∈Cv ∥Hu − Ju←v y∥F . The source therefore uses the Euclidean child-state residual gauge and is not divided by Cu . Fit Dv from source Cv to target Hv − yv0 by the same full-matrix spectral Wiener solve.
P
Thus Dv uses the same axis rule and Fourier decomposition as Gv . For ∆v = Hv − yv0 , the stored correction matrices are −1 Dv⋆ (ω) = E[∆′v (ω)Cv′ (ω)∗ ] (E[Cv′ (ω)Cv′ (ω)∗ ] + λD I) . This is a second closed-form regression, not iterative image optimisation. The JVP and VJP are matrix-free applications of the frozen local subgraph differential at the current forward point. A.7
Online Feature-Inversion Algorithm
At inversion time, the computation proceeds from the target toward the input. Choose a nonempty terminal channel set S ⊆ {1, . . . , CT } and let CS = |S|. Initialise the terminal state and seed as son T =
yT = PS HT ,
PS H T yT = , CS CS
where PS masks channels outside S. The full target has PS = I and CS = CT . A single-channel target has CS = 1. Then for each shallower selected node v whose child states have already been inverted to their boundaries: 1. attach the seed son u =
yT /CS , yu /Cu ,
u = T, u ̸= T,
to each child, and compute the online local source X
Rvon =
∗ Ju←v son u ;
u∈Cv
2. apply the first-stage map yv0 = Gv Rvon ; 3. compute the online child residual source Cvon =
X
∗ yu − Ju←v yv0 ; Ju←v
u∈Cv
25
Source-Grounded Feature Inversion
A P REPRINT
4. return yv = yv0 + Dv Cvon . The input-node state is the reported input-domain feature inverse. Only the terminal child uses CS in its seed. Every inverted nonterminal child uses its physical channel count Cu . The residual source uses the state yu itself, without seed normalisation. The calibrated Gv and Dv maps are unchanged when S changes. A.8
DAG Topology: Branches and Multi-Use Nodes
The method relies on autograd’s VJP semantics for graph topology. No special mathematical rule is introduced for residual connections, concatenations, or branching. Residual addition. For z = a + b, the VJP distributes the cotangent of z to both a and b. Each branch receives its own cotangent and is repaired at its own tensor nodes. Concatenation. For z = concat(a, b), the VJP slices the cotangent of z back into the coordinates or channels corresponding to a and b. Repairs are then applied on the branch tensors. One-to-many use. If a tensor feeds multiple downstream operations, autograd sums the cotangents contributed by each downstream user. The node repair acts on this accumulated incoming cotangent. If the capture granularity exposes separate edge tensors, the same local repair rule can be applied at those edge nodes instead. A.9
Boundary Modes
The theoretical maximal form is Full-DAG: every captured differentiable floating tensor node with a specified channel axis has a well-defined local matrix Wiener repair. Scalar nodes, non-floating tensors, and bookkeeping operations without a meaningful channel axis are traversed by autograd but are not instantiated as repair nodes. The implementation exposes three modes: • Full-DAG: instantiate every captured differentiable floating tensor node with a specified channel axis. • Coarse/stage block: instantiate the input, stem or patch embedding, and architecture block or stage edges; this is the default efficient cross-architecture mode. • Custom list: instantiate user-specified nodes after full DAG capture. Skipping nodes is an implementation choice, not a change in the local estimator. A.10
Architecture-Specific Instances
CNNs. Feature maps have shape (B, C, H, W ). The channel axis is C and the coordinate axes are (H, W ), giving a 2D Fourier transform and one C × C matrix per 2D frequency. DenseNet and ConvNeXt. The same tensor rule applies. Coarse/stage block boundaries provide a compact crossmodel instantiation, while Full-DAG can be used when memory permits. ViT token streams. For tokens with shape (B, N, D), the hidden dimension D is the channel axis and token position N is the coordinate axis. The repair uses a 1D Fourier transform over tokens and a full D × D matrix per token frequency. When a class token is present, it remains at its stored sequence position and is included in this transform unless an experiment explicitly specifies a different token convention. Attention tensors. For attention logits or weights shaped (B, h, N, N ), the head dimension h is treated as channel and query-key axes as coordinates. The repair acts on cotangents or reverse-state estimates rather than a probability matrix that is reinserted into the forward model, so row-stochastic softmax constraints are not explicitly enforced. The estimator remains algebraically well defined, but both this relaxation and the query-key frequency-diagonal approximation remain empirical questions. Vector, GAP, and MLP nodes. For (B, D) tensors, k = 0, so there is no Fourier transform and the repair is a single D × D matrix. Global-pooled tensors stored as (B, D, 1, 1) are treated equivalently because the singleton-coordinate Fourier transform is the identity. 26
Source-Grounded Feature Inversion
A.11
A P REPRINT
Restricted Operator Families
The full-channel per-frequency estimator is the final method. The ablations impose additional restrictions in both Gv and Dv : • Diagonal-channel: retain a separate map at each frequency but force every channel matrix to be diagonal. • Shared-frequency: retain full channel matrices but use one matrix shared across coordinate frequencies. • Gamma-only: force diagonal channel gains and share those gains across frequencies. • Diagonal plus low-rank: use a diagonal channel map plus a low-rank channel correction at each frequency. Diagonal-channel removes cross-channel second-order coupling, whereas shared-frequency removes frequency-specific coordinate structure. The matched quantitative and visual comparisons are reported in tables 4 and 9 and fig. 4. Those experiments evaluate not only global input-instance alignment but also the spatial and spectral structures retained by each restricted family. A.12
Complexity and Memory
Let F be the number of stored coordinate frequencies. For a node with C channels, storing full spectral statistics requires O(F C 2 ) memory, and solving by dense linear algebra costs O(F C 3 ) in the worst case. Accumulating either source-target statistic costs O(N F C 2 ) arithmetic for N calibration examples, excluding VJP/JVP and FFT costs. Applying one fitted spectral map costs O(BF C 2 ) arithmetic, in addition to coordinate-transform cost O(BCP log P ) for P coordinate positions. At each frequency, each calibration example contributes one C-dimensional Fourier coefficient, so the effective sample count for a frequency is the number of calibration examples rather than the number of coordinate positions. Vector nodes have F = 1 and are therefore comparatively cheap. ViT token nodes have C = D and F proportional to the token sequence frequency count, so calibration size and solve cost can be substantial. Engineering controls include frequency chunking, solve chunking, CPU/GPU placement of maps, cached calibration tensors, and cotangent caching. These optimisations do not change the estimator as long as they preserve the local source-target pairs defined above. JVP-FC additionally requires an autodiff path with second-order derivative support because the matrix-free JVP is evaluated through a VJP-of-probes identity. For fused attention implementations, a mathematically equivalent kernel with double-backward support must be enabled when the default kernel lacks it. A.13
Reproducibility Details
Each experiment should report: • model architecture, frozen weights, and evaluation mode, • data split for calibration and evaluation, • sampling scheme and random seeds for both splits, • image preprocessing, • calibration size and batch size, • target node and terminal seed definition, • boundary mode, • non-overlapping child-frontier construction, on • calibration seed su = Hu /Cu , nonterminal online seed son u = yu /Cu , and terminal subset seed sT = PS HT /|S|, • channel-axis convention or override, • source-normalisation setting and zero-intercept linear convention, • channel and frequency solve modes, • FFT normalisation, stored-spectrum convention, and numerical precision, • first-stage and correction ridge values and whether relative ridge is used, • nominal relative ridge coefficient ρ and effective ridge definition, • JVP-FC child-target mode, residual scale, and correction weight, • map device and chunking settings, 27
Source-Grounded Feature Inversion
A P REPRINT
• autodiff or attention-kernel settings needed for JVP double backward, • post-calibration online runtime, hardware, software version, and memory constraints. These conventions define the deployed estimator used by the supplementary theory and by every headline experiment.
B
Supplementary Theory: From the Statistical Ideal to DAG Stability
This appendix follows the derivation order in section 4. As in the main text, the estimated objects are upstream tensor states along the source-local inverse branch, not generic syntheses of the original image. It first proves the conditional-mean optimum, then connects local Wiener inversion to a covariance-induced natural adjoint. It next formalises the mean-seed observable and the two distributional Wiener projections, before deriving target-conditioned finite-DAG stability and the remaining operator boundaries. B.1
MSE-Optimal Conditional-Mean Inversion
Let Hv and Hu be square-integrable random states on the data distribution, with Hv upstream of Hu . Define m(Hu ) = E[Hv | Hu ]. For any measurable estimator T (Hu ), write Hv − T (Hu ) = Hv − m(Hu ) + m(Hu ) − T (Hu ) . Conditional-expectation orthogonality gives E [⟨Hv − m(Hu ), m(Hu ) − T (Hu )⟩] = 0. Therefore
E∥Hv − T (Hu )∥22 = E∥Hv − m(Hu )∥22 + E∥m(Hu ) − T (Hu )∥22 .
(44)
The second term is nonnegative, so the conditional mean is the MSE-optimal inverse, uniquely up to almost-sure equality. This statistical ideal uses only the downstream observation. The deployed method is input-conditioned in a different sense: the input being explained is available to instantiate the frozen forward point and its local Jacobians. It is not included as an extra variable in the conditional mean, because for a deterministic frozen model the full input already determines every forward state. The method does not estimate the full nonlinear conditional law in eq. (44). It uses the forward point to construct a sequence of local observable inverse problems on the boundary DAG. B.2
Local Wiener Inversion and the Natural Adjoint
Vectorise the local tensors and hold the Jacobian J fixed at one forward point for this unrestricted reference calculation. e v and H e u for centred parent and child deviations, and assume Write H eu = J H e v + η, H
e ∗ ] = 0. E[η H v
(45)
Multiple child states can be stacked, with J formed by vertically stacking their local Jacobians. Define the covariances ev H e v∗ ], Σv = E[H
Ση = E[ηη ∗ ],
Σu = JΣv J ∗ + Ση .
e u , the normal equation for minA E∥AH eu − H e v ∥2 is For a linear estimator AH 2 ev H e u∗ ] = Σv J ∗ . AΣu = E[H If Σu is nonsingular, the unique best linear estimator is A⋆ = Σv J ∗ Σ−1 u .
(46)
ev | H e u ], the zero-mean specialisation of the conditional-mean Under joint Gaussianity, this linear estimator equals E[H inverse. Without Gaussianity, it remains the best zero-intercept linear estimator. The same operator arises from weighted geometry when both covariances are positive definite. Use the precision metrics Mv = Σ−1 v ,
Mu = Σ−1 u , 28
Source-Grounded Feature Inversion
A P REPRINT
and the inner products ⟨a, b⟩M = a∗ M b. The Riesz adjoint J ♯ is determined by ⟨Jδv , su ⟩Mu = ⟨δv , J ♯ su ⟩Mv . Equality for all arguments gives ⋆ J ♯ = Mv−1 J ∗ Mu = Σv J ∗ Σ−1 u =A .
(47)
Thus, after the metric-induced identification of vectors and covectors, local Wiener inversion and the covariance-induced natural adjoint are the same ideal operator under the centred linear model. If a covariance is singular, the LMMSE map can be written with a pseudoinverse on its support. A positive-definite metric on the full space instead requires regularisation, for example replacing Σ by Σ + τ I with τ > 0. Centring is used only for this ideal identity. The deployed estimator retains the uncentred, zero-intercept source-target objectives defined in the Method and does not fit an additive bias. This identity motivates the geometry that an inverse correction should supply. It is not an algebraic characterisation of the deployed Gv or Dv . In a nonlinear network, the local Jacobian and appropriate covariance geometry vary with the input and boundary. Explicit deployment would require estimating and applying both metrics online. B.3
Variational Interpretation of the Mean-Seed Observable
For a child activation Hu ∈ RB×Cu ×··· , define 1 ∥Hu ∥2F . 2Cu
Ψu (Hu ) =
(48)
Its differential with respect to Hu is Hu /Cu . Therefore ∗ Ju←v
Hu Cu
is the exact adjoint pullback of a scalar child-state observable. It is also the arithmetic mean of the channel-energy pullbacks. Writing Hu,c for channel c and Ψu,c = ∥Hu,c ∥2F /2 gives Cu 1 X ∂Ψu,c Hu ∗ ∗ Ju←v = Ju←v . Cu c=1 ∂Hu Cu This equality uses only VJP linearity. For a node with multiple selected children, define X
ΨCv =
u∈Cv
1 ∥Hu ∥2F . 2Cu
Its differential with respect to Hv is Rv =
X
∗ Ju←v
u∈Cv
Hu . Cu
(49)
Thus summing child VJPs is the differential of a well-defined local observable. Dividing again by |Cv | would define a different observable and change the child gauge. The channel-subset terminal seed has the same variational interpretation. For a nonempty set S, let PS be the self-adjoint channel mask and CS = |S|. Then (S)
ΨT (HT ) =
1 ∥PS HT ∥2F , 2CS
(S)
∇HT ΨT
=
PS HT . CS
(50)
The full target is the special case PS = I and CS = CT . B.4
Distributional Wiener Projection and the Proxy Bridge
The ideal operator in eq. (47) applies a child precision before the adjoint and a parent covariance after it. The deployed source in eq. (49) instead uses an observable Euclidean VJP. Calibration therefore estimates a ridge-regularised distributional map from this source to the parent state. 29
Source-Grounded Feature Inversion
A P REPRINT
At a fixed frequency, write R = Rv′ (ω) and H = Hv′ (ω). The ridge objective min E∥GR − H∥22 + λ∥G∥2F G
has normal equation
G E[RR∗ ] + λI = E[HR∗ ].
For λ > 0, the unique solution is −1 G⋆v (ω) = SvHR (ω) SvRR (ω) + λI . (51) The full complex least-squares derivation and Fourier decomposition are given in section A.3. No Gaussian assumption is required for optimality of this ridge objective within the selected linear class. The statistics in eq. (51) encode the relation induced jointly by the input-dependent Jacobian, the data distribution, and the mean-seed observable. They do not separately identify the Jacobian or either covariance metric. The fitted Gv is therefore a closed-form distributional projection surrogate for the unavailable metric-aware inverse. The observable Rv (x) retains the source-local Jacobian through its VJP, but the fitted projection Gv is shared across samples. This distinction is minor for fixed or weakly sample-dependent local operators, where one distributional projection can capture the dominant correction. For strongly sample-dependent local operators, the relation between Rv (x) and Hv (x) can retain an operating-point-specific mismatch. Dependency support. By the chain rule, a child seed affects node v only through computational paths connecting v to that child. Entries outside the corresponding Jacobian sparsity pattern receive zero cotangent. This is a theorem-level dependency statement. State-structure proxy. Dependency support does not imply that nonzero cotangent support equals forward activation support. The working assumption is weaker: the spatial or semantic organisation of Rv remains predictive of the organisation carried by Hv . This assumption is evaluated through state-inversion diagnostics, visual structure, and transfer. Local transfer. Different terminal targets change the seed and therefore the observable source. Using one calibrated map family across compatible target depths and channel masks tests whether the dominant correction is node-local rather than a stored target-specific image. B.5
JVP-FC as a Second Residual Regression
Let yv0 = Gv Rv be the first-stage estimate. JVP-FC evaluates this estimate under the local differential of the actual source operating point and thereby constructs a second, operating-point-conditioned observable. This mechanism is especially relevant for normalisation, attention, and data-dependent gating or routing, although the residual regression itself is not restricted to those components. In the JVP below, yv0 is interpreted as a tangent vector at the fixed forward point, not as a replacement forward activation. For each selected child, define the JVP residual eu|v = Hu − Ju←v yv0 .
(52)
The exact decomposition is
eu|v = Hu − Ju←v Hv + Ju←v (Hv − yv0 ). (53) The first term is the offset or non-homogeneous base component missing from the zero-intercept differential. The residual therefore need not vanish at the exact parent state. For a positively one-homogeneous local map, Euler’s identity makes this base term zero at differentiable points. The second term transports the first-stage state error through the current sample’s local differential and exposes its source-specific child response. Pulling the residual back gives Cv =
X
∗ Ju←v eu|v .
(54)
u∈Cv
Define Φv (y) =
1 X ∥Hu − Ju←v y∥2F . 2 u∈Cv
Direct differentiation gives Cv = −∇Φv (yv0 ). The unnormalised child residual therefore corresponds to a specific Euclidean child-consistency objective. The second map solves Dv⋆ = arg min E∥DCv − (Hv − yv0 )∥22 + λD ∥D∥2F . D
30
(55)
Source-Grounded Feature Inversion
A P REPRINT
It has the same ridge-regularised frequency-wise Wiener form as Gv , with Cv as source and Hv − yv0 as target. This does not make every residual component observable. Any child-residual component in the null space of the summed pullback is absent from Cv , and the fitted map can estimate only parent-error components that are statistically predictable from the resulting source. The online version replaces each true child state by the already inverted state: X ∗ Cvon = Ju←v yu − Ju←v yv0 .
(56)
u∈Cv
This is deployable because yu is available during target-to-input recursion. When the downstream state estimate is accurate, the online residual approaches the calibration residual. Otherwise it exposes the child state that is actually propagated. The term Ju←v yv0 is a JVP at the original forward point. It is not a nonlinear evaluation of fu←v on yv0 . The correction is therefore a residual regression in local differential coordinates, not a claim that the nonlinear subgraph is homogeneous. B.6
Target-Conditioned Finite-DAG Error Propagation and Stability
The local closed-form objectives do not by themselves control how errors accumulate during the target-to-input recursion. We first give an exact recursion relative to an arbitrary target-conditioned reference family, then obtain the forward-state theorem as the full-target special case. Fix an input x and a nonempty terminal channel set S. Equip each selected node space with a fixed norm and each block between node spaces with the corresponding induced operator norm. For a child node u, define the online seed weight 1/CS , u = T, (S) αu = (57) 1/Cu , u ̸= T. For a nonterminal boundary node v, stack arbitrary child states as zCv = (zu )u∈Cv and define X X ∗ ∗ Pv(S) zCv = Ju←v αu(S) zu , Qv zCv = Ju←v zu , u∈Cv
Lv =
X
(58)
u∈Cv ∗ Ju←v Ju←v .
(59)
u∈Cv
At the fixed forward point, the complete update driven by these child states is yv0 = Gv Pv(S) zCv ,
(60)
Cv = Qv zCv − Lv Gv Pv(S) zCv , yv = Kv(S) zCv ,
(61)
Kv(S) = Gv Pv(S) + Dv Qv − Lv Gv Pv(S) .
(62)
Thus the local update is linear in the supplied child states, although its Jacobian-dependent coefficients remain inputconditioned. The displayed operator uses the unit residual and correction weights evaluated in the paper; any fixed (S) scalar weights can be absorbed into the effective correction operator in Kv . (S)
Let {H̄v } be any target-conditioned reference family satisfying (S)
H̄T
= PS HT .
(63)
(S)
(64)
Define the local defect
ϵ(S) = Kv(S) H̄Cv − H̄v(S) . v (S) (S) For online errors δv = yv − H̄v , linearity gives (S)
(S) δv(S) = ϵ(S) v + Kv δCv .
(65)
The terminal error is zero because the online and reference terminal states are both PS HT . (S)
For full-target inversion, PS = I and CS = CT . Choosing H̄v = Hv then recovers the forward-state errors and local defects used in the main text. For a strict channel subset, eq. (65) remains valid relative to any chosen target-conditioned 31
Source-Grounded Feature Inversion
A P REPRINT
reference family. The theorem controls propagation around that reference but does not define a unique semantic ground truth for channel-specific inversion. (S)
(S)
(S)
Write Kvu for the block acting on child u and κvu = ∥Kvu ∥op . Then X (S) ∥δv(S) ∥ ≤ ∥ϵ(S) κ(S) v ∥+ vu ∥δu ∥.
(66)
u∈Cv
Because the graph is acyclic and the terminal error is zero, this inequality can be unrolled over all paths from v toward the target. If w ⪰ v denotes that w is a selected descendant of v and Paths(v, w) is the set of selected-DAG paths from v to w, then X X Y (S) ∥δv(S) ∥ ≤ κab ∥ϵ(S) (67) w ∥, w⪰v, w̸=T
p∈Paths(v,w) (a,b)∈p
where the empty path for w = v has product one. If every local defect is zero, reverse topological induction gives zero error at every selected node. If the local reverse gain satisfies X sup κ(S) (68) vu ≤ q < 1, v̸=T u∈C
v
then (S)
max ∥δv(S) ∥ ≤ v
(S)
maxv̸=T ∥ϵv ∥ . 1−q
(69)
(S)
Indeed, if M = maxv ∥δv ∥ and E = maxv̸=T ∥ϵv ∥, then eq. (66) gives M ≤ E + qM . This condition is sufficient rather than necessary. When it fails, eq. (67) still identifies the boundaries and paths that can amplify local defects. These are finite-pass stability statements, not iterative convergence claims. B.7
Zero Target-Independent Injection
The two stages have the zero-intercept form yv0 = Gv Rv ,
yv = yv0 + Dv Cv .
If the selected target state is zero, its terminal seed is zero. Induction through the DAG then gives zero local VJP sources, zero first-stage states, zero JVP residual sources, and zero inverted states. The calibrated matrices can encode a calibration-distribution-dependent linear prior, but they cannot add a fixed target-independent component. This is narrower than support non-expansion. A frequency-diagonal multiplier is convolutional in coordinate space and can spread energy. The guarantee is only that every inverted component is induced by the selected target, the frozen model derivatives, and fixed zero-intercept maps. B.8
The Frequency-Diagonal Structural Boundary
The main structural restriction is coordinate-frequency diagonality. At each frequency, channels may mix through a full matrix, but different coordinate frequencies do not mix inside a repair map. For a linear operator on the selected discrete Fourier grid, this is equivalent to a matrix-valued circular convolution under that grid’s boundary convention. The equivalence describes the repair operator, not the original network. Without this restriction, a node with C channels and P coordinate positions would require an unrestricted (CP ) × (CP ) linear map. Frequency diagonality instead gives independent C × C problems over F stored frequencies. This provides the tractability needed for full-channel calibration. Whether the optimal repair is adequately approximated in this class is an empirical question. The evaluated CNN boundaries and ViT token streams support its practical use in the settings studied here. Internal attention tensors and language-transformer nodes require additional validation. The supplementary experiments next test each empirical bridge separately: observable transfer, residual repair, finite-DAG diagnostics, information flow, and the practical calibration and deployment boundaries. 32
Source-Grounded Feature Inversion
C
A P REPRINT
Supplementary Experiments
This supplement follows the claim–evidence order of the main experiments. It records the detailed evaluation protocol, expanded quantitative results, negative controls, deployment audits, and the optional breadth studies. Throughout, image-space metrics describe alignment with the known target-generating input. They are not treated as ground-truth correctness measures for a feature inverse. The progression is protocol and reuse, structural and distributional scope, mechanism and theory bridge, inverse meaning, deployment audits, decision-grounded interpretation, and optional extensions. C.1
Reproducibility, Metric Definitions, and Final-Result Criteria
Fixed protocol. Unless a row is explicitly marked as an ablation or diagnostic, all results use the final public method: local mean seeds, zero-intercept full-channel per-frequency maps for both Gv and Dv , repaired online child states, model-native block-endpoint boundaries, no activation or cotangent pre-normalisation, and relative ridge ρ = 0.01. The calibration and evaluation sets are disjoint. For the headline ImageNet experiments, 4,096 training images are sampled with seed 123 and 1,024 validation images are sampled with seed 456. Dataset-specific Pets experiments use all 3,680 available training images for calibration. Other dataset-specific calibrations use 4,096 images. We use cN /eM as shorthand for N calibration images and M evaluation images. The final runs used Python 3.10.4, PyTorch 2.5.1 with CUDA 12.1, TorchVision 0.20.1, and timm 1.0.25. Runtime measurements used one NVIDIA GeForce RTX 3090 GPU with 24 GiB of memory on a host with two Intel Xeon Gold 6426Y processors. Each run was bound to one GPU. The fitted-map placement followed the reported deployment configuration: ResNet18 maps remained on the GPU, while the larger headline map families were applied from CPU memory. Each run records the model and weight identifier, dataset root, preprocessing transform, calibration and evaluation indices, target node, fitted boundary DAG, tensor shape, channel axis, coordinate axes, frequency shape, ridge, sourcecode hashes, map metadata, and per-image metrics. The formal component validator additionally requires the same public code path, no component-specific inverse, finite outputs, and a complete target-axis record. Nondeployable oracle conditions are labelled diagnostic-only and are never used as headline method rows. Tensor and Fourier convention. A floating tensor is represented as batch, channel, and zero or more coordinate axes as defined in section A.1. Real-valued coordinate axes use an orthonormal real Fourier transform. The calibrated map is full over channels and diagonal over coordinate frequency. Vector states therefore have one frequency bin, while token streams use the token coordinate as their Fourier axis. The forward model fixes the target node and boundary topology before calibration; no inverse component is manually inserted at evaluation time. Input-instance alignment.
For flattened prediction x b and known target-generating input x, pixel cosine is cospix (b x, x) =
⟨b x, x⟩ . ∥b x∥2 ∥x∥2
(70)
The same cosine is computed after separating luminance low-pass, luminance high-pass, and chroma components. SSIM and LPIPS provide complementary structural and perceptual descriptions [20, 21]. Relative ℓ2 is ∥b x − x∥2 /∥x∥2 . These metrics quantify input-instance alignment for a known source reference. A low value for a channel or channel-subset target does not by itself invalidate the inverse, because the selected target is PS HT rather than the complete input. Representation and mechanism diagnostics. Re-encoding cosine compares the representation of the inverse with the target representation and is reported only as representation consistency. Child NMSE measures the local JVP prediction error before and after Dv . Intermediate-state relative ℓ2 measures the corresponding state-space error. Runtime is reported for post-calibration online application. Calibration complexity is described separately by calibration-set size, fitted-node count, map storage, and peak memory rather than by cross-architecture fit wall time. Map storage includes both Gv and Dv ; peak memory is measured separately during calibration and evaluation. Uncertainty and visual selection. Headline ImageNet means use all 1,024 fixed evaluation images [16, 17]. Calibration-estimation variation is assessed with independently sampled calibration subsets and is kept separate from evaluation-image uncertainty. Operating-point curves use deterministic 10,000-resample bootstrap intervals. The decision-conditioned study aggregates random controls within each image before deterministic 20,000-resample image-cluster bootstrap intervals, which prevents random-control repeats from being treated as independent images. Visual sample indices, ordering, targets, and display transformations are fixed before figure assembly. Inverse tensors are scaled only for display after all quantitative measurements. 33
Source-Grounded Feature Inversion
A P REPRINT
Table 6: Headline ImageNet terminal and boundary setups. All rows use full-state terminal targets, c4096/e1024, and registered model-native boundary DAGs. MobileNetV2 and EfficientNet-B0 factorise each stage-entry non-residual block at expansion/depthwise tensors, retain residual blocks at their add endpoints, and terminate at the final 320-channel feature block. Tensor shapes include the batch dimension; channel and coordinate axes use zero-based indexing.
C.2
Model
Captured terminal tensor
Shape
Channel axis
Coordinate axes
ResNet18 ResNet50 DenseNet121 MobileNetV2 EfficientNet-B0 RegNetY-1.6GF ConvNeXt-B Swin-T ViT-B/32
op.add_7.add:out op.add_15.add:out op.batch_norm_120.batch_norm:out features.17.conv.3:out features.7.0.block.3.1:out op.add_26.add:out op.add_35.add:out norm:out encoder.ln:out
1 × 512 × 7 × 7 1 × 2048 × 7 × 7 1 × 1024 × 7 × 7 1 × 320 × 7 × 7 1 × 320 × 7 × 7 1 × 888 × 7 × 7 1 × 1024 × 7 × 7 1 × 7 × 7 × 768 1 × 50 × 768
1 1 1 1 1 1 1 3 2
2,3 2,3 2,3 2,3 2,3 2,3 2,3 1,2 1
Fitted nodes 9 17 9 31 30 28 37 26 14
Large-Scale Atlas, No-Refit Reuse, Target Depth, and Optimisation Details
One fit, many queries. The canonical ResNet50 atlas uses one c4096 fitted map family. It serves 1,024 evaluation images, four full-state depths, four terminal channels, and one eight-channel terminal subset without refitting any map or changing the ridge. The four full-state target keys are full_n1, full_n5, full_n9, and full_n17. Their pixel cosines are 0.9448, 0.9282, 0.9339, and 0.9333, respectively. The four individual terminal channels have pixel cosines 0.8254, 0.5979, 0.9243, and 0.9199; the fixed eight-channel subset reaches 0.8932. The channel results demonstrate public-API reuse and output diversity. They are not evaluated as attempts to reproduce the complete input from one channel. Canonical iterative preimage optimisation. The external numerical baseline starts from noise and minimises the terminal feature discrepancy for 2,000 Adam steps per image [22]. The learning rate is 0.05, the total-variation and ℓ2 weights are 0.1 and 0.001, and random spatial jitter is bounded by four pixels [23]. A fixed pilot sweep from 1,000 to 3,000 steps selected this setting because additional steps produced only a small alignment change while increasing cost. The target layer, preprocessing, evaluation images, and feature objective are matched to the closed-form run. Three predeclared initialisation seeds are used. No learned decoder or saliency method is included because the intended numerical comparison is between canonical feature-preimage search and the proposed source-grounded closed-form regime. Across the matched 16-image set, iterative optimisation obtains mean target re-encoding cosine 0.9575 and feature NMSE 0.0794, but pixel cosine is 0.0351. Mean pairwise output cosine across the three initialisations is 0.1010, demonstrating that the converged preimages remain seed dependent. After calibration, the final closed-form method obtains pixel cosine 0.9388 and requires 0.9339 seconds for all 16 online queries, or 0.0584 seconds per query in the measured batched application run. Raw VJP has pixel cosine 2.5 × 10−5 on the same set. The optimisation result is not called an incorrect preimage. It demonstrates that target-only preimage search and source-local feature inversion use different information and can select different solutions of an underdetermined feature constraint. Initialisation sensitivity and post-calibration latency. The three optimisation seeds reach similar target objectives, while their outputs remain seed dependent. We therefore report the predeclared seed mean as the baseline result and retain the median only as a diagnostic. In an isolated three-repeat benchmark, closed-form batch-one online latency is 0.1385 seconds when images vary, 0.1104 seconds when depths vary, 0.1408 seconds when channels vary, and 0.1274 seconds for a mixed workload. The corresponding 2,000-step iterative-optimisation references require 23.5421, 21.7281, 23.5781, and 23.0087 seconds per query. All values in this comparison begin after calibration so that they measure the deployment-time scaling of each query type. Figure 7 combines the no-refit atlas and fixed depth/channel queries with the matched regime and online-latency comparisons. C.3
Expanded Architecture and Tensor-Component Results
Table 6 records the exact terminal tensor and axis convention for every headline ImageNet backbone [24–31]. The main paper reports nine ImageNet rows spanning eight backbone families. All rows pass the final-code validator and use c4096/e1024 with the same public recipe. The full results are reproduced in table 7 to expose architecture-dependent structure rather than reducing generality to one scalar ranking. 34
Source-Grounded Feature Inversion
a
b
One calibrated map family across depth
0.96
A P REPRINT
Different information sets select different inverse objects 0.939
1.0 target-feature match 0.958
Input-instance pixel cosine
Pixel cosine
0.95 0.94 0.93 0.92 0.91
0.6
0.4
0.2 0.035
0.0
0.90 early
c
0.8
mid-1
mid-2
deep
Iterative optimisation
Ours
Post-calibration online latency by query type Ours
d
Iterative optimisation
Source
Early
Middle
Deep
Channel
Seconds per query
101
100
10−1 Image
Depth
Channel
Mixed
Figure 7: Expanded no-refit reuse and optimisation-regime evidence. a, One ResNet50 calibration across four full-state depths. b, Descriptive input-instance alignment for the matched iterative and closed-form rows; the iterative target-feature match is annotated separately because it is a representation-consistency diagnostic. c, Post-calibration online latency across image, depth, channel, and mixed queries. d, Fixed depth and channel queries under one fitted map family.
A matched boundary audit showed that block-output-only capture was still too coarse for stage-entry MobileNetV2 and EfficientNet-B0 blocks. Stem-only changes and indiscriminate full-DAG refinement did not improve the matched c512 diagnostic. The decisive change was to expose the expansion/depthwise tensors of each non-residual stage-entry block while leaving residual blocks at their add endpoints. For EfficientNet-B0, making the squeeze-and-excitation multiplication itself a boundary caused sample-wise sign instability, so the gate and multiplication remain inside one local edge. The final c4096/e1024 rows reach pixel cosines 0.8333 and 0.7960 at the same last 320-channel targets; no shallower target was substituted. Swin-T and ViT-B/32 expose different Transformer feature geometry. Swin has stronger SSIM and high-frequency structure, whereas ViT has stronger low-frequency and global pixel alignment. The result shows that the method operates on Transformer computation graphs without asserting that hierarchy, windowing, or patch size is the unique causal explanation for the difference. Tensor-component zoo. The formal zoo contains 12 validated runs and 11 component families. It includes DenseNet concatenations, MobileNet depthwise and inverted-residual states, EfficientNet MBConv and squeeze-and-excitation states, ConvNeXt block and normalisation-adjacent tensors, Swin window-stage and patch-merging states, ViT token streams, and pooled or vector states. Representative results are shown in table 8. Every row uses the same tensor-axis interface and records manual_component_inverse=False. 35
Source-Grounded Feature Inversion
A P REPRINT
Table 7: Expanded ImageNet architecture results. Y-low and Y-high are luminance low- and high-frequency cosines. All metrics are descriptive alignment over 1,024 images. Gv
Final
Y-low
Y-high
Chroma
SSIM
LPIPS
0.9009 0.8889 0.7510 0.7810 0.6591 0.8716 0.7343 0.1783 0.3116
0.9375 0.9308 0.8245 0.8333 0.7960 0.9143 0.8450 0.5037 0.7021
0.9486 0.9178 0.8335 0.8247 0.7398 0.9500 0.8626 0.4844 0.7891
0.8624 0.8900 0.7243 0.8484 0.8581 0.6553 0.7088 0.6144 0.2480
0.9041 0.9260 0.8593 0.8244 0.7729 0.8947 0.7724 0.4514 0.6044
0.6837 0.7261 0.5878 0.7103 0.7223 0.5966 0.6166 0.5187 0.3144
0.3137 0.2772 0.4188 0.2597 0.2720 0.4231 0.3985 0.5198 0.7568
Model ResNet18 ResNet50 DenseNet121 MobileNetV2 EfficientNet-B0 RegNetY-1.6GF ConvNeXt-B Swin-T ViT-B/32
Table 8: Representative component-zoo targets. Shapes include the batch dimension. Re-encoding is a representation diagnostic and is not required to track pixel alignment monotonically. Model
Component
DenseNet121 concatenation MobileNetV2 inverted residual MobileNetV2 depthwise tensor EfficientNet-B0 MBConv/SE ConvNeXt-B normalisation-adjacent ConvNeXt-B MLP state Swin-T window stage Swin-T patch merging EfficientNet-B0 pooled vector EfficientNet-B0 logit vector
Target shape
Target example
1×256×56×56 1×160×7×7 1×32×112×112 1×192×7×7 1×56×56×128 1×56×56×512 1×14×14×384 1×28×28×384 1×1280×1×1 1×1000
dense cat residual add depthwise conv MBConv add layer norm MLP fc1 residual add merge pad average pool classifier
Pixel
Re-enc.
0.8338 0.8353 0.4674 0.1961 0.9861 0.9745 0.3080 0.0248 0.9487 0.9445 0.9475 0.9814 0.7516 0.6485 0.8924 0.9049 0.2622 −0.1710 0.2640 −0.0050
The component table establishes coverage, not uniform visual alignment. In particular, a pooled or logit vector contains a different selected feature object from a spatial tensor. Its inverse should therefore not be judged by whether it reproduces the complete input. C.4
Distribution Behaviour and Calibration Transfer
Dataset-trained representative ladder. The representative study recalibrates a dataset-trained model on each dataset while keeping the method recipe fixed. Final pixel cosines for ImageNet/Pets/CUB are 0.9308/0.9566/0.9398 for ResNet50, 0.8450/0.2876/0.6507 for ConvNeXt-B, and 0.7021/0.7590/0.7728 for ViT-B/32. The non-monotonic ConvNeXt result prevents a scalar “dataset complexity” interpretation. The supported conclusion is that feature-inverse behaviour depends jointly on the data distribution, checkpoint, and calibrated local statistics. Eight-family breadth with ImageNet weights. The separate breadth experiment applies ImageNet-weight models to Pets and CUB and recalibrates their maps on the corresponding images. All 16 rows pass the strict artifact and protocol audit. Final pixel cosine ranges from 0.3864 to 0.9451 on Pets and from 0.3684 to 0.9182 on CUB. Swin reaches 0.6757 on Pets and 0.6098 on CUB, while ViT reaches 0.7808 and 0.7193. These rows broaden the visual-distribution evidence but remain distinct from dataset-trained model comparisons. Calibration-source transfer. We fitted maps on ImageNet, Pets, or CUB and evaluated each map family on all three evaluation distributions. The model checkpoint and method remained fixed within each 3 × 3 matrix. The complete transfer matrices are shown directly in fig. 8 rather than repeated in a separate table. The matched-minus-cross average gap is small for ResNet18 (approximately 0.937 versus 0.932) and larger for ViT-B/32 (approximately 0.721 versus 0.662). Thus fixed-map transfer is more sensitive for the token model. Recipe generality across datasets does not imply that one map family is distribution invariant. Figure 8 collects the component-zoo breadth, the ImageNet-weight distribution study, and both calibration-transfer matrices. C.5
Topology, Two-Stage Mechanism, and Operator Ablations
Local topology versus a global repair. On the matched ResNet18 terminal target, raw VJP has pixel cosine 0.00019. A single global G reaches 0.07221, and a global G + D reaches 0.06714. Local Gv reaches 0.90090, while local Gv + Dv reaches 0.93795. The comparison holds operator calibration and target data fixed while changing whether 36
Source-Grounded Feature Inversion
a
A P REPRINT
b
Component coverage under one tensor-axis interface
concat/ dense connectivity depthwise/ inverted residual MBConv/ Squeeze-Excitation ConvNeXt blocks
Eight-family distribution breadth
ResNet50 DenseNet121 MobileNetV2
window-attention stages pooled/ vector state patch merging
EfficientNet-B0 RegNetY
normalization-adjacent
ConvNeXt-B
true depthwise-convolution tensors Swin-T
true depthwise and squeeze-excitation tensors ViT patch/ token states
ViT-B/32
−0.2
0.0
0.2
0.4
0.6
0.8
1.0
Pets
0.4
Pixel cosine (descriptive)
d
ImageNet
0.938
ImageNet
Pets
0.945
Pets
0.926
CUB
0.6
0.8
1.0
Final pixel cosine
ResNet18: stable across calibration sets matched 0.937 · cross 0.932
c
CUB
ViT-B/32: matched calibration matters matched 0.721 · cross 0.662
0.681
0.766
0.715
CUB
cross calibration matched calibration
0.6
0.7
0.8
0.9
Pixel cosine under evaluation distribution
0.6
0.7
0.8
0.9
Pixel cosine under evaluation distribution
Figure 8: Expanded component and distribution evidence. a, All validated component-zoo targets grouped by tensor class. b, Paired Pets and CUB results for eight ImageNet-weight backbone families under dataset-specific recalibration. c,d, Calibration-transfer profiles for ResNet18 and ViT-B/32. Each row retains the two cross-calibration values and the matched-calibration value for one evaluation distribution.
repair is factorised along the fitted DAG. It therefore isolates local topological composition from the mere existence of a closed-form map. Second-stage residual repair. On ResNet18, Gv alone reaches a pixel cosine of 0.9009 and the second stage adds 0.0367. The corresponding gain increases to 0.1107 on ConvNeXt-B and 0.3905 on ViT-B/32. Mean child NMSE decreases by 23.0%, 15.4%, and 16.2%, while intermediate-state relative ℓ2 decreases by 1.52%, 0.45%, and 5.92%. This architecture-level ordering is consistent with a larger role for JVP-FC when normalisation and attention induce strongly sample-dependent local operators. It does not isolate any individual module as the cause because architecture and component type are not independently controlled. JVP-FC therefore improves the aggregate local residual and endto-end result across all three representative architectures. Boundary-level correlations are not uniformly positive. Across 86 pooled boundaries, the Spearman correlation between the pre-correction residual and state-error reduction is 0.276 with bootstrap interval [0.068, 0.462]; its correlation with residual reduction is −0.235 with interval [−0.462, 0.004]. These results support JVP-FC as an effective second projection but not as a monotonic per-boundary causal law. Operator family. The complete matched operator comparison is reported in table 9. Full channel mixing and frequency specificity are separately required for strong input-instance alignment at the representative calibration budget. Rank-8 recovers much of the ResNet result but not the ViT result. This architecture dependence becomes stronger at small calibration budgets. Repeated ResNet18 seeds reproduce the Gamma-only result near 0.006, excluding an accidental calibration subset as the explanation. The visual outputs are retained alongside these scalar values because restricted operators can retain coarse spatial structure while differing in scale, colour, or frequency. 37
Source-Grounded Feature Inversion
A P REPRINT
Table 9: Restricted operator families. Entries are final pixel cosine at c4096/e1024. Restricted rows remain valid feature-inverse computations within their operator class; the values measure the input-aligned structures retained by that class. Operator
ResNet18
ConvNeXt-B
ViT-B/32
0.0062 0.2526 0.8337 0.2709 0.9375
0.0034 0.0961 0.4763 0.2090 0.8450
−0.0044 0.0620 0.0501 0.3366 0.7021
Gamma-only Diagonal, per-frequency Diagonal + rank-8, per-frequency Full-channel, shared-frequency Full-channel, per-frequency
Frequency alignment control. Replacing the per-frequency maps with one coordinate-collapsed map reduces pixel cosine from 0.9375 to 0.2709 on ResNet18 and from 0.7021 to 0.3366 on ViT-B/32. Permuting otherwise fitted frequency maps reduces the values further to 0.00025 and 0.0288. The result shows that map capacity alone is insufficient; the learned second-order operator must remain aligned with the coordinate frequency on which it was calibrated. Mean-seed gauge. The seed-source ablation includes heterogeneous-channel, multi-child nodes. On ResNet18, the full-state mean seed and an unweighted constant seed give numerically identical final pixel cosine, 0.9740235 in both cases. This is expected because a constant full-state rescaling is absorbed when separate zero-intercept relative-ridge maps are refitted. The mean gauge is retained because it defines the reusable terminal-subset API sT = PS HT /|S| across channel masks. It is not claimed to improve full-state quality over an arbitrary constant gauge. Unit-energy seeding is nonfinite on the tested ResNet configuration, while a fixed uniform seed produces zero output, confirming that the source must remain target dependent. Nested-target composition. Restarting the same calibrated inverse at an intermediate repaired state and continuing to the deeper output is numerically identical to the original continuation: output cosine is at least 0.999999996 on ResNet18 and ViT-B/32. Directly beginning at the shallower target gives 0.9955 output cosine to the deep continuation on ResNet18 and 0.8689 on ViT-B/32. Thus the implementation is compositionally consistent when the same repaired state is reused, while changing the declared target changes the feature-inverse object. C.6
Metric-Aware Theory Bridge and Finite-DAG Stability Diagnostics
Metric-aware local oracle. We evaluated the natural-adjoint construction on held-out local perturbations at 16 operating points. In a tractable projected problem, the local LMMSE operator and the metric-aware natural adjoint agree to numerical precision. Both reach mean NMSE 0.0291, cosine 0.9858, and relative ℓ2 0.1590. The raw Euclidean adjoint reaches NMSE 0.9999 and relative ℓ2 1.0000. A calibrated scalar rescaling improves these values to 0.5226 and 0.7191 but does not close the metric gap. The deployed observable-plus-G surrogate reaches NMSE 0.4416, cosine 0.7682, and relative ℓ2 0.6572. This experiment has a deliberately limited role. It verifies the algebraic bridge between a covariance-induced natural adjoint and a local LMMSE inverse in a tractable projected setting. It does not claim that the deployed distributional surrogate estimates Mu and Mv online or equals the local oracle at every operating point. Finite-DAG diagnostics. For each fitted boundary we recorded path depth, a conservative product-of-gain proxy, local condition statistics, child NMSE, intermediate-state error, and final pixel error. The proxy spans several orders of magnitude and describes the scale of the fitted reverse gains. However, it does not reliably rank final error across all nodes or architectures. The pooled Spearman correlation between pre-correction residual and state-error reduction is 0.276, with a 95% bootstrap interval of [0.068, 0.462]. Consequently, the empirical path-gain study is reported as a stability diagnostic, not as a contraction certificate or proof of global convergence. The formal finite-DAG result in section B.6 remains a conditional error-propagation bound. Figure 9 places the operator-capacity frontier, metric-aware oracle, boundary correlations, and seed-source controls side by side. C.7
Input Alignment, Representation Consistency, and Information-Flow Controls
Zero and shuffle controls. The complete controls are summarised in table 10. All zero targets produce exactly zero output, so the zero-intercept maps cannot inject a target-independent additive image template. Source shuffling moves 38
Source-Grounded Feature Inversion
a
b
Operator capacity and sample budget interact
0.6
Final pixel cosine
Metric-aware natural adjoint equals local LMMSE
1.0
Diag.+r8 Full/freq
Held-out local perturbation NMSE
Gamma Diagonal
0.7
A P REPRINT
0.5 0.4 0.3 0.2 0.1 0.0
0.8
0.6
0.4
0.2
0.0 512
2048
4096
Euclidean adjoint
ViT calibration images
c
Path/residual proxies are diagnostics, not guarantees
d
Residual reduction
Scalar calibrated
Deployed surrogate
Local LMMSE / natural adj.
The mean gauge defines the reusable subset API
1.0
Final pixel cosine
0.8
0.6
0.4
0.2 undefined
State-error reduction
0.0 −0.4
−0.2
0.0
0.2
0.4
Mean
Unweighted
Uniform
Unit energy
Spearman correlation with pre-correction residual
Figure 9: Operator capacity, theory bridge, and stability diagnostics. a, ViT operator-capacity by calibration-budget frontier. b, Held-out local perturbation NMSE for Euclidean, deployed, and metric-aware oracle operators. c, Pooled boundary correlations with 95% bootstrap intervals. d, Multi-child seed-source ablation.
Table 10: Information-flow controls. All rows use the matched c4096/e512 target/operator-control protocol. “Same” is the matched target and operating point. Terminal shuffle reports alignment to the fixed operating-point input and to the shuffled target image. Source shuffle reports alignment to the old input and new source. Model
Same
Zero norm ratio
Term. → input
Term. → target
Source → old
Source → new
ResNet18 Swin-T ViT-B/32
0.9380 0.3890 0.6922
0 0 0
0.9379 0.3492 0.0995
0.0225 0.0312 0.2494
0.0186 0.0096 0.0158
0.9380 0.5056 0.6922
the output toward the new operating point. Terminal-state shuffling produces architecture-dependent modulation, which separates the role of the target state from that of the source-local operators. Target–operator swap. For paired images A and B, we cross HA , HB with operator families JA , JB . ResNet18 conditions HA , JB and HB , JA align with the operator-point images at 0.9386 and 0.9386, while alignment with the target-state images is 0.0340 and 0.0316. Swin shows the same direction at 0.5509 and 0.5526. ViT has stronger target modulation: the mismatched conditions have paired identity values 0.2495/0.0959 and 0.1040/0.2358. These results show that source-local geometry selects the inverse branch while the selected terminal state modulates that branch. 39
Source-Grounded Feature Inversion
A P REPRINT
Continuous operating-point locality. On 128 fixed ResNet18 image pairs, the operating point is interpolated from A to B while the terminal target remains HA . Alignment to A decreases monotonically from 0.9402 to 0.0021, while alignment to B increases from −0.0036 to 0.9402. The two curves cross at interpolation coefficient 0.495. At every point, alignment to the current operating-point image remains between 0.9251 and 0.9402. By contrast, same-image noise with relative magnitude up to 0.1 changes alignment to the clean source by at most 0.00066. The curve therefore characterises the local branch selected by the operating point. Its child residual does not increase monotonically with mismatch and must not be used as a source-match detector. Channel diversity and representation checks. Mean pairwise pixel cosine among three fixed channel inverses is 0.4969 for ResNet18, 0.2304 for ViT-B/32, and 0.0993 for Swin-T. The outputs therefore do not collapse to one map-dependent template. Re-encoding is reported against architecture-specific random-pair nulls and remains separate from direct input alignment. The ROC AUC distinguishing matched from random-pair re-encoding is 1.000, 0.996, and 0.947 for ResNet18, Swin-T, and ViT-B/32, respectively. A high re-encoding value establishes representation consistency, while a low value does not prove that the selected feature inverse is false. Failure taxonomy. The fixed 1,024-image evaluation sets are stratified by low-frequency structure, weak highfrequency alignment, and chroma drift. ConvNeXt and Swin low-alignment cases are often concentrated in colour or fine texture, whereas deep ViT outputs are dominated more strongly by low-frequency and patch-grid structure. These categories describe architecture-dependent feature-inverse geometry. Mismatch with the complete input is not, by itself, treated as feature-inversion failure. Figure 10 expands the paired-image paths, same-image noise control, shuffle conditions, and channel-diversity summaries. C.8
Calibration, Boundary, Ridge, Online Runtime, and Memory
Calibration size. The paired token-model sweep uses calibration sizes {512, 1024, 2048, 4096, 8192}. ViT-B/32 final pixel cosine is 0.1287, 0.3626, 0.6477, 0.7021, and 0.7243. Swin-T reaches 0.0467, 0.2901, 0.4575, 0.5037, and 0.5259. The largest increase occurs between 1,024 and 2,048 images, while the 4,096-to-8,192 gain is approximately 0.022 for both models. This supports c4096 as a diminishing-return operating point, not as a universal sample optimum. Independent c4096 calibration subsets yield pixel-cosine standard deviations of 0.00030 for ResNet18, 0.00362 for ConvNeXt-B, and 0.00464 for ViT-B/32. The corresponding SSIM standard deviations are 0.00178, 0.00060, and 0.00122. Thus the reported architecture differences are larger than the observed three-subset calibration variation. Boundary density. At c512/e1024, the ResNet18 coarse boundary uses 9 fitted nodes and reaches pixel cosine 0.9293. The full-DAG mode uses 49 fitted nodes and reaches 0.9631. SSIM increases from 0.6747 to 0.7913 and LPIPS decreases from 0.3255 to 0.1664, while post-calibration online application increases from 53.9 to 74.7 ms/image. This diagnostic shows that finer boundaries can improve the inverse while raising online application, map, and topologyonboarding demands. The coarse policy is therefore a practical default, not a theoretical claim that finer factorisations cannot help. Ridge sensitivity. At c512, ResNet18 is stable over ρ ∈ {0.003, 0.01, 0.03}, with final pixel cosines 0.9247, 0.9278, and 0.9293. At c1024, ConvNeXt-B reaches 0.7233, 0.7584, and 0.7674. ViT-B/32 is more sensitive at the same small calibration budget, increasing from 0.0726 to 0.3124 and 0.5258. This interaction explains why low-capacity or low-calibration token-model results should not be used to claim universal operator compression. Online runtime, storage, and memory. The expanded resource audit is shown in table 11. Online times correspond to post-calibration application in c4096/e1024 quality runs. Storage includes both stages. Full-channel maps make calibration demand, storage, and online application architecture dependent. The practical benefit is therefore a change from repeated per-target search to direct queries after calibration, not zero total computation or memory. C.9
Deployment Validity, Oracle Gaps, and Implementation Quality Control
Public-path equivalence. Across all nine architecture implementations, the public inversion call and the directly instrumented reference call agree to pixel cosine 0.99999997 or higher. Maximum absolute differences are zero or below 6.2 × 10−7 . This audit verifies that the reported output is produced by the public API rather than a private visualisation branch. 40
Source-Grounded Feature Inversion
A P REPRINT
a Input A
b
α=0
α=0.1
α=0.5
c
Locality is path dependent
α=1
Input B
Target and source geometry have separable effects
Alignment to target-source identity
1.0
0.8
ResNet18
0.938
0.938
0.938
0.497
Swin-T
0.389
0.349
0.506
0.099
ViT-B/32
0.692
0.100
0.692
0.230
0.6
0.4
0.2 paired-image path same-image noise
0.0 0.0
0.2
0.4
0.6
0.8
1.0
ed
tch
Ma
Mismatch level
fle fle huf huf et s input urce s ource g r → Ta So ew s →n
l nne Cha rsity e div
Figure 10: Expanded operating-point locality and information-flow controls. a, Fixed-target inverses along two paired-image operating-point paths. b, Paired-image interpolation versus same-image noise. c, Matched, shuffle, and channel-diversity summaries across three architectures. Legal online child state. At deployment, each Dv source uses the repaired online child state, not a cached true forward activation. The same-map true-child condition is retained only as a nondeployable c4096/e512 diagnostic upper reference. On ResNet18, replacing the online child with the true child changes pixel cosine from 0.9382 to 0.9391 and target re-encoding from 0.6522 to 0.7570. On ViT-B/32, the same diagnostic changes pixel cosine from 0.7005 to 0.9410 and re-encoding from 0.2481 to 0.7289. The larger ViT gap identifies online child estimation as an important future improvement direction. It does not alter the legality of the deployed result. Zero intercept, scaling, and superposition. Zero targets produce zero outputs in every information-flow audit. For fixed operating-point operators, scaling the target by α ∈ {0, 0.25, 0.5, 1, 2} scales the output with cosine at numerical unity and relative errors at or below 1.7 × 10−7 for ResNet18; ViT scaling is exact to recorded precision. Cross-target superposition has output cosine 0.999999996 for both models. Weighted recomposition of a complete target from channel partitions gives cosine 0.99999 for ResNet18 and 0.99950 for ViT. After weighting for the public subset mean gauge, the remaining recomposition residual reflects finite-precision application, not an additive image template. Normalisation and residual gauge checks. The final path applies no source, activation, or cotangent prenormalisation. The Dv source is the pulled-back local child residual in the same gauge used by the online JVP, 41
Source-Grounded Feature Inversion
A P REPRINT
Table 11: Expanded resource audit. Online time is the measured 1,024-image application total divided by 1,024 after calibration; it is not isolated batch-one latency. Storage includes both Gv and Dv . Peak memory is GiB. Model ResNet18 ResNet50 DenseNet121 MobileNetV2 EfficientNet-B0 RegNetY-1.6GF ConvNeXt-B Swin-T ViT-B/32
Nodes
Online (ms/image)
Storage (GiB)
Calibration peak (GiB)
Eval peak (GiB)
9 17 9 31 30 28 37 26 14
58.4 223.7 104.1 155.0 164.0 114.2 369.7 129.0 105.1
0.83 25.42 5.84 4.35 5.43 4.29 15.51 5.98 2.99
1.24 19.44 8.79 6.46 7.63 4.79 16.69 6.57 3.79
1.71 17.09 7.42 5.39 6.79 5.21 16.78 6.56 3.61
Table 12: Decision-conditioned cross-architecture gate. Counts give the number of settings whose bootstrap interval has the required sign. The final column is the minimum–maximum prediction-flip fraction across the four tested depths at k = 16 under calibration-mean replacement. Model ResNet18 ConvNeXt-B ViT-B/32
Primary > 0
Zero > 0
Terminal negative < 0
k = 16 flip range
16/16 16/16 16/16
16/16 16/16 16/16
4/4 4/4 4/4
0.313–0.813 0.125–0.750 0.188–1.000
while its regression target is Hv − yv0 . Static checks validate the JVP direction, residual sign, stored map layout, and zero-intercept application. Runs that use true children, alternative correction weights, or nondefault internal boundaries are labelled diagnostic-only in their artifacts and are excluded from headline tables. With the legal online path and its practical headroom fixed, the next section expands the decision-grounded interpretation study without changing the fitted maps or the inversion API. Figure 11 summarises the token calibration curves, ridge sensitivity, storage demand, and online-child oracle gap. C.10
Expanded Prediction-Conditioned Feature Atlas
Protocol. The decision-conditioned experiment uses CUB-trained ResNet18, ConvNeXt-B, and ViT-B/32 checkpoints with top-1 accuracies 70.59%, 85.81%, and 70.16%. For each model, 16 correctly classified test images are fixed before feature selection. One c4096 map family is reused across four depths, all images, and all selected channel groups. The depths are 3/5/7/9 for ResNet18, 4/7/34/37 for ConvNeXt-B, and 2/6/10/14 for ViT-B/32. For predicted class ŷ and runner-up class y2 , channels are ranked by their activation–gradient contribution to the decision margin [32], ∂(zŷ − zy2 ) sc = Hl,c , . (71) ∂Hl,c Nested groups of k ∈ {1, 4, 8, 16} top-positive, top-negative, activation-matched, and 20-repeat random-matched channels are selected. Each group is inverted through the legal subset gauge PS Hl /|S|. In a parallel branch, the same channels are replaced in the original internal representation. Calibration-mean replacement is primary; zero replacement is a robustness protocol. The feature inverse is never re-input as a causal stimulus. Cross-architecture gate. The strict audit covers 36,864 intervention rows, 3,072 group inversions, and 1,536 deletion curves. Under calibration-mean replacement, the top-positive margin drop exceeds the within-image random-matched mean in all 48 model–layer–k settings. Every image-cluster bootstrap interval is above zero. Zero replacement gives the same 48/48 result. All 12 terminal top-negative controls have intervals below zero, demonstrating the expected opposite signed effect. Table 12 summarises the signed gate for each architecture. Normalised positive-minus-random effects are positive at every tested depth and group size. They range from 0.122 to 2.253 on ResNet18, 0.201 to 9.559 on ConvNeXt-B, and 0.068 to 1.805 on ViT-B/32. The effect scale differs across architectures, so the claim is based on within-image matched contrasts rather than direct comparison of raw logit units. Architecture-dependent inverse geometry. ResNet18 and ConvNeXt-B develop stronger deep common-mode inverse structure, while positive and negative groups have opposing decision effects. ViT-B/32 retains a patch-grid, low-frequency, globally mixed geometry. The shared result across architectures is the selection–inversion–intervention 42
Source-Grounded Feature Inversion
a
b
Token models reach a diminishing-return regime
A P REPRINT
Low-budget ViT is more ridge sensitive
0.7 0.8
Final pixel cosine
Final pixel cosine
0.6 0.5 0.4 0.3
0.6
0.4
0.2 0.2
ResNet18 ConvNeXt-B ViT-B/32
ViT-B/32 Swin-T
0.1
512
1k
2k
4k
8k
0.003
0.01
Calibration images
c
0.03
Relative ridge
d
Full-channel storage is architecture dependent
The online-child gap is architecture dependent
1.00 ResNet50
0.95
Deployable online child True-child diagnostic
ConvNeXt-B 0.90
Pixel cosine
Swin-T DenseNet121 EfficientNet-B0 MobileNetV2 RegNetY-1.6GF
0.85 0.80 0.75 0.70
ViT-B/32 0.65
ResNet18
0.60 100
ResNet18
101
ViT-B/32
Map storage, GiB (log scale)
Figure 11: Calibration and deployment diagnostics. a, Paired Swin-T and ViT-B/32 calibration curves. b, Ridge sensitivity at the stated small-calibration budgets. c, Full Gv + Dv storage across architecture families. d, Deployable online-child versus nondeployable true-child diagnostic.
chain, not identical visible structure. CUB segmentation and visible-part overlap remain secondary diagnostics and are not used to support the cross-architecture gate. They do not establish a unique semantic name for every channel group. Likewise, the CNN common-mode observation is consistent with prior holographic-representation evidence [33], but does not establish destructive interference without a dedicated signed cancellation experiment. C.11
Optional Generalisation and Application Extensions
These experiments test additional boundaries after the core claim is already established. They are not required for the main paradigm comparison; Figure 12 summarises the decision-conditioned, checkpoint, perturbation, and CLIP evidence. Target perturbations. ResNet18 is nearly invariant to relative Gaussian target noise up to 0.1 and to 8- or 16-bit quantisation. Its inverse remains at pixel cosine 0.9406, and output cosine to the clean inverse remains above 0.99999998 for Gaussian noise. ViT-B/32 remains close under Gaussian noise and 8-bit quantisation, but is more sensitive to channel dropout. Dropping 50% of target channels reduces pixel cosine from 0.7094 to 0.2977 and output cosine to the 43
Source-Grounded Feature Inversion
a
Early
Mid-1
Mid-2
Terminal
ViT-B/32
ConvNeXt-B
ResNet18
Source
A P REPRINT
Mean positive-minus-random effect over group sizes
b ResNet18
0.94
1.54
1.30
0.53
ConvNeXt-B
7.21
4.62
1.24
1.09
ViT-B/32
1.28
1.18
0.70
0.35
Early
Mid-1
Mid-2
Terminal
d
Target perturbation sensitivity differs by architecture
c Calibrated geometry is checkpoint specific
e
Maps A
0.939
0.017
Maps B
-0.075
0.884
Model A
Model B
Contrastive-pretrained extension
Pixel cosine
RN50 · last ViT-B/16 · last
0.8
ViT-B/16 · t2/b0 ViT-B/16 · t7/b5
0.6
ViT-B/32 · last
ResNet18 ViT-B/32
0.4
0.0
0.1
ViT-B/32 · t2/b0 ViT-B/32 · t7/b5
0.2
0.3
0.4
0.5
0.0
Target-channel dropout fraction
0.2
0.4
0.6
0.8
1.0
Final pixel cosine
Figure 12: Expanded decision-conditioned atlas and optional extensions. a, One fixed CUB image across four depths and three architectures. b, Mean positive-minus-random intervention effect over group sizes. c, Same-architecture checkpoint/map mismatch. d, Channel-dropout sensitivity. e, Contrastive-pretrained CLIP extensions across architecture and depth. clean inverse to 0.4083. This contrast is consistent with the stronger channel-coupling demand observed in the ViT operator ablation. Checkpoint specificity. Maps fitted on one ResNet18 checkpoint transfer poorly to a different same-architecture checkpoint. Matched map/model conditions reach pixel cosines 0.9388 and 0.8842, whereas mismatched conditions reach 0.0165 and −0.0754. The result confirms that calibrated maps encode checkpoint-specific local geometry rather than an architecture-level image decoder. Blind architecture onboarding. Automatic MLP-Mixer onboarding [34] captures only the stem projection rather than the intended final Mixer norm. The captured stem target itself is inverted successfully, with final pixel cosine 0.9470, SSIM 0.8299, and LPIPS 0.0991. The experiment is therefore a verified boundary-discovery failure, not a closed-form inversion failure. It defines a concrete engineering limitation: new architectures require the boundary capture policy to expose the intended target and reverse DAG. Contrastive-pretrained models. The same estimator and tensor-axis rule extend to CLIP-RN50 and CLIP ViTs [35]. CLIP-RN50 reaches final pixel cosine 0.9102 at its last target. CLIP-ViT-B/16 reaches 0.7519 at the last target and 0.8799 at an early block. For CLIP-ViT-B/32, the best last-target result is 0.5774 with a finer attention/MLP boundary, while early and middle targets reach 0.8338 and 0.7829. Overly fine token-internal splits collapse, which reinforces the practical importance of boundary selection. These results broaden the architecture and pretraining evidence but remain supplement-only because they do not change the main claim. 44
Source-Grounded Feature Inversion
A P REPRINT
References [1] Aravindh Mahendran and Andrea Vedaldi. Understanding deep image representations by inverting them. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5188–5196, 2015. [2] Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. Feature visualization. Distill, 2(11):e7, 2017. [3] Alexey Dosovitskiy and Thomas Brox. Inverting visual representations with convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4829–4837, 2016. [4] Charlie Nash, Nate Kushman, and Christopher K. I. Williams. Inverting supervised representations with autoregressive neural density models. In Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics, pages 1620–1629, 2019. [5] Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034, 2013. [6] Matthew D. Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In European Conference on Computer Vision, pages 818–833, 2014. [7] Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. Striving for simplicity: The all convolutional net. arXiv preprint arXiv:1412.6806, 2014. [8] Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. GradCAM: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision, pages 618–626, 2017. [9] Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. In Advances in Neural Information Processing Systems, volume 31, 2018. [10] David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissection: Quantifying interpretability of deep visual representations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3319–3327, 2017. [11] Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viégas, and Rory Sayres. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV). In Proceedings of the 35th International Conference on Machine Learning, pages 2668–2677, 2018. [12] Amirata Ghorbani, James Wexler, James Y. Zou, and Been Kim. Towards automatic concept-based explanations. In Advances in Neural Information Processing Systems, volume 32, 2019. [13] David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams. Learning representations by back-propagating errors. Nature, 323(6088):533–536, 1986. [14] Atilim Gunes Baydin, Barak A. Pearlmutter, Alexey Andreyevich Radul, and Jeffrey Mark Siskind. Automatic differentiation in machine learning: A survey. Journal of Machine Learning Research, 18(153):1–43, 2018. [15] Norbert Wiener. Extrapolation, Interpolation, and Smoothing of Stationary Time Series: With Engineering Applications. MIT Press, 1949. [16] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009. [17] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet large scale visual recognition challenge. International Journal of Computer Vision, 115(3):211–252, 2015. [18] Omkar M. Parkhi, Andrea Vedaldi, Andrew Zisserman, and C. V. Jawahar. Cats and dogs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3498–3505, 2012. [19] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The Caltech-UCSD birds-200-2011 dataset. Technical Report CNS-TR-2011-001, California Institute of Technology, 2011. [20] Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612, 2004. [21] Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 586–595, 2018. [22] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2015. [23] Leonid I. Rudin, Stanley Osher, and Emad Fatemi. Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena, 60(1–4):259–268, 1992. [24] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770–778, 2016. [25] Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q. Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4700–4708, 2017. [26] Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. MobileNetV2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4510–4520, 2018. [27] Mingxing Tan and Quoc V. Le. EfficientNet: Rethinking model scaling for convolutional neural networks. In Proceedings of the 36th International Conference on Machine Learning, pages 6105–6114, 2019. [28] Ilija Radosavovic, Raj Prateek Kosaraju, Ross Girshick, Kaiming He, and Piotr Dollár. Designing network design spaces. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10428–10436, 2020.
45
Source-Grounded Feature Inversion
A P REPRINT
[29] Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A ConvNet for the 2020s. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11976–11986, 2022. [30] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin Transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10012–10022, 2021. [31] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021. [32] Marco Ancona, Enea Ceolini, Cengiz Öztireli, and Markus Gross. Towards better understanding of gradient-based attribution methods for deep neural networks. In International Conference on Learning Representations, 2018. [33] Kaixiang Shu. Adjoint inversion reveals holographic superposition and destructive interference in CNN classifiers. arXiv preprint arXiv:2604.27529, 2026. [34] Ilya O. Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, Mario Lucic, and Alexey Dosovitskiy. MLP-Mixer: An all-MLP architecture for vision. In Advances in Neural Information Processing Systems, volume 34, pages 24261–24272, 2021. [35] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning, pages 8748–8763, 2021.
46