arXiv CS open access

Event-triggered Implicit Perturbation for Zeroth-Order Fine-Tuning of Spiking Transformers

2026 · arxiv_cs

arXiv CS · Papers · 2026

deep-learning machine-learning neural-networks

Conceptio Open Archive

Analyze & explore related research in Conceptio

Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.

Explore in Search →Connect via MCP / API ↗

Event-triggered Implicit Perturbation for Zeroth-Order Fine-Tuning of Spiking Transformers

arXiv:2608.21223v1 [cs.AR] 21 Aug 2026

Tengteng Lei, Prabodh Katti, Rashi Dutt, Houssem Sifaou, Tan Peng, Osvaldo Simeone, Kai Xu, Bipin Rajendran

Abstract—Zeroth-order (ZO) optimization estimates gradients using only forward-pass evaluations, making it suitable for finetuning non-differentiable, event-driven spiking neural networks (SNNs). However, its deployment on in-memory computing (IMC) accelerators is constrained by the repeated read-modify-write (RMW) operations arising from explicit weight perturbation and the prohibitive hardware footprint of random number generators (RNGs) for statistically independent per-weight perturbations. To address these challenges, we propose an implicit-perturbation ZO (IPZO) architecture in which perturbation sums computed by an event-triggered perturbation generation unit (PGU) are combined with the weighted sums produced by the IMC array, eliminating perturbation-induced RMW operations while preserving weight-stationary execution of IMC. By exploiting spike sparsity, the PGU generates and accumulates perturbation contributions only for spike-activated weight rows, reducing the required row dimension of the RNG array. An address-driven XOR recombination scheme (PGU-XOR) is further introduced to mitigate the spatial correlations caused by direct RNG reuse (PGU-Reuse). The results show that (1) PGU-XOR matches software RNGs in accuracy on Spikingformer/CIFAR-10 (76.41% vs. 76.53%) and perplexity (PPL) on SpikeGPT/WikiText-2 (54.20 vs. 53.23), whereas PGU-Reuse degrades accuracy by 9.56 percentage points and increases PPL by 11.8; (2) implemented in a TSMC 16-nm CMOS technology, PGU-XOR incurs 40.3%– 46.0% area overhead and 15.2%–48.9% energy overhead per matrix-vector multiplication (MVM) relative to PGU-Reuse, yet its faster convergence reduces the total perturbation energy to 0.51× that of PGU-Reuse at iso-accuracy; (3) IPZO reduces the perturbation energy to 0.46×–0.83× that of conventional explicit weight perturbation for a batch size of B = 64 and T = 4 time steps, with the advantage growing as BT decreases. Index Terms—On-chip learning, fine-tuning, zeroth-order optimization, random number generation, spiking transformers, inmemory computing.

I. I NTRODUCTION

HE transformer architecture has established itself as a dominant framework in artificial intelligence, driving state-of-the-art advances across fields ranging from natural language processing [1], [2] to computer vision [3], [4]. Despite this success, the intensive computation and large memory footprint of transformers often preclude their efficient deployment T. Lei, P. Katti, R. Dutt, H. Sifaou, O. Simeone, and B. Rajendran are affiliated with the Institute for Intelligent Networked Systems (INSI), Northeastern University London, London E1 8PH, U.K. T. Peng and K. Xu are with the Centre for Intelligent Information Processing Systems (CIIPS), Department of Engineering, King’s College London, London WC2R 2LS, U.K. (b.rajendran@nulondon.ac.uk). This project is funded by the Advanced Research + Invention Agency (ARIA). The work of O. Simeone and B. Rajendran was also supported by Open Fellowships of the EPSRC (EP/W024101/1, EP/X011356/1) and by the EPSRC project (EP/X011852/1). The work of O. Simeone was also supported by the European Research Council (ERC) under the European Union’s Horizon Europe Programme (grant agreement No. 101198347).

on resource-constrained edge platforms, a challenge further exacerbated by the memory-wall bottleneck of traditional Von Neumann architectures [5]. To mitigate these hardware limitations, neuromorphic accelerators integrating spiking transformers with in-memory computing (IMC) have emerged as a promising paradigm [6]–[8]. By executing matrix-vector multiplications (MVMs) directly within memory arrays, IMC eliminates prohibitive data-movement overhead between memory and computation. Binary spike activations further simplify MVM computation by reducing multiply-accumulate (MAC) operations to additions. Spike sparsity, in turn, limits synaptic computation to active events, thereby reducing unnecessary operations and energy consumption [9], [10]. While existing accelerators are highly optimized for inference [11]–[13], achieving efficient on-chip learning for adaptive and privacypreserving edge intelligence remains a formidable challenge. Compared with inference, training incurs substantially higher memory and computational overhead, placing stringent constraints on hardware implementation [14]–[16]. Zeroth-order (ZO) optimization has attracted growing attention for on-chip learning as a memory-efficient alternative to conventional backpropagation (BP) [17]–[19]. Unlike BP, which requires storing intermediate activations and computing gradients through a backward pass [20], [21], ZO optimization estimates gradients using only forward-pass evaluations [22], [23]. This distinction is particularly important for spiking neural networks (SNNs), where backpropagation through time (BPTT) requires retaining neuron states across multiple time steps, further increasing the memory overhead of training [24], [25]. The forward-only nature of ZO optimization is naturally compatible with non-differentiable activations, making it well suited for training event-driven SNNs on resource-constrained edge hardware. However, efficiently implementing ZO optimization in IMCbased accelerators is far from straightforward due to the mismatch between its algorithmic requirements and the underlying hardware architecture. Firstly, ZO optimization requires statistically independent random perturbations for every weight parameter. Although uniformly distributed perturbations have been demonstrated as an effective alternative to Gaussian perturbations [26]–[28], supporting per-weight generation of uniform random numbers still incurs a prohibitive hardware cost. Even when implemented using compact linear-feedback shift registers (LFSRs), the area footprint of a single-bit uniform random number generator (RNG) far exceeds that of a memory cell storing one weight bit in an IMC array. This footprint mismatch limits the scalability of per-weight perturbation generation in IMC-based accelerators. While

random-number reuse can alleviate the RNG overhead [26], it introduces spatial correlations among the resulting perturbations and degrades the learning accuracy with insufficient random numbers. Secondly, conventional explicit-perturbation ZO optimization (EPZO) requires repeatedly reading the entire weight matrix, applying the perturbations, and writing the perturbed weights back into the IMC array during each forward pass for gradient estimation. The resulting frequent read-modify-write (RMW) operations compromise the energy advantage of IMC by reintroducing extensive data movement [29]–[31]. To overcome these bottlenecks, we propose an implicitperturbation ZO optimization (IPZO) architecture for IMCbased SNN accelerators that combines event-triggered perturbation generation with accumulation-domain perturbation injection. Specifically, an event-triggered perturbation generation unit (PGU) generates and accumulates perturbation contributions only for spike-activated weight rows at each time step. Given the inherent sparsity of SNNs, the PGU thus reduces the required row dimension of the RNG array well below that of the weight matrix. To mitigate the spatial correlations introduced by directly reusing perturbations from the reduced RNG array across weight rows, an address-driven XOR recombination scheme is employed to construct independent perturbations across the entire weight matrix. Furthermore, accumulation-domain perturbation injection combines the perturbation sums computed by the PGU with the weighted sums produced by the IMC array. By keeping the weights stationary throughout each optimization step, IPZO eliminates repeated perturbation-induced RMW operations while preserving the energy efficiency of IMC. The effectiveness of IPZO is demonstrated through comprehensive evaluations spanning algorithmic validation and hardware implementation, highlighting its potential for efficient on-chip learning in IMC-based SNN accelerators. The main contributions of this work are summarized as follows: • We propose an IPZO architecture that injects perturbation contributions into the accumulation domain rather than the stored weight domain, preserving weight-stationary execution and eliminating the repeated RMW operations required by EPZO. • We develop an event-triggered perturbation generation unit (PGU) that generates perturbations only for spikeactivated weight rows, decoupling the RNG array size from the weight matrix dimensions. To eliminate the spatial correlations introduced by direct RNG reuse, we design an address-driven XOR recombination scheme within the PGU. • We validate the proposed framework through algorithmic evaluation and post-layout hardware implementation in TSMC 16-nm CMOS. PGU-XOR achieves performance comparable to software RNGs across SNN image classification and language modeling, while its faster convergence reduces the total perturbation energy to 0.51× that of PGU-Reuse. At the architecture level, IPZO equipped with PGU-XOR reduces perturbation energy to 0.46– 0.83× that of EPZO under practical SNN fine-tuning settings.

The remainder of this paper is organized as follows. Section II introduces the necessary preliminaries, and Section III analyzes the hardware implementation challenges of ZO optimization. Section IV presents the architectural design of the PGU within the proposed IPZO framework. Section V evaluates both the algorithmic effectiveness and hardware efficiency of the design, and Section VI concludes the work. II. P RELIMINARIES A. Spiking Transformers A standard transformer comprises multiple stacked blocks, each consisting of a multi-head self-attention (MHSA) module and a feedforward network (FFN), together with residual connections and layer normalization. In each MHSA module, the input representations are linearly projected into queries Q, keys K, and values V using the weight matrices WQ , WK , and WV , respectively. Scaled dot-product attention is then √ computed as A = softmax(QK ⊤ / dk )V , where dk denotes the head dimension [32]. The attention computation and the linear transformations within the FFN rely heavily on dense MVMs and typically require high-precision activations, posing considerable challenges for resource-constrained edge hardware. Spiking transformers extend this architecture into the temporal domain by incorporating leaky integrate-and-fire (LIF) neurons into the MHSA and FFN modules [33], [34]. Following the linear projections, Q, K and V are encoded as temporal spike sequences Qt , Kt , and Vt over T discrete time steps, where t = 1, . . . , T . At each time step, attention is computed directly in the spike domain as At = LIF LIF Qt Kt⊤ Vt [35]. The intermediate attention scores St = LIF Qt Kt⊤ are regulated by the firing dynamics of LIF neurons instead of being normalized by softmax, thereby eliminating the hardware-intensive exponential and division operations required by conventional softmax attention. B. In-Memory Computing In-memory computing (IMC) alleviates the memory-wall bottleneck by performing computations directly within memory arrays. Depending on how the MAC operations are performed, IMC can be implemented in either the analog or the digital domain. Analog IMC typically activates multiple word lines (WLs) simultaneously and accumulates the resulting currents or charges along bit lines (BLs), producing MAC outputs in the analog domain. Digital IMC instead computes bitwise products using arithmetic logic embedded within the memory macro and accumulates the resulting partial products via digital circuits (e.g., adder trees), thereby generating digitally encoded MAC outputs [36]. Despite their different circuit implementations, both commonly employ a weightstationary dataflow, in which weights remain stored within the array while input activations are streamed through it for computation. IMC can be implemented using diverse memory technologies, each offering different trade-offs among density, reliability, and manufacturability. Among volatile memories, static random access memory (SRAM) is widely adopted for its

compatibility with standard CMOS processes, mature foundry support, and robust operation, although its multi-transistor bit cell imposes considerable area overhead [37], [38]. Emerging non-volatile memories (NVMs), including resistive randomaccess memory (ReRAM), phase change memory (PCM) and magnetic random-access memory (MRAM), enable higher storage density and compact crossbar integration. However, they remain subject to technology-dependent device nonidealities, including conductance drift, limited on/off ratios, device variability, and finite write endurance [39]–[41]. When mapped onto analog IMC implementations based on these memory technologies, spiking transformers benefit not only from inherent array-level parallelism but also from simplified computations enabled by binary, event-driven spike activations. In an IMC-based spiking transformer, input activations are represented as spike trains over time steps. Accordingly, the voltage applied to each WL switches between a fixed read voltage and ground, corresponding to spike values of ‘1’ and ‘0’, respectively. This binary gating reduces each computation from MAC to a pure accumulation, with only spike-activated rows contributing to the BL currents or charges. This sparsity-aware mechanism can improve energy efficiency while retaining the parallelism and compact hardware footprint offered by analog IMC. C. Zeroth-Order Optimization ZO optimization estimates gradients from finite differences of loss evaluations, making it applicable to non-differentiable objectives in which first-order derivatives are unavailable or prohibitively expensive to obtain [42]. By relying solely on forward-pass evaluations, ZO optimization eliminates the backward pass and avoids retaining intermediate activations for gradient computation [43]. This advantage is particularly pronounced for SNNs, as BPTT must retain neuronal states at every time step, causing the activation memory to grow approximately linearly with the number of time steps T . Despite avoiding this activation-storage overhead, conventional ZO implementations still require storing a random perturbation vector whose dimensionality equals the total number of trainable model parameters. MeZO [44] reduces this requirement by regenerating the perturbation vector from a shared random seed rather than explicitly storing it. The same vector can therefore be reproduced on demand during the positive and negative forward evaluations and the subsequent parameter update, bringing the memory consumption of ZO optimization close to that of inference. For a model with parameters θ ∈ Rd and a mini-batch B = {x1 , . . . , xB } of size B sampled from the data distribution D, let LB (θ) denote the empirical loss over B. The gradient estimate ĝ of ZO optimization is then given by q

ĝ =

1 X LB (θ + ϵzi ) − LB (θ − ϵzi ) zi , q i=1 2ϵ

(1)

where q denotes the number of random perturbations for each gradient estimation, ϵ > 0 controls the perturbation magnitude, and zi ∈ Rd is sampled from isotropic distributions such as the standard Gaussian N (0, I) or uniform distribution. The

resulting gradient estimate is then used to update the model parameters according to the stochastic gradient descent (SGD) rule, θt+1 = θt − ηĝ, (2) where η is the learning rate. Each optimization step therefore requires 2q forward-pass evaluations, corresponding to the positive and negative perturbations for each zi . III. C HALLENGES A. Massive Random Number Generation Implementing ZO optimization requires an independently sampled perturbation element for every weight parameter at each optimization step, creating a substantial demand for on-chip random number generation. Although uniformly distributed perturbations have been shown to be an effective alternative to Gaussian-distributed perturbations [26]–[28], their generation still incurs prohibitive hardware overhead in a fully parallel implementation. For a perturbation vector that matches the dimensionality of the weight matrix, the area required for the LFSR-based uniform RNGs comprising multiple flipflops and feedback logic is much larger than that of the SRAM or NVM cells storing the corresponding weights. Consequently, this pronounced footprint mismatch between stochastic perturbation generation and static weight storage makes fully parallel per-weight RNGs impractical for largescale IMC-based on-chip learning. To alleviate this footprint mismatch, perturbation-efficient ZO optimization (PeZO) [26] reuses the outputs of a small uniform RNG array across multiple weights, constructing full perturbation vectors through output concatenation and cyclically shifted remapping over time. Motivated by the low intrinsic dimensionality of language-model updates, PeZO effectively reduces the required number of RNGs. Nevertheless, the resulting statistical correlations among perturbation elements can degrade learning accuracy when insufficient random numbers are provided. Moreover, the RNG configuration required to avoid such degradation varies across models and tasks, complicating hardware provisioning and limiting portability across workloads. Consequently, reducing RNG overhead while maintaining sufficient perturbation diversity remains a critical challenge for scalable IMC-based ZO training. B. Frequent Read-Modify-Write Operations Although IMC architectures derive much of their efficiency from a weight-stationary dataflow, this advantage can be undermined in the on-chip implementation of ZO training. During inference, weights are written to the memory array once and remain stationary as successive input activations are processed, allowing the weight-loading cost to be amortized over all subsequent MVMs. During explicit-perturbation ZO training, however, each optimization step perturbs the model parameters, requiring the stored weight matrix to be repeatedly read from the array, modified, and written back. A cost incurred only once during inference therefore becomes a recurring per-step overhead during training. These repeated RMW operations not only consume considerable energy but

also impose a throughput penalty because of sequential rowwise reads and writes. Furthermore, frequent reprogramming is particularly detrimental to NVM-based arrays as it exacerbates write-endurance limitations and shortens operational lifetime. As a result, explicitly injecting perturbations into the memory array disrupts the weight-stationary dataflow central to IMC efficiency, making repeated RMW operations a bottleneck for on-chip ZO training. While PeZO reduces the hardware overhead of RNGs, it still explicitly applies perturbations to the stored weights, leaving frequent RMW operations largely unchanged. To mitigate the write-endurance constraints of NVMs, a dedicated eDRAMIMC array has been introduced to store and accumulate the dynamic perturbations while retaining the weights in the RRAM crossbar [45]. Although this avoids repeatedly reprogramming the RRAM weights, the perturbations must still be read from and written to an additional memory array, shifting rather than eliminating the associated energy consumption. NoiseZO [46] instead maps the same weight matrix onto two RRAM crossbars and exploits the conductance differences induced by independent programming noise as perturbations. Although this approach avoids repeated RMW operations, it relies on device-specific write-noise characteristics, limiting its portability across memory technologies. Therefore, a solution that preserves weight stationarity without requiring dedicated perturbation storage across different IMC technologies is still lacking.

IV. A RCHITECTURE AND D ESIGN A. Implicit-Perturbation ZO Architecture In an IMC-based ZO accelerator, the random perturbation can be injected either onto the stored weights before MAC computation or onto the accumulated output afterward. Since ZO fundamentally perturbs the model weights, the most direct realization adopts the former, adding the perturbation to the stored weight matrix explicitly, as illustrated in Fig. 1a. In this EPZO scheme, the weight matrix θ is first read from the IMC array while the PGU generates the random vectors zi in parallel. The weights are then perturbed by +ϵzi and written back as θ+ϵzi for the positive evaluation. A subsequent −2ϵzi then flips them to θ−ϵzi for the negative evaluation, and a final +ϵzi restores the array to θ, a step that becomes indispensable under multiple perturbations. Driven by the input spike vector x, the MVM operation over the modified weights produces the perturbed output x · (θ ± ϵzi ) at the cost of three full-array RMW cycles per perturbation. Rather than perturbing the stored weights explicitly, the proposed IPZO architecture injects the perturbation into the accumulated output, thereby preserving the weight stationarity of the IMC array. As illustrated in Fig. 1b, the IMC array executes the standard MVM with the input spike vector x to generate the unperturbed sum x · θ, while the PGU dynamically generates random perturbations and computes the scaled perturbation sum x·ϵzi through a corresponding MVM. These two intermediate results are then merged to reconstruct

Spikes

IMC Array

PGU Array

Read Perturbation Write back MAC b

Spikes

IMC Array

PGU Array

MAC 1

1 1

Perturbation Fig. 1. Comparison of (a) EPZO and (b) IPZO architectures for computing perturbed MVM outputs. The symbols x, θ, zi , and ϵ denote the input spike vector, weight matrix, i-th random perturbation vector, and perturbation scaling factor, respectively.

the perturbed output, which is functionally equivalent to the EPZO baseline by the distributivity of the MVM, x · (θ ± ϵzi ) = x · θ ± x · ϵzi .

(3)

This separation allows IPZO to decouple the stochastic perturbation from the two-dimensional weight-matrix domain and remap it onto the one-dimensional accumulation-sum domain, leaving the stored weights untouched during a single optimization step. B. Event-Triggered Weight Perturbation 1) Dimension Reduction: When deploying the IPZO architecture on SNNs, the PGU dimension can be reduced by leveraging the event-driven sparsity of spiking activations. At each time step, only the rows activated by input spikes contribute to the MVM, while the remaining rows remain inactive. The stochastic perturbation therefore only needs to be generated for the weights associated with these active rows rather than for the entire crossbar. This sparsity-induced decoupling allows the PGU row dimension to be determined by the number of concurrently active rows rather than the physical row dimension of the IMC array.

Spikes

PGU-Reuse

IMC Array Spike Address Mapping

LFSR

LFSR shift

LFSR

Row Reduction

adc adc

2 PGU Array

accumulation 1

LFSR

adc adc

Fig. 2. Block diagram of the sparse IPZO architecture with a row-dimensionreduced PGU array. m and n denote the numbers of rows and columns of the IMC array, and k represents the number of PGU rows (k ≪ m).

Fig. 2 illustrates the resulting row-reduced PGU. Given an m × n IMC array storing n multi-bit weights per row, the input spike vector x activates k rows on average at each time step, where k ≪ m. The PGU is therefore provisioned with k physical rows, which are dynamically mapped to the active IMC rows according to their spike addresses. When the number of active rows transiently exceeds this capacity because of sparsity fluctuations, the remaining active rows are processed over successive cycles. Therefore, this rowreduced design generates perturbations covering all weights participating in the MVM while substantially reducing the hardware overhead. 2) Perturbation Independence: While row-dimension reduction effectively minimizes the PGU hardware overhead, mapping the reduced-row PGU across the full weight matrix makes it challenging to ensure independence of perturbations. A baseline solution termed PGU-Reuse realizes this mapping by sharing the perturbation vectors generated by the reducedrow PGU across multiple weight rows. As shown in Fig. 3a, PGU-Reuse partitions the m-row weight matrix into l interleaved row groups (l = ⌊m/k⌋), allowing multiple weight rows to share the same set of k ×n perturbation vectors generated by the LFSR array. To randomize the perturbation sharing pattern across iterations, the spike address ai is first shifted by an iteration-dependent offset δ, yielding an intermediate address αi = ai − δ. This intermediate address αi is then mapped to a specific PGU row through the modulo operation ri = (αi + qi ) (mod k), where the quotient qi = ⌊αi /k⌋. Although this dynamic remapping changes the assignment of shared perturbation vectors across weight rows over different iterations, the repeated reuse of the same random number bank

Spike Address Mapping

PGU-XOR LFSR

LFSR

LFSR LFSR LFSR

LFSR

Fig. 3. Perturbation generation schemes for the row-dimension-reduced PGU: (a) PGU-Reuse with group-based perturbation sharing across rows; (b) PGUXOR with address decomposition and XOR-based combination of quotientand remainder-associated LFSRs.

inevitably introduces statistical correlations. To address this, we introduce a new PGU-XOR scheme through address-driven XOR recombination, enabling statistically independent perturbations across the entire weight matrix. As shown in Fig. 3b, the address-driven XOR recombination first decomposes each spike address ai ∈ [0, m) into a quotient qi ∈ [0, l) and a remainder ri ∈ [0, k) such that ai = k · qi + ri . When k is a power of two, qi and ri are extracted directly from the high- and low-order address bits, eliminating the need for a hardware divider. Therefore, the PGU-XOR comprises two independent LFSR groups including l quotient-associated LFSRs (Q-LFSRs) and a k × n array of remainder-associated LFSRs (R-LFSRs), which generate the corresponding random numbers qrn and rrn, respectively. For a given spike address ai , the single qrn selected by qi is bitwise XORed with the corresponding row of rrn indexed by ri , producing the n-element perturbation vector associated with that address. Because every Q-LFSR and R-LFSR is configured with a distinct primitive polynomial, the resulting quotient–remainder XOR combinations generate statistically independent perturbation vectors for all m spike addresses. Consequently, PGU-XOR eliminates the perturba-

a PGU-Reuse CLU

CLU

CLU & PGU-XoR

CLU

psum

CLU

add

CLU

add

accumulation cycles sum

b stride -bit

-bit Fig. 4. (a) Circuit diagram of the perturbation accumulation architecture with a β × φ CLU array for PGU-Reuse and PGU-XOR, requiring c = k/β accumulation cycles. (b) Overlapping bit-slicing scheme using stride s to partition the w-bit CLU output rn into v pseudo-random segments.

tion correlations introduced by PGU-Reuse, thereby restoring the perturbation diversity required for effective ZO gradient estimation. 3) Accumulation Contention: While XOR recombination restores perturbation independence, it forces multiple spike addresses to share access to the Q-LFSR and R-LFSR resources. Under multi-row concurrent activation, addresses that decompose to the same remainder contend for the same RLFSR row within a single cycle, creating row-level resource contention along the XOR-and-accumulation datapath. Rather than over-provisioning the LFSR resources, we resolve this contention with an accumulation network (Fig. 4a) that serializes the contending accesses across c cycles, trading a bounded latency for a compact hardware footprint. This accumulation network is organized as a β × φ array of combinational logic units (CLUs), where β = k/c and φ = n/v are set by the accumulation cycle count c and the segmentation number v. Within each CLU of PGU-XOR, the qrn and rrn selected by the corresponding qi and ri are first gated by a mask M through AND operations, passing only those associated with valid spikes so that the network adapts to a varying number of active rows (≤ β) per cycle. The gated qrn and rrn are then combined by an XOR to produce the wide w-bit random number rn. PGU-Reuse follows the same datapath but bypasses the XOR, directly forwarding the

gated rrn as rn. As illustrated in Fig. 4b, each w-bit rn is partitioned into v overlapping u-bit segments arn with a stride of s bits, where w = s · (v − 1) + u. This overlapping bitslicing scheme improves the utilization of every generated bit. Finally, these segments arn aligned to the same column are summed by the downstream adders and accumulated over c cycles to yield the perturbation sum. The accumulation cycle count c serves as a configurable design parameter that balances hardware area against accumulation latency. A larger c shrinks the CLU array (β = k/c) but distributes the random-number generation and accumulation over more cycles. This multi-cycle accumulation, however, typically does not introduce a system-level latency penalty within the IPZO architecture. Since the IMC pipeline inherently spends multiple clock cycles on MAC operations, column-level analog-to-digital conversion and partial-sum aggregation, the PGU accumulation is overlapped within these cycles as long as c stays within the IMC pipeline latency. This relaxed timing allows c to be configured according to the available hardware budget while keeping the accumulation latency effectively hidden. C. Data Flow Fig. 5 illustrates the data flow of the PGU-XOR across the alternating forward-pass and weight-update phases. Following the ZO training framework, PGU-XOR generates perturbations for gradient estimation during the forward pass and reproduces the identical perturbations for the weight parameter update afterward. This reproducibility is achieved by holding the LFSR states constant throughout a single optimization iteration, so that the same random numbers can be regenerated on demand without being stored. The two phases, however, differ in their perturbation generation process. During the forward pass, only the spikeactivated rows participate in the computation and perturbations are therefore generated exclusively for these active rows. As illustrated in Fig. 5, three spikes are active at time step t, causing the mask Mi to assert its three least significant bits. The resulting addresses ai are then decomposed into (ri , qi ) pairs to generate the associated perturbation segments arn . In contrast to the event-driven forward pass, the update phase sequentially traverses the entire weight matrix to regenerate perturbations for every row. Accordingly, a single bit of the mask M0 is asserted while the address a0 increments sequentially from 0 to m−1. As a0 advances, r0 cycles repeatedly from 0 to k −1 whereas q0 increments every k cycles. This sequential traversal ensures that the perturbations used during parameter updates remain identical to those employed for gradient estimation. V. E XPERIMENTS AND R ESULTS A. Spatial Sparsity Analysis Fig. 6 presents the mean firing rates ρ of Q, K, and V LIF neurons in Spikingformer [34] at each time step for three representative datasets. The shaded regions indicate the variation across multiple transformer blocks. Across all three datasets, the Q, K and V LIF neurons exhibit consistently sparse firing activity throughout the inference. The mean firing rates ρ of

Time LFSR State 0 Forward Pass

Update

3 spikes 0 111 67 03742 0 127 0 307 15 0 80 0

LFSR State 1 Forward Pass

LFSR State n

Update

7 spikes 01111111 32 029 17 3 101 02901753 12 000400 0 8 0

Forward Pass

Update

1 10 0123456789

0123

rows Fig. 5. Data flow of the PGU-XOR across forward pass and parameter update phases.

Fig. 6. Mean firing rate ρ of Q, K, and V LIF neurons across transformer blocks at each time step. Shaded regions indicate its range across transformer blocks.

all three neuron types, averaged across transformer blocks, remain below 15% at every time step. Among them, the Q LIF neurons exhibit the highest firing activity, with blockwise maxima reaching approximately 20% for CIFAR-10 and CIFAR-100 and 15% for ImageNet, while the K and V LIF neurons remain predominantly below 10%. These consistently low firing rates demonstrate the inherent sparsity of spiking activity and justify the row-dimension reduction adopted in the PGU design. Guided by this observation, both PGU-XOR and PGU-Reuse are configured with k = 8 physical rows for an m × n = 128 × 16 IMC array, providing a singlecycle capacity of 6.25%. Each of the 16 columns stores an 8-bit weight, resulting in 128 bit-columns at the cell level. Rather than provisioning additional physical rows for firing rates above 6.25%, multi-cycle scheduling is employed to trade latency for area. For an instantaneous firing rate ρ, the required number of scheduling cycles is Nc = ⌈mρ/k⌉. With m = 128 and k = 8, four scheduling cycles can accommodate firing rates of up to 25%, exceeding the maximum of approximately

Fig. 7. Evolution of bit probability distribution in 128 random sequences rn generated by (a) PGU-Reuse and (b) PGU-XOR.

20% observed in Fig. 6. These additional cycles can be largely overlapped within the IMC pipeline, thereby limiting their impact on system-level latency. B. Statistical Property Evaluation Using the k = 8 configuration for a 128 × 16 weight matrix, we evaluate the statistical properties of perturbations generated by PGU-Reuse and PGU-XOR in terms of bit-level uniformity, temporal randomness, and spatial independence. In this setup, PGU-Reuse employs an 8 × 2 array of R-LFSRs, while PGU-XOR additionally incorporates 16 Q-LFSRs, each implemented as a 36-bit (w = 36) LFSR with a distinct primitive polynomial. Through overlapping bit slicing and quotient–remainder address expansion, these reduced LFSR resources are logically expanded to generate perturbations covering the entire 128 × 16 weight matrix. We first examine bit-level uniformity, which requires a balanced distribution of bits 0 and 1 in each generated sequence. Fig. 7 presents the evolution of the bit probabilities for the 128

Fig. 8. Statistical correlation properties of random sequences rn: autocorrelation coefficient of a single sequence generated by (a) PGU-Reuse and (b) PGU-XOR; cross-correlation matrix among 128 sequences generated by (c) PGU-Reuse and (d) PGU-XOR.

Fig. 9. Cross-correlation characteristics of 8-bit (u) random sequences arn extracted from a 36-bit (w) PGU-XOR sequence with different stride configurations: (a) s = 1 with 29 sequences; (b) s = 2 with 15 sequences; (c) s = 4 with 8 sequences; (d) s = 7 with 5 sequences.

perturbation sequences (rn) associated with the first column of the weight matrix, each extended to a length of 10,000. For both configurations, the probabilities of bit 0 (P (0)) and bit 1 (P (1)) rapidly converge toward 0.5 as the sequence length grows, confirming that both PGU-Reuse and PGUXOR produce uniform binary sequences. Bit-level uniformity therefore does not distinguish the two architectures, motivating a further evaluation of their temporal and spatial statistical properties. Figs. 8a and b show the autocorrelation of a single sequence generated by PGU-Reuse and PGU-XOR, respectively. In both cases, the autocorrelation peaks only at zero lag due to the self alignment, whereas the coefficients across all non-zero lags remain close to zero, indicating negligible temporal selfcorrelation. The distinction emerges in the spatial domain, where the cross-correlation among the 128 sequences differs sharply between the two architectures. As shown in Fig. 8c, PGU-Reuse exhibits pronounced periodic cross-correlations among different perturbation sequences. These structured correlations arise from the repeated reuse of shared LFSR states across parallel rows under the fixed cyclic remapping scheme. In contrast, the cross-correlation matrix of PGU-XOR (Fig. 8d) remains close to zero except along the self-correlation diagonal, indicating that the 128 sequences are mutually uncorrelated. By eliminating structured spatial correlations, PGUXOR restores statistically independent perturbations across all weight rows, thereby preserving the isotropic perturbation

directions required for unbiased ZO gradient estimation. The overlapping bit-slicing scheme introduces an additional source of correlation between the generated perturbation segments. Because adjacent segments share overlapping bits, a smaller stride inevitably leads to stronger inter-segment correlation. We therefore evaluate the influence of the stride s on the cross-correlation among the partitioned 8-bit (u = 8) perturbation segments arn , generated from a 36-bit (w = 36) LFSR output. For narrow strides of s = 1 (v = 29, Fig. 9a) and s = 2 (v = 15, Fig. 9b), the resulting segments exhibit pronounced cross-correlations exceeding 0.5 between adjacent segments along the main diagonal. Increasing the stride to s = 4 (v = 8, Fig. 9c) reduces the maximum cross-correlation between distinct segments below 0.1, while a further increase to s = 7 (v = 5, Fig. 9d) suppresses the off-diagonal correlations to nearly zero. These results reveal a clear tradeoff between statistical independence and hardware efficiency. Larger strides effectively reduce inter-segment correlation but simultaneously decrease the number of available perturbation segments. We therefore adopt s = 4, which maintains low inter-segment correlation while preserving a sufficient segment count (v = 8). C. Accuracy Evaluation The training performance of a ZO-based SNN is evaluated using different perturbation generators, including the hardware

Fig. 10. Accuracy of a three-layer SNN-MLP (1024-128-10) trained from scratch on MNIST with MeZO under four perturbation schemes including PGU-Reuse, PGU-XOR, Randint, and Randn: (a) accuracy versus training epoch and (b) final accuracy and standard deviation over 10 independent runs.

implementations PGU-Reuse and PGU-XOR, as well as two software references, namely uniformly distributed integer perturbations (Randint) and normally distributed perturbations (Randn). To ensure a fair comparison, the 8-bit integer perturbations (∈ [−128, 128)) generated by PGU-Reuse, PGUXOR √ and Randint are normalized to [−0.5, 0.5) and scaled by 12 to match the unit variance of Randn. Experiments are conducted on a three-layer SNN-based multilayer perceptron (MLP) (1024-128–10) trained on MNIST, with ten independent runs from different random initializations for each perturbation scheme. As shown in Fig. 10a, PGUXOR exhibits nearly identical convergence behavior to both software references (Randint and Randn), with all three achieving stable optimization and narrow variance envelopes across ten independent runs. In contrast, PGU-Reuse exhibits substantially larger run-to-run variability, as reflected by its considerably wider error band. This is further confirmed by the final test accuracies in Fig. 10b. PGU-XOR, Randint, and Randn reach comparable mean accuracies of 88.8%, 89.1%, and 88.7% with tightly bounded standard deviations of 0.34%, 0.31%, and 0.30%, respectively, whereas PGU-Reuse attains only 82.7% with a substantially elevated deviation of 4.10%. This degradation directly reflects the structured spatial correlations identified in Fig. 8c. By reusing random perturbations across the rows of each weight matrix, PGUReuse violates the isotropy assumption underlying ZO gradient estimation, so the perturbations span far fewer independent directions than the dimension of the parameter space. This loss of directional diversity is quantified by the effective rank of the first-layer weight update ∆W = Wfinal −Winit accumulated over training, which is bounded above by m = 128. As shown in Fig. 11a, PGU-XOR, Randint, and Randn all reach a near-full rank of about 120, indicating that the optimization explores a broad set of independent directions. PGU-Reuse, in contrast, collapses to 61.5, roughly half that of the other schemes. The 128 perturbation rows of PGU-Reuse are produced by cyclically shifting 8 rows of base random

Fig. 11. (a) Effective rank and (b) Frobenius norm ∥∆W ∥F of the weight change ∆W = Wfinal − Winit for the first layer of SNN-MLP trained from scratch using MeZO under four perturbation schemes including PGU-Reuse, PGU-XOR, Randint, and Randn. TABLE I F INE - TUNING P ERFORMANCE U SING M E ZO U NDER D IFFERENT R ANDOM N UMBERS . Spikingformer [34] CIFAR-10 Acc. (%)

Epochs†

PGU-Reuse PGU-XOR

66.85 76.41

Randint Randn

76.53 76.56

SpikeGPT [47] WikiText-2

WikiText-103

PPL

Steps†

PPL

Steps†

247 84

66.01 54.20

3944 687

94.73 83.98

4000 777

80 83

53.23 53.07

633 653

82.67 82.32

641 742

† Training epochs (Spikingformer) or steps (SpikeGPT) required to reach the

final performance of PGU-Reuse.

sequences into 16 groups, and because the shifts repeat every 8 groups, the last 8 groups merely duplicate the first 8. The row space of the perturbation matrix is therefore effectively capped at m/2 = 64, closely matching the measured 61.5 and confining every update to the same low-dimensional subspace. As shown in Fig. 11b, the Frobenius norm ∥∆W ∥F is nearly identical across all four schemes at about 1.74 × 103 . PGUReuse perturbs the weights by the same overall magnitude as the others, but concentrates this displacement in half as many independent directions. This restriction also accounts for the elevated run-to-run variability (Fig. 10a), since the alignment between the gradient directions and the low-dimensional subspace varies with the random initialization. By decorrelating the perturbation rows, PGU-XOR removes this duplication and restores near-full-rank exploration of the parameter space, which underlies its accuracy and stability on par with the software references. The impact of this rank deficiency is further examined on two large-scale spiking models including Spikingformer [34] and SpikeGPT [47], fine-tuned with ZO optimization under the same four perturbation schemes. A 66.34M-parameter Spikingformer pre-trained on ImageNet-1K is fine-tuned on CIFAR-10 with 75% of its parameters frozen, leaving 16.6M trainable parameters, whereas a 216M-parameter SpikeGPT pre-trained on OpenWebText2 is fully fine-tuned and evaluated on WikiText-2 and WikiText-103.

PGU-Reuse

PGU-XOR

CLUs

102 µm

The results are summarized in Table I. Across all three tasks, PGU-XOR consistently matches the optimization performance achieved with the software perturbation generators (Randint and Randn), whereas PGU-Reuse exhibits a clear performance degradation. On CIFAR-10, PGU-XOR achieves 76.41% accuracy, closely matching Randint (76.53%) and Randn (76.56%), whereas PGU-Reuse reaches only 66.85%, 9.56 percentage points lower than PGU-XOR. A similar trend holds on WikiText-2 and WikiText-103, where PGU-XOR attains perplexities (PPLs) of 54.20 and 83.98, respectively, close to the software references, while PGU-Reuse yields higher values of 66.01 and 94.73. With the final performance of PGU-Reuse as a common reference, PGU-XOR reaches the same accuracy on CIFAR-10 in 84 epochs compared with 247 epochs for PGU-Reuse, reducing training epochs by a factor of about 2.9. On WikiText-2 and WikiText-103, PGUXOR reaches the corresponding reference PPLs in 687 and 777 steps against 3944 and 4000 steps, corresponding to 5.7× and 5.1× fewer iterations, respectively. These results demonstrate that the spatial correlations introduced by PGU-Reuse impair convergence speed and degrade the final accuracy and perplexity at large model scales, whereas PGU-XOR preserves performance comparable to software perturbation generators across both vision and language tasks.

R-LFSRs

50 µm

Q-LFSRs

73 µm

Fig. 12. Layout comparison of the PGU-Reuse and PGU-XOR implementations in 16-nm CMOS technology under accumulation cycles of c = 8.

D. Hardware Implementation PGU-Reuse and PGU-XOR are implemented in TSMC 16nm CMOS technology using a standard-threshold-voltage digital cell library. Both designs support configurable 1-bit (BIT1) Rademacher (±1) and 8-bit (BIT8) signed perturbation generation for a 128 × 16 IMC array. All hardware results reported in this section are obtained from post-layout implementations, with throughput evaluated at a target frequency f of 1 GHz and power evaluated at the typical-typical (TT) process corner, 0.8 V supply voltage, and 25 ◦ C. The following subsections evaluate their area, throughput, and energy consumption. 1) Area and Throughput: Fig. 12 compares the post-layout implementations of PGU-Reuse and PGU-XOR under the most area-efficient configuration (c = 8), corresponding to a single-row CLU array (β = 1). The layouts are partitioned into color-coded functional regions, including the CLUs, the R-LFSRs, and the Q-LFSRs exclusive to PGU-XOR. Under a cell utilization of approximately 66%, PGU-Reuse occupies 50 µm×102 µm (≈ 5,100 µm2 ), whereas PGU-XOR occupies 73 µm × 102 µm (≈ 7,446 µm2 ), corresponding to a 46.0% area overhead introduced by the additional Q-LFSRs for XORbased perturbation generation. Beyond this single configuration, Fig. 13a evaluates the post-layout area for c ∈ {2, 4, 8} with all values normalized to the largest configuration (c = 1, PGU-XOR). Increasing c progressively reduces the size of the CLU array through β = k/c, lowering the normalized area from 1.00 to 0.47 for PGU-XOR and from 0.70 to 0.32 for PGU-Reuse, while the overhead of PGU-XOR over PGUReuse stays within 40.3%–46.0% across all configurations. Despite the additional hardware required for perturbation decorrelation, the area of PGU-XOR at c = 1 remains comparable to that of a representative 128 × 128 IMC macro,

Fig. 13. (a) Normalized post-layout area and (b) throughput of PGUReuse and PGU-XOR under different accumulation cycles c in 16-nm CMOS technology. All designs are placed at ∼ 66% cell utilization. One operation (OP) denotes one multiplication or one addition.

which can represent an 8-bit realization of the same 128 × 16 weight matrix targeted by the PGU-XOR. Such a referenced IMC macro occupies 0.124 mm2 in 65-nm CMOS technology [48], corresponding to ∼ 16,000 µm2 after scaling to 16-nm technology [49]. As c increases, the relative area cost of PGUXOR with respect to the IMC macro decreases further. Fig. 13b presents the perturbation throughput of both PGUReuse and PGU-XOR under different accumulation cycles. Since PGU-XOR introduces the additional Q-LFSRs and XOR logic without increasing the pipeline depth, both designs produce one perturbation result every Tc cycles and therefore achieve identical throughput. Within the IPZO architecture,

Fig. 15. (a) Operation energies of a 128×64 SRAM in TSMC 16-nm technology: 64-bit read er , 64-bit write ew , and 8-bit addition ea . (b) Energy comparison between IPZO (PGU-XOR, per MVM) obtained from post-layout simulation, and EPZO (RMW, per batch) estimated from (a). (c) Energy ratio R = EIPZO /EEPZO as a function of batch size and number of time steps.

Fig. 14. Post-layout (a) power and (b) energy per MVM of PGU-Reuse and PGU-XOR under different accumulation cycles c in 16-nm CMOS technology, evaluated for 8-bit signed (BIT8, open bars) and 1-bit Rademacher (BIT1, hatched bars) perturbations. One MVM denotes one spike–perturbation product x · zi , where x is the spike vector and zi is the perturbation matrix. (c) Energy cost of PGU-Reuse and PGU-XOR at iso-accuracy on CIFAR-10, normalized to PGU-Reuse.

the perturbation MVM (x · ϵzi ) computed by the PGU follows the same computational structure as the weight MVM (x · θ) in the IMC array. Each perturbation element contributes one multiplication and one addition, amounting to 2mn operations for an m × n PGU array. The throughput is thus given by Throughput =

2mn · f , Tc

(4)

where f = 1 GHz and Tc = c cycles. Increasing c serializes more accumulation operations onto a smaller CLU array, raising Tc and lowering the throughput from 4.10 TOPS at c = 1 to 2.05, 1.02 and 0.51 TOPS at c = 2, 4, and 8. Together with the area scaling at larger c, these results illustrate a direct tradeoff between hardware cost and throughput, in which the reduced throughput incurs no system-level penalty as long as the PGU sustains the throughput of the IMC array. 2) Energy Consumption: Fig. 14 presents the post-layout power and energy of PGU-Reuse and PGU-XOR under different c, evaluated in both BIT8 and BIT1 perturbation modes. The energy is reported per MVM, denoting one computation between the spike vector x and the perturbation matrix zi ∈ R128×16 . As shown in Fig. 14a, the power of both designs decreases monotonically with increasing c owing to the reduced CLU array size, falling by over 3× from c = 1 to c = 8, while BIT1 consumes about 55%–60% of the BIT8 power at every configuration. In contrast, the energy

per MVM increases with c as shown in Fig. 14b because the reduced power is outweighed by the longer cycle interval Tc . In BIT8 mode, the energy rises from 11.5 to 24.7 pJ for PGUReuse and from 17.1 to 28.5 pJ for PGU-XOR. The resulting energy overhead of PGU-XOR over PGU-Reuse therefore narrows from 48.9% at c = 1 to 15.2% at c = 8, falling below the corresponding area overhead of 40.3%–46.0% for c ≥ 2. This discrepancy arises because the Q-LFSRs occupy additional area but exhibit low switching activity during each optimization iteration, contributing much less dynamic energy than their physical footprint would suggest. Consequently, increasing c reduces area and power at a proportional cost in throughput and energy per MVM, leaving c bounded by the IMC pipeline latency and the available energy and area budget. The per-MVM energy advantage of PGU-Reuse, however, does not translate directly into lower energy during training. Fig. 14c further compares the energy required by both PGUReuse and PGU-XOR to reach the same target accuracy of 66.85% for Spikingformer fine-tuning on CIFAR-10. Because the correlated perturbations of PGU-Reuse confine the update to a low-dimensional subspace, it requires 247 epochs to reach this accuracy, whereas PGU-XOR converges within 84 epochs. Despite its higher per-MVM energy, PGU-XOR therefore reduces the total perturbation energy to 0.51× that of PGU-Reuse, demonstrating that the modest hardware cost of perturbation decorrelation can be offset by improved training convergence and ultimately yields a more favorable hardware– algorithm trade-off. The energy advantage of the proposed IPZO architecture over the conventional EPZO architecture stems from the pronounced energy disparity among the primitive operations involved in weight perturbation. Fig. 15a characterizes the

energy of these operations using a 128 × 64 SRAM implemented in TSMC 16-nm technology. A 64-bit read (er ) and a 64-bit write (ew ) consume 4.68 pJ and 4.20 pJ, respectively, whereas an 8-bit addition (ea ) consumes only 0.01 pJ. This disparity of more than two orders of magnitude reveals that memory accesses, rather than arithmetic operations, dominate the energy cost of weight perturbation. IPZO exploits this gap by replacing costly weight reads and writes with low-cost additions in the accumulation domain. Based on these primitive-operation energies, Fig. 15b compares the estimated energy of EPZO with the post-layout energy of IPZO using PGU-XOR at c = 1. In EPZO, weight perturbation is performed through an RMW operation, where the weights and perturbations are read from the memory array, added element-wise, and written back. For 8-bit weights, an m × n weight matrix contains 8mn bits and is accessed in 64bit words, requiring mn/8 read accesses each for the weights and perturbations, mn/8 write accesses, and mn additions. The energy of one RMW operation is given by eRMW = mn

2er + ew + ea , 8

(5)

where the factor of two on er accounts for reading both weights and perturbations. For the 128 × 16 weight array, this yields eRMW = 3.50 nJ. Each ZO optimization iteration requires three RMW operations for q = 1, two for the positive and negative forward passes and one for the weight restoration, resulting in a fixed energy cost of EEPZO = 3 eRMW = 10.5 nJ per optimization iteration. In contrast, the post-layout energy of PGU-XOR is ePGU-XOR = 17.1 pJ in BIT8 mode and 9.37 pJ in BIT1 mode per input per time step. Accounting for the positive and negative evaluations, the corresponding IPZO energy scales with 2BT . The different scaling behaviors of the IPZO and EPZO determine the operating regime in which each architecture is more energy-efficient. To quantify this trade-off, Fig. 15c plots the energy ratio R=

2ePGU-XOR · BT EIPZO = , EEPZO 3eRMW

(6)

as a function of B and T . Since R scales linearly with BT , the break-even condition R = 1 occurs at BT ≈ 307 in BIT8 mode and BT ≈ 560 in BIT1 mode, as indicated by the two dashed contours. Below the respective contours, IPZO is more energy-efficient, whereas above them, its accumulated perinput perturbation cost eventually exceeds the fixed RMW cost of EPZO. At B = 64 and T = 4, as adopted for Spikingformer fine-tuning, IPZO consumes only 0.83× the energy of EPZO in BIT8 mode and 0.46× in BIT1 mode. The energy advantage of IPZO becomes increasingly pronounced at smaller BT , which is particularly relevant to real-time on-chip learning on resource-constrained edge devices. These results demonstrate that injecting perturbations in the accumulation domain, rather than explicitly modifying stored weights, preserves the weightstationary execution of IMC while substantially reducing perturbation energy across practical training configurations.

VI. C ONCLUSION This work presented IPZO, which is a hardware-algorithm co-designed architecture for efficient ZO-based on-chip learning in IMC-based spiking transformers. IPZO relocates random perturbations from the stored weight domain to the accumulation domain, eliminating the repeated RMW operations of explicit weight perturbation while preserving weightstationary IMC execution. Within IPZO, the event-triggered PGU-XOR generates perturbations only for spike-activated rows, decoupling the RNG array size from the weight matrix dimensions, while address-driven XOR recombination eliminates the spatial correlations introduced by the resulting RNG reuse. PGU-XOR achieves near-full-rank exploration of the parameter space and optimization performance comparable to software RNGs across both image classification and language modeling. Post-layout implementation in TSMC 16-nm CMOS shows that, despite the additional hardware for perturbation decorrelation, PGU-XOR reduces the total perturbation energy to 0.51× that of PGU-Reuse through faster convergence. At the architecture level, IPZO with PGU-XOR reduces the perturbation energy to 0.46–0.83× that of EPZO at B = 64 and T = 4. These results demonstrate that combining accumulation-domain perturbation with sparsity-driven, decorrelated perturbation generation provides an efficient hardware approach to ZO-based on-chip learning for IMC-based SNN accelerators. R EFERENCES [1] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017. [2] T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz et al., “Transformers: Stateof-the-art natural language processing,” in Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, 2020, pp. 38–45. [3] B. Wu, C. Xu, X. Dai, A. Wan, P. Zhang, Z. Yan, M. Tomizuka, J. Gonzalez, K. Keutzer, and P. Vajda, “Visual transformers: Tokenbased image representation and processing for computer vision,” arXiv preprint arXiv:2006.03677, 2020. [4] K. Han, Y. Wang, H. Chen, X. Chen, J. Guo, Z. Liu, Y. Tang, A. Xiao, C. Xu, Y. Xu et al., “A survey on vision transformer,” IEEE transactions on pattern analysis and machine intelligence, vol. 45, no. 1, pp. 87–110, 2022. [5] C. Wolters, X. Yang, U. Schlichtmann, and T. Suzumura, “Memory is all you need: An overview of compute-in-memory architectures for accelerating large language model inference,” arXiv preprint arXiv:2406.08413, 2024. [6] Z. Song, P. Katti, O. Simeone, and B. Rajendran, “Xpikeformer: Hybrid analog-digital hardware acceleration for spiking transformers,” IEEE Transactions on Very Large Scale Integration (VLSI) Systems, vol. 33, no. 6, pp. 1596–1609, 2025. [7] T. Das, K. P. Vu, H. Chen, H. Oh, and M. Imani, “Aster: Attention-based spiking transformer engine for event-driven reasoning,” arXiv preprint arXiv:2511.06770, 2025. [8] P. Jiang, L. Wang, Y. Fang, Y. Wang, M. Zhu, X. Miao, and X. Wang, “Sparta: Spike-aware token skipping co-optimization with heterogeneous reram-cim architecture for spiking transformer acceleration,” in 2025 IEEE/ACM International Conference On Computer Aided Design (ICCAD). IEEE, 2025, pp. 1–9. [9] K. Roy, A. Jaiswal, and P. Panda, “Towards spike-based machine intelligence with neuromorphic computing,” Nature, vol. 575, no. 7784, pp. 607–617, 2019. [10] M. Davies, N. Srinivasa, T.-H. Lin, G. Chinya, Y. Cao, S. H. Choday, G. Dimou, P. Joshi, N. Imam, S. Jain et al., “Loihi: A neuromorphic manycore processor with on-chip learning,” Ieee micro, vol. 38, no. 1, pp. 82–99, 2018.

[11] B. Keller, R. Venkatesan, S. Dai, S. G. Tell, B. Zimmer, C. Sakr, W. J. Dally, C. T. Gray, and B. Khailany, “A 95.6-tops/w deep learning inference accelerator with per-vector scaled 4-bit quantization in 5 nm,” IEEE Journal of Solid-State Circuits, vol. 58, no. 4, pp. 1129–1141, 2023. [12] H. Mun, J. Meng, X. Hu, Y. Liao, C.-T. Chen, and J.-s. Seo, “Asap: A 28-nm transformer training accelerator with alternating sparsity and asymmetrical microscaling precision,” IEEE Journal of Solid-State Circuits, 2026. [13] S. Nag, G. Datta, S. Kundu, N. Chandrachoodan, and P. A. Beerel, “Vita: A vision transformer inference accelerator for edge applications,” in 2023 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE, 2023, pp. 1–5. [14] P.-S. Wu, Y.-C. Lin, and C.-H. Yang, “A 99.2 tops/w transformer learning processor with approximated attention score gradient computation and ternary vector-based speculation,” in 2024 IEEE Symposium on VLSI Technology and Circuits (VLSI Technology and Circuits). IEEE, 2024, pp. 1–2. [15] Y. Wang, X. Yang, Y. Qin, Z. Zhao, R. Guo, Z. Yue, H. Han, S. Wei, Y. Hu, and S. Yin, “A 22nm 54.94 tflops/w transformer fine-tuning processor with exponent-stationary re-computing, aggressive linear fitting, and logarithmic domain multiplicating,” in 2024 IEEE Symposium on VLSI Technology and Circuits (VLSI Technology and Circuits). IEEE, 2024, pp. 1–2. [16] L. Shen, Y. Sun, Z. Yu, L. Ding, X. Tian, and D. Tao, “On efficient training of large-scale deep learning models,” ACM Computing Surveys, vol. 57, no. 3, pp. 1–36, 2024. [17] Y. Zhang, P. Li, J. Hong, J. Li, Y. Zhang, W. Zheng, P.-Y. Chen, J. D. Lee, W. Yin, M. Hong et al., “Revisiting zeroth-order optimization for memory-efficient llm fine-tuning: A benchmark,” arXiv preprint arXiv:2402.11592, 2024. [18] K. Sugiura and H. Matsutani, “Elasticzo: A memory-efficient ondevice learning with combined zeroth-and first-order optimization,” arXiv preprint arXiv:2501.04287, 2025. [19] J. Gu, C. Feng, Z. Zhao, Z. Ying, R. T. Chen, and D. Z. Pan, “Efficient on-chip learning for optical neural networks through power-aware sparse zeroth-order optimization,” in Proceedings of the AAAI conference on artificial intelligence, vol. 35, no. 9, 2021, pp. 7583–7591. [20] D. Han and H.-J. Yoo, On-Chip Training NPU-Algorithm, Architecture and SoC Design. Springer, 2023. [21] M. Dampfhoffer, T. Mesquida, A. Valentian, and L. Anghel, “Backpropagation-based learning techniques for deep spiking neural networks: A survey,” IEEE Transactions on Neural Networks and Learning Systems, vol. 35, no. 9, pp. 11 906–11 921, 2023. [22] S. Liu, P.-Y. Chen, B. Kailkhura, G. Zhang, A. O. Hero III, and P. K. Varshney, “A primer on zeroth-order optimization in signal processing and machine learning: Principals, recent advances, and applications,” IEEE Signal Processing Magazine, vol. 37, no. 5, pp. 43–54, 2020. [23] J. Larson, M. Menickelly, and S. M. Wild, “Derivative-free optimization methods,” Acta Numerica, vol. 28, pp. 287–404, 2019. [24] Q. Meng, M. Xiao, S. Yan, Y. Wang, Z. Lin, and Z.-Q. Luo, “Towards memory-and time-efficient backpropagation for training spiking neural networks,” in 2023 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2023, pp. 6143–6153. [25] H. Zhang and Y. Zhang, “Memory-efficient reversible spiking neural networks,” in Proceedings of the AAAI conference on artificial intelligence, vol. 38, no. 15, 2024, pp. 16 759–16 767. [26] Q. Tan, S.-E. Chang, R. Xia, H. Ji, C. Yang, C. Zhang, J. Liu, Z. Zhan, Z. Fang, Z. Zou et al., “Perturbation-efficient zeroth-order optimization for hardware-friendly on-device training,” in 2025 IEEE/ACM International Conference On Computer Aided Design (ICCAD). IEEE, 2025, pp. 1–9. [27] M. Qin, S. Yin, Q. Guo, P. Liu, X. Huang, and F. Wen, “Zeroth-order forward-only snn training inspiring neuromorphic on-chip learning,” in Forty-third International Conference on Machine Learning. [28] S. Liu, X. Li, P.-Y. Chen, J. Haupt, and L. Amini, “Zeroth-order stochastic projected gradient descent for nonconvex optimization,” in 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP). IEEE, 2018, pp. 1179–1183. [29] M. E. Sinangil, B. Erbagci, R. Naous, K. Akarvardar, D. Sun, W.S. Khwa, H.-J. Liao, Y. Wang, and J. Chang, “A 7-nm compute-inmemory sram macro supporting multi-bit input, weight and output and achieving 351 tops/w and 372.4 gops,” IEEE Journal of Solid-State Circuits, vol. 56, no. 1, pp. 188–198, 2020. [30] Z. Jiang, S. Yin, J.-S. Seo, and M. Seok, “C3sram: An in-memorycomputing sram macro based on robust capacitive coupling computing

mechanism,” IEEE Journal of Solid-State Circuits, vol. 55, no. 7, pp. 1888–1897, 2020. [31] Y.-K. Yeh, J.-W. Su, T.-H. Hsu, M. Tseng, J.-C. Tien, K.-C. Chen, C.Y. Yue, Y.-E. Lin, Y.-J. Hu, L.-J. Hsieh et al., “A 16nm, 1mb, 1-to8b-configurable 444.21 tops/w fully digital sram compute-in-memory macro for hybrid snn-cnn edge computing,” in 2026 IEEE International Solid-State Circuits Conference (ISSCC), vol. 69. IEEE, 2026, pp. 526–528. [32] Q. Fournier, G. M. Caron, and D. Aloise, “A practical survey on faster and lighter transformers,” ACM Computing Surveys, vol. 55, no. 14s, pp. 1–40, 2023. [33] B. Cessac, “A discrete time neural network model with spiking neurons II. Dynamics with noise,” Journal of Mathematical Biology, vol. 62, no. 6, pp. 863–900, 2011. [34] C. Zhou, L. Yu, Z. Zhou, H. Zhang, J. Wang, H. Zhou, Z. Ma, and Y. Tian, “Spikingformer: A key foundation model for spiking neural networks,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 40, no. 3, 2026, pp. 2236–2244. [35] Z. Song, P. Katti, O. Simeone, and B. Rajendran, “Stochastic spiking attention: Accelerating attention with stochastic computing in spiking networks,” in 2024 IEEE 6th International Conference on AI Circuits and Systems (AICAS), 2024, pp. 31–35. [36] Y. Chich et al., “An 89 tops/w and 16.3 tops/mm 2 all-digital sram-based full-precision compute-in memory macro in 22nm for machine-learning edge applications,” in ISSCC, 2021, pp. 252–253. [37] C.-J. Jhang, C.-X. Xue, J.-M. Hung, F.-C. Chang, and M.-F. Chang, “Challenges and trends of SRAM-based computing-in-memory for AI edge devices,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 68, no. 5, pp. 1773–1786, 2021. [38] A. Kneip and D. Bol, “Impact of analog non-idealities on the design space of 6T-SRAM current-domain dot-product operators for in-memory computing,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 68, no. 5, pp. 1931–1944, 2021. [39] F. Aguirre, A. Sebastian, M. Le Gallo, W. Song, T. Wang, J. J. Yang, W. Lu, M.-F. Chang, D. Ielmini, Y. Yang et al., “Hardware implementation of memristor-based artificial neural networks,” Nature communications, vol. 15, no. 1, p. 1974, 2024. [40] R. Khaddam-Aljameh, M. Stanisavljevic, J. F. Mas, G. Karunaratne, M. Braendli, F. Liu, A. Singh, S. M. Müller, U. Egger, Petropoulos et al., “HERMES core–a 14nm CMOS and pcm-based in-memory compute core using an array of 300ps/LSB linearized CCO-based ADCs and local digital processing,” in 2021 Symposium on VLSI Circuits. IEEE, 2021. [41] G. Verma, A. Nisar, S. Dhull, and B. K. Kaushik, “Neuromorphic accelerator for spiking neural network using SOT-MRAM crossbar array,” IEEE Transactions on Electron Devices, 2023. [42] A. Chen, Y. Zhang, J. Jia, J. Diffenderfer, K. Parasyris, J. Liu, Y. Zhang, Z. Zhang, B. Kailkhura, and S. Liu, “Deepzero: Scaling up zeroth-order optimization for deep model training,” in International Conference on Learning Representations, vol. 2024, 2024, pp. 50 185–50 206. [43] R. C. Suwandi, F. Yin, and T.-H. Chang, “Breaking the curse of dimensionality in gaussian process training with zeroth-order adaptive perturbation,” in ICASSP 2026-2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2026, pp. 22 497–22 501. [44] S. Malladi, T. Gao, E. Nichani, A. Damian, J. D. Lee, D. Chen, and S. Arora, “Fine-tuning language models with just forward passes,” Advances in Neural Information Processing Systems, vol. 36, pp. 53 038– 53 075, 2023. [45] M. Chen, L. Zheng, J.-Y. Lin, P. D. Ye, and H. Li, “Analog multilevel edram-rram cim for zeroth-order fine-tuning of llms,” in 2025 IEEE International Memory Workshop (IMW). IEEE, 2025, pp. 1–4. [46] S. Wang, Z. Liu, C. Ding, C. Zhang, T. Wu, J. Zhou, and N. Wong, “Noisezo: Rram noise-driven zeroth-order optimization for efficient forward-only training,” in 2025 62nd ACM/IEEE Design Automation Conference (DAC). IEEE, 2025, pp. 1–7. [47] R.-J. Zhu, Q. Zhao, G. Li, and J. Eshraghian, “SpikeGPT: Generative pre-trained language model with spiking neural networks,” Transactions on Machine Learning Research, 2024. [Online]. Available: https://openreview.net/forum?id=gcf1anBL9e [48] V. Sharma, J.-E. Kim, H. Kim, L. Lu, and T. T.-H. Kim, “A reconfigurable 16kb and8t sram macro with improved linearity for multibit compute-in memory of artificial intelligence edge devices,” IEEE Journal on Emerging and Selected Topics in Circuits and Systems, vol. 12, no. 2, pp. 522–535, 2022.

[49] A. Stillmaker and B. Baas, “Scaling equations for the accurate prediction of cmos device performance from 180 nm to 7 nm,” Integration, vol. 58, pp. 74–81, 2017.

Event-triggered Implicit Perturbation for Zeroth-Order Fine-Tuning of Spiking Transformers

Analyze & explore related research in Conceptio

Related documents