Accelerating Precise End-to-End Simulation: Latency-Sensitive Many-core System Modeling Yinrong Li∗ , Zexin Fu∗ , Yichao Zhang∗ , Germain Haugou∗ , Chi Zhang∗ , Marco Bertuletti∗ , Bowen Wang∗ , Luca Benini∗† ∗ Integrated Systems Laboratory (IIS), ETH Zurich, Zurich, Switzerland † Department of Electrical, Electronic, and Information Engineering (DEI), University of Bologna, Bologna, Italy
arXiv:2605.07750v1 [cs.AR] 8 May 2026
[email protected], {zexifu, yiczhang, haugoug, chizhang, mbertuletti, bowwang, lbenini}@iis.ee.ethz.ch
Abstract—Modern large language model workloads put increasing demands on parallel compute capability and on-chip memory capacity, while also stressing fine-grained data movement and synchronization. These trends motivate exploring and designing many-core accelerators with tightly coupled scratchpad memory (SPM) for scalable compute and predictable, explicitly managed data access. However, this architectural shift raises two challenges: cycle-accurate register-transfer level (RTL) simulation becomes prohibitively slow as system complexity grows, and performance estimation requires precise modeling of latencysensitive interconnect behavior. This paper presents a fast yet accurate end-to-end modeling approach for latency-sensitive many-core architectures, targeting large-scale instances such as TeraNoC with 1024 cores and a 4 MiB globally shared L1 SPM. The approach captures timing behavior of latency-sensitive SPM accesses across multiple interconnect scales, while abstracting non-essential hardware details. Across diverse benchmarks, the model tracks a cycle-accurate RTL golden model with errors below 7%, while delivering up to 115× faster simulation. The framework also provides detailed profiling across processing elements and interconnect, enabling efficient end-to-end software development and hardware design exploration. Two case studies demonstrate its practicality: profiling-guided optimization of FlashAttention-2 to reduce interconnect stalls and synchronization overhead, and design space exploration of network-onchip (NoC) router remapping to alleviate traffic imbalance and improve throughput. Index Terms—Simulator, many-core, interconnect
I. I NTRODUCTION Recent advances in Large Language Models (LLMs) have fundamentally reshaped the design of modern accelerators. The rapid growth in model complexity places increasing demands on parallel compute capability and on-chip memory capacity [1]. At the same time, the performance of many LLM workloads, such as attention mechanisms and fused linear algebra kernels, is increasingly dominated by fine-grained data movement and synchronization across a large number of Processing Elements (PEs) [2], [3]. To efficiently support these workloads, emerging accelerator designs adopt many-core architectures with large, tightly-coupled, software-managed Scratchpad Memory (SPM) [4]–[6]. Compared to cache-based designs, SPM-centric architectures enable predictable access This work has received funding from the Swiss State Secretariat for Education, Research, and Innovation (SERI) under the SwissChips initiative. © This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.
behavior and explicit control over data movement, thereby achieving high computational efficiency [7]. This architectural shift raises new challenges. First, fast simulation and prototyping become increasingly difficult as many-core designs scale in both PE count and interconnect complexity [8]. SPM-centric architectures further enlarge the design space because software-controlled memory management interacts tightly with hardware features, making effective Design Space Exploration (DSE) critical for identifying optimal configurations. Consequently, software–hardware codesign on such systems depends on end-to-end, system-level performance evaluation with timely feedback, thus motivating the need for fast and agile modeling approaches. A second key challenge is accurately modeling fine-grained, latency-sensitive interconnect behavior in SPM-centric manycore architectures. Even single-cycle timing variations along low-latency paths can significantly affect end-to-end performance, especially under intensive word-level accesses. This challenge is compounded by multi-level interconnect hierarchies with heterogeneous topologies, making it essential to capture timing effects precisely and consistently across multiple scales. This paper presents a precise and fast modeling approach for latency-sensitive, large-scale SPM-centric many-core systems. We implement our methodology in GVSoC [9], a highspeed, event-driven simulator widely used for architectural exploration and software development, and apply it to TeraNoC [10], a 1024-core architecture featuring a 4 MiB globally shared L1 SPM and a hybrid mesh–crossbar interconnect. The key contributions are: • Developing a fast yet accurate many-core performance model that captures fine-grained, latency-sensitive access timing variations across multiple interconnect scales, spanning local crossbars and chip-level mesh Networkon-Chips (NoCs), while preserving simulation speed via appropriate abstractions. The model achieves up to 115× speedup over Register-Transfer Level (RTL) simulation, with an error margin below 7%. • Deploying the FlashAttention-2 [2] kernel on TeraNoC using the proposed model, demonstrating efficient software tuning enabled by rich, fine-grained profiling. • Identifying interconnect bottlenecks and conducting hardware DSE with the model for improved performance
under representative workloads; specifically, introducing NoC router remapping to enhance throughput and reduce latency. II. R ELATED W ORK Architecture evaluation spans a wide spectrum of abstraction levels, ranging from cycle-accurate RTL simulation to high-level analytical or functional modeling [11], [12]. Cycle-accurate RTL simulation provides a golden reference for timing fidelity, but suffers from prohibitively long runtimes when applied to large-scale many-core systems [11], making it unsuitable for rapid design iteration. Field-Programmable Gate Array (FPGA)-based prototyping offers higher execution speed [13], but typically imposes constraints on system complexity due to limited hardware resources [14], particularly for scaling the number of PEs and modeling complex memory subsystems. Moreover, both RTL simulation and FPGA prototyping suffer from limited flexibility and incur substantial design and integration effort [11]. At the other end of accuracy– speed trade-off, binary translation–based simulators enable extremely fast execution and full-system simulation [15], [16], but their highly abstracted models are typically decoupled from hardware microarchitecture, resulting in limited fidelity for performance estimation. Event-based simulators offer a balanced trade-off between accuracy and simulation speed and are therefore widely adopted for architectural analysis and DSE. Classic NoC simulators such as Garnet [17], BookSim [18], and NoXim [19] provide detailed models of router microarchitecture, routing algorithms, and arbitration policies, enabling accurate network-level evaluation across common topologies. These approaches assume simplified endpoints and traffic injection models, which limits their ability to capture the interactions between computation and interconnect behavior in tightly coupled memory systems. More recent work has pushed toward improved scalability and timing accuracy. DARSIM [20] parallelizes NoC simulation with periodic synchronization to scale to larger networks and improve throughput, while BZSim [21] accelerates simulation of large-scale many-core systems by combining analytical latency estimation with selective detailed simulation. ONNXim [22] achieves cycle-accurate modeling of memory subsystems and NoC for multi-core Neural Processing Unit (NPU) targeting Deep Neural Network (DNN) inference. Despite these advances, existing interconnect modeling approaches are primarily built around packet-level traffic and are validated in regimes where end-to-end network latency is typically on the order of tens of cycles. In contrast, latencysensitive interconnects, which support fine-grained communication between PEs and multi-banked L1 SPMs, are modeled in an abstract fashion, with limited timing accuracy. Furthermore, prior works generally lack end-to-end, systemlevel profiling under real workloads as well as the associated DSE support. Our work instead targets latency-sensitive, wordlevel SPM interconnect modeling, and emphasizes calibrated
TABLE I C OMPARISON OF REPRESENTATIVE SIMULATION APPROACHES . Simulator
Method
Gran.
Topology Error vs RTL Scope
RTL sim. cycle-accurate as-impl. as-impl. QEMU [15] binary-tran. instr. N/A Garnet [17] event-based packet multiple DARSIM [20] traffic-trace packet mesh BZSim [21] hybrid packet multiple ONNXim [22] event-based packet multiple This work event-based word multiple
reference N/A 10%–15% <20% 10%–20% 5% on NoC <7% E2E
system system NoC NoC NoC system system
Gran.: traffic granularity; N/A: not applicable; E2E: end-to-end.
end-to-end timing accuracy together with system-level profiling to enable iterative software tuning and hardware DSE for performance optimization. In this context, an error below 10% relative to cycle-accurate simulation is highly desirable. Table I summarizes representative approaches along the criteria most relevant to this work. III. S IMULATOR AND T ESTBED A RCHITECTURE This section provides a brief overview of the simulation framework used in this work, GVSoC, and the target architecture for demonstration, TeraNoC. A. GVSoC GVSoC [9] is an open-source high-speed, event-driven architectural simulator designed for timing-aware, full-system simulation. Rather than enforcing strict cycle-wise execution, GVSoC models architectural components as interacting entities whose state updates are triggered by timestamped events. This execution model enables flexible timing abstraction while preserving causal correctness across the simulated system. Simulation in GVSoC is driven by clock engines, each associated with an event queue. Events are scheduled with explicit target timestamps and processed in temporal order. When an event is triggered, its handler is executed by the corresponding clock engine to update the architectural state and therefore model the functional behavior of the component. Based on the state, the handler may schedule subsequent events with new timestamps, thereby capturing the timing behavior of hardware operations. Communication between components in GVSoC is expressed via abstract request calls, allowing architectural interactions to be modeled independently of any particular microarchitectural implementation. This interface gives model developers flexibility in selecting the desired abstraction level, ranging from coarse-grained functional interactions to finegrained, timing-aware communication. For example, a memory access can be modeled either as an instantaneous abstracted operation, or as a multi-cycle process that incurs latency and occupies bandwidth under defined timing constraints. B. TeraNoC Architecture TeraNoC [10] is designed to scale a tightly coupled, sharedL1 SPM cluster beyond 1000 cores while preserving low access latency and high bandwidth by using a hybrid mesh– crossbar interconnect. To balance single-core utilization and
x32
R ... R
R
...
SPM Bank11
SPM Bank 4
...SPM Bank 5
SPM Bank 6
SPM ... Bank 7
SPM
SPM
SPM
SPM
...
...
...
...
Core 1
Router R R x32 R
...
...
...
...
Item
Baseline configuration
Tile
PEs L1 SPM Ports
4 Snitch Cores 16 banks × 1 KiB 1 intra-Group + 2 inter-Group Ports
Group
Tiles Routers
16 Tiles 32 Routers (2 per Tile)
Cluster
Groups Topology NoC instances
16 Groups 4×4 2D mesh 32 request + 32 response NoCs
... SPM Bank10
x32
SPM Bank 9
Bank 2 Bankx32 3 Router Bank 0 x32Bank 1 Router R R R x32 R x32 R R Xbar Intra-Tile ...
x32
Level
...
...
SPM Bank 8
x32
...
TABLE II T ERA N O C CONFIGURATION USED IN EVALUATION .
...
...
x32
...
...
...
...
...
R
x32 x32 Router Router Router R RSPM R Tile 0 R SPM SPM SPM R x32 R x32 x32 R RBank14 Bank15 R Bank12 Bank13 ... ...
x32
...
...
x32
R
x32
RR
...
RR R
R
Router R R x32 R
x32
x32
...
RR
RR R
x32
...
... ...
x32
...
RR R
RR
...
Tile 3
RR
Core 0 ...
Router R R x32 R R
Tile 2 ...
...
...
RR
R
Inter-Group Arbiter
Router R R x32 R
RR R
...
Intra-Group Arbiter
...
R ...
...
...
RR
RR
x32
R
RR
...
...
Router R R x32 R
...
...
Tile 1
RR R
RR
...
x32
...
Tile 0 1 ...
RR
R R x32 RTile 7
...
RR
Tile 6 ...
R ... R
RR R
RR
...
...
Intra-Group Xbar Router x32
Router R R x32 Tile 4 R Tile 5 ...
RR
...
x32
RR
...
...
Tile 11
RR
R
...
Tile...10
RR
...
RR
R
...
Tile 9
RR R
Router R R x32 R
x32
...
RR
R R x32 RR R RR R R ... R
x32
RR
RR
x32 Group 15 Router
Router-to-Tile Xbar
...Tile 8
Tile...14
RR
Router R R x32 RTile 15
...
RR
x32
Cluster-level
Router R R x32 R Tile 13 Tile 12 ...
Core 2
...
Core 3 ...
Fig. 1. TeraNoC Architecture.
many-core scalability, the cluster is partitioned hierarchically, as shown in Fig. 1: a base-level block (Tile) integrates a subset of PEs and a portion of multi-banked L1 SPM, while multiple Tiles form a higher-level block (Group). This organization creates distinct interconnect regimes (intra-Tile, intra-Group, and inter-Group), which are central to the timing behavior targeted by our GVSoC model. Within a Tile, each PE is a single-issue, single-stage, 32-bit RV32IA Snitch core, extended with an Integer Processing Unit (IPU) (int32/16b) and a Floating-Point Unit (FPU) (fp32/16b). To tolerate memory access latency, the Load Store Unit (LSU) of each PE supports multiple outstanding transactions. The L1 SPM is software-managed and physically implemented as many small SRAM banks; banking and address interleaving expose high aggregate bandwidth, while bank conflicts and crossbar arbitration determine contention under load. PEs access the local L1 bank portion through intra-Tile crossbars to provide low-latency, high-bandwidth SPM accesses. For requests targeting banks outside the local Tile, arbitration at the Tile boundary forwards transactions to either an intra-Group crossbar for accessing other Tiles within the same Group, or injects them into the inter-Group mesh NoC for long-distance accesses. On the destination side, incoming traffic is received at the Group interface, forwarded to the target Tile, and finally delivered to the addressed SPM bank through the intra-Tile crossbar. Requests and responses traverse separate networks, allowing dataless requests and data-bearing responses to be handled independently. At the Cluster level, Groups are interconnected by a 2D mesh NoC to obtain a regular routing pattern with low wiring overhead. The mesh is implemented as multiple parallel, narrow 32-bit channels; the number of channels (and thus
routers) per Tile is a design-time parameter that can be tuned to match available wiring resources and the target bandwidth. In the baseline configuration used in this paper, the parallel mesh routers are statically mapped to the inter-Group ports of Tiles. This hybrid hierarchy enables very low latency for local and intra-Group accesses (on the order of a few cycles), while keeping remote-Group accesses bounded to modest multi-hop delays (e.g., 7 cycles between adjacent Groups and 13.7 cycles on average in the evaluated 4 × 4 mesh configuration). Table II summarizes the baseline TeraNoC configuration used in our evaluation. While we report results for this concrete setup, the GVSoC model exposes all listed quantities as configurable parameters to enable systematic DSE. IV. M ODELING & E VALUATION This section presents our modeling methodology and its end-to-end validation on the TeraNoC architecture. A. Request Propagation and Timing Modeling Building on the request-based component interface provided by GVSoC, we model SPM interconnect behavior as the propagation of abstract requests among architectural components. We focus on the functional semantics of data movement and the accumulation of timing effects along each request’s path, while abstracting away other implementation-specific details. Each request represents a memory-mapped operation issued by an active component such as a PE or a Direct Memory Access (DMA) engine. It encapsulates the information required for interconnect modeling, including the target address, operation type, payload, and timing metadata. Requests are processed and forwarded through a sequence of components that form the interconnect topology. At each hop, the component processes the request using its local state and the request’s attributes, then dispatches it to the next hop or generates a response if the destination is reached. Algorithm 1 gives an overview of this process. For each hop, the routing decision (p) is determined by the request’s target address and the component’s routing policy, while arbitration and congestion effects are evaluated using the request’s timing metadata and the component’s internal state. In our model, the timing metadata is mainly represented by two complementary terms: latency (L), which accumulates the component’s intrinsic delay and any additional waiting due to arbitration
Algorithm 1 Per-hop request processing. Processing of request r at component c // Routing decision 1: p ← c.Route(r.addr) // Timing update 2: L ← c.BaseLatency(p, r) + c.ArbitrationDelay(p, r) 3: D ← c.TransferDuration(p, r) 4: r.latency ← r.latency + L 5: r.duration ← max(r.duration, D) // State update and dispatch 6: c.UpdateState(p, r) 7: c.Dispatch(p, r)
or congestion, and duration (D), which captures the payload serialization determined by the component’s bandwidth. Accordingly, the component updates the timing metadata and its internal state before dispatching the request onward. The propagation terminates at the destination component, which serves the operation and returns the response to the initiator. Through this propagation process, end-to-end communication timing emerges naturally as the accumulation of percomponent updates, rather than being imposed as an analytically derived delay. This approach captures fine-grained latency variations arising from contention and arbitration, which are particularly critical for latency-sensitive SPM accesses. At the same time, this request-based abstraction supports modeling traffic at various granularities, from word-level accesses to coarse-grained bulk transfers, within a unified framework. B. Router-Based Interconnect Abstraction We propose a generic router template as a modeling primitive for interconnects. The router serves as an abstract, all-toall switchbox with user-defined behavior, making it a common building block across interconnect layers. A router instance can be configured with a variable number of input and output ports, routing and arbitration policies, as well as timing parameters such as bandwidth and latency, allowing flexible construction of diverse interconnect topologies. As an example, the intra-Group crossbar can be modeled using a router whose input and output ports correspond to the number of Tiles, with a bandwidth of 4 bytes/cycle and a zero-load latency of one cycle. The router implements fullyinterleaved address mapping for target selection and uses a round-robin arbitration policy to resolve contention. By instantiating the same router primitive with different architectural descriptions, the model uniformly represents both the intraGroup multi-level crossbar and the inter-Group mesh NoC. To keep the model compact and reduce simulation overhead, we avoid a one-to-one mapping between hardware blocks and model components by folding simple interconnect elements (e.g., spill registers) into adjacent router components. This approximation introduces negligible timing error while reducing the number of components and request propagation steps.
TABLE III S UPPORTED PROFILING OUTPUTS . Scope
Output
Collected information
PE
Trace Perf. counter
Instruction execution trace; per-PE state. Inst/load/store counts; stall-cycle breakdown.
Interconnect
Trace Utilization Congestion
Transfer events: enqueue, arbitrate, dispatch. Link/port busy cycles; transfer event counts. Queue occ.; port conflicts and blocking.
Memory
Trace Conflict
Memory access log. Bank conflict statistics.
the interconnect. In synchronous execution, a request is fully processed within a single event handler; multi-hop propagation across components is realized via nested request calls. This approach achieves high simulation speed, but provides limited visibility into concurrent requests, which restricts the fidelity of contention-related timing effects. In contrast, asynchronous execution allows components to buffer requests and forward them in subsequent simulation events, enabling more hardware-like modeling of arbitration, contention, and hence latency variations. The improved timing fidelity comes at higher simulation overhead compared with synchronous execution, as each request may span multiple events. In our model, we selectively apply these two strategies across the interconnect hierarchy based on contention characteristics and their impact on end-to-end performance. For intra-Group communication, which is realized by a multi-level crossbar hierarchy, we model the request path asynchronously to capture fine-grained contention and arbitration under highly concurrent, word-level accesses. In contrast, the response path within the same crossbar hierarchy typically exhibits much lower contention under typical workloads; we therefore model responses synchronously with lightweight congestion handling to reduce simulation overhead while preserving timing fidelity in the dominant contention scenarios. For the inter-Group mesh NoC, both request and response paths are modeled asynchronously. Unlike the intra-Group crossbar, mesh traffic traverses multiple hops over shared links, and response traffic can experience non-negligible contention comparable to requests. Overall, this selective combination achieves a favorable trade-off between timing accuracy and simulation performance: contention-sensitive paths use higher-fidelity asynchronous modeling, while less critical paths are simplified without compromising end-to-end performance evaluation. D. Profiling Support Our model exposes fine-grained profiling data across PEs, interconnect components, and memories, enabling insightful bottleneck analysis for both software tuning and hardware design exploration. Table III summarizes the profiling capabilities supported by our framework.
C. Timing Accuracy and Performance Trade-offs
E. Benchmark Results
GVSoC supports both synchronous and asynchronous execution strategies for modeling request propagation through
To evaluate the accuracy and efficiency of the proposed GVSoC model, we select a diverse set of workloads spanning
traditional linear algebra and signal processing, as well as modern machine learning kernels. Fig. 2 presents the accuracy and efficiency of the GVSoC model across all evaluated kernels. Fig. 2(a) compares GVSoC results against cycle-accurate RTL simulation, showing that the GVSoC model closely tracks RTL behavior with relative error consistently below 7% across all workloads. Fig. 2(b) reports the simulation runtime, demonstrating substantial acceleration over RTL simulation. Depending on the workload, the GVSoC model achieves speedups of more than two orders of magnitude, enabling rapid design iteration. V. E ND - TO - END S OFTWARE D EPLOYMENT The proposed model enables an end-to-end software development workflow on TeraNoC by combining fast simulation with actionable profiling feedback. To demonstrate its practical value, we deploy FlashAttention-2 and present a profilingguided optimization case study, which would be difficult to achieve with cycle-accurate RTL simulation at scale. FlashAttention-2 [2] adopts a tiled formulation of selfattention, where the query matrix is partitioned along the sequence dimension and the attention score computation is further decomposed into smaller tiles along the key/value dimension. This blocking strategy avoids materializing the full attention matrix and enables streaming execution with bounded on-chip storage. In this work, we follow the deployment and execution methodology described in [23] to implement the FlashAttention-2 kernel. Leveraging the model’s profiling capabilities (Table III), the execution breakdown in Fig. 3 (Base.) reveals low PE utilization with substantial interconnect-induced stalls, indicating the dominance of long-latency data accesses. The high synchronization overhead further suggests execution imbalance across PEs. Interconnect profiling attributes this imbalance to concentrated port conflicts at the remote request input ports of a small subset of Tiles, especially at the beginning of each inner tiling loop. This creates non-uniform service latency, which in turn causes progress divergence across PEs. Guided by these observations, we apply two optimizations: (i) reduce long-range traffic over inter-Group NoCs to lower round-trip latency, and (ii) smooth concurrent accesses to shared data blocks to mitigate Tile-level port conflicts. Within the existing tiling scheme, we reorganize the workload distribution across PEs so that query accesses in score computation and all accesses in online softmax remain within the local Group, effectively reducing long-latency SPM accesses. Moreover, we introduce staggered start offsets across PEs, thereby smoothing the temporal access pattern and reducing peak contention at Tile-level ports. Fig. 3 (Opt.) shows that these changes substantially reduce interconnect-induced stalls and synchronization overhead, improving overall efficiency. Overall, this case study highlights the model’s essential role in end-to-end software deployment. By enabling rapid iteration and providing insightful profiling that correlates PE execution breakdown with interconnect congestion statistics, the model guides targeted optimizations.
VI. H ARDWARE D ESIGN E XPLORATION The proposed model enables rapid, profiling-driven hardware design exploration for interconnect-centric systems. By exposing fine-grained performance metrics and supporting fast what-if evaluations, it allows researchers to identify architectural bottlenecks under interconnect-intensive workloads and to iterate on design alternatives efficiently. Using this framework, we present a representative DSE case study on TeraNoC that explores NoC router remapping to improve throughput and reduce latency by alleviating traffic imbalance. When profiling interconnect-intensive workloads with frequent global memory accesses, such as Matrix Multiplication (MatMul), we observe a substantial number of PE stalls caused by congestion in the mesh NoC despite modest overall link utilization, indicating inefficient traffic distribution. Using link utilization and congestion statistics, we further identify two forms of imbalance: (i) spatial imbalance, where utilization differs significantly across parallel meshes within the same interval, and (ii) temporal imbalance, where congestion hotspots shift over time within a mesh as access patterns evolve. These results suggest that a considerable fraction of interconnect capacity remains underutilized, motivating mechanisms that better exploit idle links to improve overall throughput. As discussed in Section III-B, TeraNoC employs multiple parallel mesh NoCs for inter-Group traffic, which are statically mapped to the inter-Group ports of Tiles. Fig. 4 (a) illustrates the baseline configuration, where each Tile is connected to two fixed NoC routers. This static connection prevents traffic injected by a Tile from being redistributed across NoC instances under unbalanced load, limiting the ability to leverage idle network capacity. To address this limitation, we introduce router remapping by inserting a crossbar-like switch between Tiles and NoC routers, which dynamically maps Tile ports to NoC routers, as shown in Fig. 4 (b). The mapping is updated every cycle using a deterministic pseudo-random schedule, which is hardware-friendly and achieves performance close to true-random remapping in our simulations. Using our model, we observe an overall performance improvement of approximately 10%, while interconnect profiling further confirms a more balanced utilization across network links. From a physical implementation perspective, the routing complexity of remapper grows quadratically with the number of ports, making a monolithic 32-to-32 full remapper impractical. To address this limitation, we decompose the remapping functionality into multiple smaller remappers with limited port counts, thereby partitioning the remapping space into independent regions, and use our framework to explore the trade-off between remapper granularity and performance. We use the optimized MatMul kernel (Section V) as the representative workload for this study. As shown in Fig. 5, the eight-port remapper delivers the best performance, while the four-port remapper is close to this optimum. Further profiling reveals that SPM accesses from Tiles with adjacent identifiers exhibit similar patterns within a given time window, whereas logically distant Tiles show more
Cycles (×10³)
(a)
RTL
75
-5.3%
-0.8%
0.2% -2.2% 1.9% 3.9% 1.0% 1.6%
0.6% 0.3%
-2.7%
-3.4%
-3.1%
50
-1.4% 2.0%
-6.1%
-5.6% -7.0%
-1.6%
25
0 (b) 103 Sim Time [min]
GVSoC
10
81×
114×
59×
35×
101×
6
f1
2
68×
67×
79×
73×
93×
50×
51×
45×
49×
113× 115×
51×
45×
50×
101
6
a
6
2
_f1
y xp
_f3
y xp
a
hn
tc
ba
_f1
m or
ra
ft_
cf
_f1
x4 di
_ ky es
ol
ch
6
6 d
2
_f1
p ot
_f3
p ot
d
g
6
f1
v_
em
f3
em
g
v_
6
2 m or
rn
ye la
_f1
m
u
m
at
6
1 l_f
6
1 l_f
u
m
at
m
) (S
) (M
m
u
m
at
1 l_f
) (L
6
m
u
m
at
3 l_f
2
2
3 l_f
u
m
at
m
)
(S
) (M
m
u
m
at
2
3 l_f
) (L m
m
6
_f3
m
m
6
2
_f1
se
_f1
ax ftm
se
so
Fig. 2. Accuracy and simulation runtime of the proposed GVSoC model across evaluated workloads.
Opt. 0
0.25
Comp. I-Stall
LSU-Stall FU-Stall
0.5
0.75
Sync 1
Fig. 3. Execution breakdown (normalized) of FlashAttention-2.
Normalized Perf
1.15
Base.
Uni Partition
Selected
★
1.10 1.05 1.00 Baseline
(a)
Opt Partition
Full
16-port 8-port
4-port
2-port
(b)
Tile 0
Router 2 Router 3
Router 31
Tile 1
Fig. 5. DSE of router remapping configurations.
Router 1
Tile 15
Router 2 Router 3 ......
Router 30
Router 0
Tile 0
......
......
...... Tile 15
Router 1
Remapper
Tile 1
Router 0
Router 30
TABLE IV P ERFORMANCE IMPROVEMENT OF N O C ROUTER REMAPPING .
Kernel Impr.
BN 11.5%
Chol. 12.2%
LN 47.1%
GEMM 12.3%
MMSE 9.5%
SM 33.8%
BN, LN, and SM denote BatchNorm, LayerNorm, and Softmax, respectively.
Router 31
Fig. 4. Router remapping architecture.
VII. C ONCLUSION
divergent traffic behavior. Exploiting this observation, we further adjust the remapper partitioning strategy to better align with traffic locality, yielding additional gains over uniform partitioning. Combining four-port remappers with optimized partitioning achieves the best overall performance and is therefore selected as the final configuration. The magnitude of improvement varies across kernels because their traffic patterns differ: kernels with bursty and spatially skewed access patterns benefit the most from remapping, as it mitigates persistent hotspots and increases effective bandwidth. Table IV summarizes the resulting performance improvements across all evaluated global interconnect–intensive workloads.
In this work, we presented an accurate and efficient endto-end modeling approach for tightly coupled, SPM-centric many-core architectures, and applied it to TeraNoC as a representative target. By capturing the micro-level timing behaviors of fine-grained, latency-sensitive SPM accesses while abstracting unnecessary hardware details, the model achieves up to 115× speedup over RTL simulation and delivers very accurate performance estimates within 7% of cycle-accurate results. The model also provides detailed profiling insights across PEs and the interconnect, substantially strengthening performance analysis for hardware and software development and optimization. We demonstrated the practical use of the framework through end-to-end deployment and optimization of FlashAttention-2, where profiling-guided tuning of SPM accesses exposes and mitigates communication bottlenecks.
Finally, we demonstrated hardware DSE via a NoC router remapping case study that alleviates traffic imbalance and improves throughput for interconnect-intensive kernels. Together, these results highlight the model’s usefulness for software– hardware co-design and DSE for performance optimization. ACKNOWLEDGMENT This work has received funding from the Swiss State Secretariat for Education, Research, and Innovation (SERI) under the SwissChips initiative. R EFERENCES [1] J. Li, J. Xu, S. Huang, Y. Chen, W. Li, J. Liu, Y. Lian, J. Pan, L. Ding, H. Zhou, Y. Wang, and G. Dai, “Large language model inference acceleration: A comprehensive hardware perspective,” 2025, arXiv:2410.04466. [2] T. Dao, “FlashAttention-2: Faster attention with better parallelism and work partitioning,” in ICLR, 2024. [3] W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, and I. Stoica, “Efficient memory management for LLM serving with PagedAttention,” in SOSP, 2023, pp. 611–626. [4] M. Cavalcante, S. Riedel, A. Pullini, and L. Benini, “MemPool: A shared-L1 memory many-core cluster with a low-latency interconnect,” in DATE, 2021, pp. 701–706. [5] C. Lai, Z. Zhou, A. Poptani, and W. Zhang, “LCM: LLM-focused hybrid SPM-cache architecture with cache management for multi-core AI accelerators,” in ICS, 2024, pp. 62–73. [6] D. C. Jung, M. Ruttenberg, P. Gao, S. Davidson, D. Petrisko, K. Li, A. K. Kamath, L. Cheng, S. Xie, P. Pan, Z. Zhao, Z. Yue, B. Veluri, S. Muralitharan, A. Sampson, A. Lumsdaine, Z. Zhang, C. Batten, M. Oskin, D. Richmond, and M. B. Taylor, “Scalable, programmable and dense: The HammerBlade open-source RISC-V manycore,” in ISCA, 2024, pp. 770–784. [7] L. Cheng, M. Ruttenberg, D. C. Jung, D. Richmond, M. Taylor, M. Oskin, and C. Batten, “Supporting dynamic task parallelism on manycore architectures with software-managed scratchpad memories,” in ASPLOS, 2023, pp. 46–58. [8] J. Zarrin, R. L. Aguiar, and J. P. Barraca, “Manycore simulation for peta-scale system design: Motivation, tools, challenges and prospects,” Simul. Model. Pract. Theory, vol. 72, pp. 168–201, 2017. [9] N. Bruschi, G. Haugou, G. Tagliavini, F. Conti, L. Benini, and D. Rossi, “GVSoC: A highly configurable, fast and accurate full-platform simulator for RISC-V IoT processors,” in ICCD, 2021, pp. 409–416. [10] Y. Zhang, Z. Fu, T. Fischer, Y. Li, M. Bertuletti, and L. Benini, “TeraNOC: A multi-channel 32-bit fine-grained, hybrid mesh-crossbar NoC for efficient scale-up,” in ICCD, 2025, pp. 610–617. [11] A. Akram and L. Sawalha, “A survey of computer architecture simulation techniques and tools,” IEEE Access, vol. 7, pp. 78 120–78 145, 2019. [12] S. Khan, S. Anjum, U. A. Gulzari, and F. S. Torres, “Comparative analysis of network-on-chip simulation tools,” IET Computers & Digital Techniques, vol. 12, pp. 30–38, 2018. [13] D. Chiou, D. Sunwoo, J. Kim, N. A. Patil, W. Reinhart, D. E. Johnson, J. Keefe, and H. Angepat, “FPGA-accelerated simulation technologies (FAST): Fast, full-system, cycle-accurate simulators,” in MICRO, 2007, pp. 249–261. [14] X. Chen, G. Zhang, H. Wang, R. Wu, P. Wu, and L. Zhang, “MRP: Mix real cores and pseudo cores for FPGA-based chip-multiprocessor simulation,” in DATE, 2015, pp. 211–216. [15] F. Bellard, “QEMU, a fast and portable dynamic translator,” in USENIX ATC, 2005, p. 41. [16] S. Riedel, F. Schuiki, P. Scheffler, F. Zaruba, and L. Benini, “Banshee: A fast LLVM-based RISC-V binary translator,” in ICCAD, 2021, pp. 1105–1113. [17] N. Agarwal, T. Krishna, L.-S. Peh, and N. K. Jha, “GARNET: A detailed on-chip network model inside a full-system simulator,” in ISPASS, 2009, pp. 33–42. [18] N. Jiang, D. U. Becker, G. Michelogiannakis, J. Balfour, B. Towles, D. E. Shaw, J. Kim, and W. J. Dally, “A detailed and flexible cycleaccurate network-on-chip simulator,” in ISPASS, 2013, pp. 86–96.
[19] V. Catania, A. Mineo, S. Monteleone, M. Palesi, and D. Patti, “Noxim: An open, extensible and cycle-accurate network on chip simulator,” in ASAP, 2015, pp. 162–163. [20] M. Lis, K. S. Shim, M. H. Cho, P. Ren, O. Khan, and S. Devadas, “Darsim: a parallel cycle-level noc simulator,” in MoBS, 2010. [21] P. Strikos, A. Ejaz, and I. Sourdis, “BZSim: Fast, large-scale microarchitectural simulation with detailed interconnect modeling,” in ISPASS, 2024, pp. 167–178. [22] H. Ham, W. Yang, Y. Shin, O. Woo, G. Heo, S. Lee, J. Park, and G. Kim, “ONNXim: A fast, cycle-level multi-core NPU simulator,” IEEE Comput. Archit. Lett., vol. 23, pp. 219–222, 2024. [23] B. Wang, M. Bertuletti, Y. Zhang, V. J. Jung, and L. Benini, “A dynamic allocation scheme for adaptive shared-memory mapping on kilo-core RV clusters,” in ASAP, 2025, pp. 9–16.