Shiftfly: Scaling the Accelerator Interconnect Past the Pod with a Shift-Routed Optical Tier

Shiftfly: Scaling the Accelerator Interconnect Past the Pod with a Shift-Routed Optical Tier∗ Eylon E. Krause

arXiv:2608.00897v1 [cs.NI] 1 Aug 2026

Weizmann Institute of Science Rehovot (7610001), Israel College of Management Academic Studies Rishon LeZion (7579806), Israel eylonkr@colman.ac.il

Abstract

CCS Concepts

Google’s TPU interconnect spent nine generations as a Cayley graph of an abelian group—a 𝑘-ary 𝑛-cube whose diameter grows as Θ(𝑁 1/𝑛 )—before TPU 8i replaced it with Boardfly, a three-tier hierarchy in which every tier is a complete graph and the pod diameter is 7 over 1,152 chips. Boardfly is well matched to a single inference pod. It does not extend: its global tier is a complete graph on groups, so reaching 𝐺 groups costs 𝐺 − 1 optical ports, and a 400,000-chip machine would need 12,499 per group against the 40 available. Another hierarchy level must therefore be stacked, and each level costs four chip hops. We propose Shiftfly, which retains Boardfly’s building block and group verbatim and replaces only the global tier with a generalized Kautz digraph, installed as a fixed permutation on the optical circuit switch that the fabric already owns. At an identical budget of 40 optical ports per group, Shiftfly is a flat graph of guaranteed diameter ⌈log𝑑 𝐺⌉, routes without tables by a shift register, and addresses content natively. Our evaluation is deliberately two-sided. Shiftfly loses at onepod scale—Boardfly achieves chip-level diameter 7 where Shiftfly needs 11— and wins beyond it, reducing worst-case distance from 23 to 19 hops at 400,000 chips with roughly 2.7× better spectral expansion at equal optical cost. The redundancy argument that motivated the design does not survive its own evaluation: localityaware placement accounts for 21.5% and 14.5% of the achievable saving on shared content on the two topologies respectively, leaving the shift algebra a residual of 2.9%. We report the metric inversion that conceals this, and conclude that the defensible case for Shiftfly is scaling and expansion rather than deduplication. Because Shiftfly is proposed as a drop-in replacement for one tier of an existing machine, we also measure whether it can be operated. Replacing a failed group costs the same 40 optical circuits in both designs; deriving the global wiring costs 28 bits of controlplane state against 550,000. Slice allocation is the one regression: an arbitrary induced subset of a shift graph is disconnected, so slices must be instantiated rather than carved, at a cost of one switch reconfiguration per allocation.

• Networks → Network design principles; Data center networks; • Computer systems organization → Interconnection architectures.

Keywords interconnection networks, network topology, degree–diameter problem, Kautz digraphs, optical circuit switching, machine-learning accelerators

1

Introduction

An accelerator interconnect is a graph, and the quantities that decide whether a machine is usable—worst-case latency, average latency, throughput under adversarial traffic, behaviour under failure— are graph invariants: diameter, mean distance, bisection width, connectivity. This paper takes that view literally, applies it to every interconnect Google has shipped for its Tensor Processing Units, and proposes a change to the most recent one. For nine generations the TPU interconnect was a torus. That was a defensible choice while the dominant traffic was nearestneighbour: dense training stencils and ring all-reduce touch only adjacent chips, so the Θ(𝑁 1/3 ) diameter of a three-dimensional torus never reaches the critical path. Mixture-of-experts routing [21] broke the assumption, because all-to-all traffic pays the diameter on every token. TPU 8i responded with Boardfly, a Dragonflyderived [13] hierarchy of complete graphs whose pod diameter is 7 irrespective of where two chips sit [5]. Boardfly is a good answer to the question it was asked. We observe that it does not answer the next one. A complete graph on 𝐺 vertices requires degree 𝐺 − 1; a group has 40 optical ports; a 400,000-chip machine has 12,500 groups. The mechanism that gives Boardfly its diameter is precisely the mechanism that prevents it from scaling, and the standard remedy—stack another hierarchy level—costs four chip hops per level. Contributions. • We catalogue every shipped TPU interconnect as a graph and audit each against the Moore bound (§3). At TPU-class radix (Δ ≈ 7) a diameter-7 graph could in principle hold 391,910 chips; Boardfly spends diameter 7 on 1,152. • We give Shiftfly (§4), which replaces only Boardfly’s global tier with a generalized Kautz digraph on the existing optical circuit switch, at an unchanged port budget. We prove its diameter, its table-free routing rule, and a criterion under

∗ Preprint, version 1.0, 2 August 2026. Not peer reviewed. Simulator, proofs and repro-

duction artefacts: https://github.com/EylonKrause/shiftfly (commit tagged v1.0). This work uses no proprietary information; every figure describing a shipped system is drawn from public announcements, and quantities inferred rather than published are marked as inferred. 1

Eylon E. Krause

Generation

Chips

v2 / v3 (2D torus) v4 (3D torus, OCS between cubes) v5e / v6e (2D torus) v5p (3D torus)† v7 Ironwood (3D torus)† v8t (3D torus + Virgo)† v8i Boardfly

256 / 1,024 4,096 256 8,960 9,216 9,600 1,152

Δ 4 6 4 6 6 6 7

2.2

𝐷 16 / 32 24 16 ≈32 ≈32 ≈33 7

Table 1: TPU interconnects as graphs. † Pod shape inferred from published chip counts.

Lemma 2.3 (Diameter decomposition). If the optical link between two groups terminates on one designated chip of one designated building block on each side, the chip-level pod diameter is 7.

which requests for the same content merge inside the fabric (§5). • We identify a port-accounting error that would silently double Shiftfly’s hardware if the comparison were done naively, and we prove the correct accounting (§4.1). • We evaluate both designs at matched optical cost with a dependency-free simulator whose Boardfly model reproduces the published diameter of 7 (§6). Shiftfly loses at pod scale and wins beyond it. • We report a negative result (§6.5): the redundancy argument that motivated the design is dominated by localityaware placement, which is topology-agnostic. We also identify a metric inversion that would have concealed this had we reported the natural efficiency ratio alone. • We measure whether the design can be operated (§7)— field replacement, control-plane state, slice allocation and growth—rather than assuming it. Field replacement ties; slice allocation is a genuine regression that we quantify and bound.

Proof. Reaching a group’s egress chip costs at most three hops: one inside the source building block, one across copper to the building block owning the optical port, one inside that block. The optical hop is one, and the destination side is symmetric, giving 3 + 1 + 3 = 7; the bound is attained. □ We adopt this accounting throughout. Traversing 𝑔 inter-group hops passes through 𝑔 + 1 groups, so chip hops = 𝜄 (𝑔 + 1) + 𝑔,

𝜄 = 3,

(1)

which reproduces 7 at 𝑔 = 1. Our simulated chip-level pod measures diameter exactly 7, which is the check that the port model is faithful (§6.2).

3 Motivation 3.1 The Moore-bound audit Theorem 3.1 (Moore bound [6, 17]). A graph of maximum Í𝐷 −1 degree Δ ≥ 3 and diameter 𝐷 satisfies 𝑁 ≤ 1 + Δ 𝑖=0 (Δ − 1)𝑖 .

All code, proofs and artefacts are public [14]. No proprietary information is used anywhere in this work; every TPU figure is drawn from public announcements, and quantities we infer rather than read are marked as inferred.

Proof. Breadth-first search from any vertex reaches at most Δ vertices at distance 1 and at most Δ(Δ − 1)𝑖 −1 at distance 𝑖, since each newly reached vertex has at most Δ − 1 unused incident edges; every vertex lies within 𝐷. □

2 Background: TPU interconnects as graphs 2.1 The torus era

Equality demands a Moore graph, and these exist only for 𝐷 = 1, for cycles, and for 𝐷 = 2 with Δ ∈ {3, 7, 57} [6]. The bound is a ceiling, so the informative quantity is the ratio to it. At Δ = 7:

Definition 2.1 (𝑘-ary 𝑛-cube). The torus 𝑇 (𝑘 1, . . . , 𝑘𝑛 ), with every 𝑘𝑖 ≥ 2, is the Cayley graph of Z𝑘1 × · · · × Z𝑘𝑛

Boardfly

Google describes Boardfly as “a building block of four fully connected chips into a fully connected group of eight boards, with 36 of such groups fully connected into a TPU 8i pod” [4]. A building block holds 4 chips; a group holds 8 building blocks, i.e. 32 chips; a pod holds 36 groups, i.e. 1,152 chips. Each building block exposes 16 external links of which 11 serve the group in copper, leaving 5 per building block and hence 40 optical ports per group. Within a pod the 36 groups are fully connected through the Apollo optical circuit switch [18], consuming 35 of those 40.

{±𝑒 1, . . . , ±𝑒𝑛 }. Î Proposition 2.2. 𝑇 (𝑘 1, . . . , 𝑘𝑛 ) has 𝑁 = 𝑖 𝑘𝑖 vertices, is 2𝑛Í regular when every 𝑘𝑖 > 2, and diam𝑇 = 𝑖 ⌊𝑘𝑖 /2⌋. A cubic torus therefore has diam𝑇 = Θ(𝑁 1/𝑛 ). with generators

Moore(7, 4) = 1,814,

Moore(7, 6) = 65,318,

Moore(7, 5) = 10,886,

Moore(7, 7) = 391,910.

Observation 3.2. Boardfly attains diameter 7 at 1,152 chips, where Δ = 7 permits diameter 4 at that order—a factor of ≈1.75. More strikingly, Δ = 7 admits diameter 7 at almost exactly 400,000 chips. Boardfly spends, at pod scale, the diameter budget that would in principle serve a machine two orders of magnitude larger.

Proof. In a Cayley graph of an abelian group with this generating set, distance is the sum of per-coordinate cyclic distances; the cyclic distance in Z𝑘 is maximised at ⌊𝑘/2⌋, and the maxima are simultaneously attainable. □

We are careful not to overclaim. Moore-optimal graphs of large diameter are not known, and the bound ignores bisection, cable length and fault tolerance—all of which Boardfly buys with its slack. The observation motivates inspection; it is not by itself an argument.

Table 1 lists the generations. Pod shapes for v5p, v7 and v8t are inferred from published chip counts and marked accordingly; the resulting diameters follow from Proposition 2.2. 2

Shiftfly: Scaling the Accelerator Interconnect Past the Pod with a Shift-Routed Optical Tier

3.2

The structural obstruction

Proof. There are 𝑑 +1 choices for 𝑢 1 and 𝑑 for each later symbol, giving the order; out-degree is the number of 𝛼 ≠ 𝑢𝐷 . For the diameter, applying the arc map 𝐷 times replaces every symbol of 𝑢 by one of 𝑣, so d(𝑢, 𝑣) ≤ 𝐷, and 𝐷 is attained by any pair sharing no admissible overlap. □

The argument is this. Boardfly’s global tier is the complete graph 𝐾36 on groups, so reaching 𝐺 groups costs 𝐺 − 1 optical ports per group. At 400,000 chips, i.e. 𝐺 = 12,500 groups of 32, this is 12,499 ports against 40 available. Full connectivity does not scale, another hierarchy level must be introduced, and by (1) every added level costs four chip hops. The property that gives Boardfly its diameter is the property that bounds its size.

4

Against the directed Moore bound 𝑁 ≤ (𝑑 𝐷+1 − 1)/(𝑑 − 1), 𝐾 (𝑑, 𝐷) is within a factor 1+𝑂 (1/𝑑): it is essentially optimal among digraphs, which is the reason to prefer it to de Bruijn here.

Design

Theorem 5.2 (Shift routing). Let ℓ be the length of the longest suffix of 𝑢 that is a prefix of 𝑣, and 𝑘 = 𝐷 − ℓ. Then

Definition 4.1 (Kautz digraph [12]). For 𝑑 ≥ 2, 𝐷 ≥ 1, the vertices of 𝐾 (𝑑, 𝐷) are words 𝑢 1 · · · 𝑢𝐷 over an alphabet of size 𝑑 + 1 with 𝑢𝑖 ≠ 𝑢𝑖+1 , and there is an arc 𝑢 → 𝑢 2 · · · 𝑢𝐷 𝛼 for each 𝛼 ≠ 𝑢𝐷 .

𝑤 ( 𝑗 ) = 𝑢 𝑗+1 · · · 𝑢𝐷 𝑣 ℓ+1 · · · 𝑣 ℓ+𝑗 ,

is a walk from 𝑢 to 𝑣 of length 𝑘, and 𝑘 = d(𝑢, 𝑣).

Definition 4.2 (Shiftfly). Retain Boardfly’s building block (4 chips, complete) and group (8 building blocks, complete; 32 chips). Replace the complete global tier by a Kautz digraph on the groups, realised as a fixed permutation installed on the optical circuit switch. Where the group count is not of the form (𝑑 + 1)𝑑 𝐷 −1 , use the Imase–Itoh digraph [7, 8] on Z𝑛 with arcs 𝑖 ↦→ −𝑑𝑖 − 𝑟 mod 𝑛, 𝑟 = 1, . . . , 𝑑, which exists at every order.

Proof. 𝑤 (0) = 𝑢, and 𝑤 ( 𝑗+1) is obtained from 𝑤 ( 𝑗 ) by deleting the leading symbol and appending 𝑣 ℓ+𝑗+1 , which is the arc map. At 𝑗 = 𝑘 we have 𝑤 (𝑘 ) = 𝑢𝑘+1 · · · 𝑢𝐷 𝑣 ℓ+1 · · · 𝑣 𝐷 , and 𝑢𝑘+1 · · · 𝑢𝐷 has length ℓ and equals 𝑣 1 · · · 𝑣 ℓ by definition of ℓ; hence 𝑤 (𝑘 ) = 𝑣. Each 𝑤 ( 𝑗 ) is a legal vertex because 𝑢𝐷 = 𝑣 ℓ ≠ 𝑣 ℓ+1 . Minimality follows because reaching 𝑣 in 𝑘 ′ steps requires the length-(𝐷 − 𝑘 ′ ) suffix of 𝑢 to equal the corresponding prefix of 𝑣, and ℓ is the largest such overlap. □

Figure 1 shows what changes. Nothing below the global tier is touched: the copper, the packaging and the 32-chip group are Boardfly’s. This is deliberate—those tiers are cheap, short-reach and well designed, and the obstruction of §3 lives entirely above them.

4.1

Theorem 5.2 is the operational point: the destination label is the route. There is no forwarding table to size, no routing protocol to converge and no state to reconcile after a reconfiguration, at any scale.

Port accounting

The comparison is worthless unless both designs are held to the same optical budget, and the natural reading of “out-degree 𝑑” understates Shiftfly’s cost by a factor of two.

Theorem 5.3 (Merge criterion). Let 𝑢, 𝑢 ′ route to a common 𝑣 under Theorem 5.2 with equal overlap. Their walks occupy the same vertex at step 𝑗 if and only if 𝑢 and 𝑢 ′ agree in their last 𝐷 − 𝑗 symbols. The first merge is therefore at 𝑗 ∗ = 𝐷 − lcs(𝑢, 𝑢 ′ ), where lcs is the longest common suffix.

Proposition 4.3. In 𝐾 (𝑑, 𝐷) with 𝐷 ≥ 2, a vertex is both a successor and a predecessor of another exactly when 𝑢 1 = 𝑢𝐷 , and then in exactly one neighbour. There are 𝑑 (𝑑 + 1) such vertices out of (𝑑 + 1)𝑑 𝐷 −1 . Undirected degree is therefore 2𝑑, or 2𝑑 − 1 on that thin set.

Proof. By Theorem 5.2, 𝑤 ( 𝑗 ) depends on the source only through 𝑢 𝑗+1 · · · 𝑢𝐷 ; the appended symbols are a function of 𝑣 alone. Equality of 𝑤 ( 𝑗 ) and 𝑤 ′( 𝑗 ) is thus equivalent to equality of those suffixes. □

Proof. 𝑣 is a successor and predecessor of 𝑢 iff 𝑢 2 · · · 𝑢𝐷 𝛼 = 𝛽𝑢 1 · · · 𝑢𝐷 −1 for admissible 𝛼, 𝛽. Matching coordinates forces 𝛽 = 𝑢 2 , 𝑢𝑖 = 𝑢𝑖+2 for all valid 𝑖, and 𝛼 = 𝑢 2 ; with the no-repeat constraint this reduces to 𝑢 1 = 𝑢𝐷 and determines 𝛼 uniquely. Counting words with 𝑢 1 = 𝑢𝐷 gives 𝑑 (𝑑 + 1). □

Theorem 5.3 says that redundant requests collapse structurally: once two requests for the same item meet, the second may be served from an in-network cache at the meeting point, with no directory, no coordination and no protocol. It is a statement about the routing algebra, and it is the property that motivated this design.

Corollary 4.4. Out-degree 𝑑 costs 2𝑑 bidirectional ports. Matching Boardfly’s 40 optical ports per group means 𝑑 = 20, not 𝑑 = 40.

Proposition 5.4 (Random labels give almost no merging). Under a uniformly random labelling, Pr[lcs(𝑢, 𝑢 ′ ) ≥ 𝑗] = 𝑑 − 𝑗 (1 + 1 1 𝑜 (1)), whence E[lcs] = 𝑑 −1 (1+𝑜 (1)) and E[ 𝑗 ∗ ] = 𝐷 − 𝑑 −1 (1+𝑜 (1)).

Every result below uses 𝑑 = 20, and our artefact asserts that the two designs’ edge counts agree to within 5% at each scale.

4.2

At 𝑑 = 20 this is E[lcs] ≈ 0.053: essentially no merging before the destination. The merge property is not free. It must be purchased by choosing the labelling so that groups likely to request the same content are suffix-close—an embedding of the observed sharing hypergraph into the suffix tree. §6.5 measures what that is worth, and the answer is sobering.

Routing

Routing is a shift register (Figure 2); the rule and its optimality are Theorem 5.2.

5

𝑗 = 0, . . . , 𝑘,

Analysis

Theorem 5.1 (Order, degree, diameter [12]). 𝐾 (𝑑, 𝐷) has 𝑁 = (𝑑 + 1)𝑑 𝐷 −1 vertices, in- and out-degree 𝑑, and diameter exactly 𝐷.

Theorem 5.5 (Arbitrary order [7, 20]). The Imase–Itoh digraph on Z𝑛 has out-degree 𝑑 and diameter at most ⌈log𝑑 𝑛⌉ for every 𝑛 ≥ 2. 3

Eylon E. Krause

Figure 1: Shiftfly changes exactly one tier. The building block and the group are Boardfly’s, unchanged; only the global tier is replaced, and it rides the optical circuit switch that already exists. Proof sketch. Iterating the arc map 𝑘 times sends 𝑖 to (−𝑑)𝑘 𝑖 − 𝑘 − 𝑗 . As the 𝑟 range over {1, . . . , 𝑑 } the reachable set after 𝑗 𝑗 𝑟 𝑗 (−𝑑) 𝑘 steps has size min(𝑑 𝑘 , 𝑛), and covers Z𝑛 once 𝑑 𝑘 ≥ 𝑛. □

Chips

Boardfly

Shiftfly

1,152 4,608 18,432 73,728 268,800 400,000 1,152,000

7 15 15 19 23 23 23

11 11 15 15 15 19 19

Í

This removes a quantisation that would otherwise strand capacity: string Kautz graphs exist only at orders (𝑑 + 1)𝑑 𝐷 −1 , and a machine is whatever size it is. Proposition 5.6 (Connectivity [8]). 𝐾 (𝑑, 𝐷) has vertex connectivity 𝑑, the maximum possible at that degree.

Table 2: Chip-level diameter at 40 optical ports per group.

Thin global tiers are the usual fragility of hierarchical fabrics; maximal connectivity is the relevant defence.

random graph over pods, which at 347 pods yields a low-diameter pod-level graph. We are not handicapping the incumbent.

6 Evaluation 6.1 Methodology

6.2

Model validation

Our chip-level Boardfly pod has 1,152 vertices, maximum degree 7, mean degree 5.84, and measured diameter exactly 7, matching the published figure and Lemma 2.3. Every conclusion below rests on this check.

Both designs receive 40 optical ports per group and identical 32chip group internals, so all comparisons are at matched optical cost (Proposition 4.3). Distances are computed exactly by breadth-first search where the instance permits and by sampling from 120–256 random sources otherwise; sampled diameters are lower bounds and are labelled as sampled. Spectral gaps are power-iteration estimates of 1−𝜆2 of the normalised adjacency operator, reported as estimates. The simulator is stdlib-only and regenerates every figure and table in this paper from one script [14]. Boardfly beyond one pod is modelled generously: the 5 optical ports per group left after intra-pod full connectivity are wired as a

6.3

Scaling

Table 2 and Figure 3 give the result. Boardfly is strictly better at the scale it was designed for: 7 against 11 chip hops at 1,152 chips, because a complete global tier crosses in one hop and a shift tier needs three. The curves cross near 4,600 chips. At the 400,000chip target the reduction is 23 → 19, or 17%; the widest gap in the 4

Shiftfly: Scaling the Accelerator Interconnect Past the Pod with a Shift-Routed Optical Tier

Figure 2: Shift routing on 𝐾 (2, 2), with the two-hop route 01 → 12 → 20. The destination label is the route: no table is consulted and no protocol converges.

Figure 3: Chip-level worst-case distance at equal optical cost. Boardfly is better at pod scale; the crossing is the result.

Figure 4: Spectral gap of the group graph. A hierarchy of near-cliques joined by few links expands poorly whatever its diameter.

sweep is 23 → 15 at 268,800 chips, where Boardfly has just paid for another hierarchy level and Shiftfly has not yet needed another symbol. Both curves are step functions, so the advantage is not monotone.

relative to expanders [2]. Boardfly beyond one pod is not an expander: it is a set of near-cliques joined by a thin random tier, which is close to the worst case for spectral expansion. The comparison therefore favours Shiftfly here, and would not against a flat high-radix fabric.

6.4

Expansion 6.5

Shiftfly’s spectral gap is roughly 2.7× Boardfly’s across the tested range (Figure 4). In hindsight this is unsurprising and it is worth stating why, because it cuts against the usual intuition about de Bruijnfamily graphs. Those families are known to have poor bisection

Sharing: a negative result

Agentic inference fans one query into many concurrent rollouts that share a prompt prefix, tool definitions and retrieved context, and therefore request the same content [19, 23, 24]. Theorem 5.3 5

Eylon E. Krause

Design

State for the global wiring

Boardfly Shiftfly

550,000 bits 28 bits

Table 4: State the control plane must hold to derive which groups are wired to which, at 12,500 groups.

Figure 5: Cost of serving one shared item to 32 agents, in tree hops per requester. Placement does most of the work, on both topologies.

𝑇 per requester Placement random locality-aware saving

Shiftfly

Boardfly

Shiftfly

2.768 2.174

2.470 2.112

2.868 2.336

2.772 2.152

−21.5%

−14.5%

Chips

Boardfly

SF naive

SF instantiated

16 36 128 512 2,048

512 1,152 4,096 16,384 65,536

1 1 3 4 4

disconn. disconn. disconn. disconn. disconn.

1 2 2 3 3

Table 5: Group-level diameter of an allocated slice. An arbitrary induced subset of a shift graph is disconnected at every size tested.

merge depth

Boardfly

Slice

and growing it must be at least as convenient as Boardfly. This section measures four such questions on a 12,500-group (400,000-chip) configuration. Three of the four favour Shiftfly or tie; one does not, and we give it its own subsection.

Table 3: Shared-content cost. Lower is better. Placement dominates.

7.1

Replacing a failed group

In both designs a replacement unit inherits the identity of the unit it replaces, and the optical circuit switch re-points that identity’s fibres at the new hardware. The cost is therefore just the degree, and the same 40-port budget binds both: 40 circuits re-established, 40 peers disturbed, in each case. Neither design is exactly regular— Shiftfly loses a port on the thin set of vertices carrying a bidirectional pair (Proposition 4.3), Boardfly wherever an inter-pod stub fails to pair—but the mean costs agree to within a few percent. Field replacement is therefore a tie, by construction rather than by luck. Nothing about a shift-routed tier makes a chip, tray or group harder to pull. What differs is that Shiftfly need not look anything up: the neighbour set of a label is a shift of that label, so the technician’s work order and the control plane’s reconciliation are the same in both designs while the state behind them is not.

says a shift-routed fabric collapses such requests structurally. We set out to quantify that advantage. It is small. Definition 6.1. For home vertex ℎ and requester set 𝑆, the unicast Í cost is 𝑈 = 𝑣 ∈𝑆 d(𝑣, ℎ) and the tree cost 𝑇 is the number of edges in the union of shortest paths from 𝑆 to ℎ—what a fabric with in-network replication pays. The efficiency is 𝜂 = 𝑈 /𝑇 . Table 3 and Figure 5 report a workload of 32 agents per query over 64 affinity clusters with 85% scheduler locality and Zipf-distributed item popularity, on 8,400 groups. Locality-aware placement removes 21.5% of tree cost on Boardfly and 14.5% on Shiftfly. Shiftfly’s residual advantage at matched placement is 2.9%, with merge depth 7.9% lower. The great majority of the achievable saving on shared content therefore comes from placement, which is topologyagnostic and equally available to Boardfly.

7.2

Control-plane state

Remark 6.2 (A metric inversion). 𝜂 must not be optimised alone. It is a ratio, so a topology is rewarded for placing requesters far from the home vertex provided their paths overlap; a network in which everything is one hop away scores the worst possible 𝜂 = 1 while paying the least. In our measurements 𝜂 ranks Boardfly higher (1.431 against 1.364) while absolute tree cost ranks it lower (2.174 against 2.112). Had we reported 𝜂 alone—the natural reading of a sharing objective—we would have drawn the opposite conclusion.

Boardfly’s intra-pod tier is derivable—a complete graph needs only pod membership—but its inter-pod tier has no closed form and must be tabulated. Shiftfly’s entire global tier follows from the pair (𝑛, 𝑑), because Theorem 5.2 makes adjacency arithmetic. Table 4 gives the resulting difference of four orders of magnitude. We do not oversell this. Boardfly could adopt a structured interpod tier and recover most of the gap—but a structured global tier over an OCS is essentially the proposal of this paper.

7

7.3

Deployment and operations

A topology that is better on paper and worse to run is not better. Shiftfly is proposed as a drop-in replacement for one tier of a machine that already exists, so the bar is concrete: servicing, allocating

Slice allocation: where Shiftfly is worse

Google allocates slices of a pod to jobs, and slices must be isolated and internally well connected. Here Boardfly has a structural advantage that Shiftfly cannot match directly: any subset of a complete 6

Shiftfly: Scaling the Accelerator Interconnect Past the Pod with a Shift-Routed Optical Tier

and we do not generalise it: the global tier is already optically circuitswitched, so the wiring is a permutation the switch installs rather than cable anyone routes. This argument is available to an operator that owns an OCS layer [18] and to essentially nobody else.

tier is itself complete, so a slice of up to 36 groups is carved out at zero cost and with diameter 1, requiring no reconfiguration at all. The corresponding naive operation on Shiftfly—take an arbitrary subset and use whichever links fall inside it—fails badly. Table 5 shows the induced subgraph is disconnected at every size we tested. Slices cannot be carved out of a shift graph. They can, however, be instantiated. Because the global tier is a permutation installed on an optical circuit switch, and because the Imase–Itoh construction exists at every order, a slice of 𝑚 groups can be given its own correctly sized shift permutation. Every slice is then a fabric in its own right rather than a fragment of a larger one, with the diameter guarantee ⌈log𝑑 𝑚⌉ intact. On those terms Shiftfly slices are better than Boardfly’s from 128 groups (4,096 chips) upward, where Boardfly’s own slice has had to become hierarchical. The cost is one OCS reconfiguration per allocation, which Boardfly avoids within a single pod. We regard this as acceptable, but only because it matches how the machine is already operated: MEMS reconfiguration takes milliseconds to seconds and happens at jobscheduling time [10, 18], against job lifetimes of minutes to days, and any slice spanning more than one pod obliges Boardfly to establish inter-pod circuits anyway. A deployment that allocates many small, short-lived slices inside one pod should keep Boardfly.

7.4

Deterministic routing. Theorem 5.2 yields a single path. Load balance will require non-minimal alternatives [9, 22], and randomising the route partly destroys the merge property of Theorem 5.3. We have not quantified that interaction, and it is the most important piece of missing work. The switch is mechanical. MEMS-based optical circuit switches reconfigure on millisecond-to-second timescales [10, 18], so the Kautz permutation is fixed at scheduling time. Label assignment is a scheduling decision, not a runtime one. The workload model is synthetic. Cluster count, agent fan-out, scheduler locality and Zipf exponent are plausible rather than measured; the conclusions of §6.5 are only as good as those four numbers.

9

Growth and software

Boardfly is defined at multiples of a pod; crossing 36 groups introduces a hierarchy level rather than widening an existing one. Shiftfly is defined at every order, so a partially populated machine of 𝑛 groups is a valid fabric with the diameter its size warrants, and growth re-installs a permutation rather than restructuring a tier. The software change is bounded by construction, because the building block and the group are untouched. Every intra-group collective—including the latency-critical small reductions the collectives engine accelerates—sees an unchanged fabric. What changes is the inter-group collective schedule, which is topology-specific in any case, and the routing layer, which gets simpler: there is no forwarding table to size, no protocol to converge, and no state to reconcile after a reconfiguration.

8

Related work

Dragonfly [13] established the high-radix hierarchical form that Boardfly follows, and HyperX [1] the flattened alternative. The degree–diameter literature supplies our ceiling [6, 17], and a line of recent topologies pursues it directly: Slim Fly from McKay–Miller– Širáň graphs [3], PolarFly from Erdős–Rényi polarity graphs [15], and Bundlefly [16]. These operate at router radix, where a switch has tens of ports; Shiftfly operates at accelerator radix, where the endpoint is the router and the port count is single-digit per chip. Kautz digraphs are due to Kautz [12]; arbitrary-order generalizations to Reddy, Pradhan and Kuhl [20] and Imase and Itoh [7, 8]. The same shift algebra underlies the Koorde distributed hash table [11], which is where we take the content-addressing observation from; our contribution is to place it in the physical topology rather than in an overlay. Google’s optical circuit switching infrastructure is described in Jupiter Evolving [18] and its use for TPU reconfiguration in TPU v4 [10]. Redundancy in LLM serving is exploited above the network by prefix caching [24] and disaggregated KV stores [19]; we ask whether it can be exploited in the fabric, and largely conclude that it is better handled by placement.

Discussion and limitations

Boardfly is right at pod scale. At 1,152 chips a complete global tier is simply the correct answer, and our own measurements say so. Nothing here argues that TPU 8i should have been built differently.

10

Slices must be instantiated, not carved. An arbitrary induced subset of a shift graph is disconnected (§7.3). This is the one operational regression, it is real, and a fleet dominated by small short-lived inpod slices should not adopt Shiftfly.

Conclusion

Google’s move from torus to Boardfly traded a Θ(𝑁 1/3 ) diameter for a constant one, and was clearly correct for a 1,152-chip inference pod. The mechanism it used—full connectivity at every tier—does not extend, because complete graphs need 𝑁 − 1 ports. Replacing only the global tier with a generalized Kautz digraph keeps the diameter logarithmic in the group count at a fixed port budget, yields table-free routing and native content addressing, and measurably improves expansion, at the cost of being worse inside a single pod. The redundancy argument that motivated the design did not survive its own evaluation: most of the available saving on shared content comes from placement, not from topology, and the natural metric for that objective inverts the ranking. We regard reporting this as more useful than the alternative.

Bisection. De Bruijn and Kautz families have asymptotically smaller bisection than a complete or expander tier [2]. We win against a hierarchy because hierarchies expand badly; against a flat high-radix fabric we would not. All-to-all-dominated training is the adversarial workload, and it is exactly the workload TPU 8t retains a torus and a flat scale-out fabric for. Cabling. Shift edges are long and irregular, which is the standard reason de Bruijn networks are not built [2]. Our mitigation is narrow 7

Eylon E. Krause

References

[14] Eylon E. Krause. 2026. Shiftfly: Simulator, Proofs and Reproduction Artefacts. https://github.com/EylonKrause/shiftfly. [15] Kartik Lakhotia, Maciej Besta, Laura Monroe, Kelly Isham, Patrick Iff, Torsten Hoefler, and Fabrizio Petrini. 2022. PolarFly: A Cost-Effective and Flexible Low-Diameter Topology. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC). [16] Fei Lei, Dezun Dong, Xiangke Liao, and Jesús Labarta. 2020. Bundlefly: A LowDiameter Topology for Multicore Fiber. In Proceedings of the 34th ACM International Conference on Supercomputing (ICS). [17] Mirka Miller and Jozef Širáň. 2013. Moore Graphs and Beyond: A Survey of the Degree/Diameter Problem. The Electronic Journal of Combinatorics, Dynamic Survey DS14 (2013). [18] Leon Poutievski, Omid Mashayekhi, Joon Ong, Arjun Singh, Mukarram Tariq, Rui Wang, Jianan Zhang, Virginia Beauregard, Patrick Conner, Steve Gribble, Rishi Kapoor, Stephen Kratzer, Nanfang Li, Hong Liu, Karthik Nagaraj, Jason Ornstein, Samir Sawhney, Ryohei Urata, Lorenzo Vicisano, Kevin Yasumura, Shidong Zhang, Junlan Zhou, and Amin Vahdat. 2022. Jupiter Evolving: Transforming Google’s Datacenter Network via Optical Circuit Switches and Software-Defined Networking. In Proceedings of the ACM SIGCOMM Conference. [19] Ruoyu Qin, Zheming Li, Weiran He, Mingxing Zhang, Yongwei Wu, Weimin Zheng, and Xinran Xu. 2025. Mooncake: A KVCache-Centric Disaggregated Architecture for LLM Serving. In USENIX Conference on File and Storage Technologies (FAST). [20] S. M. Reddy, D. K. Pradhan, and J. G. Kuhl. 1980. Directed Graphs with Minimal Diameter and Maximum Node Connectivity. Technical Report. School of Engineering, Oakland University. [21] Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. 2017. Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer. In International Conference on Learning Representations (ICLR). [22] Leslie G. Valiant. 1982. A Scheme for Fast Parallel Communication. SIAM J. Comput. 11, 2 (1982). [23] Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-Consistency Improves Chain of Thought Reasoning in Language Models. In International Conference on Learning Representations (ICLR). [24] Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark Barrett, and Ying Sheng. 2024. SGLang: Efficient Execution of Structured Language Model Programs. In Advances in Neural Information Processing Systems (NeurIPS).

[1] Jung Ho Ahn, Nathan Binkert, Al Davis, Moray McLaren, and Robert S. Schreiber. 2009. HyperX: Topology, Routing, and Packaging of Efficient Large-Scale Networks. In Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis (SC). [2] Jean-Claude Bermond and Claudine Peyrat. 1989. de Bruijn and Kautz Networks: A Competitor for the Hypercube? In Hypercube and Distributed Computers. North-Holland. [3] Maciej Besta and Torsten Hoefler. 2014. Slim Fly: A Cost Effective Low-Diameter Network Topology. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC). [4] Google. 2026. Our Eighth Generation TPUs: Two Chips for the Agentic Era. The Keyword. https://blog.google/innovation-and-ai/infrastructure-and-cloud/ google-cloud/eighth-generation-tpu-agentic-era/. [5] Google Cloud. 2026. TPU 8t and TPU 8i Technical Deep Dive. Google Cloud Blog. https://cloud.google.com/blog/products/compute/tpu-8t-and-tpu8i-technical-deep-dive. [6] Alan J. Hoffman and Robert R. Singleton. 1960. On Moore Graphs with Diameters 2 and 3. IBM Journal of Research and Development 4, 5 (1960). [7] Makoto Imase and Masaki Itoh. 1981. Design to Minimize Diameter on BuildingBlock Network. IEEE Trans. Comput. C-30, 6 (1981). [8] Makoto Imase and Masaki Itoh. 1983. A Design for Directed Graph with Minimum Diameter. IEEE Trans. Comput. C-32 (1983), 782–784. [9] Nan Jiang, John Kim, and William J. Dally. 2009. Indirect Adaptive Routing on Large Scale Interconnection Networks. In Proceedings of the 36th Annual International Symposium on Computer Architecture (ISCA). [10] Norman P. Jouppi, George Kurian, Sheng Li, Peter Ma, Rahul Nagarajan, Lifeng Nai, Nishant Patil, Suvinay Subramanian, Andy Swing, Brian Towles, Cliff Young, Xiang Zhou, Zongwei Zhou, and David Patterson. 2023. TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings. In Proceedings of the 50th Annual International Symposium on Computer Architecture (ISCA). [11] M. Frans Kaashoek and David R. Karger. 2003. Koorde: A Simple Degree-Optimal Distributed Hash Table. In Proceedings of the 2nd International Workshop on Peer-to-Peer Systems (IPTPS). [12] William H. Kautz. 1968. Bounds on Directed (𝑑, 𝑘 ) Graphs. Technical Report. Stanford Research Institute. Theory of Cellular Logic Networks and Machines, AFCRL Final Report. [13] John Kim, William J. Dally, Steve Scott, and Dennis Abts. 2008. TechnologyDriven, Highly-Scalable Dragonfly Topology. In Proceedings of the 35th Annual International Symposium on Computer Architecture (ISCA).

8

Related documents

• Optical through-silicon waveguides for 3D chip-to-chip interconnections / Francesco Villasmunta ; Gutachter: Harald Schenk, Patrick Heise, Andreas Mai#623828
• Cavity-Optomechanics with Silica Microtoroids : Quantum-Coherent Coupling and Optomechanically Induced Transparency#320099
• QPS-ToR: A Parallel Iterative Switching Algorithm for Reconfigurable Optical Datacenter Switching#673463
• Nothing Breaks: No Single Peer Can Soundly Gate Post-Quantum Delivery#667917
• The 'Light-Well': a tuneable free-electron light source on-a-chip#260690