X RONOS: Heterogeneity-Aware Tensor Parallelism for Collaborative LLM Fine-Tuning on Edge CPUs Wonmi Choi1,∗ , Sunjae Park1,∗ , Dohyeok Kwon1 , Zhixiong Niu2 , Yeonho Yoo3 , Chuck Yoo1 , Gyeongsik Yang1
arXiv:2609.19909v1 [cs.DC] 17 Sep 2026
1
Dept. of Computer Science and Engineering, Korea University
Abstract—Collaborative fine-tuning on edge devices adapts large language models to domain-specific data while keeping each device’s data local. State-of-the-art (SOTA) collaborative finetuning techniques are largely designed for GPU-based edge devices and rely on pipeline parallelism (PP). However, many edge platforms, including IoT gateways, smart-home hubs, and invehicle computers, are primarily CPU-based. This paper reports that PP is ineffective on CPU-based edge devices because the same CPU handles both model computation and communication, which causes severe CPU contention. Our analysis shows that this leads to 5.75× higher computation stall ratios than on GPU devices on average. Tensor parallelism (TP) can alleviate this contention by separating computation and communication, but existing TP techniques assume homogeneous devices. On heterogeneous CPU edge devices, we find that this assumption causes faster workers to remain idle for up to 34% while waiting for slower devices at synchronization points. To address the limitations, we propose X RONOS, a collaborative fine-tuning framework for heterogeneous CPU edge devices. X RONOS uses TP as its execution backbone and combines lightweight profiling with heterogeneity-aware tensor partitioning to reduce the straggler bottleneck. Across diverse devices, models, and benchmark tasks, X RONOS reduces fine-tuning time by 18% (TP) to 56% (PP) and the ratio of device idle time by ∼5.9× over SOTA techniques, while maintaining the accuracy. Index Terms—Collaborative fine-tuning, Distributed training modeling, Heterogeneous edge devices, Edge computing
I. I NTRODUCTION Collaborative fine-tuning enables multiple edge devices to jointly update a large language model (LLM) without centralizing the raw data. This capability is becoming increasingly important for edge AI services such as autonomous driving, mobile platforms, and IoT environments [1], [2]. In such settings, pretrained LLMs often require domain-specific adaptation and continual updates from newly generated user data. Because this data frequently contains sensitive personal information, sending the raw data to a centralized cloud for fine-tuning is undesirable [1], [3]. Collaborative fine-tuning addresses this challenge by allowing training to proceed directly on edge devices while keeping the raw data local [4]. Existing studies on collaborative fine-tuning have mainly focused on distributing fine-tuning across edge devices [5]–[9]. Most target GPU-based edge platforms and rely on pipeline parallelism (PP), which improves device utilization and reduces iteration time by overlapping computation and communication. However, collaborative fine-tuning is also needed on CPU-based edge devices. In real deployments, many edge platforms, including Siemens’ IoT gateways [10], smart-home hubs [11], and in-vehicle computers [12], are primarily CPUbased. Even when GPUs or NPUs are available, they are ∗ Equal contribution.
2
Microsoft Research Asia
3
Dongguk University
often reserved for latency-critical inference rather than finetuning [13], [14]. As a result, many edge environments require collaborative fine-tuning on CPUs. In this paper, we study collaborative fine-tuning on heterogeneous CPU-based edge devices, a setting that remains largely unexplored. We begin with a system-level analysis of PP-based techniques on CPU devices (§III). Our analysis shows that the key advantage of PP, computation–communication overlap, does not hold in this setting. On CPU-based edge devices, the same processor handles both model computation and communication, causing resource contention. This contention prevents effective overlap and leads to an average 5.75× higher computation stall ratio than on GPU devices. These results indicate that directly applying GPUoriented PP designs to CPU edge environments is inefficient. We therefore explore an alternative strategy that reduces CPU contention. Tensor parallelism (TP) is a promising candidate because it does not rely on computation–communication overlap. Instead, TP separates computation and synchronization into sequential phases and improves training speed through tensor partitioning and collective communication. This structure makes TP a better fit for CPU-based edge devices, where reducing CPU contention is critical. However, we find that directly applying existing TP techniques to heterogeneous CPU edge devices is still ineffective. Our experiment with Megatron-LM [15] shows that the fastest device is idle for up to 34% of an iteration. In other words, more than one-third of the fast workers are waiting for synchronization rather than performing fine-tuning. This is not merely a local inefficiency: as TP proceeds synchronously, such waiting time directly translates into wasted compute capacity and longer iteration time for the whole system. This inefficiency arises as most TP techniques are designed for homogeneous datacenter settings with GPUs of similar compute and memory capabilities, whereas edge devices exhibit significant computational heterogeneity. To address these challenges, we propose X RONOS, a collaborative fine-tuning system for LLMs on heterogeneous CPUbased edge devices. X RONOS adopts TP as its backbone to avoid CPU contention. It also introduces heterogeneityaware planning to minimize idle time caused by stragglers. Specifically, it profiles a small set of representative, nonrepeated LLM layers and uses the results to predict perdevice computation cost and memory usage. Based on these predictions, it searches for a partitioning strategy that reduces the idle time and thereby increases the training speed. The key contributions of this study are: • Present a system-level analysis of collaborative LLM finetuning on CPU-based edge devices and show why PP is
© 2026 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. This paper has been accepted for publication in IEEE MASCOTS 2026.
Output
Linear
Dropout Dropout All-reduce
Worker 2
Layer Norm Layer Norm
Replicated tensor
Transformer blocks x B Drop Layer out Norm Drop out
Layer Norm
Drop out
Layer Norm
Communication All-reduce
Worker 1
Layer Norm
All-reduce
B. Parallelization Strategy We focus on two representative strategies for collaborative fine-tuning on edge workers: PP and TP. Note that although other parallelization strategies are in principle possible, PP and TP capture the main design trade-offs relevant to our edge-device settings. To the best of our knowledge, prior edge collaborative fine-tuning systems have primarily built on PP, making it the most relevant baseline and TP the natural alternative to examine. 1) PP: PP divides the model into a sequence of stages, where each stage contains consecutive layers and is assigned to a single worker. Fig. 1a shows a three-worker example. The model is partitioned from the embedding layer to the output layer into stage 0, stage 1, and stage 2, which are mapped to worker 0, worker 1, and worker 2, respectively.
Worker 0
All-reduce All-reduce
II. BACKGROUND A. Edge Collaborative Fine-tuning Edge collaborative fine-tuning updates a shared LLM across multiple trusted edge devices connected through a local network. One device acts as the coordinator, orchestrating training, profiling the participating devices when needed, and selecting a parallelization strategy for fine-tuning. Other devices act as workers. Throughout the paper, we use worker to denote a device that stores part or all of the model and participates in collaborative fine-tuning. Fine-tuning proceeds over epochs and iterations. An epoch is one full pass over the fine-tuning dataset. Each epoch consists of multiple iterations, and each iteration processes one global batch. The global batch size is the total number of training examples processed by all workers in an iteration. Depending on the parallelization strategy, the global batch may be further subdivided; e.g., PP splits it into micro-batches. In each iteration, each worker performs forward and backward computation on its assigned portion of the model, such as a pipeline stage in PP or a set of partitioned tensors in TP, and exchanges intermediate tensors (e.g., activations and gradients) with other workers over the network. The per-worker iteration time thus consists of local computation time and communication time. Execution is synchronous: the coordinator starts the iteration, and the system advances to the next iteration after all workers finish the current one. So, the end-to-end iteration time is determined by the slowest worker.
GeLU
(b) Execution workflow.
Fig. 1: PP example with three workers. ineffective in this setting. • Design X RONOS, a TP-based collaborative finetuning system that combines lightweight profiling with heterogeneity-aware tensor partitioning. • Demonstrate that X RONOS improves iteration time by up to 56% and reduces idle time ratio by up to 5.9×, across diverse device combinations and workloads.
Linear
(a) Model partitioning. Partitioned tensor
(a) Model partitioning.
LayerNorm
Linear
Dropout Dropout All-reduce
Self attention
Stage 2 Output layer
Forward for micro-batch i Communication for i Backward for micro-batch i intermediate results Worker 0 … 0 1 2 3 0 (Stage 0) Worker 1 … 0 3 1 0 1 2 (Stage 1) Worker 2 Overlap … 0 0 1 1 2 (Stage 2)
LayerNorm
Transformer layer 0 Stage 1 Transformer layer 1~3
Transformer blocks layers xx N Transformer B
i
Embedding
Stage 0 Embedding layer
Drop out
Output
Drop out
Output
Drop out
Output
(b) Execution workflow.
Fig. 2: TP example with three workers. During each iteration, PP splits the global batch into multiple micro-batches and pipelines them across the stages (Fig. 1b). Each worker executes forward and backward passes only for its own stage. During the forward pass, a worker sends activation tensors to the next stage; during the backward pass, it sends gradient tensors to the previous stage. PP attempts to overlap this communication with computation across different micro-batches and stages. For example, while worker 1 in Fig. 1b computes the forward pass of one micro-batch, it can simultaneously receive activations for another micro-batch from worker 0. By hiding part of the communication latency behind computation, PP aims to reduce the overall iteration time. Existing collaborative fine-tuning systems predominantly utilize PP [5]–[7], [16]. 2) TP: Unlike PP, TP does not assign disjoint groups of layers to workers. Instead, TP partitions tensors within selected layers across workers; we refer to these as partitioned tensors. Throughout the rest of the paper, we call a layer partitioned to be distributed across workers via tensor partitioning, and replicated when the layer is fully maintained on every worker. In TP, compute- and memory-intensive layers, such as embedding, self-attention, and linear layers, are partitioned, as their large tensors benefit from distributed storage and parallel computation. In contrast, layers and operations with small parameter sizes or limited parallelization benefits, such as layer normalization, dropout, and the final output layer, are typically replicated to avoid unnecessary synchronization overhead. Fig. 2a shows a TP example with three workers. Colored blocks denote partitioned tensors distributed across workers, while gray blocks denote replicated layers. For each partitioned layer, every worker computes its assigned tensor partition in parallel. The workers then communicate (yellow boxes) to synchronize and aggregate partial results through collective communication (e.g., all-reduce) to produce the activations or gradients required for subsequent computation (Fig. 2b). Replicated layers are executed independently on each worker between synchronization points. In contrast to PP that overlaps communication and computation across microbatches, TP separates the two. III. S YSTEM - LEVEL A NALYSIS OF E DGE C OLLABORATIVE F INE - TUNING ON CPU D EVICES This section addresses two questions. First, does PP retain its main benefit on CPU-based edge workers? Second, if PP is ineffective, can conventional TP serve as a direct replacement? To answer the questions, we first analyze PP on GPU and
2 1
Expected time
0 GPT-2- RoBERTa Medium -Base
Fig. 3: Normalized CPU iteration time of PP (§III-B1).
GPT-2- RoBERTa Medium -Base GPU worker 9.03% CPU worker 65.52% CPU/GPU 7.26×
12.63% 53.57% 4.24×
Metric
Model
PP−TP PP
Context GPT-2-Medium 95.02% switches RoBERTa-Base 97.70% Comp. GPT-2-Medium 19.98% stall RoBERTa-Base 14.97%
CPU workers and then examine TP on CPU workers with and without hardware heterogeneity. The results motivate the design of X RONOS. A. Experiment Setup 1) Workload: We fine-tune GPT-2-Medium [17] and RoBERTa-Base [18] in FP32 with a sequence length of 32 and a global batch size of 8. For PP, each global batch is split into 8 micro-batches to expose the intended pipeline overlap. We use Gloo [19] for inter-worker communication and finetune the models on the CoLA task from the GLUE benchmark [20]. We use a single downstream task in this section, as the goal is to isolate system behavior rather than compare task accuracy. Unless otherwise stated, all results are averaged over the iterations within a single epoch. 2) Comparison: We compare representative PP and TP techniques. For PP, we use Asteroid [5], a state-of-the-art (SOTA) heterogeneity-aware pipeline parallelism technique. For TP, we use Megatron-LM [15], a widely used tensor parallelism technique for homogeneous GPU workers. 3) Devices: For the PP analysis in §III-B, we compare two scenarios: (1) three GPU workers of Jetson Orin Nano devices and (2) three CPU workers of Raspberry Pi 5 devices. By comparing GPU and CPU workers, we identify the problem of existing techniques on CPU workers. For the TP analysis in §III-C, we compare two scenarios: (1) three homogeneous CPU workers (Raspberry Pi 5) and (2) three heterogeneous CPU workers (two Raspberry Pi 5 of four-core ARM Cortex-A76, 2.4 GHz and one ASUS MiniPC of four-core Intel N100, 3.4 GHz). We identify the problem of existing TP techniques for heterogeneous CPU workers. B. Analysis 1: PP Loses Its Main Advantage on CPU Workers 1) Iteration time: We compare the average iteration time of PP on GPU and CPU workers. Iteration time is defined as the end-to-end latency of a single iteration, including both computation and communication, and is determined by the slowest worker under synchronous execution. We first measure the execution time of a single transformer block from GPT-2-Medium and RoBERTa-Base on CPU and GPU workers without collaborative fine-tuning to quantify the compute speed difference between CPU and GPU. We observe that the transformer blocks take 3.35× and 2.51× longer on CPU workers than on GPU workers. Next, we measure the iteration time during collaborative fine-tuning. To isolate the overhead beyond the inherent CPU–GPU speed difference, we normalize the CPU iteration time by (1) the single-block speed ratios (3.35× and 2.51×) and (2) the GPU iteration time. A normalized value of 1 indicates that the CPU worker has the same iteration time as the GPU worker, after accounting for the difference in compute speed between the GPU and the CPU.
40 30 20 10 0
0 ASUS Rasp. Rasp. MiniPC Pi 5-1 Pi 5-2
(a) GPT-2-Medium
Idle time fraction (%)
TABLE I: Computation TABLE II: CPU overheads stall ratio of PP (§III-B2). between PP and TP (§III-C1).
Idle time fraction (%)
Normalized CPU iteration time
3
40 30 20 10 0
0 ASUS Rasp. Rasp. MiniPC Pi 5-1 Pi 5-2
(b) RoBERTa-Base
Fig. 4: Idle time ratio of TP (§III-C2). Fig. 3 shows that the normalized CPU iteration time reaches 2.33 for GPT-2-Medium and 1.70 for RoBERTa-Base. This result indicates that PP introduces additional slowdown beyond the compute speed difference between CPU and GPU workers. We next investigate the cause of this slowdown. 2) Computation–communication overlap: We next analyze why PP slows down disproportionately on CPU workers. We measure the computation stall ratio, defined as (backend stall cycles/CPU cycles) × 100, measured through the Linux perf_event_open interface [21]. A higher value indicates that the worker spends a larger fraction of execution time stalled rather than making forward progress. Table I reports the computation stall ratio for PP on (1) GPU workers, (2) CPU workers, and (3) their comparison (CPU divided by GPU). CPU workers show, on average, 5.75× higher stall ratios than GPU workers—7.26× for GPT2-Medium and 4.24× for RoBERTa-Base. So, PP in CPU workers poorly overlaps communication with computation. The reason is architectural. On GPU workers, model computation runs on the GPU while communication is handled by the host CPU, so the two can proceed largely in parallel. On CPU workers, however, the same processor should execute both computation and communication. As a result, the overlap of PP turns into contention for the same CPU cores and runtime resources, greatly reducing the benefit of pipelining. C. Analysis 2: Existing TP Remains Inefficient on Heterogeneous CPU Workers We consider TP as an alternative, as it separates computation from communication (§II-B) and is inherently better suited for CPU-based workers. We thus ask two questions: (1) does TP reduce CPU contention, and (2) if so, do existing TP techniques remain effective for CPU workers? 1) CPU contention: We compare TP and PP on three homogeneous CPU workers (Raspberry Pi 5). We measure two metrics: (1) the number of context switches and (2) the computation stall ratio, both obtained using perf_event_open. The number of context switches reflects OS scheduling overhead; a higher value indicates greater interference between computation and communication tasks. Table II reports the relative reduction of TP compared to PP, computed as (PP−TP)/PP. TP reduces context switches by 95.02% for GPT-2-Medium and 97.70% for RoBERTaBase. It also reduces the computation stall ratio by 19.98% and 14.97%, respectively. The reductions explain that, to mitigate CPU contentions, TP is a better backbone for collaborative fine-tuning on CPU workers than PP. 2) Idle time ratio: However, lower CPU contention alone does not make existing TP techniques sufficient for CPU workers. We evaluate Megatron-LM, a representative TP technique, on three heterogeneous CPU workers (two Raspberry Pi 5 and one ASUS MiniPC). We measure the idle time ratio, defined
Phase 1: Worker profiling
Phase 2: Heterogeneity-aware planning
LLM to fine-tune Embedding layer (emb) B transformer blocks (blk)
Profiling set
Output layer (out)
out
emb blk
Synthetic inputs layer ℓ
Profiling
Phase 3: TP-based fine-tuning Xronos coordinator
Xronos coordinator
Model partition units
Partitioned model under p*
Step 1 Step 2
λ Initialize fine-tuning
Step 3
Worker d
.. . Step h Optimal partitioning ratio p* Step 8
Xronos profiler
Hyperparameters
Collaborative fine-tuning
Local dataset
d d d Mw, ℓ Ma,ℓ Mt,ℓ
T ℓd Fine-tuning time
Peak memory usage
Fig. 5: X RONOS architecture and end-to-end workflow. as the fraction of an iteration during which a worker waits at synchronization points for other workers to catch up. Fig. 4 shows that the two Raspberry Pi 5 workers have idle time ratios below 5% for both models, indicating that they are almost fully utilized throughout the iteration. In contrast, the faster ASUS MiniPC worker remains idle for 34% of the iteration for GPT-2-Medium and 23% for RoBERTa-Base. For GPT-2-Medium, more than one-third of the fastest worker’s iteration time is spent waiting rather than performing useful computation. Because TP proceeds synchronously, this waiting time not only affects local workers but also directly increases the end-to-end iteration time. The high idle ratios are due to the design assumptions of existing TP techniques. Most TP techniques are developed for relatively homogeneous GPU clusters and thus rely on uniform assignment of partitioned tensors [15], [22], [23]. With heterogeneous CPU workers, however, such uniform assignment creates stragglers: slower workers receive similar amounts of work despite their differences in compute capability, while faster workers repeatedly wait at synchronization points. In summary, our analysis in this section suggests that an efficient collaborative fine-tuning system for CPU workers should satisfy two requirements: (1) it should avoid CPU contention inherent in PP, and (2) it should account for worker heterogeneity when determining the amount of computation assigned to each worker to reduce idle time. IV. X RONOS D ESIGN Fig. 5 shows the X RONOS architecture and its end-to-end workflow, which consists of three phases: (1) worker profiling, (2) heterogeneity-aware planning, and (3) TP-based finetuning. In the profiling phase, each worker runs the X RONOS profiler to measure its layer-wise fine-tuning time and peak memory usage, and then sends the results to the coordinator. In the planning phase, the coordinator collects these profiling results and derives TP strategy that matches each worker’s compute and memory capacity. In the fine-tuning phase, the coordinator materializes the selected tensor layout and launches collaborative fine-tuning across the workers. A. Worker Profiling To capture worker heterogeneity with low overhead, X RONOS profiles a small representative subset of the target LLM rather than the entire model. We decompose the model into three components: an embedding layer (emb), B transformer blocks (blk), and a final output layer (out). While emb and out are structurally similar across many LLMs, the internal
composition of a transformer block depends on the model family. For example, a GPT-2 block contains ten layers—one self-attention layer, three linear layers, two layer-normalization layers, three dropout layers, and one GELU activation— whereas a LLaMA block contains eight layers—one selfattention layer, four linear layers, two layer-normalization layers, and one SiLU activation [24]. Given an LLM to fine-tune, the X RONOS profiler builds a profiling set consisting of one emb, one representative blk, and one out. Let k denote the number of layers inside the representative transformer block. The profiling set therefore contains k + 2 layers. Instead of profiling the full model, the profiler runs a few iterations (e.g., 10) of fine-tuning on the profiling set, which we empirically find sufficient to accurately estimate device capabilities across different LLMs. We use synthetic inputs [25] because the goal of this phase is to characterize performance rather than task accuracy. During profiling, worker d records two quantities for each layer ℓ: (1) layer-wise fine-tuning time and (2) peak memory usage. First, let Tℓd denote the time required for worker d to run layer ℓ once. Because the profiling set contains k + 2 layers, profiling yields k + 2 values of Tℓd per worker. Second, let Mℓd denote the peak memory usage of layer ℓ on worker d. We decompose this quantity into three separate parts: memory for model weights, activations, and temporary d d d kernel buffers, denoted by Mw,ℓ , Ma,ℓ , and Mt,ℓ , respectively. Thus, for each of the k + 2 layers, the profiler records three d d d memory metrics—Mw,ℓ , Ma,ℓ , and Mt,ℓ —resulting in 3(k+2) distinct metrics per worker. Note that we exclude communication time from our profiling metrics because it remains constant across different partitioning ratios. In TP, workers synchronously exchange activation and gradient tensors after computing their local partitions, so the communication time is bounded by the slowest worker. Let Bd denote the network bandwidth of worker d. The communication time Tcomm becomes Vcomm /min Bd . Here, d∈D Vcomm denotes the communication volume, which is determined only by the batch size and the model’s original hidden dimension [15]. So, it remains consistent across workers and partitioning ratios. The communication time is thus determined by min Bd , i.e., the minimum worker bandwidth, which is also d∈D independent of the partitioning ratio and consistent for a given set of edge workers [26]. Therefore, Tcomm remains constant across partitioning ratios and is not profiled separately.
B. Heterogeneity-aware Planning Using the profiles, the coordinator determines how many partitioned tensors to place on each worker. As described in §II-B, each worker stores (1) replicated tensors for layers that are not partitioned and (2) a worker-specific fraction of partitioned tensors. The coordinator chooses these fractions to reduce straggler-induced waiting while respecting per-worker memory limits. 1) Partition ratio: Let pd denote the fraction of all partitioned tensors assigned to worker d. In TP, tensors cannot be split arbitrarily; instead, they are assigned in indivisible chunks that must reside on a single worker. We call such a chunk a partition unit. For example, in self-attention, all tensors associated with one attention head form a partition unit, since they jointly produce one output. Accordingly, TP assigns partition units to each worker in integer numbers [15]. In LLMs, each layer contains the same number of partition units, H, which defines the granularity of workload distribution. If worker d receives h partition units, its partition ratio is pd = h/H. The feasible set of partition ratios is then: pd ∈ P = {h/H | h ∈ {0, 1, . . . , H}} .
(1)
2) Problem formulation: Suppose that N workers are available, where worker d ∈ {1, . . . , N } has memory capacity d Mcap . Our goal is to choose partition ratios p∗ = (p∗1 , . . . , p∗N ) that minimize the bottleneck in worker time. Under the TP setting we target, synchronization cost is identical across workers for a given model and batch configuration, so the worker with the longest local fine-tuning time determines the end-to-end iteration time. We therefore formulate the problem: p∗ = arg min max T d (pd ) (2) {p1 ,...,pN } 1≤d≤N
s.t.
PN
d=1 pd = 1 d d Mtotal (pd ) ≤ Mcap ,
pd ∈ P,
(3)
∀d ∈ {1, . . . , N } (4) ∀d ∈ {1, . . . , N }. (5)
Eq. (2) minimizes the maximum fine-tuning time across workers, where T d (pd ) denotes the fine-tuning time of worker d under partition ratio pd . Eq. (3) ensures that all tensors to be partitioned are fully distributed across workers. Eq. (4) enforces the memory constraint of each worker, where d (pd ) denotes the total memory usage of worker d. Mtotal Finally, Eq. (5) restricts each pd to the discrete candidate set P defined in Eq. (1). 3) Partition-aware cost estimation: To solve Eq. (2)–Eq. (5), X RONOS estimates two quantities for each worker: its fined tuning time T d (pd ) and its peak memory usage Mtotal (pd ). Both are derived from the profiling results obtained in §IV-A. T d (pd ) estimation. We estimate the worker-side finetuning time as d d d T d (pd ) = pd · Temb + Tout + B · Tblk (pd ).
(6)
The first term scales the embedding-layer cost by pd because worker d stores only a fraction pd of the embedding tensors.1 The second term is independent of pd because the output layer 1 As the embedding computation is proportional to the number of tensor elements, the execution time scales linearly with the tensor size.
is replicated on every worker. The last term captures the cost of the B transformer blocks, where P P d Tblk (pd ) = ℓ∈blkpart pd · Tℓd + ℓ∈blkrepl Tℓd . (7) Here, the first sum corresponds to partitioned layers within one transformer block and scales with pd , whereas the second sum corresponds to replicated layers and is independent of pd . All Tld values are known from the worker profiling results. d Mtotal (pd ) estimation. The total peak memory usage is the sum of (1) weights Mwd (pd ), (2) activations Mad (pd ), and (3) temporary buffer memory Mtd (pd ) under pd : d Mtotal (pd ) = Mwd (pd ) + Mad (pd ) + Mtd (pd ).
(8)
First, Mwd (pd ) is derived as: d d Mwd (pd ) = pd · Mw,emb + Mw,out P P d d + B pd ℓ∈blkpart Mw,ℓ + ℓ∈blkrepl Mw,ℓ . (9) Weights remain resident in memory throughout the entire fine-tuning. So, the total static memory usage is obtained by summing the embedding, output, and B transformer block tensors, where the embedding tensors and partitioned tensors in each transformer block scale with pd , while replicated tensors are independent of pd . Similar to the weights, activations also remain in memory throughout the fine-tuning process. Thus, Mad (pd ) follows the same structure as Eq.(9), replacing each weight term Mw with the corresponding activation term Ma . Lastly, Mtd refers to the peak memory usage from temporary buffers used to store intermediate results during computation (e.g., matrix multiplications or gradient computations). Unlike weights or activations above, the buffers are shortlived and exist only during the execution of a specific tensor computation. They are released immediately upon completion of computation and reused by subsequent operations. So, temporary buffers do not accumulate across layers, and the peak temporary memory is determined by the largest buffer during one fine-tuning iteration, which is calculated as follows: n d d Mtd (pd ) = max pd · Mt,emb , Mt,out , o d d max pd · Mt,ℓ , max Mt,ℓ . ℓ∈blkpart
ℓ∈blkrepl
(10) 4) TP strategy search: We search for the TP strategy to determine the final partition ratios p∗ , by solving Eq. (2)–(5). Finding the global optimum requires exploring all possible allocations of H partition units across N workers. However, −1 the resulting search space of size H+N leads to exponenN −1 tial time complexity, which quickly becomes prohibitive even for moderate H and N . To make the search practical, we use a greedy partition-unit assignment strategy that allocates one partition unit at a time, reducing the number of candidate evaluations to O(HN ). The search starts from an empty partition assignment, i.e., p∗ = 0, and considers all workers as candidates for receiving partition units. At each step, X RONOS tentatively assigns one additional partition unit, ∆p = 1/H, to each candidate worker and verifies whether the resulting partition ratio satisfies the worker’s memory constraint. Workers that cannot accommodate the additional unit are excluded from further consideration.
TABLE III: Model specifications. Model
Arch.
Params
# Transformer blocks
RoBERTa-Base GPT-2-Medium MobileLLaMA-1.4B
Encoder Decoder Decoder
125M 345M 1.4B
12 24 24
TABLE IV: Device specifications. Device Raspberry Pi 5 Orange Pi 5+ LattePanda Mu ASUS MiniPC
CPU processor
Memory
ARM A76 (2.4GHz) ARM A76/A55 (2.4/1.8GHz) Intel N100 (3.4GHz) Intel N100 (3.4GHz)
8 GB 16 GB 8 GB 16 GB
The partition unit is then assigned to the feasible worker that yields the smallest estimated fine-tuning time after receiving the unit. Because TP execution is synchronous, the overall iteration time is determined by the slowest worker. By always assigning to the fastest worker, the search balances the load across workers and directly reduces the bottleneck. The process continues until all H units are assigned. If all partition units are successfully assigned, the resulting ratios p∗ are used as the TP strategy for fine-tuning. If no feasible worker remains before all units are allocated, the search returns null, indicating that the available workers do not provide enough aggregate memory to host the target model. C. TP-based Fine-tuning After searching p∗ , the X RONOS coordinator materializes the TP layout by placing replicated and partitioned tensors on workers according to p∗ . It then initializes collaborative fine-tuning, including the hyperparameters (e.g., global batch size, sequence length, learning rate, and optimizer) and dataset. Once initialization finishes, the coordinator launches synchronized TP-based fine-tuning across the workers. V. E VALUATION We implement X RONOS in PyTorch with ∼2K lines of code. For inter-worker communication, we use Gloo backend [19] and PyTorch all-reduce operations [27]. A. Experiment Setup 1) Comparison: We compare X RONOS with two representative SOTA techniques: Asteroid [5] and Megatron-LM [15]. Asteroid is a PP-based collaborative fine-tuning technique for edge environments, which partitions the model according to device compute capability and memory capacity. MegatronLM is a widely used TP technique designed for high-end GPU clusters. It distributes computation uniformly across workers without accounting for device heterogeneity. We include Megatron-LM to evaluate the impact of heterogeneity-aware planning in X RONOS. While other techniques exist [6], [7], [23], many are either not publicly available or follow designs similar to the two; thus, we use the two as baselines. 2) Workloads: We use three LLMs: RoBERTa-Base, GPT2-Medium, and MobileLLaMA-1.4B [28]. Their architectures, parameter sizes, and number of transformer blocks are summarized in Table III. We select the three because: 1) edge fine-tuning typically targets models under 2B parameters [29], and 2) they cover a wide range of model scales, from 125M (RoBERTa-Base) to 1.4B (MobileLLaMA-1.4B).
We fine-tune the models on three GLUE benchmark tasks [20]: CoLA (linguistic acceptability), SST-2 (sentiment analysis), and MRPC (paraphrase detection). We set the sequence lengths to 32, 64, and 128 for CoLA, SST-2, and MRPC, respectively. The minimum sequence length required to preserve model accuracy varies across tasks [30]; our choices fall within the ranges while covering diverse lengths. All experiments use FP32 precision with a global batch size of 8. For PP, each global batch is divided into 8 micro-batches. We use AdamW with a learning rate of 2 × 10−5 . 3) Devices: We evaluate the following three scenarios: • Set-A (three devices): one ASUS MiniPC and two Raspberry Pi 5 devices. • Set-B (four devices): one LattePanda Mu, one Orange Pi 5+, and two Raspberry Pi 5 devices. • Set-C (five devices): one ASUS MiniPC, one LattePanda Mu, one Raspberry Pi 5, and two Orange Pi 5+ devices. Table IV summarizes the specifications of each device type. The devices are heterogeneous in architecture, i.e., ARM (Raspberry Pi 5, Orange Pi 5+) and x86 (LattePanda Mu, ASUS MiniPC), with CPU frequencies ranging from 2.4 to 3.4 GHz and memory capacities from 8 to 16 GB. The differences result in heterogeneity in both compute capability and memory capacity across scenarios. All devices are connected via 1 Gbps Ethernet, similar to other studies [6], [31]. 4) Metrics: We measure and report the following items: • Main results: we report the following metrics. – Iteration time: average time per iteration. – Computation stall ratio: amount of backend stall cycles divided by entire CPU cycles (§III-B2). – Number of context switches: total number of context switches (§III-C1) compared to PP (Asteroid). – Idle time ratio: fraction of an iteration during which a device remains idle (§III-C2). • Micro-benchmarks: we report the following metrics. – Estimation error of fine-tuning time: the percentage error calculated by MAPE between the estimated value (§IV-B) and the measured ground-truth. – Accuracy–time analysis: fine-tuning accuracy over elapsed time. We compare time-to-accuracy, the time required to reach a target accuracy (e.g., 90%). Unless otherwise noted, all metrics are averaged over the iterations within a single epoch. Among the three models, we report all metrics for RoBERTa-Base and GPT-2-Medium across all scenarios. For MobileLLaMA-1.4B, we report results only for Set-C, as the others cannot load the model due to limited memory. The remaining setup follows §III-A. B. Main results 1) Iteration time: Figs. 6a, 6b, and 6c present the iteration time (y-axis) on three benchmark tasks (x-axis) for RoBERTaBase, GPT-2-Medium, and MobileLLaMA-1.4B, respectively. The bars are grouped by scenarios (Set-A, Set-B, and Set-C). Across all experiment cases, X RONOS achieves the best (lowest) iteration time. For RoBERTa-Base in Fig. 6a, X RONOS reduces iteration time by 31% (up to 53% on Set-A of CoLA task) and 14% (up to 18% on Set-B of SST-2) on average compared to Asteroid and Megatron-LM, respectively. For GPT-2-Medium, X RONOS reduces the iteration time by 28% (up to 43% on Set-A of CoLA) and 14% (up to 18%
Iteration time (s)
Xronos
10 8 6 4 2 0
Set-A
Set-B
Asteroid
30
Set-C
Set-A
Megatron-LM Set-B
Set-C
20 10 0 CoLA SST-2 MRPC CoLA SST-2 MRPC CoLA SST-2 MRPC
CoLA SST-2 MRPC CoLA SST-2 MRPC CoLA SST-2 MRPC
(a) RoBERTa-Base
(b) GPT-2-Medium
50 40 30 20 10 0
Set-C
×
×
×
CoLA SST-2 MRPC
(c) MobileLLaMA-1.4B
Fig. 6: Iteration time comparison. × marks: out-of-memory error (§V-B1). Xronos 60
Computation stall ratio (%)
60
Asteroid
60
40
40
40
20
20
20 0
0
0 Set-A
Set-B Set-C
(a) RoBERTa-Base
Set-A
Set-C
Set-B Set-C
(b) GPT-2-Medium
(c) Mobile LLaMA-1.4B
Fig. 7: Computation stall ratio comparison (§V-B2). TABLE V: Context switches comparison. All values are averaged across all devices and tasks. (§V-B3) Scenario
Method
RoBERTa -Base
GPT-2 -Medium
MobileLLaMA -1.4B
Set-A
Xronos Asteroid
8,190 1,225,716
20,595 1,029,182
– –
Set-B
Xronos Asteroid
17,498 1,108,873
41,145 1,064,570
– –
Set-C
Xronos Asteroid
15,023 1,166,493
44,540 1,133,191
38,942 1,240,182
on Set-A of SST-2) on average. Also, for MobileLLaMA1.4B, X RONOS improves iteration time by 39% (up to 56% on CoLA) on average compared to Asteroid. Note that MegatronLM fails due to an out-of-memory error because it does not account for device heterogeneity, including memory capacity. The results show that X RONOS is more effective in terms of fine-tuning speed than existing techniques. 2) Computation stall ratio: Figs. 7a, 7b, and 7c show the computation stall ratio (y-axis) of X RONOS and Asteroid for RoBERTa-Base, GPT-2-Medium, and MobileLLaMA-1.4B, respectively. For each scenario (Set-A, Set-B, and Set-C), we measure the stall ratio for every device on each task (CoLA, SST-2, and MRPC) and report the average across all devices and tasks. Note that MobileLLaMA-1.4B reports values only for Set-C, as it can run only in this scenario, where the memory of devices is sufficient to load the model. X RONOS achieves lower computation stall ratios than Asteroid—on average, 20%, 23%, and 18% for RoBERTa-Base, GPT-2-Medium, and MobileLLaMA-1.4B, respectively. 3) Context switches: Table V shows the number of context switches. Each value is measured in the same way as the computation stall ratio above. X RONOS significantly reduces the number of context switches compared to Asteroid. Specifically, the reduction is 98.82%, 96.73%, and 96.86% on average for RoBERTa-Base, GPT-2-Medium, and MobileLLaMA1.4B. The improved computation stall ratio and reduced context switches show that X RONOS effectively alleviates CPU contention in PP by utilizing TP on CPU devices. 4) Idle time ratio: Figs. 8a and 8b show the idle time ratio of X RONOS and Megatron-LM for RoBERTa-Base and GPT-
2-Medium, respectively, on three tasks (x-axis). We exclude MobileLLaMA-1.4B because Megatron-LM fails to fine-tune it due to out-of-memory errors. X RONOS maintains the idle time ratio below 6% across both models and all configurations. In contrast, Megatron-LM shows ∼24% and ∼34% idle time ratios for RoBERTa-Base and GPT-2-Medium (with averages of 18% and 26%), respectively. This corresponds to a 4.6× and 5.9× reduction in idle time with X RONOS. Overall, X RONOS consistently shows higher utilization on heterogeneous CPU workers than existing techniques in our setting. C. Micro-benchmarks 1) Time estimation error: Fig. 9 presents the estimated finetuning time (y-axis) against the measured ground-truth finetuning time (x-axis) across scenarios. Each point represents one LLM-task pair in each scenario and is measured using the optimal partition ratio p∗ chosen by X RONOS. As other partition ratios exhibit similar trends, we report results with p∗ as representative. The diagonal line denotes the ideal estimation case (y = x), where the estimated fine-tuning time exactly matches the measured time. Thus, points closer to this line indicate higher estimation accuracy. Most points lie close to the diagonal line—the average estimation error is 15.32%, ranging from 13.28% (RoBERTaBase on CoLA in Set-A) to 16.65% (MobileLLaMA-1.4B on MRPC in Set-C). The results indicate that, although the estimates are not perfectly accurate, using profiling results from the profiling set yields sufficiently reliable estimates to improve fine-tuning time, as explained in §IV-B. 2) Accuracy-time analysis: Fig. 10 presents the accuracy achieved during five epochs of collaborative fine-tuning. We report RoBERTa-Base on the CoLA dataset under the Set-A scenario as a representative case; other settings show similar trends. X RONOS reaches the target accuracy in the shortest time. Specifically, with 90% accuracy as the target, X RONOS reduces time-to-accuracy by 53.6% and 16.4% compared with Asteroid and Megatron-LM, respectively. The results show that X RONOS effectively reduces fine-tuning time while maintaining the accuracy comparable to existing techniques. VI. R ELATED WORK Collaborative fine-tuning. PipeDream [32] is an early PPbased technique on GPUs without heterogeneity-aware planning. Asteroid [5] and PAC [6] both profile devices and incorporate heterogeneity-aware planning under PP using the same methodology. As PAC shares a similar design with Asteroid and lacks an open-source implementation, we use Asteroid as the representative baseline in our experiments. DGPAS [7] further extends PP-based training to hybrid GPU+CPU devices with heterogeneity-aware scheduling. Al Maruf et al. [16] studies pipelined execution on CPUs but
20 10 0
(a) RoBERTa-Base
30
Set-B
Set-C
20 10 0
(b) GPT-2-Medium
Fig. 8: Idle time ratio comparison (§V-B4). focuses on exploiting multiple cores within a single device rather than coordinating training across multiple edge devices, still using PP. Overall, existing techniques are dominated by PP and largely rely on overlapping computation and communication. In contrast, X RONOS targets collaborative finetuning on heterogeneous CPU devices and leverages TP with heterogeneity-aware planning. TP. Several studies have improved TP, largely developed for training on datacenter GPUs. Their primary goal is to improve scalability by sharding large tensors and optimizing communication. Megatron-LM [15], Optimus [22], and TAPAS [23] are representative techniques. As such systems are typically composed of similar devices, uniform sharding is commonly used in practice. Thus, these techniques are not designed for heterogeneous CPU-only edge devices, which can lead to high idle time, as we demonstrate. In contrast, X RONOS targets heterogeneous CPUs and allocates tensor partitions adaptively to device capacity, which differs from existing TP techniques. VII. C ONCLUSION This study proposes X RONOS, a collaborative fine-tuning system for LLMs on heterogeneous CPU-based edge devices. Through system analysis, we identify that pipeline parallelism suffers from CPU resource contention, while existing tensor parallelism frameworks fail to account for device heterogeneity. To address these challenges, X RONOS adopts tensor parallelism with a heterogeneity-aware partitioner that assigns non-uniform workloads across devices. Our evaluations demonstrate that X RONOS reduces iteration time ∼56% over Asteroid and reduces idle time ∼5.9× over Megatron-LM. ACKNOWLEDGMENT This research was supported by Basic Science Research Program through National Research Foundation of Korea (NRF), funded by Ministry of Education (MOE) (RS-2021NR060143), by NRF grant funded by Korea government (MSIT) (RS-2024-00336564), by IITP-ICT Creative Consilience Program grant funded by MSIT (IITP-2026-RS2020-II201819), by IITP grant funded by MSIT (RS-202625518394), and by ANCHOR program through Seoul ANCHOR Center, funded by MOE and Seoul Metropolitan Government (2026-ANCHOR-01-003-09). Corresponding authors: Gyeongsik Yang and Chuck Yoo. R EFERENCES [1] X. Wang et al., “Empowering edge intelligence: A comprehensive survey on on-device AI models,” ACM Computing Surveys, vol. 57, no. 9, 2025. [2] W. Choi et al., “Intelligent packet processing for performant containers in IoT,” IEEE Internet of Things Journal, vol. 11, no. 24, 2024. [3] C. Shin et al., “Prediction-based GPU sharing for distributed training,” Future Generation Computer Systems, p. 108413, 2026. [4] Z. Lin et al., “Split learning in 6G edge networks,” IEEE Wireless Communications, vol. 31, no. 4, pp. 170–176, 2024.
30 20 10 0
MobileLLaMA-1.4B MRPC GPT-2-Medium MRPC
Set-A Set-B Set-C Ideal (y = x)
RoBERTa-Base CoLA
0 10 20 30 Measured fine-tuning time (s)
Fig. 9: Estimation accuracy (§V-C1).
1.0 Accuracy (%)
Set-C
Set-A
Predicted fine-tuning time (s)
Set-B
Idle time fraction (%)
Set-A
Megatron-LM
40
C oL SS A T M -2 R P C C oL SS A T M -2 R P C C oL SS A T M -2 R PC
30
C oL SS A T M -2 R P C C oL SS A T M -2 R P C C oL SS A T M -2 R PC
Idle time fraction (%)
Xronos
40
0.9 0.8
Xronos Asteroid Megatron-LM
0.7 0.6 0
100 200 300 400 Elapsed time (min)
Fig. 10: Accuracy-time analysis (§V-C2).
[5] S. Ye et al., “Asteroid: Resource-efficient hybrid pipeline parallelism for collaborative DNN training on heterogeneous edge devices,” in MobiCom, 2024, pp. 312–326. [6] B. Ouyang et al., “Pluto and charon: A time and memory efficient collaborative edge ai framework for personal LLMs fine-tuning,” in ICPP, 2024, pp. 762–771. [7] J. Li et al., “DGPAS: DQN-GRU guided distributed DNN pipeline training and adjacent scheduling in edge networks,” Computer Networks, p. 111592, 2025. [8] J. Yoon et al., “Edgepipe: Tailoring pipeline parallelism with deep neural networks for volatile wireless edge devices,” IEEE Internet of Things Journal, vol. 9, no. 14, pp. 11 633–11 647, 2021. [9] W. Choi et al., “Harmonia: Accurate federated learning with all-inclusive dataset,” in IEEE CLOUD, 2024, pp. 302–304. [10] Siemens, “SIMATIC IOT2050: Edge and cloud connectivity,” accessed: 2026-06-03. [Online]. Available: https://www.siemens.com/en-us/prod ucts/simatic-iot-gateways/iot2050/ [11] Home Assistant, “Home Assistant Green,” accessed: 2026-06-03. [Online]. Available: https://www.home-assistant.io/green/ [12] “Nuvo-2610VTC Series: In-vehicle computer (Intel Atom x6425E),” accessed: 2026-02-04. [Online]. Available: https://www.neousys-tech. com/en/product/product-lines/in-vehicle-computing/nuvo-2610vtc-intel -atom-x6425-invehicle-computer [13] “Integrated GPU Chipset—Qualcomm Adreno GPU,” https://www.qualcomm.com/processors/adreno, accessed: 2026-0218. [14] “A full stack platform for Edge AI,” accessed: 2026-02-17. [Online]. Available: https://developers.google.com/coral [15] M. Shoeybi et al., “Megatron-lm: Training multi-billion parameter language models using model parallelism,” arXiv preprint arXiv:1909.08053, 2019. [16] M. Al Maruf et al., “Optimizing DNN training with pipeline model parallelism for enhanced performance in embedded systems,” Journal of Parallel and Distributed Computing, vol. 190, p. 104890, 2024. [17] A. Radford et al., “Language models are unsupervised multitask learners,” OpenAI blog, vol. 1, no. 8, p. 9, 2019. [18] Y. Liu et al., “RoBERTa: A robustly optimized BERT pretraining approach,” arXiv preprint arXiv:1907.11692, 2019. [19] “Gloo,” accessed: 2026-02-22. [Online]. Available: https://github.com/p ytorch/gloo/ [20] A. Wang et al., “GLUE: A multi-task benchmark and analysis platform for natural language understanding,” in EMNLP workshop BlackboxNLP, 2018, pp. 353–355. [21] “perf event open(2),” accessed: 2026-03-07. [Online]. Available: https: //man7.org/linux/man-pages/man2/perf event open.2.html [22] Q. Xu and Y. You, “An efficient 2d method for training super-large deep learning models,” in IPDPS, 2023, pp. 222–232. [23] Z. Shi et al., “TAPAS: Fast and automatic derivation of tensor parallel strategies for large neural networks,” in ICPP, 2025, pp. 804–815. [24] Z. Lu et al., “Demystifying small language models for edge deployment,” in ACL, 2025, pp. 14 747–14 764. [25] Y. Zhong et al., “DistServe: Disaggregating prefill and decoding for goodput-optimized large language model serving,” in OSDI, 2024. [26] A. Devraj et al., “Efficient allreduce with stragglers,” arXiv preprint arXiv:2505.23523, 2025. [27] “Pytorch distributed,” accessed: 2026-03-04. [Online]. Available: https://docs.pytorch.org/docs/stable/distributed.html [28] X. Chu et al., “Mobilevlm: A fast, strong and open vision language assistant for mobile devices,” arXiv preprint arXiv:2312.16886, 2023. [29] Z. Liu et al., “MobileLLM: Optimizing sub-billion parameter language models for on-device use cases,” in ICML, 2024. [30] S. Goyal et al., “Power-bert: Accelerating bert inference via progressive word-vector elimination,” in ICML, 2020, pp. 3690–3699. [31] G. Bartolomeo et al., “Oakestra: A lightweight hierarchical orchestration framework for edge computing,” in USENIX ATC, 2023, pp. 215–231. [32] D. Narayanan et al., “PipeDream: Generalized pipeline parallelism for DNN training,” in SOSP, 2019, pp. 1–15.