Multi-Node Transformers for Object Detection and Captioning Across Distributed Edge Networks

arXiv:2611.11291v1 [cs.DC] 8 Nov 2026

Multi-Node Transformers for Object Detection and Captioning Across Distributed Edge Networks Saurabh Kumar

Dewant Katare

Prakriti Jain

Walmart Sunnyvale, USA saurabh.kumar4@walmart.com

Delft University of Technology Delft, the Netherlands d.katare@tudelft.nl

Heidelberg University Heidelberg, Germany prakriti.jain@stud.uni-heidelberg.de

Abstract—This paper presents a unified transformer-based framework for object detection and dense region captioning across distributed edge networks. The architecture uses a shared visual encoder followed by detection, region extraction, and autoregressive caption-decoding stages, allowing localization and region description to reuse the same image features. This design removes the second image-encoding path required by decoupled detection and captioning pipelines. To support execution on resource-limited devices, a workload-allocation formulation assigns model stages and input streams across connected edge nodes while accounting for execution time, memory capacity, processor utilization, power limits, and intermediate tensortransfer cost. The framework is evaluated on a six-device edge testbed comprising four NVIDIA Jetson Orin Nano and Orin NX inference nodes and two Raspberry Pi 5 auxiliary CPU nodes, with TensorRT FP16 and INT8 execution on the NVIDIA devices. On a Jetson Orin Nano, the unified INT8 model achieves 38.1 FPS with 3.5 GB peak memory use and 11.8 W measured power. Under four concurrent streams, the proposed allocation achieves 35.2 FPS and 28.4 ms mean end-to-end latency with an internode transfer rate of 6.8 MB/s. Index Terms—Distributed Machine Learning, Edge Systems, Edge AI, Detection, Image Captioning

I. I NTRODUCTION Edge vision systems increasingly combine spatial perception with semantic description in applications such as traffic monitoring, industrial inspection, and autonomous sensing. Object detection provides localized object information, while region captioning adds textual descriptions for users and downstream text-based systems. Executing both tasks at the edge can reduce raw-data transfer and support latency-sensitive operation. Existing pipelines commonly use separate detection and captioning models, causing the same image to be encoded more than once. This repeated processing increases computation, memory use, and energy demand. Lightweight object detectors have been studied for edge deployment [1]–[3], while recent vision-text systems address traffic captioning, remote-sensing captioning, and multimodal object understanding [4]–[6]. These directions are usually considered separately rather than within a shared edge-oriented inference graph. Transformer-based vision-text models also remain expensive for low-power hardware because visual encoding, token processing, and autoregressive decoding compete for limited memory and compute resources. Prior work reduces this cost

through token sampling, simplified attention, pruning, and quantization [7]–[10]. Distributed inference provides another option by partitioning models across connected devices [11]– [16]. However, the benefit of partitioning depends on where the model is divided: large intermediate tensors increase transfer time, while detection and autoregressive captioning impose different execution loads. Static placement therefore perform poorly when stream load or link conditions change. This paper presents a unified transformer-based framework for object detection and dense region captioning across heterogeneous edge nodes. A shared visual encoder processes each input image once, after which detection, region extraction, and caption decoding operate on the resulting features. The model is divided into assignable execution stages, and a communication-aware allocation formulation selects their placement according to execution time, memory capacity, processor load, power limits, and inter-node transfer cost. The framework is deployed with TensorRT FP16 and INT8 and evaluated under changing stream loads and network conditions on a six-node heterogeneous edge testbed. The main contributions are: A unified transformer model that shares visual feature extraction between object detection and dense region captioning. • A multi-node allocation formulation for assigning encoding, detection, region extraction, and caption decoding across heterogeneous edge nodes while accounting for memory, load, power, and communication constraints. • A TensorRT FP16/INT8 evaluation covering task quality, throughput, latency, memory use, power consumption, and inter-node traffic under different stream and network conditions. •

The paper is organized as follows. Section II reviews object detection, dense region captioning, efficient transformer models, and distributed edge inference. Section III presents the shared visual architecture, distributed execution model, communication-cost formulation, and workload allocation problem. Section IV describes the dataset, hardware testbed, deployment settings, baselines, and evaluation metrics. Section V presents the experimental results and discussion. Section VI summarizes the findings and future work.

II. R ELATED W ORK A. Edge Detection and Vision-text Models Edge object detection has been studied through lightweight architectures, hardware-aware implementation, and task offloading [1], [2], [14]. These approaches reduce local inference cost but mainly address localization and classification. Vision-text models extend visual processing with captioning, grounding, question answering, and object-level reasoning [4]– [6], [17]. When detection and caption generation are deployed as independent models, however, each task may use a separate visual encoder, increasing computation and memory demand. The present work instead shares one visual representation between localization and dense region captioning. B. Efficient Transformer Execution Vision-transformer execution on edge hardware has been reduced through attention reformulation, token sampling, pruning, redundancy removal, and quantization [7]–[10], [18]. These methods lower the cost of processing a single model, but they do not determine how heterogeneous stages such as visual encoding, detection, region extraction, and autoregressive decoding should be assigned across several devices. Their execution characteristics also differ: detection follows a largely fixed computation path, whereas caption-decoding cost varies with the number of retained regions and generated tokens. C. Distributed Transformer Inference Distributed inference partitions model execution across connected devices to reduce the load on individual nodes. Prior studies consider transformer partitioning, collaborative edge execution, multi-tier inference, and hybrid data-model parallelism [11]–[13], [15], [19]. The benefit of model partitioning depends on both local execution cost and the size of intermediate tensors transferred between nodes. Large feature tensors can offset the gain from offloading, particularly under restricted bandwidth. Existing work does not jointly address shared visual encoding, variable region-caption decoding, and communicationaware stage placement in a multi-node edge setting. The framework in this paper combines these elements by sharing one visual encoder between detection and captioning and selecting stage placement according to processing time, memory demand, node load, power limits, and intermediate tensortransfer cost.

A. Execution and Placement Model Let N = {n1 , . . . , nN } denote the set of physical edge devices and Ncmp ⊆ N denote the subset of devices eligible for computational-stage placement. Let S = {s1 , . . . , sK } denote the set of active streams, where stream sk offers frames at rate fk . After stale-frame removal, let f˜k ≤ fk denote the rate admitted to the inference pipeline. For a compute node ni ∈ Ncmp , mi denotes available memory, u0,i background processor utilization, and pi the configured power limit. The link from node ni to node nj has measured bandwidth bij and link delay dij . The inference pipeline contains the stage set G = {genc , gdet , greg , gcap }, corresponding to shared encoding, detection, region extraction, and caption decoding, respectively, as illustrated in Fig. 1. The encoder produces a visual feature tensor F from input image I. Detection produces retained detections D containing region locations and associated predictions. Region extraction obtains aligned region features R directly from F and D on the local device, and the autoregressive decoder generates one caption for each retained region. Because region features are sliced directly from F, the captioning branch avoids a second image crop and redundant visual encoding stage. The captioning branch uses a two-layer autoregressive Transformer decoder with hidden dimension d = 256 and four attention heads. Region features R condition the decoder, which predicts tokens over a WordPiece vocabulary of V = 30,522. The decoder is trained using token-level crossentropy loss. Binary variable xg,i ∈ {0, 1} indicates whether stage g is assigned to compute node ni ∈ Ncmp , while zk,i ∈ {0, 1} specifies the ingress node for stream sk . A common computational-stage placement is used by all active streams, while zk,i specifies the ingress assignment of each stream. Each computational stage receives exactly one assignment. If qg denotes the memory required by the parameters and execution buffers of stage g, a placement is feasible only when the cumulative memory of stages assigned to node ni does not exceed mi . B. Caption-Aware Processing Cost The encoder, detector, and region-extraction stages have profiled mean per-frame execution times τ̄g,i on compute node ni . Caption-decoding runtime varies with both the number of retained regions and the generated sequence length. For sequential region decoding, the mean caption-stage service time on node ni is modeled as

III. S YSTEM M ODEL AND P ROBLEM F ORMULATION

τ̄cap,i = τinit,i (Mavg ) + Mavg Tavg τtok,i ,

The system processes multiple camera streams across a network of edge devices. Each frame is encoded once and then passes through detection, region extraction, and autoregressive region-caption decoding. The allocation problem determines where the computational stages execute while accounting for processing time, memory limits, processor load, power limits, and inter-node communication cost.

where Mavg is the profiled mean retained-region count, Tavg is the mean caption length in tokens, τinit,i is the regionconditioning latency, and τtok,i is the mean token-decoding latency. The aggregate service time assigned to node ni is X τ̄i = xg,i τ̄g,i . (2) g∈G

(1)

Fig. 1: End-to-end framework and distributed execution pipeline across heterogeneous edge devices. The shared Swin-T encoder (genc ) produces feature map F, which is reused locally by the detection (gdet ) and region extraction (greg ) stages on Edge Compute Node 1. The MINLP placement partitions execution at the greg → gcap boundary, transferring compact region embeddings R across the inter-node link to the autoregressive caption decoder (gcap ) on Edge Compute Node 2 while auxiliary nodes perform out-of-band telemetry.

Since only admitted frames enter the computational pipeline, node utilization is estimated using the aggregate admitted processing rate: ! K X ρi = u0,i + f˜k τ̄i , ni ∈ Ncmp . (3) k=1

The distinction between fk and f˜k separates offered workload from processed workload when the input rate exceeds available processing capacity. This is consistent with the staleframe removal policy used in the experiments. To account for increasing delay as utilization approaches capacity, the node residence-time cost is approximated as τ̄i Tnode,i = , ρi ≤ ρmax < 1. (4) 1 − ρi

the compact region embeddings R, avoiding the prohibitive bandwidth cost of transmitting raw feature maps F. The total inter-stage communication cost Tcomm is the sum of transfer times across consecutive stage boundaries assigned to different compute nodes. Similarly, Rtrans denotes the aggregate rate of input and intermediate payloads transmitted between nodes for the admitted stream workload. For stream sk , end-to-end processing latency combines ingress transfer, node residence times, inter-stage communication, and pipeline synchronization: X Te2e,k = Tin,k + Tnode,i + Tcomm + Tsync . (6) i:ni ∈Ncmp

The optimization considers the mean end-to-end latency K

1 X T̄e2e = Te2e,k . K

This expression is used as a placement-cost approximation rather than an exact queueing result and is validated against measured node delays in Section V. C. Communication and End-to-End Cost The payload transmitted across consecutive stage boundaries depends on the selected placement. An ingress transfer carries the input frame, a genc → gdet boundary transfers the shared feature tensor F, a gdet → greg boundary transfers serialized detection records D, and a greg → gcap boundary transfers aligned region features R. If vg,h denotes the payload size between consecutive stages g and h, the inter-node transfer time from node ni to node nj is 8vg,h . (5) Tijg,h = dij + bij The factor 8 converts payload size from bytes to bits. As shown in Fig. 1, partitioning across heterogeneous devices (e.g., placing {genc , gdet , greg } on Node 1 and gcap on Node 2) restricts the transferred payload across the inter-node link to

(7)

k=1

Node power is estimated from idle and peak measurements as Pi (X) = Pidle,i + ρi (Ppeak,i − Pidle,i ) .

(8)

This linear relation is used only as a first-order surrogate within the placement objective; experimentally reported power values are obtained from direct device measurements. D. Workload Allocation To limit uneven resource use, the allocation penalizes memory and processor load differences across participating compute nodes. Let Nact ⊆ Ncmp denote the compute nodes assigned at least one computational stage. Processor-load imbalance is quantified by Uload =

max ρi −

i:ni ∈Nact

min

i:ni ∈Nact

ρi ,

(9)

with Umem defined analogously using the occupied-memory fraction of each active compute node.

Let X = [xg,i ] and Z = [zk,i ] collect the stage and stream assignments, respectively. The objective jointly minimizes normalized mean latency, network-transfer rate, memory imbalance, and processor-load imbalance: min X,Z

J =α

T̄e2e Rtrans +β + γUmem + δUload . Tref Rref

The placement is subject to X xg,i = 1,

(10)

n1 , n2 Jetson Orin Nano n3 , n4 Jetson Orin NX n5 , n6 Raspberry Pi 5

Device

zk,i = 1,

∀sk ∈ S, (11)

xg,i qg ≤ mi ,

Device

Memory

Role/Mode

8 GB 16 GB 8 GB

INT8/FP16, 15 W INT8/FP16, 25 W CPU auxiliary

∀g ∈ G,

i=1

X

Nodes

TABLE II: Representative per-stage execution times (τ̄g,i ) across edge devices.

i:ni ∈Ncmp N X

TABLE I: Six-device edge testbed configuration.

genc

gdet

greg

gcap

Jetson Orin NX (INT8) 5.4 ms 2.3 ms 1.0 ms 3.8 ms Jetson Orin Nano (INT8) 11.2 ms 4.8 ms 2.1 ms 8.1 ms Raspberry Pi 5 (CPU) 148.5 ms 38.5 ms 14.2 ms 42.0 ms

∀ni ∈ Ncmp ,

g∈G

ρi ≤ ρmax , Pi (X) ≤ pi ,

∀ni ∈ Ncmp , ∀ni ∈ Ncmp .

The variables satisfy xg,i , zk,i ∈ {0, 1}. The normalization constants are fixed to Tref = 100 ms and Rref = 50 MB/s for all experiments. These constants scale the latency and network-transfer terms before application of the objective weights. The binary assignments, utilization-dependent residence time, and placement-dependent communication cost result in a mixed-integer nonlinear program (MINLP). The allocation problem is implemented in Pyomo and solved using SCIP 8.0. The objective weights are fixed to α = 0.40, β = 0.30, γ = 0.15, and δ = 0.15 for all experiments. SCIP is configured with a 5.0 s time limit and a relative optimality-gap target of 1%. The best feasible incumbent is used when the time limit is reached before the target gap. IV. E XPERIMENTAL S ETUP AND E VALUATION The evaluation examines task-level prediction quality and system-level execution behavior under different precision modes, stream loads, network conditions, and stage placements. Experiments use the Visual Genome benchmark and a six-device edge testbed comprising four NVIDIA Jetson inference nodes and two Raspberry Pi 5 auxiliary CPU nodes. A. Dataset and Workload Profiling The experiments use the Visual Genome dataset [20], which provides localized object annotations and natural-text region descriptions for joint localization and dense region-caption evaluation. We use an image-level 80/10/10 split for training, validation, and testing. All annotations associated with an image remain within the same subset to prevent image-level overlap. Input images are resized to 640 × 640 using bilinear interpolation and normalized with ImageNet channel statistics. Color jitter and random horizontal flipping are applied during training. Region descriptions are processed using WordPiece tokenization with a maximum sequence length of Tmax = 16 tokens; longer descriptions are truncated and shorter sequences are padded.

The allocation model in Section III uses profiling measurements obtained from the validation subset and target devices. After applying a detection confidence threshold of 0.5 and non-maximum suppression with an IoU threshold of 0.6, the mean retained-region count is Mavg = 14.2, while the mean caption length is Tavg = 8.5 tokens. For each profiled stage– device pair, mean execution time τ̄g,i is measured over 500 timed executions following 50 warm-up iterations. TensorRT FP16 and INT8 profiling is performed on the NVIDIA devices, while Raspberry Pi 5 measurements use CPU execution. Concurrent workloads are generated by replaying validation images as K ∈ {2, 4, 6, 8} independent streams. Each stream operates at a configured rate selected from fk ∈ {10, 15, 20, 30} FPS. These replay streams are used to vary aggregate system load; no temporal modeling or frame-toframe consistency is assumed. B. Testbed and Deployment Configuration The physical testbed contains six edge devices, summarized in Table I. Jetson Orin Nano and Orin NX devices provide GPU-accelerated inference, while the Raspberry Pi 5 devices serve as auxiliary CPU edge nodes for runtime profiling and system monitoring. Their stage measurements are also used to characterize when CPU-only execution would violate the utilization limits of the placement model. The NVIDIA devices run JetPack 6.2 with Jetson Linux 36.4.3 and TensorRT 10.3. Raspberry Pi 5 devices run the 64bit Raspberry Pi operating system and use ONNX Runtime 1.18 on the ARM64 CPU with NEON acceleration. The placement policy evaluates measured execution time, available memory, background utilization, configured power limits, and link conditions rather than assigning stages according to fixed hardware tiers. Table II reports representative execution times for the four computational stages. The Raspberry Pi measurements characterize the cost of CPU-only execution relative to the accelerated Jetson platforms. The Raspberry Pi 5 can execute the profiled stages on the CPU, but its service times are substantially higher than those of the INT8 Jetson devices. Region extraction is the least costly

profiled Pi stage at 14.2 ms, compared with 38.5 ms for detection, 42.0 ms for caption decoding, and 148.5 ms for shared encoding. For a given workload, a stage–device combination is considered infeasible when its predicted utilization violates the constraint ρi ≤ ρmax . The Pi measurements therefore provide a CPU-edge reference and allow infeasible CPU placements to be screened as the aggregate stream rate increases. The highthroughput placement experiments reported in Section V use the GPU-accelerated NVIDIA nodes for computational-stage execution. Stages assigned to NVIDIA devices are exported to ONNX using opset 17 and compiled with TensorRT 10.3. Both FP16 and INT8 execution are evaluated. INT8 engines use explicit QuantizeLinear and DequantizeLinear nodes inserted during post-training quantization. Quantization parameters are estimated from 1000 unlabeled images selected from the training subset, with no overlap with validation or test data. Operations without an INT8 implementation execute using a supported higher-precision kernel, and the resulting layer precisions are recorded during engine construction. The nodes communicate through a managed 1 Gbps Ethernet switch. Controlled link conditions are configured using tc-netem. Evaluated bandwidths are bij ∈ {10, 50, 100, 500, 1000} Mbps, with added delays dij ∈ {1, 5, 15, 30, 50} ms. Each experimental configuration is repeated 10 times under identical stream, precision, and network settings. C. Baselines and Evaluation Metrics The proposed allocation policy is evaluated against four placement baselines. Single-node execution places all computational stages on one feasible NVIDIA node. Static pipeline placement uses a fixed assignment across the available NVIDIA compute nodes and does not change with stream or network conditions. Greedy load-based placement assigns each stage to the feasible compute node with the lowest measured utilization, while memory-guided placement uses available memory without accounting for intermediate tensor size or link bandwidth. All placement methods use the same model parameters, precision mode, input streams, detection thresholds, NMS settings, and caption-decoding procedure. At the model level, the proposed shared-encoder architecture is compared with a decoupled baseline containing a Swin-B detector and an independent ViT-B caption encoder. Both systems use the same training and evaluation subsets. Because the architectures have different encoder capacities, this comparison measures end-to-end deployment cost rather than attributing the full difference to encoder sharing alone. Localization quality is reported using mAP50 , while regioncaption quality is measured using CIDEr. System measurements include end-to-end latency, completed-frame throughput, inter-node transfer rate, peak per-node memory use, node utilization, and power consumption. End-to-end latency is measured from frame availability at the ingress node until detection and all retained region captions are complete. Throughput counts frames for which the complete pipeline

TABLE III: Single-node performance across model and precision configurations (K = 1, node n1 ). Architecture

Mode mAP50 CIDEr FPS Mem.

Decoupled baseline FP16 Proposed unified FP16 Proposed unified INT8

31.4 31.1 30.6

94.2 93.8 92.1

Power

14.2 6.2 GB 14.8 W 22.5 3.9 GB 13.2 W 38.1 3.5 GB 11.8 W

finishes per second. Network traffic includes both ingress and intermediate transfers between compute nodes. Jetson memory measurements refer to shared system memory, and node utilization refers to measured GPU utilization on the NVIDIA compute nodes. Raspberry Pi CPU utilization and process memory are used only for auxiliary profiling and feasibility characterization. Modeled power used by the allocation objective is kept separate from direct device measurements. The terms Umem and Uload represent max–min differences across active computational nodes and are not interpreted as statistical variances. The evaluation addresses three questions: whether shared encoding and reduced-precision execution lower deployment cost while preserving task quality; how bandwidth, link delay, and concurrent stream load affect distributed execution; and whether the proposed allocation improves latency, network traffic, and load distribution relative to the placement baselines. V. E XPERIMENTAL R ESULTS AND D ISCUSSION The evaluation examines model efficiency, placementpolicy performance, sensitivity to network bandwidth, streamload scaling, and the accuracy of the utilization-dependent residence-time model. The Raspberry Pi 5 nodes are used as lightweight control and monitoring endpoints in the testbed; the high-throughput stage-placement results reported in this section use the GPU-accelerated NVIDIA nodes. A. Model Efficiency Table III compares the proposed unified model with the decoupled dual-backbone baseline on a single Jetson Orin Nano node. The proposed model uses a shared Swin-T encoder and contains 48.2 million parameters, whereas the decoupled baseline uses a Swin-B detector with 88 million parameters and an independent ViT-B caption encoder with 86 million parameters. Because the systems differ in both feature-sharing structure and encoder capacity, the comparison represents their complete deployment cost rather than the isolated effect of encoder sharing. The unified FP16 model increases throughput from 14.2 to 22.5 FPS while reducing peak memory from 6.2 to 3.9 GB. The corresponding differences in mAP50 and CIDEr are 0.3 and 0.4 points, respectively. INT8 execution further increases throughput from 22.5 to 38.1 FPS, corresponding to a 1.69× increase over unified FP16 execution. Peak memory decreases from 3.9 to 3.5 GB and measured power draw decreases from 13.2 to 11.8 W. Relative to unified FP16, INT8 reduces mAP50 by 0.5 percentage points and CIDEr by 1.7 points. INT8 therefore increases throughput by 69.3% over unified

TABLE IV: Placement performance under four concurrent streams (bij = 100 Mbps, dij = 5 ms, INT8). Strategy

Latency FPS

Traffic

Single node 88.4 ms 11.3 0.0 MB/s Static pipeline 42.1 ms 23.8 14.2 MB/s Greedy load-based 36.5 ms 27.4 18.6 MB/s Memory-guided 49.2 ms 20.3 28.1 MB/s Proposed MINLP 28.4 ms 35.2 6.8 MB/s

TABLE V: Effect of inter-node bandwidth on mean latency (K = 4, dij = 5 ms, INT8).

Memory Power

Bandwidth

Single

7.8 GB 4.1 GB 4.0 GB 3.6 GB 3.7 GB

10 Mbps 50 Mbps 100 Mbps 1000 Mbps

88.4 ms 142.1 ms 188.5 ms 88.4 ms 51.2 ms 64.0 ms 88.4 ms 36.5 ms 49.2 ms 88.4 ms 26.1 ms 27.8 ms

14.9 W 13.5 W 13.0 W 12.8 W 12.1 W

Greedy

Memory Proposed 52.3 ms 31.1 ms 28.4 ms 24.2 ms

TABLE VI: System performance under increasing concurrent stream load. FP16 while incurring a limited reduction in localization and caption quality. B. Placement-Policy Performance Table IV compares the placement methods using four concurrent streams, each configured at 30 FPS, giving an offered load of 120 FPS. Network bandwidth is fixed at 100 Mbps with an added link delay of 5 ms. Throughput counts frames for which detection and all retained region captions are completed. When the offered rate exceeds processing capacity, stale replay frames are discarded and the most recent available frame is processed; the same policy is applied to all placement methods. Single-node execution reaches 7.8 GB peak memory, leaving limited capacity for other system and runtime activity. Under the four-stream workload, it achieves 11.3 FPS with a mean latency of 88.4 ms. The proposed allocation reduces mean latency to 28.4 ms and achieves 35.2 FPS, with measured latencies ranging from 26 to 31 ms across repeated runs. Throughput is 47.9% higher than greedy load-based placement and 73.4% higher than memory-guided placement. For this configuration, the selected assignment partitions the execution path at the gdet → greg boundary between GPU-accelerated edge nodes. Serialized detection records at this boundary are smaller than the encoder feature tensor and aligned region features, reducing communication demand. The resulting transfer rate is 6.8 MB/s, compared with 18.6 MB/s for greedy placement and 28.1 MB/s for memoryguided placement. The power values in Table IV correspond to the measured draw of the Jetson node carrying the largest computational load. The Raspberry Pi 5 nodes are not assigned computeintensive stages in this high-rate configuration. Their measured CPU execution times are used during profiling to identify stage–device combinations that violate the configured utilization constraint before placement optimization. C. Effect of Network Bandwidth Table V reports mean end-to-end latency as inter-node bandwidth is varied. The stream count, INT8 precision, and added link delay remain fixed. Single-node latency remains unchanged because no intermediate tensors are transferred between devices. Distributed placements are more sensitive to bandwidth because part of their end-to-end latency depends on inter-node payload transfer.

K

Offered (FPS)

Completed (FPS)

Latency (ms)

Drop (%)

Peak ρ

2 4 6 8

60 120 180 240

58.4 35.2 38.6 39.1

21.2 28.4 34.1 41.5

2.7% 70.7% 78.6% 83.7%

0.42 0.68 0.81 0.89

At 10 Mbps, the proposed policy reduces mean latency by 63.2% relative to greedy placement and by 72.3% relative to memory-guided placement. As bandwidth increases, the difference between the distributed strategies decreases because communication contributes a smaller fraction of total latency. The proposed policy maintains the lowest latency among the compared strategies at each reported bandwidth, showing the importance of intermediate payload size when link capacity is restricted. D. Stream Load and Residence-Time Model The residence-time approximation in Eq. (4) is evaluated by comparing predicted and measured node delays over utilization levels from 0.10 to 0.88. Across feasible stage–node combinations, the model achieves a mean absolute error of 2.34 ms, supporting its use as a placement-cost approximation within the evaluated utilization range. The experiments use ρmax = 0.90, which prevents placements from approaching the singular point of Eq. (4) and leaves capacity for runtime variation. rate is defined as Drop (%) = 100 ×  The frame-drop  f 1 − completed , representing the fraction of offered frames foffered that are not completed by the pipeline. Table VI reports system performance as the number of concurrent streams increases from K = 2 to K = 8, with each stream configured at 30 FPS (bij = 100 Mbps, dij = 5 ms, INT8). At K = 2, the system completes most of the offered workload, reaching 58.4 FPS with a 2.7% drop rate. As the offered load increases, completed throughput approaches a saturation region of approximately 35–39 FPS, while mean latency rises from 28.4 ms at K = 4 to 41.5 ms at K = 8. Peak utilization increases from 0.42 to 0.89 and remains below the configured limit of ρmax = 0.90. The increasing drop rate reflects offered load beyond the sustained processing capacity rather than a violation of the configured utilization constraint. Overall, the unified INT8 model reaches 38.1 FPS on a single Orin Nano, while the proposed distributed placement

achieves 35.2 FPS and 28.4 ms mean latency under four concurrent streams. At 10 Mbps, communication-aware placement reduces mean latency by 63.2% relative to greedy placement and by 72.3% relative to memory-guided placement. VI. C ONCLUSION AND F UTURE W ORK This paper discussed a unified transformer-based framework for object detection and dense region captioning across distributed edge nodes. A shared visual encoder eliminates repeated image encoding, while a workload-allocation formulation assigns encoding, detection, region extraction, and caption decoding according to execution time, memory capacity, processor utilization, power limits, and inter-node transfer cost. The experimental platform contained four GPU-accelerated NVIDIA edge nodes and two Raspberry Pi 5 auxiliary CPU nodes used for profiling and feasibility characterization. TensorRT FP16 and INT8 execution was evaluated on the Jetson Orin Nano and Orin NX devices. On a Jetson Orin Nano, the unified INT8 model achieved 38.1 FPS with 3.5 GB peak memory use and 11.8 W measured power. Under four concurrent streams, the proposed placement achieved 35.2 FPS and 28.4 ms mean end-to-end latency with an internode transfer rate of 6.8 MB/s. At 10 Mbps bandwidth, mean latency was reduced by 63.2% relative to greedy placement and by 72.3% relative to memory-guided placement. These results support the use of shared visual encoding, reduced-precision execution, and communication-aware stage placement for joint object detection and dense region captioning on resource-constrained edge hardware. Future work will consider online stage migration, node-failure recovery, feature-transfer compression, and larger heterogeneous edge– cloud deployments. R EFERENCES [1] R. Al Amin, M. Hasan, V. Wiese, and R. Obermaisser, “Fpga-based real-time object detection and classification system using yolo for edge computing,” IEEE Access, vol. 12, pp. 73 268–73 278, 2024. [2] P. Mittal, “A comprehensive survey of deep learning-based lightweight object detection models for edge devices,” Artificial Intelligence Review, vol. 57, no. 9, p. 242, 2024. [3] D. Katare and A. Y. Ding, “Energy-efficient edge approximation for connected vehicular services,” in 2023 57th Annual Conference on Information Sciences and Systems (CISS). IEEE, 2023, pp. 1–6. [4] Q. M. Dinh, M. K. Ho, A. Q. Dang, and H. P. Tran, “Trafficvlm: A controllable visual language model for traffic video captioning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024, pp. 7134–7143. [5] Y. Bazi, L. Bashmal, M. M. Al Rahhal, R. Ricci, and F. Melgani, “Rsllava: A large vision-language model for joint captioning and question answering in remote sensing imagery,” Remote Sensing, vol. 16, no. 9, p. 1477, 2024. [6] Y. Zang, W. Li, J. Han, K. Zhou, and C. C. Loy, “Contextual object detection with multimodal large language models,” International Journal of Computer Vision, vol. 133, no. 2, pp. 825–843, 2025. [7] J. Chen, X. Zhang, W. Zhou, and W. Shi, “Ta-asf: Attention-sensitive token sampling and fusing for visual transformer models on the edge,” in 2024 IEEE/ACM Symposium on Edge Computing (SEC). IEEE, 2024, pp. 123–134. [8] S. A. Koohpayegani and H. Pirsiavash, “Sima: Simple softmax-free attention for vision transformers,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2024, pp. 2607– 2617.

[9] N. J. Eliopoulos, P. Jajal, J. C. Davis, G. Liu, G. K. Thiravathukal, and Y.-H. Lu, “Pruning one more token is enough: Leveraging latencyworkload non-linearities for vision transformers on the edge,” in 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). IEEE, 2025, pp. 7153–7162. [10] D. Choi and H. Kim, “Gradq-vit: Robust and efficient gradient quantization for vision transformers,” in Proceedings of the AAAI conference on artificial intelligence, vol. 39, no. 15, 2025, pp. 16 019–16 027. [11] C. Hu and B. Li, “When the edge meets transformers: Distributed inference with transformer models,” in 2024 IEEE 44th International Conference on Distributed Computing Systems (ICDCS). IEEE, 2024, pp. 82–92. [12] D. Wen, G. Liang, T. Li, L. Chen, J. Li, and T. Li, “Easyvit: An adaptive collaborative edge computing framework for vision transformer,” IEEE Internet of Things Journal, 2025. [13] Y. Li, Y. Xue, J. Zeng, D. Zheng, L. Feng, and H. Xing, “Collaborative transformer inference in multi-tier multi-node edge networks,” IEEE Network, 2026. [14] Y. Yuan, S. Gao, Z. Zhang, W. Wang, Z. Xu, and Z. Liu, “Edgecloud collaborative uav object detection: Edge-embedded lightweight algorithm design and task offloading using fuzzy neural network,” IEEE Transactions on Cloud Computing, vol. 12, no. 1, pp. 306–318, 2024. [15] K. Chaitra, M. Basthikodi, and B. Poornima, “Scalable parallelization of machine learning models using hybrid data-model partitioning,” in 2025 Second International Conference on Computing, Semiconductor, Mechatronics, Intelligent Systems and Communications (COSMIC). IEEE, 2025, pp. 103–108. [16] D. Katare, E. Marin, N. Kourtellis, M. Janssen, and A. Y. Ding, “Arasec: Adaptive resource allocation and model training for serverless edge computing,” IEEE Internet Computing, 2024. [17] Y. Feng, Y. Liu, S. Yang, W. Cai, J. Zhang, Q. Zhan, Z. Huang, H. Yan, Q. Wan, C. Liu et al., “Vision-language model for object detection and segmentation: A review and evaluation,” arXiv preprint arXiv:2504.09480, 2025. [18] Q. Wang, X. Zou, C. Li, Y. Peng, H. Wang, Y. Wen, M. Yeerlan, and C. Chen, “Cst-vit: Cascaded spatio-temporal redundancy elimination for efficient vision transformers on edge iot devices,” IEEE Internet of Things Journal, 2025. [19] Z. Ding, M. Zhu, Z. Tang, S. Wei, and Y. Liu, “A distributed framework for privacy-enhanced vision transformers on the edge,” in Proceedings of the Tenth ACM/IEEE Symposium on Edge Computing, 2025, pp. 1–16. [20] R. Krishna, Y. Zhu, O. Groth, J. Johnson, K. Hata, J. Kravitz, S. Chen, Y. Kalantidis, L.-J. Li, D. A. Shamma et al., “Visual genome: Connecting language and vision using crowdsourced dense image annotations,” International journal of computer vision, vol. 123, no. 1, pp. 32–73, 2017.

Related documents

• LUMA: Benchmarking Segmentation via a Lightweight Universal Mask Adapter#1458371
• Revealing Multi-View Hallucination in Large Vision-Language Models#820396
• Seeing Before Synthesizing: VLM-Guided Transition Event Discovery for Weakly-Supervised Dense Video Captioning#927047
• SPHQuant: Efficient extreme low bit weight quantization for Vision-Language Models#1051273
• Mask-supervised Object-centric Representation Learning with LeJEPA#1202134