CloakLM: Obfuscating GPU Memory Layout to Mitigate Model Ex-filtration for Serving Kunal Jain
Seokjin Go
Divya Mahajan
Georgia Institute of Technology USA [email protected]
Georgia Institute of Technology USA [email protected]
Georgia Institute of Technology USA [email protected]
Abstract
show near-native performance alongside strong resistance to PCIe snooping and HBM dump attacks, demonstrating that inference-time model exfiltration can be made substantially less practical in real-world deployments.
Large foundation models deployed on third-party and shared accelerator infrastructure face a practical risk of model exfiltration that existing defenses do not fully address [30]. In common serving deployments, the model provider controls the VM or bare-metal serving stack and trusts the drivers and runtime of its own accelerators, but does not control the surrounding hardware substrate: the host–GPU interconnect, the accelerator fabric, and neighboring infrastructure components remain outside the tenant’s trust boundary. Prior work has demonstrated that both attack surfaces are practically exploitable. Hermes [39] shows that a passive observer with access to the PCIe bus can achieve lossless DNN reconstruction from transferred packets alone. TunnelS [36] shows that a non-participating host with driver-level access can exfiltrate HBM contents at high throughput by exploiting PCIe underutilization, without interrupting inference. Beyond these direct hardware paths, co-tenant VMs reachable over the shared frontend or management network can access memory-mapped interfaces or improperly segmented RDMA regions without requiring physical co-location. These attacks share a common enabler: ML frameworks allocate model weights in large, contiguous, and repeatedly accessed memory regions to maximize throughput. This regularity makes intercepted PCIe transfers and raw HBM captures structurally rich enough to reconstruct layer boundaries, tensor shapes, and model parameters. We present CloakLM, a software-only memoryobfuscation framework that removes this structural regularity without modifying the inference stack’s logical view of memory. CloakLM operates across three tiers: PCIe traffic shaping to reduce extraction bandwidth on the host–GPU path; inter- and intra-layer weight shuffling to break the correspondence between observed access sequences and model structure; and physical HBM page remapping to ensure raw memory captures no longer yield contiguous, reconstructable model segments. The authorized execution sees a valid virtual memory layout and incurs negligible overhead; whereas unauthorized observers see fragmented, semantically incoherent state. CloakLM integrates with vLLM and PyTorch, requires no hardware changes, and is complementary to confidential computing. We evaluate CloakLM on distributed inference workloads using LLaMA and Qwen models and
1
Introduction
Large foundation models [10, 20, 26] are increasingly deployed as remote services on accelerators hosted in thirdparty datacenters and shared GPU clusters [19]. Providers such as OpenAI [26], Anthropic [1], and Mistral [15, 16] relinquish physical control of the hardware, yet the deployed model represents their primary intellectual property and economic asset. While the community has extensively studied secure VMs, data isolation, and user-data confidentiality, the security of model weights as they traverse and reside in accelerator memory remains largely unaddressed. Trust boundaries in third-party inference deployments. In practice, a model provider is granted control over a virtualized host or VM instance provisioned by the cloud operator. We treat this VM and its user-space serving stack as trusted, it executes the provider’s own software and is comparable to bare-metal access granted by a CSP. The trust boundary lies below and around this VM along two dimensions. On the software side, we trust the accelerator drivers, firmware, and runtime stack on GPUs that are explicitly part of the serving deployment, these are provisioned to the model owner and execute software under their control. On the hardware side, the physical accelerator fabric, interconnects, peer hosts, and nodes on the same front-end network are not under the model provider’s control: an attacker need not subvert any trusted software component to reach model weights if they have access to the underlying hardware substrate or the network plane connecting it. As shown in Figure 1, this boundary has material consequences as a model transitions from training to inference. During training, parameters are short-lived, continuously evolving, and confined to a small number of tightly managed clusters; this transience lets providers exercise close operational control over where and how parameters reside. Instead, at inference time stable checkpoints are replicated, persistently resident on third-party and disaggregated accelerators, and kept resident for long periods while serving 1
Preprint, 2026
Anonymous Submission Weights transfer for model execution
Router
Training Loop
HBM
Norms
FFN
SMs
SMs
GPU
GPU
Continuous memory allocation
Router
FFN
Original model architecture and weights exfiltrated
Dump of victim HBM
attack
HBM
Attention
Trust boundary shifts from model owner to infra provider
Host (Attacker)
2 HBM
Attention
Controlled Training Infrastructure
HBMs content extracted
E1 E2 E3 E4
Model Weights in Transient State
1
2
Frontier Trained Model Intellectual Property
Attention
PCIe Snooping
Host (Victim)
E1 E2 E3 E4
FFN
Attention
Embedding
Attention
Experts
Malicious colocated Host
Privately operated Host and GPU for Serving
Frontier Model uploaded to Cloud Service Providers
Training models incurs huge costs: 1. Human capital 2. Hardware (GPUs, NW cards, etc) 3. OpEx: power, cooling, etc
Untrusted weights transfer by attacker
Reconstruction Algorithm
Inference on Cloud Service Providers
Figure 1. Model lifecycle and inference-time memory exposure. During training, model parameters are transient and reside on tightly controlled infrastructure. Once deployed for inference, stable checkpoints cross the trust boundary and become persistently resident on third-party, disaggregated accelerators. Co-tenant VMs, non-participating GPUs on the same fabric, and physical bus observers can all access or intercept model weights, through direct memory channels, shared interconnects, or PCIe snooping, without participating in or disrupting the inference workload. requests. Even when the serving VM is fully trusted, any actor with access to the same hardware substrate, accelerator HBM, or co-resident VM can potentially observe, access, or extract model weights. A concrete and expanding attack surface. We identify three classes of inference-time model exfiltration attacks, each realistic in today’s cloud and datacenter deployments. The first targets host- and VM-level access: a malicious cotenant or non-participating host can reach HBM over the shared management network, or exploit idle PCIe bandwidth through driver-level modifications, without touching the serving stack [36]. The second targets the accelerator fabric: a non-participating GPU on the same NVLink domain or interconnect fabric can access peer device memory or intercept its traffic at the device level, without requiring host-side code execution or network reachability [25]. The third involves physical hardware access: an attacker with physical proximity to the server can instrument the infrastructure directly, for example by inserting a PCIe snooping device, and reconstruct model parameters from DMA traffic or device memory without involving the host software stack [39]. All three positions share the same capability and limitation: they obtain raw memory contents or raw bus transfers outside the serving stack, without the framework-level metadata such as virtual memory layout, virtual-to-physical page mappings, and tensor placement records. Why ML software stacks make raw memory access sufficient. Even though the attacker does not have frameworklevel metadata, in an unprotected deployment this limitation
is immaterial. Frameworks such as PyTorch and vLLM allocate model weights in large, contiguous physical regions that are persistently resident and repeatedly accessed in a predictable order to maximize throughput. This regularity makes both memory contents and bus-visible transfers semantically rich: contiguity, stable access order, and tensor regularity are alone sufficient to infer layer boundaries, tensor shapes, and parameter values, the framework metadata is effectively recoverable from the raw data itself, without access to the serving stack [31, 36, 39]. Protecting model-resident memory beyond confidential computing. Confidential Computing (CC) provides strong hardware-backed isolation when the entire execution environment supports it, and CloakLM is designed to complement it. CC requires the trust boundary to extend across every participating component: a single non-enclave node or heterogeneous accelerator breaks the chain, and protects HBM through access controls rather than encrypting memory contents at runtime [11]. Physical access to the device through hardware probing or firmware-level bypass can therefore still recover plaintext weights directly from HBM, even on CC-enabled hardware. More broadly, CC addresses data in motion across the protected fabric, but does not reduce an attacker’s ability to reconstruct the model from weights already resident in device memory. CloakLM targets this complementary surface: by fragmenting and permuting the physical layout of model weights in HBM, raw
2
CloakLM
Preprint, 2026
memory captures, however obtained, yield structurally incoherent data rather than reconstructable model parameters, on any infrastructure regardless of enclave support. CloakLM: obfuscation as an accelerator-memory defense. CloakLM reshapes how model weights are laid out and transferred across the accelerator memory path. Tier 0 applies controlled PCIe traffic shaping to reduce extraction bandwidth and fidelity on the host–GPU path. Tier 1 randomizes layer placement so that observed access sequences no longer reveal execution order or contiguous tensor regions. Tier 2 remaps the physical HBM pages backing model weights so that raw memory captures no longer yield contiguous, semantically coherent segments. CloakLM is a software-only solution integrated with PyTorch allocators, the CUDA runtime, and vLLM, while preserving compatibility with distributed inference. Authorized execution retains a valid logical memory view and nearnative performance; unauthorized observers who obtain raw memory dumps or bus-level captures see fragmented, incoherent state that resists reconstruction without the serving stack’s layout metadata. CloakLM is designed specifically against attackers who access raw physical memory or bus transfers from outside the serving environment. We do not defend against full compromise of the host OS, hypervisor, or firmware of the serving stack, nor against attackers who can directly read virtual-to-physical mappings or modify inference kernels, such adversaries can defeat any softwareonly layout defense. Our goal is cost imposition, not secrecy: making accelerator-side model extraction substantially less practical for the attacker classes described above. Contributions. We make the following contributions: • Defining accelerator-side attack vectors for model exfiltration. We identify and characterize three concrete attacker positions, co-tenant or non-participating hosts VMs via the management network, non-participating accelerators via shared backend network fabric, and physical bus access, and show why existing defenses such as confidential computing and network isolation are either limited in deployment or do not practically fully address them. • Characterizing inference-time model exposure. We show how contiguous virtual-to-physical placement and stable HBM residency in current serving stacks make raw memory access, without framework metadata, sufficient for practical model reconstruction. • A multi-tier software defense against acceleratorside exfiltration. CloakLM disrupts extraction signals across PCIe-visible transfers, runtime-visible layout, and physical HBM placement by fragmenting, shuffling, and remapping model weights, without hardware changes or secure enclaves. • Performance-preserving design for memory-bound workloads. Authorized execution retains a valid logical memory view and efficient GPU access behavior despite
obfuscated physical placement, with no kernel modifications required. • End-to-end implementation and evaluation on real inference stacks. CloakLM integrates with PyTorch and vLLM [17], supporting distributed multi-GPU execution, and is evaluated on LLaMA and Qwen, demonstrating strong extraction degradation at near-native inference performance.
2
Background
2.1
AI Infrastructure and Model Ownership
Model weights are critical intellectual property. Foundation models represent the primary intellectual property of modern AI companies [1, 16, 26]. Training a state-of-theart model requires months of engineering effort, curated datasets at scale, and compute budgets measured in millions of dollars. The resulting weights encode this accumulated investment in a compact, executable form. For many providers, the trained model is the entire product: once ex-filtrated, it can be reused, fine-tuned, or served by a competitor at negligible marginal cost, eliminating the original provider’s competitive advantage entirely. Protecting model weights is therefore a core business and national security requirement, not merely a privacy concern [30]. This work focuses on the inference. While protecting model weights across the full deployment lifecycle is important, training and inference present different exposure profiles that call for different approaches. Training workloads tend to run on more homogeneous, purpose-built clusters where infrastructure configuration is more tightly controlled. NVIDIA GPUs currently dominate this space, accounting for approximately 92% of the accelerator market [3]. Inference, by contrast, must scale across heterogeneous environments, rented accelerators, shared datacenters, and geographically distributed deployments and operates on stable, long-lived model checkpoints that remain unchanged across millions of requests. These two properties, the breadth of heterogeneous deployment and the stability of inference checkpoints, make inference both a high-risk and the most tractable phase for software-based weight protection. We focus on inference throughout this paper and leave training-time defenses to future work. 2.2
Inference Deployment and the Trust Boundary
Inference is deployed on heterogeneous, third-party infrastructure. To meet inference demand, providers rely on Cloud Service Providers (CSPs) such as AWS, Azure, and Google Cloud, renting accelerators hosted in third-party datacenters. The provider retains control of the software stack but cedes physical control of the hardware: the underlying servers, PCIe fabric, and accelerator interconnects are owned and operated by the CSP. Co-located tenants share network segments, accelerator racks, and in some configurations, the 3
Preprint, 2026
Anonymous Submission
same physical hosts. This deployment model exposes model weights to infrastructure operators and co-located adversaries whose presence and capabilities the model provider cannot audit or control. Inference checkpoints are stable and persistently resident. Unlike training, where parameters are updated continuously and intermediate states are transient, inference operates on fixed model checkpoints that remain resident in accelerator memory for extended periods. A single deployment may serve millions of requests with the same weights continuously loaded in GPU HBM. This stability creates a persistent, high-value target: an adversary with access to the accelerator or its interconnects can observe the same data repeatedly, aggregate observations over time, and correlate partial captures to reconstruct full model state. 2.3
Hopper and Blackwell Confidential Compute [11], address inference-time exposure by establishing a hardwareenforced enclave around CPU and GPU execution. Within this perimeter, PCIe transfers between host and GPU (data and kernel information) are encrypted in transit, and GPU kernels execute inside an enclave that excludes the hypervisor and CSP from the trusted computing base. NVLink connections between GPUs within the same CC domain are similarly encrypted and authenticated. HBM contents are access-controlled but not encrypted at rest. CC enforces access control over HBM, preventing unauthorized code from reading device memory, but does not encrypt the underlying DRAM cells. Weight values reside in HBM as plaintext; the access-control layer governs who may read them, not whether the bits themselves are protected at the storage level. This is an architectural consequence of the GPU compute model: kernels require low-latency, highbandwidth access to weights on every forward pass, and encrypting HBM at rest would impose prohibitive overhead on every memory access.
System and GPU Architecture for Serving
Weights are loaded once and persistently resident. LLM inference is GPU-centric: model weights are transferred from host DRAM to GPU HBM at service startup and remain resident throughout the serving lifetime. Inference kernels read weights from HBM on every forward pass, but the parameters themselves are never modified. HBM therefore functions not as transient working memory but as a stable, read-only store of model parameters, continuously present in device memory for the duration of the deployment. Frameworks allocate parameters in large contiguous regions. Inference frameworks allocate model parameters using bulk allocators optimized for throughput. PyTorch’s caching allocator reserves large contiguous virtual memory regions and maps them to physical HBM pages in a size-ordered, predictable fashion [5]. vLLM’s block allocator similarly assigns contiguous physical blocks to layer weight tensors to maximize memory bandwidth utilization during kernel execution [17]. As a result, a model’s weight tensors occupy large, contiguous, physically predictable regions of HBM, laid out in the order in which they are loaded and subsequently accessed. Inference computation spans shared interconnects. Modern inference deployments disaggregate computation across accelerators connected by high-speed fabrics. Within a host, GPUs communicate with the CPU over PCIe; across GPUs within a rack, NVLink provides high-bandwidth device-to-device transfers; across racks, RDMA-capable networks carry model shards and activations between nodes [4, 24]. These interconnects are shared infrastructure: in cloud deployments, the physical fabric and switching equipment are operated by the CSP, and multiple tenants’ workloads may traverse the same switches and fabric segments. 2.4
3
Threats for Model Exfiltration
Production inference deployments expose model weights through multiple simultaneous attack surfaces. Weights transferred to GPU HBM at service startup remain persistently resident and are repeatedly accessed in predictable patterns throughout the serving lifetime, making them an unusually stable target. The interconnect transfers that carry these weights between host and GPU are unencrypted and structurally regular, reflecting the deterministic access patterns of inference kernels. The shared management and frontend networks that connect co-located infrastructure provide additional reachability to the accelerator and its memory. Together, these properties place model weights in simultaneous contact with three distinct attacker positions, each operating at a different layer, each requiring a different kind of defense. 3.1
Attack Surfaces
In this work, the trust boundary runs at the serving software stack: the host OS, drivers, and runtime are trusted; the physical hardware and all non-serving infrastructure are not. Recent work has shown that accelerators, interconnects, and co-tenant VMs are increasingly exploitable attack surfaces. Figure 2 illustrates all three attacker positions relative to the trusted serving environment. 1 Co-tenant or non-participating host. A compromised VM or non-participating host on the same infrastructure can reach model weights through two complementary paths. Over the shared front-end or management network, a cotenant with authenticated access can exploit reachability to the serving host directly: the XZ Utils backdoor [13] illustrates this entry point, where a single compromised package granted network access to otherwise isolated hosts, and prior
Confidential Computing
CC establishes a hardware-enforced trust perimeter. Confidential computing (CC) platforms, such as NVIDIA 4
CloakLM
Preprint, 2026
Serving Attack Surfaces Tackled by CloakLM 1
Datacenter
Malicious co-tenant with network-accessible memory
Host
Host
PCIe Reads Exploits recheability over shared network to access HBM [TunnelS, XZ-Utils]
2
PCIe
Model Configs
GPU Drivers
Exfiltrate HBM contents due to bus underutilization [ReDMArk]
GPU GPU
Serving GPU Stack (Trusted)
3
GPU
HBM
HBM
GPU GPU
NVLink
Infrastructure Outside Enclave
Serving Infrastructure (Untrusted)
Passive snooping on physical substrate such as HBM [Hermes]
1
2
PCIe
GPU
Physical device access PCIe Snopper
Kernel Graphs
Serving Host Software Stack (TEE) — Model Configs — Serving Software & Metadata
Serving Software & Metadata
Serving GPU Stack (Trusted)
Non-participating GPUs on the same fabric NvLINK RDMA
3
Datacenter
Serving Host Software Stack (Trusted)
— GPU Drivers — Kernel Graphs
GPU
GPU
HBM
HBM
NVLink
CC Rack (Network Trusted; HBM Data unencrypted)
Raw Byte Extraction Layer 1 Layer 2 Layer 3
011001001 100110001 111100100
The trust boundary at serving software stack: host OS, drivers, and runtime are trusted; the physical hardware substrate and all non-serving infrastructure are not.
Confidential Computing enclave upto rack level but is incompatible with infrastructure external to the trust boundary and incurs a latency overhead.
1 A malicious co-tenant or non participating hosts, ○ 2 A malicious GPU on the Figure 2. We evaluate three attack surfaces: ○ 3 An adversary with physical access to the infrastructure. The adversary in all three scenarios is same accelerator fabric and, ○ able to extract raw byte data of a model resident in HBM or in motion across the network. CloakLM tackles these adversary 3 with latency overheads and by break assumptions around memory layout and transfers. Confidential computing mitigates ○ requires the enclave to be extended for the entire boundary of trust. cross-tenant work [32, 33] has shown that network adjacency alone is sufficient to leak program state from victim tenants. At the driver level, a non-participating host need not use the management network at all: TunnelS [36] demonstrates that driver-level modifications can exfiltrate HBM contents at high throughput by exploiting idle PCIe windows, without executing any code on the victim host or interrupting its workload. 2 Non-participating GPU on the same accelerator fabric. A GPU belonging to a different tenant on the same NVLink domain or accelerator fabric represents a distinct attacker position from the host-level adversary above [25]. Operating at the device level, it has potential visibility into peer GPU memory through fabric-level access mechanisms, without requiring host-side code execution or management network reachability. This arises naturally in multi-tenant accelerator clusters where GPUs from different workloads share the same NVLink based backend network infrastructure. 3 Physical hardware access. An attacker with physical access to the server or datacenter infrastructure can instrument the hardware directly—for example, by inserting a PCIe snooping device into the interconnect to intercept
DMA transfers between host and GPU, or by attaching instrumentation to the accelerator to read HBM contents without involving the host software stack. Unlike the VM- or driver-level adversaries above, this attacker operates below the software layer entirely. Hermes [39] demonstrates the severity of the interconnect vector, achieving lossless DNN model reconstruction by passively analyzing PCIe packets alone, without any interaction with the GPU or inference stack. All three attacker positions share the following capability: they obtain raw memory contents or raw bus transfers from outside the serving stack. They see physical bytes, either captured from the PCIe bus or read directly from HBM, without access to the framework-level metadata that would otherwise be needed to interpret them: the PyTorch virtual memory layout, the virtual-to-physical page mapping, and the placement decisions that record which tensor occupies which address range. In an unprotected deployment, this limitation is immaterial due to the predictable layout of current ML infrastructure. As established in Section 2, ML frameworks allocate model weights in large, contiguous physical regions that are persistently resident and repeatedly accessed in a predictable order. The regularity of this layout means that raw physical bytes alone are sufficient to infer layer boundaries, tensor 5
Preprint, 2026
Anonymous Submission
shapes, and parameter values, the framework metadata is effectively recoverable from the data itself [36, 39]. 3.2
The security goal of CloakLM is not to make weight extraction impossible: an adversary with unrestricted physical control of a device can always recover data at some level. Rather, CloakLM aims to impose cost, to ensure that raw physical bytes, whether captured from the bus or dumped from HBM, do not yield an interpretable, deployable model without effort commensurate with training from scratch. We treat reconstruction cost as the operative security metric: CloakLM succeeds if the effort required to recover a functionally accurate model from raw observations exceeds the effort of training an equivalent model independently.
Limitations of Existing Defenses
Enclave based mechanisms leave most inference deployments exposed. CC and TEE offer hardware enclave but impose non-trivial latency overhead (∼10-20% [2, 29]), limiting deployment in latency-sensitive production settings. As a result, the majority of inference today runs without CC protection, leaving interconnect transfers and HBM contents exposed to the attack surfaces described above. Where CC is deployed, it protects data in motion and enforces access control over HBM, but does not encrypt HBM contents at rest, a potential residual exposure for adversaries with physical device access below the access-control layer. The CC perimeter is bounded by vendors’s interconnect domain. CC enclave boundary encompasses the CPU– GPU PCIe link and NVLink connections within a domain. It does not extend to the broader network fabric. In multi-rack or heterogeneous deployments, inter-rack communication traverses InfiniBand or Ethernet without CC-provided encryption. NVIDIA Vera Rubin NVL72 [23] is the largest CCenabled protected domain with a full rack-scale NVLink fabric, but multi-rack and heterogeneous accelerator deployments remain outside this boundary. A single non-CCcapable node in a distributed inference cluster is sufficient to break the end-to-end isolation guarantee. CC and CloakLM therefore address complementary threat classes: CC enforces who may access the accelerator and encrypts data in motion, while CloakLM degrades the utility of raw memory contents for adversaries who operate at or below the access-control boundary. 3.3
3.4
Model Exfiltration Attacks
Model extraction via PCIe snooping with Hermes [39]. An adversary may insert a PCIe snooping device into the interconnect to intercept packets exchanged between the host and the GPU. Prior work demonstrates that PCIe packets expose physical addresses corresponding to transferred data [39]. When combined with the observation that GPU memory allocations for model parameters are typically large and contiguous, these traces enable accurate reconstruction of both model architecture and weights. Leveraging this property, along with the assumption that GPU memory allocations for model layers and parameters are largely contiguous, Hermes trains reconstruction models that can recover architectures and weights with high accuracy, even for previously unseen models. These techniques further assume that layers are transferred to the GPU in the same order in which they appear in the model graph. Model extraction via GPU device memory access with TunnelS [36]. Beyond interconnect observation, an adversary with physical access to the accelerator can directly extract GPU high-bandwidth memory (HBM) contents without interfering with normal execution. Such attacks operate via DMA-based reads over the PCIe fabric or external instrumentation of the accelerator memory interface, achieved by modifying device drivers, exploiting covert memory-access side channels, or introducing malicious OS APIs, without modifying the user-facing host software stack or executing malicious GPU kernels, and are therefore consistent with our trusted-host assumption. The resulting memory dumps expose all data resident in GPU memory, including model parameters and activations. As with PCIe-based attacks, existing reconstruction techniques rely on frameworks allocating parameters in large, contiguous HBM regions, preserving the semantic structure needed to identify and recover individual layers and tensors.
Threat Model
We consider an adversary operating in the same physical datacenter as a deployed inference service, occupying one or more of the following positions: (1) physical or firmwarelevel access to the PCIe bus or accelerator interconnect, enabling passive observation of DMA transfers; (2) a nonparticipating or compromised host on the same infrastructure, enabling bus-level reads of HBM contents via underutilization exploits or (3) non participating GPUs on the same accelerator fabric, allowing NVLink or RDMA based unauthorized memory accesses. Figure 2 illustrates these positions. We assume that the host executing the inference workload, including the CPU, operating system, hypervisor, GPU drivers, and software stack, is trusted and uncompromised. The adversary operates outside the serving stack, leveraging physical or bus-level access to observe data transfers or extract device memory contents. We explicitly exclude attacks that rely on host-side compromise, software privilege escalation, or malicious kernel execution.
4
CloakLM
4.1
Memory Management in Serving
LLM serving frameworks manage GPU memory through a multi-tiered hierarchy: vLLM orchestrates resource allocation, PyTorch handles device-level memory management, 6
CloakLM
Preprint, 2026
Inference Engine (vLLM) A A A A A A A A A A A A A A A A A A A A A A A
CloakLM
Initializer
layer 1layer 2layer 3
Default
A A A A A A
Inference Frameworks Delegate tensor management to PyTorch
Model
PyTorch Tensor Management torch.nn: torch.Tensor: self.tensors[] self.data_ptr self.forward() self.shape self.device(cuda) self.device(cuda) self.dtype
Torch leverages caching allocator to minimize fragmentation
PyTorch Memory Allocator Reserve virtual memory
1
Physical Page Allocation Queue
nn contains multiple “Tensor”. Tensors live on GPUs, PyTorch only sees their virtual address
Default
Virtual Addresses
GPU HBM
By default, layers are initialized sequentially, thus virtual address are
Page allocation & deallocation
3
Couple virtual and physical pages
Shuffle queue
Allocate random physical pages
2’
CloakLM
contiguous across layers. CloakLM shuffles the layout of the model in virtual address space
Allocate physical pages
2
Default
Inference engine like vLLM allocate model layers sequentially using torch.nn modules
CloakLM
GPU Driver APIs 1 cuMemAddressReserve 2 cuMemCreate 3 cuMemMap
GPU HBM
By default continuous virtual addresses map to consecutive physical pages with 1 2 3 CloakLM shuffles allocation requests to obfuscate model across memory with 1
2’
3
Figure 3. LLM frameworks such as vLLM use PyTorch to allocate and manage logical tensors via a CUDA caching allocator. By default, PyTorch reserves large contiguous regions of GPU virtual memory and maps it to contiguous physical pages, creating predictable patterns exploitable by memory-based attacks. In contrast, CloakLM introduces inter-layer and page-level obfuscation, randomizing both virtual-to-physical mappings and tensor layout to disrupt structural regularity while preserving inference correctness. Leveraging Virtual–Physical Address Decoupling. The decoupling of virtual and physical address spaces provides significant flexibility in how data is placed and managed within GPU memory. Applications operate exclusively on virtual addresses, while the runtime and device driver transparently manage the underlying physical page mappings. Consequently, physical memory layouts can be modified, such as rearranging, remapping, or shuffling pages, without affecting application correctness or developer-visible semantics. This indirection creates an opportunity to introduce defenses that disrupt an adversary’s view of GPU-resident data while preserving the abstraction expected by existing LLM serving frameworks. This property forms a key enabling mechanism for CloakLM.
and the GPU runtime maps virtual to physical pages in HBM. This hierarchy determines where model weights and KV caches reside, how long they persist, and which surfaces are visible to an accelerator-level adversary. Figure 3 illustrates this stack. User Frontend: vLLM. Frameworks such as vLLM expose a high-level interface that allows users to configure and instantiate LLMs with minimal effort (e.g., llm = vllm.LLMEngine (...)). Internally vLLM orchestrates a complex sequence of memory allocations and data movements, including loading model weights, managing key–value (KV) cache blocks for auto-regressive decoding, and tracking active tensors. vLLM delegates managing device memory and these low-level responsibilities to the PyTorch backend. Developer Frontend: PyTorch. PyTorch further abstracts GPU memory management by exposing tensor-based APIs that allow developers to allocate device memory using simple, declarative commands (e.g., tensor = torch.tensor(size, device='cuda')) Internally, PyTorch employs a caching allocator designed to minimize the frequency of allocation and deallocation requests issued to the GPU over the PCIe. To achieve this, PyTorch’s caching allocator exploits the separation between virtual and physical address spaces. It reserves large contiguous regions of virtual address space upfront while mapping physical HBM pages on demand. While this design improves performance, it also results in longlived physical pages that may be reused across tensors, requests, or users—an important consideration under an accelerator-level adversary.
4.2
CloakLM’s Defense
CloakLM’s goal is to ensure that raw memory observations, whether captured from the PCIe bus or extracted from HBM, do not yield a reconstructible model by introducing structured memory obfuscation during model loading and execution. The memory layout properties that make inference weights an attractive target, stable residency, contiguous allocation, predictable access order, also make them amenable to software-level obfuscation. CloakLM is designed to degrade the effectiveness and accuracy of model ex-filtration, extraction, and reconstruction attacks while preserving inference correctness and imposing negligible performance overhead. 7
Preprint, 2026
Anonymous Submission
Production LLM inference at scale typically occupies dedicated nodes, as model sizes and memory requirements preclude node-level co-tenancy for most serving configurations. In shared deployments where co-tenants exist, their traffic independently adds noise to the observable bus. Since PCIe bandwidth is a local bus resource shared only among colocated tenants, the additional traffic issued by CloakLM occupies only the idle headroom already available on the link. Because inference places modest demands on PCIe bandwidth, this additional traffic does not measurably affect inference latency or throughput. Even in deployments where interconnect congestion is disabled, the remaining two defense tiers, virtual address shuffling and physical page remapping, described below, independently disrupt weight reconstruction from raw bus or HBM observations. Inter-layer randomization via virtual address shuffling. PCIe-based extraction attacks assume that model layers are transferred to the GPU in architectural order and occupy sequentially increasing virtual addresses. CloakLM violates both assumptions by randomizing the order in which layers are instantiated and mapped during initialization, decoupling logical layer index from virtual address placement. By violating these assumptions, inter-layer randomization obscures the correspondence between observed PCIe transfers or virtual memory regions and the logical model structure. As a result, attackers are forced to solve a substantially harder reconstruction problem without a reliable notion of layer boundaries or ordering. As shown in Table 1a, default allocators produce monotonically increasing addresses that directly expose model order. An attacker observing PCIe traffic under this layout can infer that adjacent address regions correspond to consecutive layers and estimate layer sizes from address extents. In contrast, CloakLM permutes the order in which tensors are transferred and mapped, decoupling virtual address ordering from model structure. Adjacent virtual regions no longer imply adjacency in the model graph, nor do address ranges reveal layer sequencing. This shuffling is applied once at initialization, the only phase in which the full set of weight transfers crosses the PCIe bus. After initialization, weights remain resident in HBM and inference relies on a small number of stable virtual handles for kernel dispatch; re-shuffling during steadystate inference therefore provides little additional protection against interconnect-based attacks. A potential recovery vector [7] is kernel launch metadata, which could in principle reveal execution order. This signal is largely unavailable in modern LLM inference: production pipelines rely on CUDA Graphs, which capture and replay kernel sequences from a single profiling pass and expose no runtime trace of layerlevel dispatch order. Physical page remapping. Virtual address shuffling disrupts layer ordering but does not break the physical contiguity of individual tensors. CloakLM addresses this with
Figure 4. PCIe utilization for inference. Left: Bus utilization is relatively low, with the maximum utilization happening when the model weights are transferred during initialization. The peak of 6GBps observed in this case is much lower than the maximum bandwidth of the link, 32GBps. Right: Prior model extraction attacks have taken advantage of this free resource to extract the model quickly [36].
An adversary reconstructing weights from bus intercepts or HBM dumps relies on three assumptions: that layers appear in model order, that tensors occupy contiguous physical regions, and that layout persists long enough to correlate observations across time. CloakLM targets each assumption directly, operating entirely within the serving software stack without relying on cryptographic isolation or trusted hardware. It applies obfuscation at three complementary levels of the memory hierarchy: 1. Interconnect congestion: Saturates idle PCIe bandwidth to raise the cost and reduce the signal of bus-level observation. 2. Inter-layer virtual address shuffling: Randomizes the order in which layers are initialized and mapped, decoupling model structure from observable memory layout. 3. Physical page remapping: At initialization and then periodically randomizes the mapping of virtual addresses to physical HBM pages, breaking spatial contiguity and temporal consistency. Figure 3 illustrates how CloakLM modifies the default memory allocation of vLLM and PyTorch across these three levels. Interconnect obfuscation via PCIe congestion. Inference transfers model weights to HBM once at initialization and accesses them in place thereafter. As shown in Figure 4, PCIe utilization during steady-state inference remains well below link capacity, a window that attacks such as TunnelS exploit to exfiltrate HBM contents with minimal contention. CloakLM closes this window by issuing parallel, benign read and write operations on the PCIe interconnect during inference, inflating observable traffic, reducing the signal-tonoise ratio available to a snooping adversary, and introducing contention that directly slows memory dump throughput. 8
CloakLM
Layer Index 1 2 3 4 5
Preprint, 2026
Addresses Default CloakLM 0x13554 0xF7980 0x13584 0xF8280 0x135B4 0x13524 0x135E4 0x507E4 0x13614 0x207E4
Tensor (Page) T1 (P1) T1 (P2) T2 (P1) T2 (P2) T2 (P3)
Jumps Default CloakLM — — 0x00030 0x90000 0x00030 -0x3CFC0 0x00030 0x3D2C0 0x00030 -0x30000
(a) Virtual address layout. Default allocators produce a uniform 0x30 stride that exposes layer order; CloakLM shuffles layer placement to produce variable, non-informative jumps.
Page Index Default CloakLM 0x0027a 0x00134 0x0027c 0x002d4 0x0027e 0x00332 0x00280 0x002fa 0x00282 0x00330
Contiguous? Default CloakLM — — Yes No Yes No Yes No Yes No
(b) Physical page allocation. Default allocators assign sequential 2 MB pages to tensors; CloakLM remaps tensors to non-contiguous physical pages, breaking spatial layout assumptions.
Table 1. CloakLM disrupts the contiguous memory layout that reconstruction attacks depend on, at two levels of the memory hierarchy. Red indicates the predictable structure exploited by attackers; green indicates the obfuscated layout produced by CloakLM.
periodic randomization of the mapping between virtual address ranges and physical HBM pages using CUDA Virtual Memory Management (VMM) APIs. As shown in Table 1b, default allocators assign contiguous 2 MB pages to tensors sequentially; CloakLM remaps these to non-contiguous, nonsequential physical pages, eliminating the spatial regularity on which HBM dump reconstruction depends. From the inference engine’s perspective, tensor pointers and access semantics remain unchanged, CUDA VMM makes the remapping transparent to the application. An adversary observing physical HBM instead encounters fragmented, time-varying layouts that invalidate assumptions of spatial locality and temporal persistence. Physical page remapping introduces no additional memory footprint; The only overhead arises from synchronization points needed to safely remap pages, which can be amortized over long inference intervals or triggered selectively based on the anticipated attack model (e.g., continuous snooping versus one-shot memory dumping). Because inference workloads are dominated by repeated reuse of resident weights, these remapping operations do not affect steadystate throughput or inference latency. This exposes a tunable tradeoff: operators can adapt reshuffling frequency to their deployment risk without modifying the model or inference stack. We evaluate this tradeoff in Section 5 showing that physical page shuffling imposes negligible overhead on endto-end inference latency when performed once every 1,000 inference iterations, relative to the scale of billions of inference requests in production serving deployments. Performance impact of memory obfuscation. LLM inference issues large, streaming memory requests spanning gigabytes of weights and KV caches. Although DRAM access latency can be affected by bank conflicts and row-buffer misses, these effects are most pronounced under small, serialized accesses. LLM inference is bandwidth-bound: kernels saturate HBM bandwidth across many parallel requests rather than
exploiting physical page contiguity. When CloakLM randomizes physical page placement, large accesses distribute naturally across HBM banks, maintaining uniform bandwidth utilization. Section 5 confirms empirically that physical page remapping incurs negligible impact on end-to-end inference latency.
5
Evaluation
We evaluate CloakLM against two concrete prior attacks, a memory dump attack based on TunnelS [36] and a PCIe snooping attack based on Hermes [39]. We evaluate our strategy across three broad metrics: (i) accuracy of model extracted, (ii) impact of extraction process, and (iii) latency impact on end-to-end inference latency. Setup and Hardware. We perform our experiments on L40s GPUs with 46GB HBM memory and PCIe 4.0 connectivity to the CPU with 16 lanes (32GB/s uni-directional bandwidth). We conduct our experiments on three models: Llama 3.1 (8B parameters), Qwen 3 (14B parameters) and Qwen 3 MoE (30B parameters, 3B active parameters) for the attacks using vLLM framework, evaluate that CloakLM significantly reduces the risk of model exfiltration and has no adverse effects on the end-to-end latency of LLMs. Our set up is detailed in Table 2. Implementation Details. CloakLM integrates its defence strategies directly into the PyTorch and vLLM codebases, exposing a simple one-line API for service providers in ∼300 lines of code. All changes are backwards compatible across vLLM versions. Since the host is trusted, we seed the PRNG from /dev/urandom prior to deployment: virtual addresses are enumerated, a random permutation is computed, and physical pages are allocated accordingly, all before any weight transfer ensuring the attacker never observes a structured PCIe memory layout. 5.1
Attack Evaluation
Threat assumptions. Existing PCIe- and memory- based attacks rely on two key structural assumptions: (1) model 9
Preprint, 2026
Anonymous Submission
Model Model Parameters Tensor Virtual-Physical Contiguity Shuffling Name Type #Billion Parallelism Default CloakLM Latency(s) Llama 3.1 Dense 8 1 91.41% 2.38% 1.58 Qwen 3 Dense 14 1 95.57% 1.57% 1.61 Qwen 3 Dense 14 2 2.81 Qwen 3 MoE 30 2 2.95 Table 2. We evaluate 3 models across 4 configurations (dense and MoE, varying tensor parallelism), reporting the percentage of virtual address ranges backed by contiguous physical pages under the default and CloakLM allocator. CloakLM reduces virtual–physical contiguity from over 90% to below 3%, disrupting layout-based reconstruction. Since shuffling is performed independently per GPU, similar reductions hold under TP2. Shuffling latency remains modest at 1.5–3 seconds per configuration.
Figure 5. Iteration-level prefill and decode latency with and without CloakLM. Across all evaluated models and parallelism settings, CloakLM incurs no measurable runtime overhead, with latency distributions closely matching default execution. or I/O behavior [22, 31, 36, 39]. Our goal is to prevent reconstruction even under strong attacker knowledge. HBM dump attack based on TunnelS [36]. TunnelS demonstrates that full HBM exfiltration is achievable from a non-participating host using a modified NVIDIA driver, without access to the victim host or its workload, by exploiting idle PCIe bandwidth during inference. We obtain GPU memory dumps using the technique described in TunnelS, then attempt reconstruction by grouping physically contiguous HBM pages into segments and enumerating permutations until a candidate ordering reproduces the known correct output. Under default vLLM, we successfully reconstruct models up to 11 GB (16 LLaMA decode layer) within approximately eight hours, as shown in Figure 6. Under CloakLM, physical page dispersion eliminates stable contiguous segments entirely, rendering reconstruction infeasible within the same time budget for models as small as 64 MB. CloakLM therefore removes the structural foundations that state-of-the-art reconstruction attacks depend on, even when the attacker has strong prior knowledge of the model. PCIe Snooping Attack based on Hermes [39]. Hermes [39] demonstrates that a passive adversary with physical access to the PCIe bus can reconstruct both the architecture and weights of a DNN model hosted on a GPU, without any interaction with the victim system. The adversary interposes a snooping device, such as a PCIe interposer or rogue
Figure 6. Extracting models larger than 64 MB in size is impractical with CloakLM. However, it is possible to extract models as large as 11 GBs with naive methods.
parameters are allocated in large, contiguous memory regions and (2) the attacker can recover meaningful virtual-tophysical layout information. Under these conditions, attackers exploit layout regularities to identify layer boundaries and reconstruct parameters. Unless stated otherwise, we assume the attacker knows the target model architecture and has access to at least one correct input–output pair for validation. This gives the adversary maximal leverage and represents a best-case setting for reconstruction. Our defense does not rely on obscuring the architecture: prior work has shown that architectures can often be inferred from memory traces, kernel launches, 10
CloakLM
Preprint, 2026
peripheral, on the host-to-device interconnect, capturing raw DMA transfer packets as tensors (activations, weights, gradients) flow between host memory and the GPU during inference. By reverse-engineering the traffic patterns (packet sizes, transfer sequences, timing intervals) Hermes reconstructs the layer topology and ultimately recovers full model weights with lossless fidelity. The attack is entirely passive, requires no software foothold on the victim host, and leaves no detectable trace in system logs. Fully reproducing this attack requires custom PCIe snooping hardware. We instead emulate it by (i) instrumenting the PyTorch/CUDA runtime to capture tensor transfer traces during DNN inference, recording the sequence, size, and timing of every host-to-device and device-to-host DMA transfer, and (ii) replaying these traces through a packet generation model that faithfully reconstructs the PCIe traffic stream as it would appear to a physical interposer, following the threat model described in Hermes. This emulation allows us to evaluate reconstruction fidelity and measure the effect of CloakLM’s defenses without requiring physical snooping hardware. We evaluate reconstruction fidelity under three conditions: (i) default serving with no defenses; (ii) CloakLM with virtual memory shuffling only; and (iii) CloakLM with both virtual shuffling and PCIe congestion. Under default serving, we reproduce the Hermes result: lossless reconstruction of a CNN-based binary classifier. Virtual memory shuffling alone defeats the attack, degrading classifier accuracy from ∼0.9 to ∼0.5 and collapsing the reconstructed model to near-random performance, by destroying the correspondence between observed DMA addresses and model layers. Adding PCIe congestion introduces structured noise into the transfer stream, perturbing packet timing and sizes to further degrade any residual signal available to a more adaptive adversary.
5.2
35.5 s for 30B), as shown in Figure 8. Virtual address shuffling introduces negligible initialization overhead. Physical page shuffling requires approximately 1.5 s per GPU and is performed independently on each GPU in multi-GPU deployments, as reported in Table 2. Shuffling Overhead in Multi-GPU Environments. Table 2 shows that HBM shuffling incurs modest overhead, requiring approximately 1.5 seconds per GPU on average. In multi-GPU deployments, shuffling is currently performed independently on each GPU. Virtual and Physical Address Shuffling. Virtual address shuffling does not affect model upload time or steady-state iteration latency. Across all evaluated models and parallelism configurations, performance remains unchanged, as confirmed by Figure 8. However, as illustrated in Table 1a, the virtual addresses corresponding to model layers are permuted, disrupting their original ordering and significantly increasing the difficulty of inferring layers from PCIe traffic. Physical memory shuffling further fragments the memory layout. As shown in Table 2, the number of contiguous virtual address ranges backed by contiguous physical pages drops substantially under CloakLM. This fragmentation directly undermines key assumptions made by prior model reconstruction attacks that rely on large contiguous memory regions to infer model structure. Configurable security–overhead tradeoff. As shown in Figure 7, shuffling frequency directly governs the security– overhead tradeoff: more frequent reshuffling provides stronger protection against continuous observation but increases amortized overhead. In practice, a single shuffle at initialization is sufficient to disrupt one-shot HBM dump attacks, while periodic reshuffling can be enabled to defend against adversaries with sustained observation capability. Operators can tune reshuffling frequency to match their deployment risk profile without modifying the model or inference stack.
Performance Evaluation
Inference latency. As shown in Figure 5, physical memory shuffling introduces no measurable overhead to per-iteration prefill or decode latency across all evaluated models and configurations, dense, MoE, single-GPU, and multi-GPU. Figure 7 confirms that performance degradation is confined to the shuffling operation itself and does not affect steady-state execution. Moreover, since token packets during inference are small (on the order of MBs), PCIe congestion likewise introduces no measurable degradation in time-to-first-token or time-between-tokens. Initialization overhead. PCIe congestion increases link utilization to approximately 30 GB/s during model loading, as shown in Figure 9. This directly slows PCIe-based extraction: the time required to exfiltrate a 48 GB model increases from approximately 45 seconds to over 8 minutes. The corresponding initialization overhead scales linearly with model size at approximately 1.1 s/GB (9.1 s for 8B, 16.1 s for 14B,
6 6.1
Discussion Security Guarantees and Non-Goals
In security, as new defenses are deployed, adversaries adapt and develop increasingly sophisticated attacks. Our goal is not to claim absolute protection against model exfiltration, but to raise the cost and complexity of practical attacks that exist today or are feasible under current deployment models, specifically, accelerator-centric attacks that exploit visibility into interconnect traffic and device memory during inference. We acknowledge that determined adversaries may eventually develop techniques that circumvent our defenses. We focus on inference-time threats. During inference, model weights are static, long-lived, and repeatedly accessed, making the network fabric and accelerator memory hierarchy the most exposed attack surfaces for model exfiltration. 11
Preprint, 2026
Anonymous Submission
Figure 7. We evaluate workloads across a range of QPS values and input/output sequence lengths uniformly sampled between 100 and 200 tokens. End-to-end performance remains stable when physical shuffling is performed once every 1,500 iterations. Performance degradation increases proportionally with shuffling frequency. In practice, a single shuffle during initialization is sufficient to obfuscate HBM contents, while additional runtime shuffling can be enabled to provide stronger security guarantees. and low-noise interconnect traffic, thereby degrading attack accuracy and increasing reconstruction effort. A key design constraint is that defenses must preserve inference correctness and performance, avoiding mechanisms that introduce measurable latency, reduce throughput, or require changes to model execution semantics. CloakLM thus bridges the gap between purely performance-driven systems and heavyweight confidential computing solutions such as full memory encryption or trusted accelerator enclaves. Figure 8. Impact of CloakLM on model initialization and loading latency. PCIe congestion increases loading time proportionally to model size, while virtual shuffling introduces negligible overhead.
(a) Normal execution.
6.2
Out-of-Scope Attack Vectors
The following attack vectors are orthogonal to the focus of this paper. Timing-based attacks. These attacks exploit congestion and co-tenancy on shared network components to infer victim model properties [14, 28], injecting PCIe traffic and measuring latency variations across inference stack peripherals (host–device links, RDMA NICs, NVMe SSDs). Such techniques are limited to coarse properties like model size or architecture, making them substantially weaker than the weight-recovering attacks discussed above. Timing side channels over NVLink have also been explored [6, 35]. Residual memory and cache attacks. These attacks exploit residual memory state or shared caches under cotenancy, relying on insufficient memory isolation or incomplete hardware state clearing in public cloud environments [12, 27]. While powerful, they assume logical colocation rather than physical access to interconnects or device memory. Other side-channel attacks. Additional side channels include power consumption [8], GPU context-switch behavior [31], and application-level observables [22], typically leaking indirect information under strong assumptions about workload scheduling or observability. Participating host-side attacks. CPU-based attacks spanning memory [21], cache [34, 37], and microarchitectural channels [38] are extensively studied, with well-established
(b) Model extraction.
Figure 9. PCIe traffic with interconnect congestion. Both look same and model extraction is severely delayed, from 45 seconds to extract 48GBs to over 8 minutes! Training workloads, by contrast, continuously update parameters, rendering intermediate snapshots transient and requiring fundamentally different threat models; protecting against training-time exfiltration is left for future work. Rather than aiming for cryptographic secrecy, CloakLM focuses on structural obfuscation: breaking the assumptions that existing extraction attacks rely on, such as contiguous memory layouts, stable layer ordering, 12
CloakLM
Preprint, 2026
defenses [9, 18]. We explicitly assume the host executing inference is secure.
7
[11] Zhongshu Gu, Enriquillo Valdez, Salman Ahmed, Julian James Stephen, Michael Le, Hani Jamjoom, Shixuan Zhao, and Zhiqiang Lin. 2025. NVIDIA GPU Confidential Computing Demystified. arXiv:2507.02770 [cs.CR] https://arxiv.org/abs/2507.02770 [12] Yanan Guo, Zhenkai Zhang, and Jun Yang. 2024. GPU Memory Exploitation for Fun and Profit. In 33rd USENIX Security Symposium (USENIX Security 24). USENIX Association, Philadelphia, PA, 4033–4050. https://www.usenix.org/conference/usenixsecurity24/ presentation/guo-yanan [13] Red Hat. 2024. CVE-2024-3094. https://access.redhat.com/security/ cve/cve-2024-3094 [14] Xing Hu, Ling Liang, Shuangchen Li, Lei Deng, Pengfei Zuo, Yu Ji, Xinfeng Xie, Yufei Ding, Chang Liu, Timothy Sherwood, and Yuan Xie. 2020. DeepSniffer: A DNN Model Extraction Framework Based on Learning Architectural Hints. In Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems (Lausanne, Switzerland) (ASPLOS ’20). Association for Computing Machinery, New York, NY, USA, 385–399. doi:10.1145/3373376.3378460 [15] Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2023. Mistral 7B. arXiv:2310.06825 [cs.CL] https://arxiv.org/abs/2310.06825 [16] Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, MarieAnne Lachaux, Pierre Stock, Sandeep Subramanian, Sophia Yang, Szymon Antoniak, Teven Le Scao, Théophile Gervet, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2024. Mixtral of Experts. arXiv:2401.04088 [cs.LG] https://arxiv.org/abs/2401.04088 [17] Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient Memory Management for Large Language Model Serving with PagedAttention. In Proceedings of the 29th Symposium on Operating Systems Principles (Koblenz, Germany) (SOSP ’23). Association for Computing Machinery, New York, NY, USA, 611–626. doi:10.1145/3600006.3613165 [18] Fangfei Liu, Hao Wu, Kenneth Mai, and Ruby B. Lee. 2016. Newcache: Secure Cache Architecture Thwarting Cache Side-Channel Attacks. IEEE Micro 36, 5 (2016), 8–16. doi:10.1109/MM.2016.85 [19] McKinsey and Company. 2025. The next big shifts in AI workloads and hyperscaler strategies. https://www.mckinsey.com/industries/ technology-media-and-telecommunications/our-insights/the-nextbig-shifts-in-ai-workloads-and-hyperscaler-strategies [20] Meta. 2025. The Llama 4 herd: The beginning of a new era of natively multimodal AI innovation. https://ai.meta.com/blog/llama-4multimodal-intelligence/ [21] Onur Mutlu and Jeremie S. Kim. 2020. RowHammer: A Retrospective. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 39, 8 (2020), 1555–1571. doi:10.1109/TCAD.2019.2915318 [22] Hoda Naghibijouybari, Ajaya Neupane, Zhiyun Qian, and Nael AbuGhazaleh. 2018. Rendered Insecure: GPU Side Channel Attacks are Practical. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security (Toronto, Canada) (CCS ’18). Association for Computing Machinery, New York, NY, USA, 2139–2153. doi:10.1145/3243734.3243831 [23] NVIDIA. 2026. https://www.nvidia.com/en-us/data-center/solutions/ confidential-computing/ [24] Zhenghang Ren, Yuxuan Li, Zilong Wang, Xinyang Huang, Wenxue Li, Kaiqiang Xu, Xudong Liao, Yijun Sun, Bowen Liu, Han Tian, Junxue Zhang, Mingfei Wang, Zhizhen Zhong, Guyue Liu, Ying Zhang, and
Conclusions
This paper identifies inference-time model exfiltration as a critical and underexplored security risk in accelerator-centric AI deployments. As model providers increasingly rely on third-party and disaggregated infrastructure, model weights are exposed to interconnect- and memory-level attacks that bypass traditional software isolation. We present CloakLM, a lightweight, performancepreserving defense framework that disrupts the structural assumptions exploited by existing extraction attacks. By combining interconnect congestion, randomized layer initialization, and dynamic GPU page remapping, CloakLM obfuscates memory layouts and data movement patterns without modifying inference kernels or degrading throughput. Our results show that these techniques substantially raise the cost and complexity of model reconstruction while maintaining production-grade performance. We hope CloakLM motivates further exploration of systems-level defenses for model IP protection in shared AI infrastructure.
References [1] Anthropic. 2025. Claude Sonnet 4.5. https://www.anthropic.com/ news/claude-sonnet-4-5 [2] Marcin Chrapek, Marcin Copik, Etienne Mettaz, and Torsten Hoefler. 2025. Confidential LLM Inference: Performance and Cost Across CPU and GPU TEEs. arXiv:2509.18886 [cs.PF] https://arxiv.org/abs/2509. 18886 [3] Carbon Credits. 2026. NVIDIA Controls 92% of the GPU Market in 2025 and Reveals Next Gen AI Supercomputer. [4] Michael Davies, Neal Crago, Karthikeyan Sankaralingam, and Christos Kozyrakis. 2025. LIMINAL: Exploring The Frontiers of LLM Decode Performance. arXiv:2507.14397 [cs.AR] https://arxiv.org/abs/2507. 14397 [5] Zachary DeVito. 2022. A guide to PyTorch’s CUDA Caching Allocator. https://zdevito.github.io/2022/08/04/cuda-caching-allocator.html [6] Sankha Baran Dutta, Hoda Naghibijouybari, Arjun Gupta, Nael AbuGhazaleh, Andres Marquez, and Kevin Barker. 2023. Spy in the GPUbox: Covert and Side Channel Attacks on Multi-GPU Systems. In Proceedings of the 50th Annual International Symposium on Computer Architecture (Orlando, FL, USA) (ISCA ’23). Association for Computing Machinery, New York, NY, USA, Article 45, 13 pages. doi:10.1145/ 3579371.3589080 [7] Jonah Ekelund, Stefano Markidis, and Ivy Peng. 2025. Boosting Performance of Iterative Applications on GPUs: Kernel Batching with CUDA Graphs. In 2025 33rd Euromicro International Conference on Parallel, Distributed, and Network-Based Processing (PDP). 70–77. doi:10.1109/PDP66500.2025.00019 [8] Yansong Gao, Huming Qiu, Zhi Zhang, Binghui Wang, Hua Ma, Alsharif Abuadbba, Minhui Xue, Anmin Fu, and Surya Nepal. 2024. DeepTheft: Stealing DNN Model Architectures through Power Side Channel. In 2024 IEEE Symposium on Security and Privacy (SP). 3311– 3326. doi:10.1109/SP54263.2024.00250 [9] Michael Godfrey and Mohammad Zulkernine. 2014. Preventing cachebased side-channel attacks in a cloud environment. IEEE Transactions on Cloud Computing 2, 4 (2014), 395–408. doi:10.1109/TCC.2014. 2358236 [10] Google. 2025. Gemini. https://gemini.google.com/ 13
Preprint, 2026
Anonymous Submission
Kai Chen. 2025. Enabling efficient GPU communication over multiple NICs with FuseLink. In Proceedings of the 19th USENIX Conference on Operating Systems Design and Implementation (Boston, MA, USA) (OSDI ’25). USENIX Association, USA, Article 6, 18 pages. [25] Benjamin Rothenberger, Konstantin Taranov, Adrian Perrig, and Torsten Hoefler. 2021. ReDMArk: Bypassing RDMA Security Mechanisms. In 30th USENIX Security Symposium (USENIX Security 21). USENIX Association, 4277–4292. https://www.usenix.org/conference/ usenixsecurity21/presentation/rothenberger [26] Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El-Kishky, Aidan McLaughlin, Aiden Low, AJ Ostrow, Akhila Ananthram, Akshay Nathan, Alan Luo, Alec Helyar, Aleksander Madry, Aleksandr Efremov, Aleksandra Spyra, Alex Baker-Whitcomb, Alex Beutel, Alex Karpenko, Alex Makelov, Alex Neitz, Alex Wei, Alexandra Barr, Alexandre Kirchmeyer, Alexey Ivanov, Alexi Christakis, Alistair Gillespie, Allison Tam, Ally Bennett, Alvin Wan, Alyssa Huang, Amy McDonald Sandjideh, Amy Yang, Ananya Kumar, Andre Saraiva, Andrea Vallone, Andrei Gheorghe, Andres Garcia Garcia, Andrew Braunstein, Andrew Liu, Andrew Schmidt, Andrey Mereskin, Andrey Mishchenko, Andy Applebaum, Andy Rogerson, Ann Rajan, Annie Wei, Anoop Kotha, Anubha Srivastava, Anushree Agrawal, Arun Vijayvergiya, Ashley Tyra, Ashvin Nair, Avi Nayak, Ben Eggers, Bessie Ji, Beth Hoover, Bill Chen, Blair Chen, Boaz Barak, Borys Minaiev, Botao Hao, Bowen Baker, Brad Lightcap, Brandon McKinzie, Brandon Wang, Brendan Quinn, Brian Fioca, Brian Hsu, Brian Yang, Brian Yu, Brian Zhang, Brittany Brenner, Callie Riggins Zetino, Cameron Raymond, Camillo Lugaresi, Carolina Paz, Cary Hudson, Cedric Whitney, Chak Li, Charles Chen, Charlotte Cole, Chelsea Voss, Chen Ding, Chen Shen, Chengdu Huang, Chris Colby, Chris Hallacy, Chris Koch, Chris Lu, Christina Kaplan, Christina Kim, CJ Minott-Henriques, Cliff Frey, Cody Yu, Coley Czarnecki, Colin Reid, Colin Wei, Cory Decareaux, Cristina Scheau, Cyril Zhang, Cyrus Forbes, Da Tang, Dakota Goldberg, Dan Roberts, Dana Palmie, Daniel Kappler, Daniel Levine, Daniel Wright, Dave Leo, David Lin, David Robinson, Declan Grabb, Derek Chen, Derek Lim, Derek Salama, Dibya Bhattacharjee, Dimitris Tsipras, Dinghua Li, Dingli Yu, DJ Strouse, Drew Williams, Dylan Hunn, Ed Bayes, Edwin Arbus, Ekin Akyurek, Elaine Ya Le, Elana Widmann, Eli Yani, Elizabeth Proehl, Enis Sert, Enoch Cheung, Eri Schwartz, Eric Han, Eric Jiang, Eric Mitchell, Eric Sigler, Eric Wallace, Erik Ritter, Erin Kavanaugh, Evan Mays, Evgenii Nikishin, Fangyuan Li, Felipe Petroski Such, Filipe de Avila Belbute Peres, Filippo Raso, Florent Bekerman, Foivos Tsimpourlas, Fotis Chantzis, Francis Song, Francis Zhang, Gaby Raila, Garrett McGrath, Gary Briggs, Gary Yang, Giambattista Parascandolo, Gildas Chabot, Grace Kim, Grace Zhao, Gregory Valiant, Guillaume Leclerc, Hadi Salman, Hanson Wang, Hao Sheng, Haoming Jiang, Haoyu Wang, Haozhun Jin, Harshit Sikchi, Heather Schmidt, Henry Aspegren, Honglin Chen, Huida Qiu, Hunter Lightman, Ian Covert, Ian Kivlichan, Ian Silber, Ian Sohl, Ibrahim Hammoud, Ignasi Clavera, Ikai Lan, Ilge Akkaya, Ilya Kostrikov, Irina Kofman, Isak Etinger, Ishaan Singal, Jackie Hehir, Jacob Huh, Jacqueline Pan, Jake Wilczynski, Jakub Pachocki, James Lee, James Quinn, Jamie Kiros, Janvi Kalra, Jasmyn Samaroo, Jason Wang, Jason Wolfe, Jay Chen, Jay Wang, Jean Harb, Jeffrey Han, Jeffrey Wang, Jennifer Zhao, Jeremy Chen, Jerene Yang, Jerry Tworek, Jesse Chand, Jessica Landon, Jessica Liang, Ji Lin, Jiancheng Liu, Jianfeng Wang, Jie Tang, Jihan Yin, Joanne Jang, Joel Morris, Joey Flynn, Johannes Ferstad, Johannes Heidecke, John Fishbein, John Hallman, Jonah Grant, Jonathan Chien, Jonathan Gordon, Jongsoo Park, Jordan Liss, Jos Kraaijeveld, Joseph Guay, Joseph Mo, Josh Lawson, Josh McGrath, Joshua Vendrow, Joy Jiao, Julian Lee, Julie Steele, Julie Wang, Junhua Mao, Kai Chen, Kai Hayashi, Kai Xiao, Kamyar Salahi, Kan Wu, Karan Sekhri, Karan Sharma, Karan Singhal, Karen Li, Kenny Nguyen, Keren Gu-Lemberg, Kevin King, Kevin Liu, Kevin Stone, Kevin Yu, Kristen Ying, Kristian Georgiev, Kristie Lim, Kushal Tirumala, Kyle Miller, Lama Ahmad, Larry Lv, Laura Clare, Laurance Fauconnet, Lauren Itow, Lauren
Yang, Laurentia Romaniuk, Leah Anise, Lee Byron, Leher Pathak, Leon Maksin, Leyan Lo, Leyton Ho, Li Jing, Liang Wu, Liang Xiong, Lien Mamitsuka, Lin Yang, Lindsay McCallum, Lindsey Held, Liz Bourgeois, Logan Engstrom, Lorenz Kuhn, Louis Feuvrier, Lu Zhang, Lucas Switzer, Lukas Kondraciuk, Lukasz Kaiser, Manas Joglekar, Mandeep Singh, Mandip Shah, Manuka Stratta, Marcus Williams, Mark Chen, Mark Sun, Marselus Cayton, Martin Li, Marvin Zhang, Marwan Aljubeh, Matt Nichols, Matthew Haines, Max Schwarzer, Mayank Gupta, Meghan Shah, Melody Huang, Meng Dong, Mengqing Wang, Mia Glaese, Micah Carroll, Michael Lampe, Michael Malek, Michael Sharman, Michael Zhang, Michele Wang, Michelle Pokrass, Mihai Florian, Mikhail Pavlov, Miles Wang, Ming Chen, Mingxuan Wang, Minnia Feng, Mo Bavarian, Molly Lin, Moose Abdool, Mostafa Rohaninejad, Nacho Soto, Natalie Staudacher, Natan LaFontaine, Nathan Marwell, Nelson Liu, Nick Preston, Nick Turley, Nicklas Ansman, Nicole Blades, Nikil Pancha, Nikita Mikhaylin, Niko Felix, Nikunj Handa, Nishant Rai, Nitish Keskar, Noam Brown, Ofir Nachum, Oleg Boiko, Oleg Murk, Olivia Watkins, Oona Gleeson, Pamela Mishkin, Patryk Lesiewicz, Paul Baltescu, Pavel Belov, Peter Zhokhov, Philip Pronin, Phillip Guo, Phoebe Thacker, Qi Liu, Qiming Yuan, Qinghua Liu, Rachel Dias, Rachel Puckett, Rahul Arora, Ravi Teja Mullapudi, Raz Gaon, Reah Miyara, Rennie Song, Rishabh Aggarwal, RJ Marsan, Robel Yemiru, Robert Xiong, Rohan Kshirsagar, Rohan Nuttall, Roman Tsiupa, Ronen Eldan, Rose Wang, Roshan James, Roy Ziv, Rui Shu, Ruslan Nigmatullin, Saachi Jain, Saam Talaie, Sam Altman, Sam Arnesen, Sam Toizer, Sam Toyer, Samuel Miserendino, Sandhini Agarwal, Sarah Yoo, Savannah Heon, Scott Ethersmith, Sean Grove, Sean Taylor, Sebastien Bubeck, Sever Banesiu, Shaokyi Amdo, Shengjia Zhao, Sherwin Wu, Shibani Santurkar, Shiyu Zhao, Shraman Ray Chaudhuri, Shreyas Krishnaswamy, Shuaiqi, Xia, Shuyang Cheng, Shyamal Anadkat, Simón Posada Fishman, Simon Tobin, Siyuan Fu, Somay Jain, Song Mei, Sonya Egoian, Spencer Kim, Spug Golden, SQ Mah, Steph Lin, Stephen Imm, Steve Sharpe, Steve Yadlowsky, Sulman Choudhry, Sungwon Eum, Suvansh Sanjeev, Tabarak Khan, Tal Stramer, Tao Wang, Tao Xin, Tarun Gogineni, Taya Christianson, Ted Sanders, Tejal Patwardhan, Thomas Degry, Thomas Shadwell, Tianfu Fu, Tianshi Gao, Timur Garipov, Tina Sriskandarajah, Toki Sherbakov, Tomer Kaftan, Tomo Hiratsuka, Tongzhou Wang, Tony Song, Tony Zhao, Troy Peterson, Val Kharitonov, Victoria Chernova, Vineet Kosaraju, Vishal Kuo, Vitchyr Pong, Vivek Verma, Vlad Petrov, Wanning Jiang, Weixing Zhang, Wenda Zhou, Wenlei Xie, Wenting Zhan, Wes McCabe, Will DePue, Will Ellsworth, Wulfie Bain, Wyatt Thompson, Xiangning Chen, Xiangyu Qi, Xin Xiang, Xinwei Shi, Yann Dubois, Yaodong Yu, Yara Khakbaz, Yifan Wu, Yilei Qian, Yin Tat Lee, Yinbo Chen, Yizhen Zhang, Yizhong Xiong, Yonglong Tian, Young Cha, Yu Bai, Yu Yang, Yuan Yuan, Yuanzhi Li, Yufeng Zhang, Yuguang Yang, Yujia Jin, Yun Jiang, Yunyun Wang, Yushi Wang, Yutian Liu, Zach Stubenvoll, Zehao Dou, Zheng Wu, and Zhigang Wang. 2025. OpenAI GPT-5 System Card. arXiv:2601.03267 [cs.CL] https://arxiv.org/abs/2601.03267 [27] Tyler Sorensen and Heidy Khlaaf. 2024. LeftoverLocals: Listening to LLM Responses Through Leaked GPU Local Memory. arXiv:2401.16603 [cs.CR] https://arxiv.org/abs/2401.16603 [28] Mingtian Tan, Junpeng Wan, Zhe Zhou, and Zhou Li. 2021. Invisible Probe: Timing Attacks with PCIe Congestion Side-channel. In 2021 IEEE Symposium on Security and Privacy (SP). 322–338. doi:10.1109/ SP40001.2021.00059 [29] Yifan Tan and Zeyu Mi. 2024. Performance Analysis and Optimization of Nvidia H100 Confidential Computing for AI Workloads. In 2024 IEEE International Symposium on Parallel and Distributed Processing with Applications (ISPA). 1426–1432. doi:10.1109/ISPA63168.2024.00192 [30] National Telecommunications and Information Administration. [n. d.]. Dual-Use Foundation Models With Widely Available Model Weights Report. https://www.ntia.gov/programs-and-initiatives/artificialintelligence/open-model-weights-report 14
CloakLM
Preprint, 2026
[36] Zhenkai Zhang, Tyler Allen, Fan Yao, Xing Gao, and Rong Ge. 2023. TunneLs for Bootlegging: Fully Reverse-Engineering GPU TLBs for Challenging Isolation Guarantees of NVIDIA MIG. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security (Copenhagen, Denmark) (CCS ’23). Association for Computing Machinery, New York, NY, USA, 960–974. doi:10.1145/3576915.3616672 [37] Zirui Neil Zhao, Adam Morrison, Christopher W. Fletcher, and Josep Torrellas. 2024. Last-Level Cache Side-Channel Attacks Are Feasible in the Modern Public Cloud. In Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2 (La Jolla, CA, USA) (ASPLOS ’24). Association for Computing Machinery, New York, NY, USA, 582–600. doi:10.1145/3620665.3640403 [38] Zirui Neil Zhao, Adam Morrison, Christopher W. Fletcher, and Josep Torrellas. 2025. From Colocation to Exfiltration: Practical Cache SideChannel Attacks in the Modern Public Cloud. IEEE Micro 45, 4 (2025), 95–102. doi:10.1109/MM.2025.3574715 [39] Yuankun Zhu, Yueqiang Cheng, Husheng Zhou, and Yantao Lu. 2021. Hermes Attack: Steal DNN Models with Lossless Inference Accuracy. In 30th USENIX Security Symposium (USENIX Security 21). USENIX Association, 1973–1988. https://www.usenix.org/conference/ usenixsecurity21/presentation/zhu
[31] Junyi Wei, Yicheng Zhang, Zhe Zhou, Zhou Li, and Mohammad Abdullah Al Faruque. 2020. Leaky DNN: Stealing Deep-Learning Model Secret with GPU Context-Switching Side-Channel. In 2020 50th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN). 125–137. doi:10.1109/DSN48063.2020.00031 [32] Zhenyu Wu, Zhang Xu, and Haining Wang. 2012. Whispers in the hyper-space: high-speed covert channel attacks in the cloud. In Proceedings of the 21st USENIX Conference on Security Symposium (Bellevue, WA) (Security’12). USENIX Association, USA, 9. [33] Yuval Yarom and Katrina Falkner. 2014. FLUSH+RELOAD: a high resolution, low noise, L3 cache side-channel attack. In Proceedings of the 23rd USENIX Conference on Security Symposium (San Diego, CA) (SEC’14). USENIX Association, USA, 719–732. [34] Yinqian Zhang, Ari Juels, Michael K. Reiter, and Thomas Ristenpart. 2014. Cross-Tenant Side-Channel Attacks in PaaS Clouds. In Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security (Scottsdale, Arizona, USA) (CCS ’14). Association for Computing Machinery, New York, NY, USA, 990–1003. doi:10.1145/2660267.2660356 [35] Yicheng Zhang, Ravan Nazaraliyev, Sankha Baran Dutta, Andres Marquez, Kevin Barker, and Nael Abu-Ghazaleh. 2025. NVBleed: Covert and Side-Channel Attacks on NVIDIA Multi-GPU Interconnect. arXiv:2503.17847 [cs.CR] https://arxiv.org/abs/2503.17847
15