Lifecycle-Aware Dynamic Analysis for Secure ML Model Execution Gabriele Digregorio∗ Politecnico di Milano
Marco Di Gennaro∗ Politecnico di Milano
Francesco Pastore Politecnico di Milano
arXiv:2606.19023v1 [cs.CR] 17 Jun 2026
Stefano Longari Politecnico di Milano
Michele Carminati Politecnico di Milano
Abstract
ecuting ML models obtained from public repositories or shared through other channels. Recent research has highlighted the risks associated with running untrusted models [22, 34, 35, 46, 72], while major model-sharing platforms (e.g., the Hugging Face Hub) have increasingly expanded their set of built-in or third-party ML model artifact scanners to mitigate these threats [10, 30, 33, 54, 69]. The risks posed by malicious models range from unauthorized file access and data exfiltration to arbitrary code execution [11–18]. As a result, downloading and running an untrusted ML model increasingly resembles executing untrusted software at large. Although malicious software detection is a mature and well-studied area, the problem of ML model scanning remains underexplored. Existing model scanners provide limited and fragile protection. Most tools [38,68] either focus narrowly on Pickle-based artifacts or rely on static, signature-driven rules that struggle to generalize across different file formats (e.g., .pt, .keras), ML frameworks (e.g., Pytorch, TensorFlow, and Keras), and evolving attack techniques. Consequently, these scanners are largely reactive, detecting known attack patterns while failing to identify new attacks that exploit previously unexplored techniques or abuse framework features [72]. Empirical studies [7, 22] further show that current solutions suffer from false negatives and false positives in real-world deployments. As a result, recent works [22] have emphasized the need to draw inspiration from decades of research in malware detection when addressing ML model security. At the same time, it is crucial to account for the distinctive properties that distinguish ML models from generic software.
The growing reliance on pre-trained Machine Learning (ML) models has introduced new attack surfaces. Recent vulnerabilities demonstrate that malicious behavior can be embedded within model artifacts, often bypassing existing defenses. Current model-scanning solutions primarily rely on static, format-specific rules or known attack signatures, which limit their ability to generalize across frameworks and to detect novel exploitation paths. In contrast, we propose a solution that focuses on the effects an attack has on the host system executing the model and builds on foundational intuitions about ML model execution. In particular, we observe that ML models operate within well-defined lifecycle phases and that, within each phase, interactions with the host system are highly structured and predictable. We translate these intuitions into M OAT, a dynamic lifecycle-aware approach for securing ML model execution, and instantiate this design in R E -M OAT, our reference implementation. We evaluate R E -M OAT across multiple ML frameworks using 77,974 real-world model artifacts from the Hugging Face Hub, 31 Proofs-of-Concept (PoCs) from CVEs, and 334 models from a state-of-the-art dataset, and compare it against state-of-the-art model-scanning solutions. Our results show that our approach detects all evaluated attack classes while maintaining a close-to-zero false-positive rate, validating our intuitions and motivating dynamic analysis for securing ML model execution.
1
Stefano Zanero Politecnico di Milano
Introduction
The adoption of Machine Learning (ML) is growing rapidly, driven by platforms for sharing pre-trained models [29, 31, 37, 59]. This development mirrors traditional software engineering, where open-source repositories facilitate the reuse and adaptation of code. Similarly, today’s ML ecosystem relies on the sharing and reuse of model artifacts. Despite the clear advantages, this transition has raised concerns about the security implications of loading and ex-
Our Intuition. Malicious ML models can be effectively identified by the effects they induce on the host system at runtime (dynamic analysis), leveraging the characteristic properties of ML models. This runtime perspective can complement static analysis, addressing cases that signature- and format-specific tools miss. In particular, ML model execution is organized around a small number of well-defined lifecycle phases (e.g., loading, inference, and training), and different ML artifacts should not be treated as entirely distinct software instances.
* These authors equally contributed to this work.
1
Rather, they represent variations within the capability space exposed by a specific framework. As a consequence, the set of interactions that model execution is expected to have with the host system is narrow and highly predictable, making deviations a strong signal of potentially malicious behavior (allowlisting approach).
Remote Storage Å Model Hub
Training / Fine-Tuning Training Dataset (e.g., Labels, Samples)
Ô Optimization
Input Data (e.g., Image, Text)
Gradient Updates
Fetch
Download Artifacts
Æ Local Cache
Memory Loading
K Deserialization
Inference
j Execution
torch.load() .keras, .pt, .bin keras.models.load_model()
Results
Predictions (e.g., Classes)
Forward Pass
LIFECYCLE TEMPORAL FLOW
Figure 1: Lifecycle of an ML model, from artifact retrieval to prediction.
Our Approach. We translate these intuitions into M OAT, a lifecycle-aware approach for securing ML model execution by enforcing execution boundaries over system-level interactions. M OAT is structured around four components: (i) an action abstraction that maps low-level execution events to securityrelevant interactions with the host system; (ii) the execution boundaries definition, which specifies the actions expected during each lifecycle phase; (iii) an orchestrator, which coordinates the analysis across different lifecycle phases and (iv) a tracer that observes system-level interactions at runtime and detects boundary violations. We realize this approach in R E -M OAT, a reference implementation that instantiates these components by tracing system calls and mapping them to security-relevant actions.
• We design and implement R E -M OAT, a syscall-based reference implementation of M OAT that traces system calls at runtime and maps them to security-relevant actions. R E M OAT supports multiple ML frameworks and file formats. • We empirically validate our intuitions by evaluating R E M OAT on 77,974 real-world models, 31 PoCs, and 334 models from a state-of-the-art dataset. Open Science. After peer-reviewed publication, we will provide all artifacts necessary to reproduce our findings.
Evaluation. We evaluate R E -M OAT across multiple frameworks and lifecycle phases against 23 PoCs from 8 public CVEs, 4 PoCs derived from a vulnerability not yet publicly disclosed at the time of analysis, 77,974 real-world models downloaded from the Hugging Face Hub, and 4 PoCs from a state-of-the-art attack [72]. We further compare R E -M OAT with state-of-the-art approaches on 334 models from a dataset used in prior work [38]. Under our threat model, which targets model behaviors that aim to compromise the host system, R E M OAT exhibits no observed false negatives or false positives (0%) on the evaluated datasets, thereby improving detection coverage while reducing spurious alerts compared to existing approaches. Notably, under the same threat model, R E -M OAT outperforms approaches specifically tailored to Pickle artifacts when evaluated on Pickle-based models, while remaining agnostic to the underlying model format. These results demonstrate a robust and generalizable defense for ML model execution across diverse attack patterns, ranging from unsafe serialization to code reuse and framework-specific vulnerabilities, and across different model formats, from frameworkspecific representations to standard serialization formats such as Pickle.
2
Background
We provide background on ML model sharing (or ML model supply chain), the approaches adopted by different ML frameworks, and the associated risks.
2.1
ML Models’ Lifecycle
ML models are typically used through a sequence of welldefined stages that constitute their lifecycle, as illustrated in Figure 1. In this work, we refer to these stages as lifecycle phases, each representing a logically distinct execution stage in which the model and the ML framework perform a specific, semantically coherent set of operations. Across ML frameworks, three primary lifecycle phases are commonly observed: loading, inference, and training. The loading phase involves reading a model artifact from persistent storage and reconstructing an in-memory model object for subsequent use. During inference, the model processes inputs to produce predictions without modifying its parameters. The training phase instead consists of updating the model’s parameters based on training data.
Contributions. We make the following contributions:
2.2
• We derive foundational intuitions under which the structured and predictable execution of ML models enables the definition of execution boundaries. These boundaries define which interactions with the host system are expected, allowing deviations to be systematically identified as suspicious.
ML Model Sharing
Instead of training models from scratch, practitioners increasingly download and reuse third-party ML model artifacts from dedicated repositories (“model hubs”) and frameworkspecific distribution channels [29, 31, 37, 59]. This ecosystem resembles traditional software distribution. However, the primary shared object is often a model artifact (e.g., .pt, .keras, .h5), which is loaded through ML frameworks rather than inspected as source code.
• We translate these intuitions into M OAT, a lifecycle-aware approach for securing model execution by enforcing execution boundaries over interactions with the host system. 2
Model Persistence and Loading. Many popular frameworks support persistence mechanisms that range from weightsonly storage (largely data-oriented) to full-object serialization that may reconstruct executable objects at load time. For example, PyTorch documents multiple ways to save and load models and explicitly discusses the security implications of its serialization semantics [57, 58]. Keras similarly provides whole-model saving and loading APIs, as well as weights-only workflows, and introduces security-oriented settings such as safe_mode intended to restrict unsafe deserialization paths [9, 41]. TensorFlow supports SavedModel and checkpoint-based mechanisms that restore model graphs and parameters using framework-defined loaders [1, 63, 64].
2.3
formalize for ML model execution, grounded in the intuitions of § 6.
3
Related Work
Prior work spans artifact scanners that analyze model files prior to execution, restricted loading that constrains loading at runtime, empirical analysis of model behavior, and broader analyses of the ML model supply chain. In this section, we review these directions. Artifact Analysis and Scanners. Due to the widespread use of Pickle, many tools focus on detecting unsafe deserialization patterns in Pickle-based artifacts. Trail of Bits’ Fickling [68], for example, performs program analysis of Pickle files by observing the imports and operations triggered during deserialization and comparing them against manually curated allowlists of trusted ML libraries. While effective at identifying unexpected imports, this approach relies on predefined allowlists and remains tightly coupled to Pickle semantics and known library behaviors. Model-sharing platforms integrate scanners into their upload pipelines. The Hugging Face Hub [31], for instance, applies automated inspection to uploaded artifacts, including generic malware detection via ClamAV [10] and VirusTotal [69], Pickle-specific analysis to enumerate imported modules [30], and third-party tools such as M ODEL S CAN [54] from Protect AI [55] and scanners provided by JFrog [33]. These static scanners analyze model files before execution and rely on predefined signatures, heuristics, or frameworkspecific rules to flag suspicious constructs, such as the use of specific serialization mechanisms or model components (e.g., Lambda layers). As with traditional malware scanning, their effectiveness depends on the completeness and maintenance of their rulesets and is inherently limited to the formats and threat patterns they support.
Risks of Model Sharing
Model artifacts are not passive inputs: they are processed by ML frameworks, creating an attack surface that spans multiple lifecycle phases. During loading, framework loaders deserialize artifacts, resolve model components, and reconstruct in-memory objects; after loading, phases such as inference and training may invoke framework functionality on attackercontrolled state. Across these phases, malicious behavior can manifest as host-level effects, including file-system access, network communication, process creation, or modification of persistent state. Building on insights from open-source software supply chains [23, 42, 51], recent work frames this as an ML model supply chain risk [22, 46], in which untrusted artifacts are executed through complex framework loading and execution paths.
2.4 Behavioral Profiling and Host-Based Detection To clarify how the approach in § 7 can be categorized, we briefly review behavioral profiling and host-based detection. A behavioral profile characterizes the expected runtime behavior of software through its observable interactions with the host system, such as system calls and the file, network, and process operations they generate. Profiling of this kind underlies Host-based Intrusion Detection Systems (HIDS), which monitor host activity and flag deviations from expected behavior [5, 26]. HIDS typically follow misuse-based approaches, which match activity against signatures of known attacks, or anomaly-based approaches, which model normal behavior and treat departures from it as suspicious [60]. The practical value of anomaly-based detection depends on the profiling strategy: when profiles are too heterogeneous, normal behavior varies so widely that benign and malicious activity cannot be separated, which is the classical source of the high false-positive rates reported in operational settings. Through this lens, our approach is a form of anomaly-based behavioral profiling, and several of its components can be cast as a HIDS; what distinguishes it is the profiling strategy that we identify and
Restricted Loading Environments. P ICKLE BALL [38] analyzes serialized Pickle programs together with the source code of the library that produced the model and derives modelspecific policies that constrain the operations permitted during deserialization. These policies are then enforced at load time, restricting the model’s behavior rather than merely flagging it. While this approach reduces reliance on fixed signatures and provides stronger guarantees than pure artifact scanning, it remains limited to Pickle-based artifacts and focuses exclusively on the loading phase of the model lifecycle. Similarly, beyond its program analysis capabilities, Fickling also supports runtime loading restrictions by hooking the Pickle library and enforcing security checks. Empirical and Dynamic Analyses. Casey et al. [7] conducted the first large-scale exploit instrumentation study of ML supply chain attacks on the Hugging Face Hub. Their analysis systematically examined thousands of publicly available models, identified the serialization formats in use, and 3
empirically tested exploitability by injecting malicious payloads into models serialized with unsafe mechanisms. Their approach combines Python-level tracing with operating system–level syscall tracing (via strace) during model loading, enabling the observation of interactions such as file operations, process execution, and network access. However, malicious behavior is identified by searching for the presence of a predefined set of syscalls and Python-level execution primitives (e.g., socket, connect, execve, chmod, as well as exec and eval), which are treated as indicative of compromise attempts. While effective for large-scale measurement, this strategy relies on fixed indicators of suspicious behavior rather than on a structured characterization of expected versus unexpected actions during model execution. Their results show that a majority of models using unsafe serialization formats are exploitable and that existing platform-level scanners fail to flag a substantial fraction of vulnerable or malicious artifacts.
4
a single serialization format and primarily act at load time, so they cannot reason about malicious behavior triggered during inference or training. (ii) Trust-shifting mitigations. Some mitigations reduce the amount of executable or framework-resolved state embedded in an artifact, but do not eliminate the underlying trust problem. Weights-only sharing, for instance, reduces loading-time risks, but shifts trust to external components: architectures, custom layers, preprocessing logic, and dependencies must still be distributed separately. (iii) Reactive and incomplete scanner coverage. Scanners that generalize across formats, such as those integrated into the Hugging Face Hub [31] (e.g., M ODEL S CAN [54] from Protect AI), typically depend on signatures, heuristics, or frameworkspecific rules. Their coverage is therefore incomplete and difficult to maintain as frameworks evolve. Recent work systematically bypasses state-of-the-art scanners across alternative loading paths and risky-function gadgets [45], while prior analysis uncovered Keras vulnerabilities that evaded existing framework mitigations and exposed inconsistencies in scanner support across formats [22].
Motivation
Relevance and Timeliness. The attack surface described in § 2.3 poses an increasingly urgent problem. Recent work [22] has shown that neither framework-level mitigations nor hubintegrated scanners reliably detect malicious model artifacts. This is not merely a limitation of a specific unsafe format: even data-oriented formats can enable arbitrary code execution through framework-level loading logic. At the same time, code-based formats remain widespread for sharing ML models, despite being intrinsically difficult to secure. Pickle is the most prominent example: code execution during deserialization is part of its semantics, and its documentation explicitly warns against loading untrusted artifacts [56]. Nevertheless, Pickle remains embedded in several ML persistence workflows, either directly or indirectly [57], creating practical exploitation paths and contributing to documented vulnerabilities in model-loading flows [11–13, 15–18]. Moreover, the attack surface extends beyond loading: recent attacks also trigger malicious behavior during inference by abusing legitimate framework APIs [14, 72]. Recent talks at venues such as DEF CON 33 [52] and Black Hat USA 2025 [36] have further highlighted attacks against ML model formats and frameworks, while model hubs continue to integrate new scanners, such as the VirusTotal integration announced by Hugging Face in October 2025 [6]. These developments suggest that model security is an active and evolving arms race between framework hardening, platform scanning, and adversarial abuse.
(iv) Unresolved precision–recall tradeoff. Even when detection succeeds, existing approaches still struggle to balance false positives and false negatives. Scanners may conservatively misclassify benign models containing Lambda layers while missing other malicious artifacts [22], and P ICKLE BALL still reports false-positive rates above 20% [38]. Motivating Example. A concrete instance of this dynamic is CVE-2025-1550 [13], disclosed in early 2025. Unlike earlier exploits relying on the serialization of executable code (e.g., pickled bytecode or Lambda layers), this vulnerability exploited a previously unexplored execution path in the loading logic of the supposedly safe .keras format, even under Keras safe_mode [19]. By abusing how framework internals resolve and instantiate model components, the attack achieved arbitrary code execution at load time. Following the disclosure, Hugging Face integrated new detection rules from Protect AI that explicitly referenced the CVE [50], yet other scanners deployed on the Hub still did not flag instances of the same vulnerability weeks later [22]. The episode illustrates the core limitation of static, rule-based scanning: detection is reactive and presupposes prior knowledge of the specific exploitation technique. Toward a Behavior-Based Defense. The above observations motivate a novel defense that does not rely on a specific file format, serialization mechanism, scanner signature, or known vulnerability. Instead of asking how an attack is encoded inside a model artifact, we focus on the effects this attack, performed via model execution, has on the host system, enforcing execution boundaries (i.e., an execution allowlist) during specific lifecycle phases. We envision this defense as part of practical deployments that can combine static and dynamic analysis. Static analysis can cheaply reject known malicious
Open Problems. We identify four main open problems from the approaches reviewed in Related Work (§ 3). (i) Limited coverage across formats and lifecycle phases. Existing defenses are often tied to a specific artifact format or execution phase. For example, Pickle-oriented approaches such as Fickling [68] and P ICKLE BALL [38] are confined to 4
patterns, while restricted loaders can reduce exposure to specific dangerous mechanisms. Our dynamic approach, instead, is particularly useful for previously unseen attacks, framework abuse, and attacks that manifest outside loading, where static or loader-specific defenses may provide limited coverage.
Observing Security-Relevant Events. Security-relevant actions = Interactions with the host system (files, network, processes, devices). applied to ML models execution
Predictable ML Model Behavior. Limited heterogeneity in execution actions across model instances during specific lifecycle phases. enables
5
Boundaries defined over interactions with the host system can distinguish expected vs. unexpected actions.
Threat Model
Unexpected actions ≈ malicious behavior across large model populations.
Expanding the threat model defined by Digregorio et al. [22], which focuses on arbitrary code execution during the loading phase, we consider the threat posed by malicious ML model artifacts that aim to compromise the user’s system during one or more phases of the model lifecycle.
Figure 2: Overview of the core intuitions behind lifecycleaware execution boundaries for ML models.
Attacker’s Target. We focus on the execution pipeline composed of the ML model’s lifecycle phases defined in § 2.1 (i.e., loading, inference, and training), executed locally on a user’s machine. The system targeted by the attacker consists of: (i) the user environment, including the operating system with an installed ML framework; (ii) the model artifact, represented as a serialized file such as .keras [39] or .h5 [39]; and (iii) the phase-specific framework mechanisms used to operate on the model. Concretely, this includes framework-provided functions such as keras.models.load_model() [65] during the loading phase and keras.Model.predict() [40] during inference. We focus on scenarios in which a user executes a model obtained from an external source.
While we assume the ML framework itself to be trustworthy, the threat model allows an attacker to abuse both modelembedded code and legitimate framework code paths. In particular, an adversary may embed malicious routines within a model artifact or reuse existing framework functionality in unintended ways to achieve malicious effects, as demonstrated by recent vulnerabilities such as CVE-2025-9905 [17], CVE-2025-9906 [18], and CVE-2025-8747 [16].
6
Foundational Intuitions
This section provides an overview of the fundamental intuitions underlying this work, which form its conceptual core. Here, we introduce the notions used throughout the paper, such as software execution actions, and formalize the distinction between expected, unexpected, and malicious actions. We then observe that, by leveraging the structured lifecycle of ML models, execution boundaries can be instantiated in a precise and enforceable manner across different execution phases. A graphical summary of the conceptual flow underlying this work is shown in Figure 2.
Attacker’s Goal. We consider an attacker who crafts a malicious model artifact to compromise the victim’s system during one of the model lifecycle phases. The attacker’s primary objective is to induce attacker-controlled behavior that escapes the execution context of the model and results in control over the host system, enabling arbitrary interactions with system resources, including but not limited to arbitrary code execution, within the privileges of the user executing the model. These attacks exploit vulnerabilities in the deserialization process or in the ML framework and target the host system, rather than the model’s predictions, as done by model-level attacks (e.g., poisoning attacks) [4].
6.1
Observing Security-Relevant Events
We adopt a dynamic approach to secure ML model execution. Unlike existing static approaches, we shift the focus away from attack-specific reasoning and instead define an allowlistbased approach centered on security-relevant events triggered during model execution. These events are captured at the boundary between the ML execution environment and the host system, rather than through internal framework states or model-specific abstractions. We focus on events that directly impact host security, such as file-system access, network communication, process creation, and device interaction. An alternative approach would be to instrument the ML framework, for example, by monitoring internal APIs or execution paths during model loading, inference, or training. Under our threat model, however, this strategy is unreliable. Even assuming the framework to be trustworthy, attackers
Attacker’s Capabilities and Assumptions. The attacker can create, modify, and distribute ML model artifacts but has no prior access to the target’s system. We consider two distribution channels: (1) public repository poisoning, in which a malicious model is uploaded to a trusted platform (e.g., Hugging Face Hub [31], Kaggle [37], or GitHub) under the guise of a legitimate resource; and (2) direct delivery, in which the artifact is sent to the victim through private channels such as email or cloud storage. We do not make specific assumptions about whether the victim enables framework-level mitigations such as safe_mode [41] in Keras or weights_only [58] in PyTorch. Prior work [22] has shown that such mechanisms are not always sufficient to prevent attacks at model loading time and, in some cases, may create a false sense of security. 5
can exploit legitimate framework code paths to induce unintended effects [72], thereby bypassing in-framework monitoring. Recent vulnerabilities illustrate this limitation. For instance, CVE-2025-9906 [18] demonstrates how legitimate Keras code paths can be abused during model loading to disable the safe_mode security mechanism. Any observation or enforcement mechanism implemented purely within the framework would be exposed to the same manipulation.
both execution sources can ultimately lead to arbitrary code execution [12, 16, 18], we treat them uniformly and do not attempt to distinguish their provenance. For brevity and readability, throughout this section and the rest of the paper, we refer to execution actions as being “performed by an ML model” or, interchangeably, by “an ML framework.” 6.2.2
6.2
Execution Actions
To capture their security-relevant semantics, actions are identified not solely by their type (e.g., file read or file write) but also by contextual attributes that determine their effect on the system. For example, reading model.pt and reading /etc/passwd constitute distinct actions, even though both are file-read operations. Similarly, accessing model.pt in read versus write mode represents different actions.
We define an action as any operation executed by a software during its runtime that results in an interaction with the host system on which it runs (e.g., file-system access, network communication, process creation, or memory management). We denote by A the universe of such actions. The classification of actions as expected or unexpected is inherently contextual. We therefore define an expected action as an action that is consistent with the intended behavior of the software under a given operational context, as determined by its specification, typical usage, or an explicit security policy defined by a system administrator. Formally, we denote by Aexp ⊆ A the set of actions expected under a given context, and by Aunexp = A \ Aexp the set of unexpected actions. For example, it might be considered expected for a text editor to perform a write operation on the path of the edited file (the executed action, which includes information about the accessed path) during the saving phase (a lifecycle phase of the text editor), provided that the system policy allows such file manipulation (as defined by a system administrator). Concretely, writing to a user file such as /home/user/note.txt (opened as part of the edit) would be considered expected. In contrast, a write operation targeting an unrelated file path (e.g., /etc/passwd) would be considered unexpected. According to the threat model defined in § 5, we further define the subset of malicious actions Amal ⊆ Aunexp as those actions whose goal is to harm the system. 6.2.1
Action Granularity
6.3
Predictable ML Model Behavior
Generic software is characterized by high heterogeneity across instances, as distinct programs may legitimately perform radically different actions. A software instance denotes a specific realization of a software program, belonging to one or more categories providing similar functionality (e.g., nano is an instance of the text editor category). This variability introduces two challenges: (i) distinguishing expected from unexpected actions is difficult without detailed knowledge of the specific instance; and (ii) the boundary around malicious actions lacks clear delineation. An action cannot be classified as malicious per se, but only relative to the execution characteristics and intended functionality of the specific instance: a text editor may legitimately access certain cache paths that another would not, even within the same category. As a consequence, static allow/deny policies alone are insufficient for malware detection and are better suited to expressing and auditing administrator security policies. This challenge has motivated a broad line of malware-analysis research, from signature-based detection to ML-based approaches that distinguish benign from malicious behaviors [3, 27, 53]. In contrast, ML exhibits limited heterogeneity across instances: a model artifact is not a distinct software instance but a variation within the capability space exposed by its ML framework, a general property we confirm across frameworks (§ 8). This determines our profiling strategy that governs anomaly-based detection (§ 2.4): observe the behavior of a framework that is executing an ML model within a certain lifecycle phase. Because behaviors within framework–phase pairs are narrow and stable, expected behaviors (profiles) can be expressed as explicit allowlists, and any action outside them can be treated as anomalous. The low heterogeneity of these profiles is what allows such allowlists to separate benign from malicious behavior, rather than incurring the falsepositive rates that burden anomaly-based detection when the
Action Provenance in ML Model Execution
In the ML model execution domain, a significant portion of the executed actions is determined by code from the ML framework. Indeed, each lifecycle phase of an ML model is realized through calls to framework APIs, which execute within the framework’s runtime environment while operating on a specific model instance (e.g., during loading, inference, or training). However, additional actions may also originate from code specific to the model instance itself, such as custom model definitions or auxiliary logic serialized within the model artifact. As a result, both framework code and modelembedded code contribute to the observed execution behavior. Moreover, as clarified in the threat model in § 5, an attacker may abuse both model-embedded code and legitimate framework code paths to achieve malicious effects. Since 6
ML Execution Environment
1
K
j
Ô
Loading
Inference
Training
Action Space A
This component enables reasoning about software behavior in terms of semantically meaningful actions with the host system, abstracting low-level execution events (e.g., system calls) into high-level events that collectively instantiate the action space A . This abstraction is necessary because the same low-level event may correspond to different actions depending on its context, and conversely, the same action may arise from different low-level events. Crucially, high-level actions must be defined with sufficient granularity to preserve security-relevant context (see § 6.2.2, Action Granularity).
Unexpected Aunexp
Expected Aexp
q Malicious Amal
Physical Host > CPU
GPU
:
¾
Z
RAM
Disk
NIC
Figure 3: Overview of the action-space abstraction. ML lifecycle phases induce actions in a global action space A , which are classified as expected, unexpected, or malicious and manifest as interactions with the host system.
2
Execution Boundaries. This intuition enables the definition of clear execution boundaries for ML model execution. The boundary between expected and unexpected actions during a given lifecycle phase can be defined once and reused across a large number of model instances. Moreover, the limited heterogeneity makes it feasible to draw this boundary so that Aunexp is a minimal superset of the malicious set defined in § 6.2, closely approximating it: an action deemed unexpected across a large model population renders any model performing it suspicious. For example, a model that connects to a remote server when loaded from a local file can be considered suspicious. Figure 3 provides a conceptual representation of this intuition.
Approach and Implementation
In this section, we describe how we translate the foundational intuitions introduced in § 6 into a systematic approach. We first outline the reference design (M OAT) that a system based on these intuitions should follow. We then present our reference implementation (R E -M OAT), with its technical details, which realizes this design and is used to experimentally validate our intuitions.
7.1
Execution Boundaries Definition
For each framework–phase pair, our reference design specifies the set of actions (i.e., interactions with the host system) that a model is expected to perform (i.e., Aexp as in § 6.2, Execution Actions). This set constitutes the execution boundaries for that pair. Execution boundaries may incorporate both actions that are generally expected based on standard practice and domain knowledge (which are typically shared across different frameworks) and actions that depend on framework-specific behavior. The latter may be derived from empirical observation of legitimate model executions. This empirical derivation, partially supported by expert knowledge, is our primary method and proceeds in three steps. First, a representative set of benign models is executed under the target lifecycle phase, and the observed low-level events are mapped into actions using the 1 Action Abstraction. Second, correlated actions are aggregated into the same candidate boundary entry; for example, multiple file accesses within the same directory can be represented as a single directory-level entry. Finally, an analyst manually validates each candidate entry. The manual validation step serves two purposes. It confirms that a candidate boundary entry is legitimate under the assumption that the model artifact is untrusted: the analyst verifies that the entry cannot be abused in its current form; otherwise, the corresponding actions must be aggregated differently or represented at a finer granularity. Following the previous example, this may require one entry for each accessed file rather than a single entry allowing access to the entire directory. Validation also distinguishes genuine modelphase behavior from lazy-initialization effects introduced by libraries or runtime environments; when such effects are legitimate but unrelated to the phase under analysis, they can be excluded by letting the orchestrator handle the corresponding initialization before monitoring begins. Once execution boundaries are defined for a given lifecycle phase, any action performed during that phase that falls outside the expected set (i.e., belongs to Aunexp ) is treated as suspect. Depending on the deployment context, such violations may trigger an alert or halt the model execution.
profiles are too heterogeneous [60].
7
Action Abstraction
Reference Design (M OAT)
Our reference design conceptually includes four core components: 1 abstraction of low-level events into higher-level actions, 2 definition of execution boundaries, 3 orchestration and isolation of ML model lifecycle phases, and 4 tracing and enforcement of execution boundaries. 7
Offline Definitions
While this process could potentially be further automated, we consider such automation an engineering effort and therefore out of scope for this work.
1
+ Action Abstraction
Orchestrator
In the proposed design, the orchestrator controls ML model execution and isolates the lifecycle phase under analysis, ensuring that observed behavior is interpreted within the correct semantic context, as the same model may interact with the host system in different ways across lifecycle phases. In addition, phase isolation reduces noise introduced by framework routines and initialization.
Framework 1
Z NETWORK
socket()
3
E Exec Boundaries Definition 2
open()
¾ FILESYS
clone()
X PROCESS
K
j
Ô
Loading
Inference
Training
Framework 2 K
j
Ô
Loading
Inference
Training
Figure 4: Offline definitions of R E -M OAT, showing the Action Abstraction and the Execution Boundaries Definition. Runtime Implementation
4
Tracer
X Orchestrator
3
Lifecycle Management · Boundary Selection · Monitoring Setup
This component enables the observation of the interactions between the executing ML model and the host system. To do this, multiple tracing mechanisms can be used (see Appendix A), each offering different trade-offs in terms of runtime overhead, deployability, and engineering effort. The foundational intuitions are largely orthogonal to the concrete monitoring mechanism employed.
7.2
ML Execution Environment K
j
Ô
Loading
Inference
Training
Û Tracer
4
Intercepts Syscalls · Resolves Arguments · Applies Syscall-to-Action
Reference Implementation (R E -M OAT)
¾
>
Z
X
_
FILESYS
DEVICE
NETWORK
PROCESS
SYSTEM
Boundary Enforcement (Executed Actions vs. Aexp )
Implementation Choices. While M OAT describes a general approach that can be applied across different systems, the concrete implementation of the 1 Action Abstraction and the 4 Tracer inherently depends on the underlying architecture and operating system. Our reference implementation, R E -M OAT, targets Linux-based operating systems on x86-64. This choice is motivated by the widespread adoption of Linux in ML environments and by the fact that major ML frameworks provide first-class support for Linux platforms (e.g., after version 2.10, TensorFlow no longer provides native GPU support on Windows [28]).
Physical Host > CPU
: RAM
¾ Disk
Z NIC
Figure 5: Runtime architecture of R E -M OAT, showing the Orchestrator and the Tracer. either anonymous memory allocation or access to a file, depending on whether a file descriptor is provided and on flags such as MAP_ANONYMOUS. Similarly, clone and clone3 may correspond to thread creation or process creation depending on flags such as CLONE_THREAD. Figure 6 (appendices) summarizes the action categories used in R E -M OAT, including sub-actions specializing each category. This set is not intended to be exhaustive, nor to represent a canonical taxonomy of system behavior. Rather, it is designed to be sufficiently expressive to capture the securityrelevant actions exercised by the ML frameworks during specific lifecycle phases. We consider refinements of this set to be possible but orthogonal to the core ideas explored in this work, and dependent on specific use cases.
Architecture. R E -M OAT consists of four main components, referenced using the same naming conventions introduced in the reference design (§ 7.1): 1 Action Abstraction, 2 Execution Boundaries Definition, 3 Orchestrator, and 4 Tracer. The first two components are defined offline prior to execution, whereas the latter two are active at runtime. Figure 4 provides a high-level overview of the offline configuration and Figure 5 shows the runtime architecture. 1
GPU
Action Abstraction
The syscall-to-action mapping associates each system call available on a Linux x86-64 system with one or more abstract interactions with the host system (i.e., an action in A ). The mapping explicitly accounts for the fact that the same system call may correspond to different actions depending on its arguments. For example, the mmap system call may represent
2
Execution Boundaries Definition
We instantiate the boundary-definition process described in § 7.1 by profiling benign model executions and manually 8
validating distinct action patterns. We first construct baseline boundaries from 600 programmatically generated benign artifacts, comprising 100 models for each of the following categories: PyTorch models relying exclusively on built-in libraries (i.e., fully self-contained Pickle artifacts that also include the model architecture), PyTorch models using custom classes (weights only), Torchvision models (weights only), Keras models in the .keras format (self-contained), Keras models in the legacy .h5 format (self-contained), and TensorFlow models in the SavedModel format (self-contained). To cover framework paths that may not be exercised by generated artifacts, we further analyze approximately 1,000 models from the Hugging Face Hub. We execute all these artifacts under R E -M OAT across the corresponding lifecycle phases and record the observed actions. For each new candidate entry, we manually verify that the action is legitimate under the assumption that the model artifact is untrusted and identify whether its root cause lies in framework behavior or runtime initialization. Importantly, boundary construction is performed before evaluating malicious PoCs and in-the-wild models, and no action observed from malicious artifacts is used to define or update the boundaries. A high-level representation of the boundary definitions for each framework–phase pair is shown in appendices (Listings 1, 2, 3, 4, 5), while the complete definitions are included in the released code. 3
Lastly, this component selects the boundary definition corresponding to the target framework–phase pair from those defined by the Execution Boundaries Definition ( 2 ).
4
Tracer
The tracing mechanism is built on top of libdebug [20, 21], an open-source programmatic debugger that exposes ptracebased debugging functionality through Python APIs, including breakpoint, signal handling, and system-call tracing. We use its support for user-defined callbacks to install a callback at the entry of each system call executed by the target process, i.e., the process running the ML framework during the lifecycle phase under analysis. Within the callback, the intercepted system call and its arguments are mapped to the corresponding high-level interaction with the host system using the Action Abstraction ( 1 ). The resulting action is then checked against the expected execution boundaries defined through the Execution Boundaries Definition ( 2 ). If a boundary violation is detected, a security action is triggered. In R E -M OAT, this action consists of logging the violation together with relevant contextual information, such as the system call identifier and its arguments.
7.2.1
Orchestrator
Runtime Overhead
When measured on the boundary-definition dataset using a laptop with an Intel Core i7-9750H CPU (12 cores, up to 4.5 GHz) and 64 GB of RAM, with CPU-only execution for the ML frameworks, setup overhead ranges from 0.126 ± 0.064s to 2.273 ± 0.104s, depending on the framework–phase pair, and is constant across runs. This setup phase includes the initialization performed by the orchestrator before tracing begins, including the lazy-loading initialization identified during boundary construction. Tracing overhead ranges from 0.001 ± 0.000s to 3.224 ± 0.913s. For inference, overhead has low correlation with model size because matrix operations do not generate system calls. For loading, overhead grows approximately linearly with model size, as I/O-related system calls scale with artifact size; however, the relative overhead decreases as the baseline loading time dominates. These results make R E -M OAT suitable as an artifact scanner, which is the scope of our implementation and the focus of our experimental evaluation. We also recognize the potential application of our approach to real-time monitoring. In that setting, the overall approach would remain unchanged, but the syscall tracing mechanism should differ (i.e., the 4 Tracer): instead of ptrace-based tracing, which incurs frequent context switches, a production-oriented implementation should rely on lower-overhead monitoring mechanisms (Appendix A).
It is responsible for coordinating execution and monitoring, invoking the right framework API (e.g., loading a PyTorch model via torch.load) while enabling tracing. It isolates lifecycle phases by precisely delimiting the execution window to be monitored. Since the code invoking the framework API is controlled by the orchestrator itself, R E -M OAT uses process control and synchronization mechanisms to mark the beginning and the end of the lifecycle phase under analysis. In particular, two signals are injected immediately before and after the execution of the corresponding framework API, serving as delimiters for the analysis. These signals are intercepted by the Tracer ( 4 ), which starts and stops system-call tracing accordingly. A high-level example is shown below: ... os.kill(os.getpid(), signal.SIGUSR1) model = framework.load("path_to_model") os.kill(os.getpid(), signal.SIGUSR2) ...
To further limit the noise during the lifecycle phase under analysis, the orchestrator also injects all code required to complete initialization prior to invoking the framework API. This includes framework imports, device initialization (e.g., GPU availability checks), and other setup operations. 9
7.3
Manual Effort Estimation
selecting the appropriate framework versions to ensure that each vulnerability could be reproduced.
Across the five framework–phase pairs considered in our implementation, namely PyTorch loading and inference, Keras loading and inference, and TensorFlow inference (see § 8), each boundary contains between 4 and 21 entries. Many entries are shared across frameworks or phases, yielding 27 unique entries overall. Lazy-initialization handling is harder to quantify: a single initialization statement may cover multiple lazy-loading effects, while a single library may require multiple initialization steps. Under a conservative accounting, we estimate ≈ 20 such initializations. In our experience, validating a previously unseen action requires between 10 minutes and 3 hours for an experienced analyst, depending on the complexity of the framework path that produced it. Overall, an analyst with at least three years of experience in ML and systems security can add support for a new framework from scratch, covering loading and inference, in roughly one to two working days. Reducing this manual validation effort, for example, through automated profiling, clustering, or root-cause analysis of newly observed benign actions, is left as future work.
8
(ii) Large-scale In-the-Wild Analysis. We acknowledge that some models may require additional dependencies at runtime. Supporting such dependencies would require the development of an automated dependency-resolution mechanism, which is outside the scope of this work and of the experiment itself. We therefore perform the analysis using only the mandatory dependencies. To ensure transparency and avoid inaccurate results, we implement a mechanism that logs execution information during analysis and excludes models with missing dependencies from classification by R E -M OAT. Indeed, missing dependencies may cause exceptions before a model’s actual behavior can be observed, potentially leading to false negatives. As discussed in § 8.2, this limitation does not prevent us from analyzing the vast majority of the identified models (77,974 out of 88,789), which is largely compatible with the scope of the test (i.e., evaluation over a large number of in-the-wild models). Further discussion on the dependency limitation, as well as possible directions for automating module resolution (which pertain to the reference implementation rather than to the proposed technique), is provided in § 10. (iii) State-of-the-Art Comparison. The smaller number of artifacts and the limited set of dependencies compared to largescale analysis make it possible to identify and manually install all required external dependencies. We also update the orchestrator to account for the import and initialization of these additional libraries. These choices enable a comprehensive and fair comparison with the state of the art across the entire dataset, rather than a reduced subset that could introduce bias. Despite the additional dependencies, no changes are made to the execution boundaries.
Experimental Evaluation
Our experimental evaluation is guided by three research questions, each addressed by a dedicated experiment and corresponding subsection. Specifically, these are: RQ1. Is our solution able to identify attacks exploiting novel and previously unseen vulnerabilities? RQ2. How does our solution perform against real-world models collected in the wild?
Scope. Our evaluation does not restrict the analysis to a specific model architecture or family. R E -M OAT applies to any artifact supported by the tested frameworks and included in the evaluated datasets or hosted on Hugging Face. This includes artifacts spanning different architectural classes and application domains, from convolutional and fully connected networks to transformer-based language models.
RQ3. How does our solution compare against state-of-theart approaches? Loading Settings. To maximize compatibility, we disable framework-provided security mechanisms (e.g., Keras safe_mode and PyTorch weights_only). While reducing the attack surface, these mechanisms can impose practical limitations on loading certain (even benign) model artifacts, which may lead users to disable them. Consequently, this choice ensures that potential threats are not excluded from the analysis and aligns with our threat model.
Result Validation. We manually validate all results by confirming that models flagged as malicious indeed exhibit malicious behavior. In addition, we verify that the observed execution-boundary violations are consistent with the exploits embedded in the artifacts.
Dependencies Management. We install, in the containerized environment used for our experiments, all mandatory dependencies required to execute the evaluated models across different lifecycle phases (e.g., the official Keras, PyTorch, TensorFlow, and Torchvision libraries). We then proceed as follows for each experiment:
8.1
Unseen Exploits Simulation (RQ1)
Our goal is to assess the ability of R E -M OAT (and of its underlying intuitions) to detect attacks that exploit novel vulnerabilities in ML frameworks. Relying on PoCs from real-world CVEs and on the recently proposed TensorAbuse technique by Zhu et al. [72], which affects popular ML frameworks, we simulate a scenario in which a defender must protect against
(i) Unseen Exploits Simulation. No additional dependencies beyond the mandatory ones were required, except for TensorFlow I/O [66]. Dependency management primarily involved 10
an attacker exploiting such vulnerabilities. We evaluate M OAT using framework versions that are vulnerable to these vulnerabilities or by disabling the security mechanisms implemented to mitigate them. Because our solution does not rely on how an exploit is implemented, but rather on its observable effects on the host system, no vulnerability-specific rules are hardcoded into R E -M OAT. As a result, rolling back framework versions approximates the scenario in which such exploits are encountered for the first time, prior to public disclosure and the availability of tailored detection mechanisms. This analysis is further complemented by an evaluation against a vulnerability not publicly disclosed at the time of testing, which abuses TFSMLayer in Keras. We identified this vulnerability during the development of R E -M OAT while analyzing the behavior of benign models using this layer. Thanks to the highly informative nature of our approach (which reports the type of violation, accessed paths, and related contextual details), we identified security concerns related to TFSMLayer. In particular, this layer loads external SavedModel artifacts during Keras model loading, which cannot be considered secure [67], as demonstrated by TensorAbuse [72]. This behavior effectively bypasses safe_mode when TFSMLayer is used. We privately reported these observations to the Keras team, who confirmed the issue (already known internally). This vulnerability provides a concrete case study that closely mirrors our motivating example, representing a novel exploitation path that is not publicly disclosed at the time of our evaluation. To avoid any potential bias, we do not update the execution boundaries after developing the PoCs, and we use all vulnerabilities to implement exploits following the same state-of-theart attack scenarios as TensorAbuse. 8.1.1
Table 1: Results for attacks based on TensorAbuse and framework vulnerabilities. Columns are attacker goals: EC (ExecuteCodeByInjection), LF (LeakFile), LIP (LeakIP), and GS (GetShell). ✓indicates detection. Due to vulnerability constraints, some attacks cannot be implemented (NA) or can only be partially implemented († ). Vulnerability TensorAbuse [72] (TensorFlow) CVE-2024-3660 [11] (Keras) CVE-2025-1550 [13] (Keras) CVE-2025-8747 [16] (Keras) CVE-2025-9905 [17] (Keras) CVE-2025-9906 [18] (Keras) CVE-2025-12058 [12] (Keras) CVE-2025-49655 [15] (Keras) CVE-2025-32434 [14] (Pytorch) TFSMLayer Abuse (Keras)
8.1.2
| | | | | | | | | |
EC
LF
LIP
GS
✓ ✓ ✓ NA ✓ ✓ NA ✓ NA ✓
✓ ✓ ✓ NA ✓ ✓ ✓† ✓ NA ✓
✓ ✓ ✓ NA ✓ ✓ NA ✓ NA ✓
✓ ✓ ✓ ✓† ✓ ✓ NA ✓ ✓† ✓
Framework Vulnerabilities
We focus on PyTorch and Keras which, together with TensorFlow covered by the previous experiment, span three of the most widely used ML frameworks [32, 49]. We identify vulnerabilities targeting these two frameworks by querying CVE.org using the keywords ‘keras’ and ‘pytorch’, and then selecting those that are compatible with our threat model. We initially focus on vulnerabilities that enable arbitrary code execution and then extend the analysis to vulnerabilities that expose primitives compatible with TensorAbuse-style attacks. Overall, we identify eight CVEs: seven affecting Keras and one affecting PyTorch. All Keras vulnerabilities manifest during the model loading phase, whereas the PyTorch vulnerability is triggered during inference (forward call). For each CVE, we take the official PoCs (which typically only spawn /bin/sh or write to mock files or SSH keys), whenever available, and extend them to implement complete attack scenarios inspired by TensorAbuse, reusing the same attackercontrolled servers. When a vulnerability does not permit the full realization of a specific attack, we implement the closest achievable behavior. For example, when a vulnerability enables only arbitrary file writing, we use it to write to the authorized_keys, simulating a subset of the GetShell attack.
TensorAbuse Attacks
Zhu et al. [72] show that TensorFlow models can embed malicious behavior by abusing legitimate APIs, enabling systemlevel actions such as file-system access and network communication. This threat is triggered during the inference phase. For this test, we use the PoCs provided in the accompanying repository of the original paper [72]. The dataset includes four distinct exploits, each modeling a different attacker objective: ExecuteCodeByInjection, in which a malicious model creates a Python file that overrides imports of widely used libraries; LeakFile, which simulates data exfiltration by reading a sensitive file from the victim system (e.g., an SSH private key) and transmitting it to a remote attacker-controlled server; LeakIP, which triggers a request to a malicious server in order to disclose the victim’s IP address; and GetShell, which appends a public key to the victim’s authorized_keys file, exfiltrates the username to a remote server, and simulates an attacker establishing SSH access using the injected key. R E -M OAT detects all attacks during the inference phase; the results are reported in the first row of Table 1.
In addition, we include PoCs implementing the same attack scenarios using TFSMLayer abuse, which was not publicly disclosed at the time of testing. This vulnerability affects Keras and manifests during model inference. In total, the dataset for this test is composed of 27 PoCs. R E -M OAT successfully detects all attacks, confirming the effectiveness of its design, prioritizing actual exploit behavior over vulnerability-specific details. 11
8.2
Large-Scale In-the-Wild Analysis (RQ2)
specialized to any particular model format. In the following, we discuss additional considerations regarding the experimental settings related to the dataset and threat model, followed by the corresponding results. We refer the reader to Appendix B for further considerations on weights_only and M ODELT RACER. Dataset Considerations. The original dataset contains 253 benign models and 84 malicious models, for a total of 337 models 1 . Several malicious models are sourced from the repositories used to construct the MalHug dataset [71]. MalHug also includes Keras models that are not supported by some of the defenses considered in the comparison (e.g., P ICKLE BALL and PyTorch’s weights_only mode) and were therefore not included in the original P ICKLE BALL’s dataset. We manually analyzed the dataset prior to experimentation. During this analysis, we identified three malicious models whose payloads rely on the nt module, an internal implementation module used by os on Windows platforms. As all experiments were conducted on Linux, we excluded these three models from our evaluation. We also identified a model labeled as malicious in the dataset (twitter-roberta-base-sentiment.bin from the oceanhacktitude/tinymodel Hugging Face repository2 ) that exhibited no malicious behavior; a detailed discussion is provided in Appendix B. We therefore reclassified it as benign. After these adjustments, the final dataset consists of 254 benign and 80 malicious models. Threat Model Considerations. Among malicious models, 18 are of the form eval("print(’message’)") or similar variations. The print operation has no impact on the system an attacker aims to compromise under our threat model, and eval alone does not execute operating-system commands or access system resources. In other words, while eval can be abused for system compromise, its use alone does not constitute such a compromise, nor does it introduce capabilities beyond those already permitted by Pickle (i.e., Pickle already allows Python code execution). One additional model contains strings commonly associated with exploit code, but consistently raises a SyntaxError across different environments before executing any payload and therefore does not represent an actual threat. One model compiles malicious code and returns a code object that would require explicit execution. This is incompatible with our threat model, which assumes artifacts are loaded and used as ML models. Finally, another model only attempts to execute a Python script that is not included in the dataset, resulting in an exception. This behavior is not malicious per se, as it depends entirely on the contents of an external file that is not present and could be either benign or malicious. Accordingly, under our threat model, we reclassify these 21 models from malicious to benign, resulting in a dataset
We test our approach against a large number of models collected from the Hugging Face Hub. The goal is to assess the applicability of our approach to in-the-wild models, where false positives are more likely due to the lack of control over the samples w.r.t. controlled experimental settings. We use the Hugging Face API to enumerate and download repositories containing PyTorch and Keras models. We select these two frameworks due to their widespread adoption in ML [32, 49], the high availability of publicly shared artifacts, and the large number of CVEs disclosed in 2025 (for Keras, see § 8.1) or reliance on unsafe serialization formats (for PyTorch). We select repositories whose last update date falls within 2025 and further filter them using framework-related keywords (i.e., "keras" and "pytorch"). For each selected repository, we further filter the contained files by serialization format, considering .bin, .pt, and .pth files for PyTorch models, and .h5 and .keras files for Keras models. While this filtering strategy may exclude some valid model artifacts, it is consistent with the goals of our evaluation, which aim to assess our approach on a large and diverse set of real-world model files rather than to analyze all models available on the platform. Moreover, due to computational constraints, we discard artifacts larger than 16 GiB and restrict the analysis to the loading phase. In total, we analyze 40,162 repositories, from which we extract 88,789 artifacts. From the analysis we exclude models that rely on external dependencies and therefore cannot be fully executed (runtime dependencies are discussed in § 10). After this final filtering step, we retain 77,974 models (≈88%). Among these, R E -M OAT flags 23 models as suspicious. We manually inspect all flagged cases and confirm that they exhibit malicious behavior, ranging from OS-level command execution to reverse shells, resulting in a false-positive rate of 0%. We are in contact with Hugging Face, which is aware of this evaluation and its results.
8.3
State-of-the-Art Comparison (RQ3)
To compare R E -M OAT against state-of-the-art approaches, we employ the same dataset recently used by P ICKLE BALL [38]. This allows a direct comparison with P ICKLE BALL and its evaluation baselines: M ODEL S CAN [54] (v0.8.7), M OD ELT RACER [7], and PyTorch’s restricted unpickling mode (weights_only=True, v2.8.0) [58]. For completeness, we also include Fickling [68] (v0.1.7), which provides analysis utilities and a decompiler for Pickle artifacts, as well as a runtime mechanism targeting ML models that hooks the Pickle library to enforce security checks during loading. We note that this evaluation focuses on Pickle-based model artifacts and that three of the compared approaches (P ICKLE BALL, weights_only, and Fickling) operate exclusively on Pickle-based artifacts. In contrast, R E -M OAT is not
1 The original paper reports 252 benign models; however, the official repository contains 253 samples, all of which we include in our analysis. 2 https://huggingface.co/oceanhacktitude/tinymodel
12
9
Table 2: Comparison of detection outcomes under two threat models. We report, on the left, the results obtained using the ground truth defined by our threat model (see Section 5), and, on the right, the results obtained using the original ground truth adopted by prior work (e.g., P ICKLE BALL). Tool
#FP
#FN
FPR
FNR
M ODEL S CAN [54] M ODELT RACER [7] weights_only [58] P ICKLE BALL [38] Fickling [68] R E -M OAT (ours)
35 | 16 1|1 117 | 96 73 | 52 117 | 96 0|0
5|7 17 | 38 0|0 0|0 0|0 0 | 21
12.7% | 6.3% 0.4% | 0.4% 42.5% | 37.8% 26.5% | 20.5% 42.5% | 37.8% 0% | 0%
8.5% | 8.8% 28.8% | 47.5% 0% | 0% 0% | 0% 0% | 0% 0% | 26.2%
Discussion
The experimental results covering three complementary research questions ((i) unseen exploits simulation over 31 PoCs, (ii) large-scale analysis of 77,974 in-the-wild artifacts, and (iii) state-of-the-art comparison over 334 models) confirm that our solution provides an effective way to detect malicious behavior in ML model artifacts. The observed behavior provides evidence that, in the context of ML model execution, it is possible to define execution boundaries such that (i) malicious behavior manifests as violations of lifecycle-specific execution boundaries, while (ii) benign models largely remain within them. Beyond raw detection accuracy, the results demonstrate that the proposed approach generalizes across multiple dimensions. Table 3 summarizes the covered settings and the corresponding takeaways.
comprising 275 benign models and 59 malicious models. We acknowledge that prior work adopts a different labeling. For transparency, we report results under both threat models using the corresponding labels. While R E -M OAT is capable of monitoring actions such as writes to stdout, we deliberately do not modify execution boundaries when changing the ground truth, and instead consider the resulting additional false negatives to be an explicit design choice.
Explainability and Diagnostic Value. Although not the primary focus of this work, we observe that our approach can also support the understanding of malicious behavior. Boundary violations, by their nature, provide contextual information (e.g., violation type and accessed paths), offering actionable insights into how attacks manifest during execution. Within the scope of this work, this capability supported both the cross-checking of malicious model implementations against their observed boundary violations and the discovery of the TFSMLayer abuse described in § 8.1.
Results. Table 2 reports the results under the two threat models discussed. We present the number of false positives (#FP), false negatives (#FN), the false positive rate (FPR), and the false negative rate (FNR) for each evaluated tool.
10
Limitations and Future Work
Under the threat model this work assumes (reported on the left), R E -M OAT is the only approach that exhibits neither false positives nor false negatives. In contrast, approaches based on statically extracted signatures or allowlists (e.g., M ODEL S CAN, P ICKLE BALL, weights_only, and Fickling) exhibit a substantial number of false positives while maintaining a low or zero false-negative rate. This behavior is expected: the malicious models in the dataset have been available for some time and exploit well-known vulnerabilities for which existing signatures and allowlists have already been updated. Finally, M ODELT RACER produces a single false positive (i.e., a model we reclassified from malicious to benign; see Appendix B for further details); however, it suffers from false negatives due to its limited threat assumptions, as it restricts detection to network activity, process execution, and permission-modification events identified via four system calls and a few other commands [38].
Anti-Dynamic Analysis. Our method is potentially exposed to anti-dynamic analysis techniques, a well-studied class of evasions in the malware analysis literature [2,8,24]. The reference implementation, R E -M OAT, does not implement explicit countermeasures against such techniques and therefore does not claim resilience against an adaptive adversary seeking to evade runtime monitoring. At the same time, to the best of our knowledge, there is currently no evidence of ML model artifacts in the wild employing anti-dynamic analysis techniques. We therefore view the absence of explicit anti-evasion mechanisms in R E -M OAT as a reflection of the current threat landscape rather than a design oversight. Nevertheless, understanding whether and how classical anti-dynamic analysis techniques could be adapted to the structured execution environment of ML models remains an important direction for future work.
Under the threat model prior work assumes (reported on the right), R E -M OAT exhibits 21 false negatives, which are explicitly described in the Threat Model Considerations paragraph. Regarding the other tools, the results trend remains largely unchanged: signature- and allowlist-based methods reduce their false positives while maintaining a low or zero number of false negatives. In contrast, M ODELT RACER exhibits an increase of exactly 21 false negatives.
Execution Boundaries Maintenance. In our experimental evaluation, we employed multiple versions of the same frameworks without requiring any redefinition or update of the execution boundaries, demonstrating their robustness. However, we cannot exclude the possibility that significant changes in framework implementations may eventually require updates to the associated boundaries. Given the allow-list nature of the expected action set, unreflected changes in framework 13
Table 3: Generality of execution-boundary enforcement across evaluation dimensions. Dimension
Evaluated settings
Takeaway
ML frameworks
PyTorch, Keras, TensorFlow
Viewing models as variations within the capability space exposed by a framework is generally valid and not tied to a specific framework.
Serialization formats
Pickle, .keras, .h5, Saved- Focusing on runtime behavior (effects on the host) rather than artifact Model structure generalizes across heterogeneous serialization formats.
Framework versions
Keras 3.8 (TF 2.17), Keras 3.12 Expected actions are stable across framework versions: execution(TF 2.20), PyTorch 2.5.1, Py- boundary definitions required no changes, reducing maintenance burden. torch 2.9.1, TF 2.17
Framework–phase com- PyTorch (loading, inference), The intuitions underlying our approach remain valid not only across binations Keras (loading, inference), Ten- frameworks but also across lifecycle phases. sorFlow (inference) Attack classes
Code injection, code reuse, By focusing on what a benign model should do at runtime (allowlistframework vulnerabilities, etc. based approach) rather than on how an attack is implemented, the approach remains effective across diverse exploitation techniques.
In-the-wild models
77,974 artifacts; 23 flagged, all confirmed malicious
Execution boundaries derived from limited empirical data generalize well to real-world model artifacts beyond controlled settings, enabling low false-positive rates at scale.
Dependency
10 dependencies (beyond main framework libraries) installed during evaluation in § 8.3 (e.g., Ultralytics, Flair).
After updating the orchestrator to correctly handle dependency initialization and imports, execution boundaries remained unchanged. This suggests that execution-boundary definitions are largely independent of specific dependency sets.
11
behavior may manifest as false positives, with new benign actions flagged as malicious. Nevertheless, updating execution boundaries requires limited effort, mitigating the practical impact of this limitation. Moreover, this issue is not unique to dynamic analysis: static analysis techniques and rule-based scanners similarly require continuous maintenance to remain effective as frameworks evolve.
Conclusion
ML model sharing is increasingly part of the software supply chain, but existing defenses remain largely tied to specific formats, loading mechanisms, or known attack patterns. This work shows a complementary perspective: instead of reasoning about how malicious behavior is encoded inside an artifact, we reason about the effects that model execution produces on the host system. We introduced M OAT, a lifecycle-aware approach that secures ML model execution by enforcing phase-specific execution boundaries over host interactions, and instantiated it in R E -M OAT, a syscall-based reference implementation. The key observation is that ML models are not arbitrary software instances: they execute through a small number of welldefined lifecycle phases, and within each framework–phase pair their host interactions are structured enough to admit compact, reusable boundaries. Our evaluation validates this across multiple dimensions. R E -M OAT detects all evaluated attacks across 31 PoCs, including framework vulnerabilities, code-reuse attacks, and inference-time threats, without relying on vulnerabilityspecific signatures. It scales to 77,974 artifacts from the Hugging Face Hub, flagging 23 models we confirmed as malicious, and outperforms state-of-the-art scanners under our threat model. These results indicate that lifecycle-aware dynamic analysis offers a practical and generalizable basis for securing ML model execution.
Artifact Compatibility. Model artifacts may be tied to specific versions of an ML framework or of the Python interpreter, which can complicate their execution and their analysis. A possible mitigation is to maintain analysis environments supporting multiple frameworks and interpreter versions and to select the appropriate environment based on artifact metadata or on errors observed during preliminary execution attempts. In our evaluation of specific attacks and CVEs, we manually identified the appropriate framework versions and did not implement an automatic version-selection mechanism. We consider this an engineering effort rather than a conceptual limitation of our intuitions. Runtime Dependencies. Dynamic analysis depends on the availability of runtime dependencies: if an artifact relies on external libraries, these must be available in the analysis environment for execution to proceed. Automated mechanisms could be explored to detect missing dependencies and provision them on demand within isolated environments. 14
More broadly, model scanning should not be treated solely as a static artifact-inspection problem. As ML frameworks evolve and attacks increasingly abuse legitimate execution paths, defenses must monitor what model execution does to the host system. M OAT is a step in this direction, showing that runtime effects, interpreted through the structure of the ML lifecycle, can expose malicious behavior across formats, frameworks, and attack techniques.
[9] François Chollet et al. Keras, 2015. https://keras. io. [10] Cisco Talos (ClamAV Team). ClamAV: Open-Source Antivirus Toolkit. https://docs.clamav.net/, 2025. Accessed: 2025-12-30. [11] CVE Program. CVE-2024-3660. https://www.cve. org/CVERecord?id=CVE-2024-3660, 2024.
References
[12] CVE Program. CVE-2025-12058. https://www.cve. org/CVERecord?id=CVE-2025-12058, 2025.
[1] Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, Manjunath Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, Derek G. Murray, Benoit Steiner, Paul Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. TensorFlow: a system for largescale machine learning. In Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation, OSDI’16, page 265–283, USA, 2016. USENIX Association.
[13] CVE Program. CVE-2025-1550. https://www.cve. org/CVERecord?id=CVE-2025-1550, 2025. [14] CVE Program. CVE-2025-32434. https://www.cve. org/CVERecord?id=CVE-2025-32434, 2025. [15] CVE Program. CVE-2025-49655. https://www.cve. org/CVERecord?id=CVE-2025-49655, 2025. [16] CVE Program. CVE-2025-8747. https://www.cv e.org/CVERecord?id=CVE-2025-8747, 2025. Accessed: 2025-12-30.
[2] Amir Afianian, Salman Niksefat, Babak Sadeghiyan, and David Baptiste. Malware Dynamic Analysis Evasion Techniques: A Survey. ACM Comput. Surv., 52(6), November 2019.
[17] CVE Program. CVE-2025-9905. https://www.cve. org/CVERecord?id=CVE-2025-9905, 2025.
[3] Ömer Aslan Aslan and Refik Samet. A Comprehensive Review on Malware Detection Approaches. IEEE Access, 8:6249–6271, 2020.
[18] CVE Program. CVE-2025-9906. https://www.cve. org/CVERecord?id=CVE-2025-9906, 2025.
[4] Battista Biggio and Fabio Roli. Wild patterns: Ten years after the rise of adversarial machine learning. Pattern Recognition, 84:317–331, 2018.
[19] Gabriele Digregorio. CVE-2025-1550 - Bypassing Keras safe_mode for Arbitrary Code Execution. https: //github.com/io- no/CVE- Reports/issues/2, 2025. Accessed: 2025-12-30.
[5] Robert A. Bridges, Tarrah R. Glass-Vanderlan, Michael D. Iannacone, Maria S. Vincent, and Qian (Guenevere) Chen. A survey of intrusion detection systems leveraging host data. ACM Comput. Surv., 52(6), November 2019.
[20] Gabriele Digregorio, Roberto Alessandro Bertolini, Francesco Panebianco, and Mario Polino. libdebug: Build Your Own Debugger. https://libdebug.org, 2024.
[6] Adrien Carreira and Bernardo Quintero. Hugging face and virustotal collaborate to strengthen ai security. Hugging Face Blog, October 2025. Accessed: 2026-06-03. [7] Beatrice Casey, Joanna C. S. Santos, and Mehdi Mirakhorli. A Large-Scale Exploit Instrumentation Study of AI/ML Supply Chain Attacks in Hugging Face Models. arXiv preprint, abs/2410.04490, 2024.
[21] Gabriele Digregorio, Roberto Alessandro Bertolini, Francesco Panebianco, and Mario Polino. Poster: libdebug, Build Your Own Debugger for a Better (Hello) World. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, CCS ’24, page 4976–4978, New York, NY, USA, 2024. Association for Computing Machinery.
[8] Xu Chen, Jon Andersen, Z Morley Mao, Michael Bailey, and Jose Nazario. Towards an understanding of antivirtualization and anti-debugging behavior in modern malware. In 2008 IEEE international conference on dependable systems and networks with FTCS and DCC (DSN), pages 177–186. IEEE, 2008.
[22] Gabriele Digregorio, Marco Di Gennaro, Stefano Zanero, Stefano Longari, and Michele Carminati. On the (In)Security of Loading Machine Learning Models . In 2026 IEEE Symposium on Security and Privacy (SP), pages 214–231, Los Alamitos, CA, USA, May 2026. IEEE Computer Society. 15
[23] Ruian Duan, Omar Alrawi, Ranjita Pai Kasturi, Ryan Elder, Brendan Saltaformaggio, and Wenke Lee. Towards Measuring Supply Chain Attacks on Package Managers for Interpreted Languages. In NDSS 2021, 2021.
[35] Wenxin Jiang, Nicholas Synovic, Rohan Sethi, Aryan Indarapu, Matt Hyatt, Taylor R. Schorlemmer, George K. Thiruvathukal, and James C. Davis. An Empirical Study of Artifacts and Security Risks in the Pre-trained Model Supply Chain. In SCORED 2022, pages 105–114, 2022.
[24] Manuel Egele, Theodoor Scholte, Engin Kirda, and Christopher Kruegel. A survey on automated dynamic malware-analysis techniques and tools. ACM Comput. Surv., 44(2), March 2008.
[36] Zhou Ji’an and Song Lishuo. Safe Harbor or Hostile Waters: Unveiling the Hidden Perils of the TorchScript Engine in PyTorch. Presentation at Black Hat USA https://i.blackhat.com/BH-USA-25/Presenta tions/US-25-Jian-Lishuo-Safe-Harbor-or-Hos tile-Waters.pdf, August 2025. Accessed: 2026-0603.
[25] Falco Project. Falco: Cloud Native Runtime Security. https://github.com/falcosecurity/falco, 2025. Version 0.42.0. [26] S. Forrest, S.A. Hofmeyr, A. Somayaji, and T.A. Longstaff. A sense of self for unix processes. In Proceedings 1996 IEEE Symposium on Security and Privacy, pages 120–128, 1996.
[37] Kaggle, Inc. Kaggle Models. https://www.kaggle.c om/models, 2025. Accessed: 2025-12-12. [38] Andreas D. Kellas, Neophytos Christou, Wenxin Jiang, Penghui Li, Laurent Simon, Yaniv David, Vasileios P. Kemerlis, James C. Davis, and Junfeng Yang. PickleBall: Secure Deserialization of Pickle-based Machine Learning Models. In Proceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security, CCS ’25, page 3341–3355, New York, NY, USA, 2025. Association for Computing Machinery.
[27] Daniel Gibert, Carles Mateu, and Jordi Planes. The rise of machine learning for detection and classification of malware: Research developments, trends and challenges. Journal of Network and Computer Applications, 153:102526, 2020. [28] Google. Install TensorFlow with pip. https://ww w.tensorflow.org/install/pip, 2025. Accessed: 2025-12-29.
[39] Keras Developers. Serialization and Saving — Keras Documentation. https://keras.io/guides/serial ization_and_saving/, 2023. Accessed: 2025-12-30.
[29] Google Research, Brain Team. TensorFlow Hub: Reusable Machine Learning Modules. https://ww w.tensorflow.org/hub, 2025. Accessed: 2025-1212.
[40] Keras Developers. Model Training APIs — Keras Documentation. https://keras.io/api/models/mode l_training_apis/, 2025. Accessed: 2025-12-30.
[30] Hugging Face, Inc. Security and Pickle Files — Hugging Face Hub Documentation. https://huggingf ace.co/docs/hub/en/security-pickle, 2024. Accessed: 2025-12-30.
[41] Keras Developers. Whole model saving & loading Keras. https://keras.io/api/models/model_ saving_apis/model_saving_and_loading/, 2025. Accessed: 2025-12-12.
[31] Hugging Face Inc. Hugging Face Hub Documentation. https://huggingface.co/docs/hub/index, 2025. Accessed: 2025-12-12.
[42] Piergiorgio Ladisa, Henrik Plate, Matias Martinez, and Olivier Barais. SoK: Taxonomy of Attacks on OpenSource Software Supply Chains . In 2023 IEEE Symposium on Security and Privacy (SP), pages 1509–1526, Los Alamitos, CA, USA, May 2023. IEEE Computer Society.
[32] JetBrains. Python Developers Survey 2024. https: //lp.jetbrains.com/python-developers-surve y-2024/, 2024. Accessed: 2026-01-15.
[43] Linux Developers. ptrace(2) — Linux manual page. https://man7.org/linux/man-pages/man2/ptra ce.2.html, 2024. Accessed: 2025-12-29.
[33] JFrog Ltd. Software Supply Chain Solutions for DevOps and Security — JFrog. https://jfrog.com/, 2025. Accessed: 2025-12-30.
[44] Linux Developers. seccomp(2) — Linux manual page. https://man7.org/linux/man-pages/man2/secc omp.2.html, 2025. Accessed: 2025-12-29.
[34] Wenxin Jiang, Nicholas Synovic, Matt Hyatt, Taylor R. Schorlemmer, Rohan Sethi, Yung-Hsiang Lu, George K. Thiruvathukal, and James C. Davis. An Empirical Study of Pre-Trained Model Reuse in the Hugging Face Deep Learning Model Registry. In Proceedings of the 45th International Conference on Software Engineering, ICSE ’23, page 2463–2475. IEEE Press, 2023.
[45] Tong Liu, Guozhu Meng, Peng Zhou, Zizhuang Deng, Shuaiyin Yao, and Kai Chen. The art of hide and seek: Making pickle-based model supply chain poisoning stealthy again, 2025. 16
[46] Sarah Meiklejohn, Hayden Blauzvern, Mihai Maruseac, Spencer Schrock, Laurent Simon, and Ilia Shumailov. Position: Machine Learning Models Have a Supply Chain Problem. In Forty-second International Conference on Machine Learning Position Paper Track, 2025.
[58] PyTorch developers. Save and Load the Model — PyTorch Tutorials 2.7.0+cu126 documentation. https: //docs.pytorch.org/tutorials/beginner/basi cs/saveloadrun_tutorial.html, 2026. Accessed: 2026-01-31.
[47] Microsoft. Sysmon for Linux. https://github.com /microsoft/SysmonForLinux, 2025.
[59] PyTorch Foundation. PyTorch Hub. https://pytorc h.org/hub/, 2025. Accessed: 2025-12-12.
[48] Microsoft Sysinternals. Sysmon v15.15. https://le arn.microsoft.com/en-us/sysinternals/downl oads/sysmon, 2024. Accessed: 2025-12-27.
[60] Hami Satilmiş, Sedat Akleylek, and Zaliha Yüce Tok. A systematic literature review on host-based intrusion detection systems. IEEE Access, 12:27237–27266, 2024.
[49] Paul Mooney. 2022 Kaggle Machine Learning & Data Science Survey. https://kaggle.com/competition s/kaggle-survey-2022, 2022. Kaggle.
[61] Husain Sharaf, Imtiaz Ahmad, and Tassos Dimitriou. Extended berkeley packet filter: An application perspective. IEEE Access, 10:126370–126393, 2022.
[50] Sean Morgan. 4M Models Scanned: Protect AI + Hugging Face 6 Months In. https://huggingface.co/b log/pai-6-month, April 2025. Accessed: 2025-12-30.
[62] Sysdig Inc. Sysdig: Cloud Security Starts at Runtime. https://sysdig.com/, 2025. Accessed: 2025-12-27. [63] TensorFlow Developers. SavedModel format guide — TensorFlow Documentation. https://www.tensor flow.org/guide/saved_model, 2024. Accessed: 2025-12-12.
[51] Marc Ohm, Henrik Plate, Arnold Sykosch, and Michael Meier. Backstabber’s Knife Collection: A Review of Open Source Software Supply Chain Attacks. In DIMVA 2020, pages 23–43, 2020.
[64] TensorFlow Developers. Training checkpoints — TensorFlow Documentation. https://www.tensorflow .org/guide/checkpoint, 2024. Accessed: 2025-1212.
[52] Cyrus Parzian. Loading Models, Launching Shells: Abusing AI File Formats for Code Execution. Presentation at the DEF CON 33 Hacking Conference https://media.defcon.org/DEF%20CON%2033/DE F%20CON%2033%20presentations/Cyrus%20Parzi an%20-%20Loading%20Models%2C%20Launching% 20Shells%20Abusing%20AI%20File%20Formats%2 0for%20Code%20Execution.pdf, 2025. Accessed: 2025-08-21.
[65] TensorFlow Developers. tf.keras.models.load_model — TensorFlow API Documentation. https://www.tens orflow.org/api_docs/python/tf/keras/models /load_model, 2025. Accessed: 2025-12-30. [66] TensorFlow I/O Contributors. TensorFlow I/O: Dataset, streaming, and file system extensions maintained by TensorFlow SIG-IO. https://github.com/tensorf low/io, 2018.
[53] Feargus Pendlebury, Fabio Pierazzi, Roberto Jordaney, Johannes Kinder, and Lorenzo Cavallaro. TESSERACT: Eliminating Experimental Bias in Malware Classification across Space and Time. In 28th USENIX Security Symposium (USENIX Security 19), pages 729–746, Santa Clara, CA, August 2019. USENIX Association.
[67] TensorFlow Security Team. TensorFlow Security Policy. https://github.com/tensorflow/tensorflow/b lob/master/SECURITY.md, 2025. Accessed: 2025-0118.
[54] Protect AI. ModelScan: Open source protection against model serialization attacks. https://github.com/p rotectai/modelscan, 2025. Accessed: 2025-12-30.
[68] Trail of Bits. Fickling: A Tool for Manipulating and Analyzing Python Pickle Programs. https://github .com/trailofbits/fickling. Accessed: 2025-1230.
[55] Protect AI. Protect AI — The Platform for AI Security. https://protectai.com/, 2025. Accessed: 2025-1230.
[69] VirusTotal. VirusTotal. https://www.virustotal.c om/, 2025. Accessed: 2025-12-30.
[56] Python Software Foundation. pickle — Python object serialization. https://docs.python.org/3/librar y/pickle.html, 2025. Accessed: 2025-12-12.
[70] Chris Wright, Crispin Cowan, Stephen Smalley, James Morris, and Greg Kroah-Hartman. Linux Security Modules: General Security Support for the Linux Kernel. In 11th USENIX Security Symposium (USENIX Security 02), San Francisco, CA, August 2002. USENIX Association.
[57] PyTorch Developers. Serialization semantics — PyTorch Documentation. https://docs.pytorch.o rg/docs/2.7/notes/serialization.html, 2025. Accessed: 2025-12-12. 17
[71] Jian Zhao, Shenao Wang, Yanjie Zhao, Xinyi Hou, Kailong Wang, Peiming Gao, Yuanchao Zhang, Chen Wei, and Haoyu Wang. Models Are Codes: Towards Measuring Malicious Code Poisoning Attacks on Pre-trained Model Hubs. In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, ASE ’24, page 2087–2098, New York, NY, USA, 2024. Association for Computing Machinery.
These mechanisms enable the enforcement of access control policies on operations such as file access, process creation, and network communication. However, they typically require system-wide configuration and administrative privileges, and their coarse granularity can complicate lifecyclephase–specific enforcement. Existing Monitoring Tools. Several host-based security and observability tools build on the mechanisms discussed above. For example, Falco [25] and Sysdig [62] leverage kernel modules or eBPF to provide runtime security analytics on Linux, while Sysmon exposes similar behavioral signals on Windows [48] and Linux [47]. Although effective for generalpurpose threat detection, these systems are not designed for lifecycle-aware monitoring of ML model execution.
[72] Ruofan Zhu, Ganhao Chen, Wenbo Shen, Xiaofei Xie, and Rui Chang. My Model is Malware to You: Transforming AI Models into Malware by Abusing TensorFlow APIs. In 2025 IEEE Symposium on Security and Privacy (SP), pages 486–503, 2025.
A
Tracer Implementations
B Additional Considerations on State-of-theArt Comparison
In this appendix, we survey concrete mechanisms that can be used to observe system-level interactions generated during ML model execution and briefly discuss their respective tradeoffs. These mechanisms represent possible implementation choices for the Tracer ( 4 , see § 7).
In the following, we present additional considerations regarding the model mislabeled as malicious in the P ICKLE BALL [38] dataset, as well as weights_only and M ODEL T RACER, complementing the State-of-the-Art Comparison in § 8.3. Considerations on a Mislabeled Sample. During our manual analysis of the P ICKLE BALL dataset, we identified one sample labeled as malicious (twitter-roberta-base-sentiment.bin from the oceanhacktitude/tinymodel Hugging Face repository3 ) that does not contain any malicious payload. Thereby, it cannot be considered malicious under either our threat model or that adopted by prior work. Consistently, the sample is not flagged as malicious by scanners supported by the Hugging Face platform. Nonetheless, the sample is classified as unsafe by P ICKLE BALL, weights_only, Fickling, and M ODELT RACER. The first three tools, which rely on statically constructed allowlists, do not include certain modules required by the model. For example, Fickling flags the model as unsafe due to its use of the transformers module, which is considered “outside the standard library.” In contrast, M ODELT RACER classifies the model as unsafe because it observes a socket system call during the loading phase. This system call is triggered by a urllib probe originating from a dependency of the huggingface_hub module, which is imported indirectly during model loading as part of the model’s dependency chain. As a result, the observed system call is caused by the dependency chain rather than by the model itself. With respect to R E -M OAT, although it relies on system-call tracing, the orchestrator performs initialization steps that preload dependencies, including those that import the urllib module, thereby effectively isolating the lifecycle phase under analysis. This prevents the corresponding system call from
Ptrace-Based. A natural instantiation consists of monitoring system calls issued by the process executing the ML model. Since interactions with the host system (e.g., file access, network communication, process creation, or memory mapping) ultimately manifest as system calls, this approach provides fine-grained visibility into model and framework behavior. Traditional implementations rely on mechanisms such as ptrace [43], which allow a user-space monitor to intercept system calls and inspect their arguments. While expressive and precise, this approach may incur non-negligible overhead due to frequent context switches. This is the solution chosen for our reference implementation. eBPF-Based. Extended Berkeley Packet Filter (eBPF) [61] enables user-defined programs to execute safely within the kernel and to attach to observation points such as system calls, tracepoints, kernel functions, and Linux Security Module (LSM) hooks [70]. Compared to ptrace-based tracing, eBPF can significantly reduce overhead by executing monitoring logic in-kernel and avoiding continuous user-kernel transitions. Seccomp-Based. seccomp [44] represents a restricted use of BPF programs, designed specifically to filter system calls invoked by a process. Seccomp filters are evaluated at the syscall boundary and can be installed statically or dynamically. However, unlike general-purpose eBPF tracing, seccomp policies offer limited semantic visibility. As a result, expressing lifecycle-aware execution boundaries or distinguishing between benign and unexpected uses of the same system call (e.g., file accesses to different paths) is challenging. Kernel Modules. Kernel-level approaches include custom kernel modules and security frameworks such as LSMs [70].
3 https://huggingface.co/oceanhacktitude/tinymodel
18
ExecutionBoundaries: FILESYSTEM_READ: - path in PythonEnvironment - path in ModelFolder
being attributed to the model execution and being flagged as a violation. Considerations on weights_only. This mode is explicitly designed to “limit the functions executed during unpickling to only those necessary for loading weights” [58]. As a consequence, complete model artifacts that contain not only weight definitions but also model architecture, or that rely on third-party libraries, are not intended to be supported by this design. It is therefore expected that weights_only blocks the loading of such artifacts. While, for the sake of comparison, we adopt the same evaluation metrics used in the original P ICKLE BALL work and classify these cases as false positives, this interpretation should be considered in light of the intended scope of the weights_only mechanism.
FILESYSTEM_MODIFY: - path in TorchTemporaryLoadingPaths DEVICE_ACCESS: - fd in {STDOUT, STDERR} PROCESS_CONTEXT: - allow all SYSTEM_GET_INFO: - allow time-related syscalls
Listing 1: High-level execution boundaries for the loading phase of PyTorch models.
Considerations on M ODELT RACER. M ODELT RACER adopts a dynamic tracing approach. Specifically, it treats the occurrence of system calls and commands such as socket, connect, execve, chmod, exec, and eval during the modelloading phase as indicators of malicious behavior. As a dynamic analysis technique, M ODELT RACER faces challenges similar to ours in handling external dependencies. However, to avoid introducing additional noise or bias into its implementation, we evaluate M ODELT RACER exactly as released by the authors, without modifying its dependency-handling mechanisms.
ExecutionBoundaries: FILESYSTEM_READ: - path in PythonEnvironment DEVICE_ACCESS: - fd in {STDOUT, STDERR} PROCESS_CONTEXT: - allow all
Listing 2: High-level execution boundaries for the inference phase of Pytorch models.
Filesystem Actions READ: read files, list directory contents
ExecutionBoundaries: FILESYSTEM_READ: - path in PythonEnvironment - path in ModelFolder - path == /usr/lib/x86_64-linux-gnu/libc.so.6 - path == /proc/meminfo - path == /etc/ld.so.cache - path == /etc/ld.so.preload - allow working-directory queries
MODIFY: create, write, delete, or rename files and directories EXEC: execute programs or scripts stored in files
Device Actions ACCESS: access to hardware devices (e.g., GPU)
Network Actions ACCESS: socket creation, binding, and connection establishment
Process Actions
FILESYSTEM_EXEC: - path == /usr/lib/x86_64-linux-gnu/libc.so.6
CROSS_PROCESS: signals, IPC, synchronization primitives PROCESS_CREATE: creation of a new process
PROCESS_EXEC: - path == /usr/bin/uname - path == /usr/sbin/uname - path == /usr/local/bin/uname - path == /usr/local/sbin/uname
THREAD_CREATE: creation of a new thread EXEC: execution of a new program in a process context CONTEXT: modification of process-local state
System Actions GET_INFO: retrieval of system-wide information
DEVICE_ACCESS: - fd in {STDOUT, STDERR}
CONTROL: modification of system-wide parameters
PROCESS_CONTEXT: - allow all
Figure 6: Categories of actions used in the reference implementation (R E -M OAT). Each category abstracts one or more system calls based on their semantics and arguments, and constitutes the basis for defining execution boundaries.
SYSTEM_GET_INFO: - allow system-information syscalls - allow time-related syscalls
Listing 3: High-level execution boundaries for the loading phase of Keras models. 19
ExecutionBoundaries: FILESYSTEM_READ: - path in PythonEnvironment - path in ModelFolder - path == /usr/lib/x86_64-linux-gnu/libc.so.6 - path == /etc/ld.so.cache - path == /etc/ld.so.preload FILESYSTEM_EXEC: - path == /usr/lib/x86_64-linux-gnu/libc.so.6 PROCESS_EXEC: - path == /usr/bin/uname - path == /usr/sbin/uname - path == /usr/local/bin/uname - path == /usr/local/sbin/uname - allow memory-protection changes DEVICE_ACCESS: - fd in {STDOUT, STDERR} PROCESS_CONTEXT: - allow all SYSTEM_GET_INFO: - allow system-information syscalls
Listing 4: High-level execution boundaries for the inference phase of Keras models. ExecutionBoundaries: FILESYSTEM_READ: - path in PythonEnvironment - path in ModelFolder - path == /usr/lib/x86_64-linux-gnu/libc.so.6 - path == /etc/ld.so.cache - path == /etc/ld.so.preload - path == /sys/devices/system/cpu/cpu0/tsc_freq_khz - path == /sys/devices/system/cpu/online - path == /proc/cpuinfo - path == /proc/sys/vm/overcommit_memory FILESYSTEM_EXEC: - path == /usr/lib/x86_64-linux-gnu/libc.so.6 PROCESS_EXEC: - path == /usr/bin/uname - path == /usr/sbin/uname - path == /usr/local/bin/uname - path == /usr/local/sbin/uname - path == /usr/lib/x86_64-linux-gnu/libc.so.6 - allow syscall mprotect DEVICE_ACCESS: - fd in {STDOUT, STDERR} PROCESS_CONTEXT: - allow all SYSTEM_GET_INFO: - allow system-information syscalls
Listing 5: High-level execution boundaries for the inference phase of TensorFlow models.
20