Detecting and Understanding Vulnerabilities in Fully Homomorphic Encryption Frameworks Yiteng Peng∗ , Dongwei Xiao∗ , Zhibo Liu† , Zhenlan JI‡ , Shuai Wang∗ ∗ The Hong Kong University of Science and Technology
{ypengbp, dxiaoad, shuaiw}@cse.ust.hk † State Key Laboratory of Novel Software Technology, Nanjing University
[email protected] ‡ Nara Institute of Science and Technology
arXiv:2606.22519v1 [cs.CR] 21 Jun 2026
Abstract—Fully homomorphic encryption (FHE) allows computations to be performed directly on encrypted data without decryption, offering strong privacy guarantees for sensitive data analysis. This capability is important for privacy-sensitive applications like secure cloud computing, finance, and healthcare. The complexity of FHE schemes, however, has hindered their practical adoption. To make FHE accessible to a broader range of developers, a new generation of specialized frameworks has emerged to translate high-level FHE programs into complex FHE operations, introducing a new programming paradigm. However, the inherent complexity of FHE frameworks makes them prone to incorrect implementation logic. Unlike mere crashes, logic bugs in these frameworks can silently corrupt encrypted computation, potentially leading to severe financial losses and security vulnerabilities in FHE-enhanced applications. In this work, we introduce HERTA, the first automated testing tool tailored for FHE frameworks. HERTA leverages metamorphic testing to uncover deep-seated implementation bugs and vulnerabilities across the multi-layered FHE software stack. To that end, we design a set of novel metamorphic relations (MRs) derived specifically from FHE semantics. These MRs stress the most challenging aspects of the pipeline, enabling automated correctness testing without the need for a manual ground truth. Our evaluation of HERTA on 3 leading industry frameworks discovered 21 previously unknown bugs, several of which have already been confirmed and fixed by developers. Furthermore, our hazard analysis reveals the critical security impact these bugs pose to the integrity and availability of FHE-based services.
I. I NTRODUCTION Fully homomorphic encryption (FHE) has emerged as a transformative paradigm in the privacy-preserving computation field [1–3]. By enabling computations directly on encrypted data without decryption, FHE addresses the longstanding dilemma between data utility and privacy, facilitating critical applications in privacy-preserving machine learning, secure genomic analysis, and confidential cloud computing [4, 5]. With the maturation of underlying FHE schemes, the demand for deploying FHE in real-world production systems is growing rapidly. Despite its promise, developing FHE-enhanced applications remains difficult for non-cryptographers. Developers must navigate complex cryptographic parameters and manage ciphertext noise [6]. To bridge this gap, vendors like Google and IBM have developed high-level FHE frameworks [7, 8].
These frameworks take high-level programs as input and automatically lower them into cryptographic primitives or circuits, handling the heavy intermediate passes of parameter selection, relinearization, and bootstrapping insertion. While significantly lowering the adoption barrier, these frameworks inevitably introduce an intricate multi-layered software stack. Since the underlying cryptographic primitives lack native support for common non-linear functions, such as sine and maximum, FHE frameworks perform sophisticated adaptations to accommodate high-level inputs. Furthermore, to mitigate the inherent performance overhead of homomorphic computation, domain-specific optimizations, such as lookup table (LUT) fusion [9] and array layout permutation [7], are crucial. The complexity is further exacerbated by the need to interface with diverse underlying cryptographic libraries [10, 11] and varying lower-level FHE schemes [2, 12]. Given the sensitive nature of input data and the critical domains served by FHE [13, 14], ensuring the correctness of these frameworks is of paramount importance. Despite these high stakes, automated testing for FHE frameworks remains largely unexplored and presents unique challenges. The FHE compilation and execution pipeline, spanning frontend parsing, intermediate optimization, and backend cryptographic execution, is highly complex. Moreover, bugs in FHE frameworks often manifest as “silent errors” that subtly corrupt encrypted computations rather than obvious failures like crashes or hangs. Such silent failures are particularly insidious, as they can lead to significant security vulnerabilities in privacy-sensitive applications, as will be shown in our hazard analysis in Sec. VI-D. We introduce HERTA, the first automated and systematic testing tool designed to uncover bugs across the hierarchical stack of modern FHE frameworks. HERTA is designed on the basis of metamorphic testing (MT), a powerful software testing scheme that checks program correctness through the output consistency under specific input transformations, known as metamorphic relations (MRs) [15, 16]. Specifically, we equip HERTA with novel FHE-specific MRs that target three critical stages of the compilation and execution stack: 1) frontend mutations: targeting ciphertext types and dataflow structures to systematically transform seed programs; 2) intermediate
Runtime
Compiler Front-End
Intermediate Representation
@fhe.compiler({"x": "enc."}) def f(x): y = np.maximum(x, 1) return y + 42
%0 = "FHE.apply_lookup_table"(%arg0, %cst)... %c42_i7 = arith.constant 42 : i7 %1 = "FHE.add_eint_int"(%0, %c42_i7)... return %1 : !FHE.eint<6>
Parser
Param. Search
Optimization
Adaptation
Cryptographic Parameters
Backend Code Generation
Key Generation
Enc. Input
Compiled FHE Program Deployed with FHE libraries Executing on hardware Decrypted Result
Fig. 1: Overview of the pipeline in the FHE frameworks.
configurations: employing priority-driven pairwise testing to achieve comprehensive coverage of the vast optimization space despite high execution overhead; and 3) backend retargeting: implementing equivalent transformations across disparate FHE schemes and hardware backends. By applying these hierarchical MRs, HERTA can detect both silent logic errors and crashes that may expose new exploitation vectors, which helps protect the integrity and availability of FHE-enhanced applications. We implemented HERTA and evaluated it on three leading FHE frameworks, specifically Zama’s Concrete [9], Google’s HEIR [8], and IBM’s HELayers [7]. Our extensive testing campaign successfully uncovered 21 bugs distributed across various layers of the FHE frameworks, including 16 logic errors leading to silent incorrect calculations and 5 crashes during compilation and execution. We have reported these issues to the respective developers, with several of them already confirmed. Furthermore, we conducted a pioneering hazard analysis on the discovered bugs, demonstrating how these bugs can potentially compromise the integrity and availability of real-world privacy-preserving applications. In summary, we make the following contributions: • We advocate for a new initiative in security assurance for FHE frameworks, identifying bugs that can silently corrupt computations or cause exploitable crashes in complex FHE frameworks. • We design HERTA, an automated testing framework based on MT. It is equipped with hierarchical, FHE-specific novel MRs and design optimizations. It enables fine-grained and effective testing of the entire FHE framework stack. • We conduct a systematic evaluation on three major FHE frameworks developed by industry, identifying 21 previously unknown bugs. Our analysis and exploitation further reveal that these bugs may constitute severe security hazards in real-world, privacy-preserving applications. II. P RELIMINARY A. Fully Homomorphic Encryption Homomorphic encryption (HE) enables computations on encrypted data without accessing underlying plaintexts [17]. For a function f and sensitive input x, the client encrypts x̃ = Enc(x), the server evaluates ỹ = f˜(x̃), and correctness guarantees Dec(ỹ) = f (x). Modern FHE schemes are based on hard problems like Learning With Errors (LWE) [18]. The
fundamental challenge is noise accumulation during homomorphic operations, which grows with each operation and causes decryption failure when exceeding a threshold. Gentry’s bootstrapping [1] refreshes noisy ciphertexts by homomorphically evaluating the decryption circuit, though at significant computational cost. Modern FHE schemes target different data types and operations. BFV [19] and TFHE [12] support exact arithmetic on modular integers and bits, respectively, while CKKS [2] enables approximate real-number computations for machine learning. TFHE’s programmable bootstrapping (PBS) evaluates non-linear functions via lookup tables during noise refresh [20, 21]. BFV and CKKS use SIMD packing to encode plaintext vectors into ciphertext slots, requiring expensive rotation operations for cross-slot dependencies [22]. B. FHE Frameworks While promising for privacy, implementing applications directly on raw cryptographic primitives is difficult as it requires manual handling of complex ring arithmetic and polynomial operations. Libraries such as SEAL [10], OpenFHE [11], and TFHE-rs [23] alleviate this problem to some extent by implementing common FHE primitives like addition, multiplication, and rotation for various FHE schemes, such as BFV, CKKS, and TFHE. However, developers are still required to possess deep cryptographic expertise to select secure encryption parameters, manually manage noise growth, and adapt non-linear functions using a limited set of supported operations. Improper configuration can lead to decryption failures or performance degradation of several orders of magnitude. To democratize FHE, industry leaders developed high-level frameworks like Google’s HEIR [8], Zama’s Concrete [9], and IBM’s HELayers [7], abstracting underlying FHE libraries. Fig. 1 demonstrates a high-level workflow of FHE frameworks.1 These frameworks accept high-level programs, such as Python, MLIR, or domain-specific APIs, as input and automatically lower them into optimized invocations of lowlevel FHE primitives in the FHE libraries, such as OpenFHE or SEAL. By abstracting away the underlying cryptographic complexities, they allow developers to focus on core business logic. Since FHE operations are 103 × to 106 × slower than 1 We will use the term “compiled program” to refer to the output of FHE frameworks, which will be API calls to underlying FHE libraries in our context. The term “runtime execution” refers to the execution of API calls on FHE libraries.
plaintext execution [24, 25], these frameworks apply crucial optimizations like array layouts, LUT fusion, and bootstrapping scheduling to enhance efficiency. FHE frameworks perform automatic analysis to synthesize cryptographic parameters satisfying security requirements while accommodating noise constraints. Since user programs exhibit diverse computational depth and characteristics, static policies are insufficient. Modern frameworks employ sophisticated techniques to precisely estimate noise accumulation and optimize bootstrapping scheduling, balancing between conservative strategies that guarantee correctness but incur prohibitive performance penalties and aggressive optimizations that maximize efficiency. Beyond parameter selection, FHE frameworks bridge the semantic gap between high-level program semantics and restricted FHE arithmetic. This includes mapping native data types to cryptographic representations (for example, ConcreteML [26] quantizes floating-point values for TFHE [12]) and approximating unsupported operators like non-linear functions via polynomial approximation [27] or LUTs [20]. These adaptations create a vast design space for trading off accuracy and efficiency. Frameworks also implement multilayered optimizations, from generic passes like dead code elimination to domain-specific transformations such as tensor layout optimization [7] and LUT fusion [9]. Note that modern FHE frameworks adopt modular designs supporting backend retargetability, allowing programs to target different hardware like CPUs and GPUs, various cryptographic libraries like SEAL [10] and OpenFHE [11], or FHE schemes like BFV [19] and BGV [28]. While this flexibility enhances portability, it creates a deep and heterogeneous software stack where each layer, from semantic bridging and parameter selection to optimization and code generation, introduces distinct failure possibilities. C. Metamorphic Testing Traditional testing relies on test oracles (ground truth) to verify output correctness. However, in complex domains like compilers and cryptography, determining correct outputs is often intractable, a challenge known as the oracle problem [29]. Metamorphic testing (MT) [15] addresses this by verifying invariant properties termed metamorphic relations (MRs). Formally, given a source test case Is and transformation T , MT generates a follow-up test case It = T (Is ). The system executes both to obtain outputs Os and Ot , and an MR prescribes the expected relation R between these outputs. Any violation indicates a potential defect, bypassing the need for reference implementations. Example. Consider testing a sine function implementation. We can use the mathematical property sin(x) = sin(π − x) as an MR. Given source input Is = x, we apply transformation T (x) = π − x to generate follow-up input It = π − x. After computing Os = sin(Is ) and Ot = sin(It ), the MR requires Os = Ot . If this equality fails, the sine implementation contains a bug.
III. M OTIVATION A. Vulnerability and Significance FHE has transitioned from a theoretical construct to a cornerstone of privacy-preserving computation, driven by major industrial frameworks such as Zama’s Concrete, Google’s HEIR, and IBM’s HELayers. With increasing adoption in high-stakes domains like finance and healthcare [13, 14], the correctness of their implementation becomes paramount. Unlike traditional crashes, vulnerabilities in FHE frameworks often manifest as “silent errors”, which are logic bugs that corrupt encrypted results without outward failure. The realworld impact of such errors can be catastrophic: in finance, a silent miscalculation during encrypted auditing could lead to manipulated credit assessments while the source code appears semantically correct; in healthcare, it could lead to incorrect medical diagnoses based on corrupted genomic analysis, directly endangering patient safety. Given these significant economic and safety risks, systematically detecting these logic flaws is necessary to safeguard the integrity of privacysensitive applications. B. Research Challenges Testing FHE frameworks poses unique challenges compared to traditional compiler testing, stemming from the novel cryptographic characteristics discussed in Sec. II. To make this powerful paradigm accessible to developers lacking deep cryptographic expertise, modern FHE frameworks abstract away the underlying mathematical complexity. They attempt to encapsulate intricate computations within the rigid constraints of cryptographic primitives. This abstraction requires a complex compilation process, which must not only translate familiar high-level constructs into low-level arithmetic circuits but also meticulously manage properties unique to FHE, most notably ciphertext noise growth. These cryptographic constraints introduce tight coupling and potential failure modes at every level of the framework’s hierarchical stack, from the high-level language frontend to the backend scheme selection. Traditional compiler testing or analysis methods, however, are ill-equipped to systematically explore this intricate space, as they lack the domain awareness necessary to reason about such FHE-specific, crosslayer concepts. Below, we provide three concrete examples to illustrate fundamental testing challenges: logic errors in highlevel lowering, complex configuration interactions, and crossbackend semantic inconsistencies. C. Challenge I: High-Level Program Lowering The first challenge in testing FHE frameworks stems from two fundamental properties of the underlying cryptography. As mentioned in Sec. II, a unique challenge of FHE is the management of ciphertext noise, where excessive noise accumulation can render ciphertexts undecryptable. Therefore, an essential responsibility of an FHE framework is to manage the noise budget by synthesizing appropriate encryption parameters and strategically inserting bootstrapping operations to reset noise levels. Furthermore, the framework must bridge
scalar = fhe.constant(4) arr = fhe.array([0, 1, x, 2, 3]) z = scalar * arr return z[2] + x
t = np.abs(-2) | np.abs(15) v = np.maximum(x, -13 + y) w = (v >= t) return t + w + x
# backend = SealCkksContext () v = 12 + (5 - x) w = -14 - (-7 - v) return w + (-7 - v)
(a) Logic error in high-level program lowering. The Concrete framework mishandles the dataflow when mixing encrypted inputs with array packing, leading to a silent bug.
(b) Assertion failure triggered by optimization choice. Enabling the specific THREE_TLU_CASTED strategy causes an internal compiler error on valid code.
(c) Semantic divergence across backends. The identical HELayers code yields correct results on OpenFHE but fails silently on the SEAL backend.
Fig. 2: Simplified motivating examples discovered by HERTA. These cases demonstrate that vulnerabilities permeate the entire multi-layered FHE framework, from high-level code lowering (a) and optimization configuration (b) to backend switching (c).
the gap between high-level data structures and low-level polynomial ring representations via correct and efficient encoding. This involves non-trivial transformations, such as representing floating-point numbers on integer rings through scaling, or packing multiple data elements into single ciphertexts to maximize throughput. The interplay between data representation and noise management further creates complex dependencies that the framework must navigate correctly. This complexity often manifests as silent bugs where an incorrect implementation leads to precision loss or corrupted results without obvious manifestations. Generating test cases that can systematically stress these interconnected properties is a crucial yet challenging task for uncovering deep, lowering-related vulnerabilities. Consider the program from the Concrete framework [9] in Fig. 2a: the program attempts to construct an array by mixing an encrypted scalar input x with plaintext constants in line 2. This operation forces the framework to handle data layout transformations, while managing the broadcasting of operations in line 3. Although the framework handles the array creation and element extraction correctly, it fails to maintain semantic consistency when the extracted value is added back to the original input x. This bug exemplifies the difficulty in managing hybrid dataflows and packing strategies. Without targeted test cases that specifically exercise these aspects, such logic errors about computational depth and data encoding would remain hidden. D. Challenge II: Configuration Complexity Unlike traditional compilation, where most operations can be deterministically mapped to assembly instructions, highlevel FHE operations, such as comparison, maximum, ReLU, lack direct primitive equivalents in the underlying cryptographic schemes. Instead, FHE frameworks must synthesize these operations into sequences of supported primitives. This synthesis is not unique; a single high-level operator can often be lowered via multiple distinct strategies. For instance, a ReLU function can be approximated using a polynomial [30], or evaluated precisely via LUTs [31]. Each lowering strategy offers distinct trade-offs in accuracy, performance, and noise consumption, and frameworks often provide users with configuration options to select among these strategies. This configurability creates a combinatorial explosion of compilation paths where specific settings can interact unexpectedly with program constructs. Consequently, effective testing requires more than valid input generation; it neces-
sitates a systematic exploration of this vast configuration space. The code in Fig. 2b demonstrates a subtle bug arising from such configuration interactions. It executes successfully by default but crashes with an internal assertion failure when the min_max_strategy_preference is set to THREE_TLU_CASTED, i.e., instructing the framework to synthesize the min and max operations using three LUTs. This demonstrates how valid programs can fail due to specific configuration-dataflow interactions. Since exhaustively exploring this combinatorial space is prohibitively expensive, we need a prioritized testing strategy to navigate its complexities. E. Challenge III: Inconsistent Backend/Scheme Modern FHE frameworks are designed to be modular, often supporting multiple cryptographic schemes, such as BGV, CKKS, and TFHE, and backend libraries, including OpenFHE [11], SEAL [10], HElib [32]. As discussed in Sec. II, FHE schemes differ fundamentally in their capabilities; some, like BGV, perform exact integer arithmetic, while others, like CKKS, perform approximate arithmetic on real numbers. Furthermore, different backend libraries encapsulate FHE primitives differently, exposing unique parameters and exhibiting distinct performance and noise management characteristics even when implementing the same scheme. While this modularity offers flexibility, it creates unique challenges in finding subtle bugs that lead to cross-stack inconsistencies. Naı̈vely cross-checking outputs across different backends or schemes is insufficient, as these components have inherent semantic differences. For instance, checking consistency between an exact integer scheme and an approximate arithmetic scheme requires a nuanced oracle that tolerates mathematical divergence. Such semantic gaps complicate testing oracle design. Moreover, translating a single highlevel program into correctly configured primitives for different backends is a complex and error-prone process. Fig. 2c illustrates a semantic divergence we discovered in HELayers. A valid CKKS program executes correctly with OpenFHE but silently computes incorrect results on the SEAL backend. This discrepancy highlights the difficulty of ensuring semantic preservation across heterogeneous cryptographic foundations. Uncovering these bugs necessitates systematically exploring the combinatorial space of backend and scheme configurations to detect such subtle but critical inconsistencies. The cross-product of multiple schemes, diverse backend libraries, and varying hardware targets creates a vast testing
① FHEAware Seed Generation
IV. M ETHODOLOGY Study Scope. To systematically address challenges in Sec. III, we introduce HERTA, a testing tool tailored for FHE frameworks. In general, HERTA exposes two categories of implementation defects: Logic Errors. These bugs occur when the program executes to completion, but the decrypted result is mathematically incorrect. Such errors often stem from the intricate compilation and runtime pipeline, where the complexities of data representation, noise management, and scheme-specific optimizations can lead to silent miscompilations. Given that FHE frameworks are increasingly deployed in privacysensitive domains (Sec. III-A), these silent errors are critical, as they undermine the computation and lead to flawed usage. • Crashes. This category encompasses explicit failures where the FHE framework or the compiled program terminates abnormally. These defects typically manifest as internal assertion failures or memory corruption events that occur either during the complex compilation phases or at runtime. •
Main Users. HERTA is designed as a defensive tool for FHE framework developers and vendors, not intended for malicious exploitation. Our work assists them in testing their frameworks before release and during maintenance. As shown in Sec. VI-A, major framework developers have confirmed our approach’s utility by responding to our bug reports. HERTA uncovers subtle logic errors, particularly silent miscompilations, that are difficult to detect with existing techniques. These findings are highly critical, as they often indicate potential vulnerabilities that could be exploited in deployed privacypreserving applications [33, 34]. Formulation. We formalize the behavior of an FHE framework as a function F : P ×C → R, where P denotes the space of valid input programs, C represents the configuration space, and R is the space of decrypted results. For a given program P ∈ P and configuration Γ ∈ C, the framework execution F(P, Γ) produces a result R ∈ R. We model a framework configuration as Γ = (σ, β, Ccrypto , Cada , Copt ), which comprises the following components: the FHE scheme σ (such as BGV, CKKS, or TFHE), the backend implementation β (for example, SEAL, OpenFHE, or hardware targets like GPU), the cryptographic parameters Ccrypto (including polynomial modulus degree), the adaptation strategies Cada for non-native operations (such as the choice of approximation method for non-linear functions), and the optimization settings Copt (for instance, LUT fusion). This formalization enables defining MRs as transformations on either the program space P or the configuration space C, establishing rigorous equivalence conditions for detecting framework defects. Pipeline Overview. An overview of HERTA’s pipeline is illustrated in Fig. 3. The pipeline consists of four main components: an FHE-aware seed generator and three mutation stages that mutate the seed program with three MRs, respectively.
Validity Check
② Type and Dataflow Mutation ③ Compilation Configuration Mutation ④ Backend and Scheme Mutation
Compile & Execute
surface where semantic preservation is not guaranteed and traditional strict-equality oracles are hardly applicable.
Crash
Logic Error
Fig. 3: Overview of HERTA’s testing pipeline ➀ FHE-aware Seed Generation. This stage generates a seed program, P , with FHE-specific constructs, such as complex data types and operations requiring significant adaptation. This seed program will be used as the basis for subsequent metamorphic mutations to stress-test the FHE framework. ➁ MR1: Type and Dataflow Mutation. This MR transforms P into a semantically equivalent P ′ with FHE-specific mutations, checking if F(P, Γ) = F (P ′ , Γ). It targets FHE-specific data types and expression structures to test the framework’s management of encrypted data and computational flow. ➂ MR2: Compilation Configuration Mutation. This component systematically mutates compilation configurations, such as cryptographic parameters Ccrypto , the adaptation strategies Cada and optimization settings Copt . It uses prioritized combinatorial testing to effectively explore the vast configuration space and test the framework’s robustness across diverse settings. ➃ MR3: Backend and Scheme Mutation. This MR tests for cross-stack inconsistencies by mutating the FHE scheme σ and the backend β. By validating the equivalence of results across different cryptographic foundations, it ensures correctness across the heterogeneous FHE ecosystem, including software implementations and hardware accelerators. For each generated test case, which consists of a seed program P and a related configuration Γ, HERTA executes it and its mutated variants through the FHE framework under test. Any explicit failure, such as a segmentation fault or an assertion failure during either process, is immediately reported as a crash. If all executions are successful, their decrypted results are compared. A discrepancy between their outputs reveals a logic error. A. FHE-aware Seed Generation Design Goal. The primary objective of seed generation is to produce a corpus of initial FHE programs P that are not only syntactically and semantically valid but also specifically crafted to stress the FHE framework stack. In contrast to traditional compiler testing, our process prioritizes the generation of diverse, FHE-specific constructs. This includes deliberately generating programs with deep multiplication paths to challenge noise management, as multiplication operations significantly contribute to noise growth in FHE and thus holistically test the framework’s noise analysis and cryptographic parameter selection logic. Additionally, the programs incorporate a rich variety of operations that lack direct mappings to FHE primitives, such as comparisons and non-linear functions, which require complex adaptation strategies.
Grammar of FHE Programs. To systematically guide this process, we define a context-free grammar, ΛHERTA , which abstracts the syntax of high-level FHE programming languages. ΛHERTA is not designed to cover every syntactic detail of a specific FHE framework. Instead, it is intentionally simple yet expressive enough to capture the essential FHE programming features, ensuring that generated programs are adaptable across different frameworks. Fig. 4 shows the selected syntax of ΛHERTA . The grammar explicitly distinguishes two concepts critical to FHE: the privacy type ω, which separates plaintext from encrypted values, and the indirect operators π, which represent high-level operations lacking direct support from FHE primitives and thus requiring adaptation. Program Statement
P s
::= ::=
Assignment Variable Constant BasicOp IndirectOp BasicType PrivacyType Type Expression
assign v c ⊗ π φ ω τ e
::= ::= ::= ::= ::= ::= ::= ::= ::=
s; P | s assign | other structure supported by FHE frameworks v := e valid variable name valid values of type φ +|−|× operators not directly supported int | float plaintext | encrypted {ν : φ | ν : ω} c | v | e1 ⊗ e2 | π(e1 , . . . )
Fig. 4: Selected abstract grammar for FHE programs. Program Generation Algorithm. The generation of a program begins with the start symbol P and recursively expands non-terminal symbols by replacing them with symbols from the right-hand side of applicable production rules, continuing until only terminal symbols remain. For non-terminals with multiple production options, the choice is made based on dynamically adjusted probability weights that prioritize FHEspecific constructs. Specifically, the generator maintains a context tracking the current circuit depth and the distribution of operator types. As the program grows, probability weights are dynamically adjusted. Initially, the generator biases towards arithmetic operations (such as multiplication) to rapidly accumulate multiplicative depth; as the multiplicative depth reaches a configurable threshold, the bias shifts towards frameworkspecific indirect operations (such as comparisons) to increase functional diversity without exceeding cryptographic constraints. This strategy stresses the noise management of FHE frameworks while enriching seed diversity for subsequent metamorphic mutations. To ensure seed validity, we adopt lightweight self-checking during framework execution: seeds that cause compilation or runtime failures (such as exceeding the noise budget) are discarded and regenerated, whereas unexpected crashes are reported as potential bugs. When the framework provides a plaintext reference, the seed’s decrypted result will also be checked. Alg. 1 outlines the core logic of our expression generation process. After A DJUST O P W EIGHTS in line 2 dynamically calibrates operation probabilities as discussed above, the G EN -
E XPR function invokes S ELECT O P to determine the next operation type. This selection is driven by dynamically adjusted weights in line 3 that prioritize arithmetic operations early to accumulate depth, while shifting towards indirect operations π later to maximize feature coverage. Subsequently, operands are chosen via a dual-mode heuristic. The generator stochastically toggles between selecting high-depth variables in line 7 to stress noise management and infrequently used variables in line 9 to foster diverse combinations of multiplication depths. Finally, the symbol table Σ and annotation set A are updated to reflect the new state, enabling context-sensitive decisions in subsequent expressions. Upon completing the recursive construction of program P , HERTA appends a default configuration Γ, which results in a complete executable seed (P, Γ) and serves as the foundation for the MRs below. Algorithm 1 Expression Generation Strategy 1: function G EN E XPR(Symbol Table Σ, Annotation Set A, Max Depth Dmax ) 2: W ← A DJUST O P W EIGHTS(A, Dmax ) 3: op ← S ELECT O P(W ) 4: A ← U PDATE A NNOTATION(A, op) ▷ Record used operations 5: for i in 1 . . . N UM O P E XPR(op) do 6: if C HOOSE M AX D EPTH VAR() then 7: opExpri ← G ET M AX D EPTH VAR(Σ) 8: else 9: opExpri ← G ET L EAST U SEDVAR(Σ) 10: Σ[opExpri ].usage ← Σ[opExpri ].usage + 1 11: expr ← op(opExpr1 , . . . ) 12: return expr, A
B. MR1: Type and Dataflow Mutation Design Goal. This MR addresses Challenge I (Sec. III-C) by stressing the FHE framework’s implicit data layout and noise management. It systematically mutates the seed program P into equivalent variants P ′ with data-centric transformations that directly impact how data is represented and how computations are orchestrated in the encrypted domain. It then validates whether F(P, Γ) = F (P ′ , Γ). Specifically, this MR mutates programs with two complementary focuses. It modifies data types and layouts, such as variable bit-widths and tensor shapes, to stress the framework’s data packing and representation logic. It also reshapes the dataflow structure of the computation graph to challenge the framework’s noise management capability. Mutation Operators. To systematically explore the program space sensitive to FHE properties, we introduce a set of specialized mutation operators divided into two categories: type mutations and dataflow mutations.2 Type mutations. This category stresses the framework’s ciphertext data representation and packing logic. Key operators include: ➀ Data type alteration, which promotes variable types and bit-widths (such as Secret[i16] → Secret[i32]) to test the framework’s parameter selection logic under varying data range constraints; ➁ Data layout mutation, which systematically transforms scalar operations into vectorized 2 Here “type” refers to data types and layouts in the FHE context.
tensor computations. For example, this operator can employ a “wrap-compute-unwrap” pattern, transforming x × y into [1,x,2]×y [1]. This embeds scalars into arrays before computation and then extracts the corresponding result, forcing the FHE framework to exercise complex ciphertext packing and index resolution paths; and ➂ Plaintext-ciphertext swapping, which toggles the encryption status of operands between plaintext and ciphertext representations to verify the correctness of arithmetic involving mixed privacy levels. Although simple to implement, these transformations can trigger diverse optimization paths by forcing the framework to re-synthesize cryptographic parameters and adapt data packing strategies. Dataflow mutations. This category targets the computational graph structure to challenge the framework’s noise management and optimization capabilities. We design the following mutators: ➃ Expression reshaping, which transforms algebraic expressions, such as rewriting v × v as v 2 , or v + v as 2 × v. This transformation potentially alters multiplication depth and exercises different lowering paths; ➄ Redundant operation insertion, which injects redundant arithmetic operations in the ciphertext domain, such as transforming x + y into (x + 0enc ) + (y × 1enc ). This preserves the mathematical outcome while forcing the framework to manage additional noise accumulation, which stresses the framework’s noise estimation accuracy and management capability; and ➅ Equivalent nonlinear insertion, which replaces a variable with an equivalent non-linear function, such as v → max(v, MIN_VAL). As the transformed expression cannot be directly mapped to FHE primitives, the framework must invoke complex adaptation strategies, thereby testing the adaptation layer. C. MR2: Compilation Configuration Mutation Design Goal. This MR addresses Challenge II (Sec. III-D) by systematically mutating framework configurations Γ. Unlike MR1, which alters the program P , this relation focuses on the compilation configurations of FHE frameworks. Specifically, it mutates the adaptation strategies Cada that control the lowering of high-level non-linear operations, the optimization settings Copt that govern passes like subexpression elimination, and the cryptographic parameters Ccrypto such as polynomial modulus degree when applicable. Note that this MR does not mutate the scheme σ or backend β, which are handled by MR3. Regardless of the specific configuration Γ′ applied, a correct FHE framework should yield consistent results, i.e., F(P, Γ) = F (P, Γ′ ). Technical Challenge. As highlighted in Sec. III-D, the configuration space of FHE frameworks is combinatorially large, with dozens of configurable parameters that interact in subtle and non-obvious ways. For instance, a high-level comparison operation can be lowered using multiple adaptation strategies, each presenting different trade-offs in performance, precision, and noise management. When compounded with global optimization flags or adjustable cryptographic parameters when applicable, the number of possible configuration combinations can easily scale into thousands or tens of thousands. Exhaustively executing all configurations is computationally
Algorithm 2 Prioritized Configuration Testing 1: function P RI C ONFIG MT(Program P , Initial Configuration Γ, Max Tests Nmax ) 2: R ← E XECUTE(P, Γ) 3: Ops ← R ETRIEVE O PERATORS(P ) 4: P arams ← G ET R ELEVANT PARAMETERS(Ops) 5: P aramList ← G EN P RIORITIZE S UITE(P arams, Γ, Nmax ) 6: for each Γ′ in P aramList do 7: R′ ← E XECUTE(P, Γ′ ) 8: if Execution Failed then 9: R EPORT P OTENTIAL C RASH( ) 10: if R′ ̸= R then 11: R EPORT I NCONSISTENCY( ) 12: return
infeasible due to the high cost of FHE execution (often several orders of magnitude greater than plaintext). Naı̈ve random sampling also proves inefficient, often wasting resources on configurations that do not impact the given program’s behavior. This necessitates a targeted strategy to efficiently navigate promising regions of the configuration space. Prioritized Combinatorial Testing. To address the challenge of combinatorial explosion, HERTA employs a two-pronged strategy that integrates systematic coverage of combinatorial testing with a diversity-guided prioritization heuristic. This approach efficiently navigates the unified parameter space, spanning Cada , Copt , and Ccrypto , by ensuring diversity of the selected configurations. The foundation of our strategy is pairwise (2-way) combinatorial testing. As a widely adopted method [35], this technique is proven to effectively detect interaction-triggered faults while significantly reducing the configuration search space by generating a candidate suite that covers every possible pair of parameter values. We choose 2-way coverage as it provides a practical balance: empirical studies show that most configuration bugs involve relatively few parameter interactions [36, 37], while higher-way coverage incurs prohibitive costs. However, given the high computational cost of FHE execution, running the full pairwise suite remains impractical. To address this, HERTA further refines the test suite through a usage-aware pruning mechanism followed by diversity-guided selection. To prune parameters that may not influence the program’s behavior, HERTA excludes parameters that do not correspond to any operators present in the seed program P . For instance, if a seed program contains only linear arithmetic, there is no need to test adaptation strategies for non-linear operations like LUT configurations. This prevents wasting resources on logic paths that are never exercised. Another prioritization heuristic is based on maximizing configuration diversity. When selecting the next configuration from the candidate suite, HERTA selects the one that maximizes distance from previously executed configurations. By prioritizing configurations that differ significantly from prior tests, HERTA increases the likelihood of uncovering corner cases where multiple configurations interact in unexpected ways. Alg. 2 outlines this prioritized combinatorial testing process. Line 2 derives a reference output by executing the seed
program P with the default configuration Γ. Line 3 retrieves an operator set Ops that records all operators in P . The function G ET R ELEVANT PARAMETERS then prunes the full configuration space by only selecting parameters relevant to the operators used in P , particularly the adaptation strategies. Based on this refined scope, line 5 enumerates all pairs of configurations and orders them according to the Hamming distance from previously selected configurations, then selects the top Nmax configurations to form a prioritized test suite P aramList, where Nmax is a user-defined budget to limit the total number of tests. For each variant configuration, HERTA executes the program under Γ′ and reports any crashes or inconsistencies (lines 6–11). D. MR3: Backend and Scheme Mutation Design Goal. This MR addresses Challenge III (Sec. III-E) by validating the framework’s correctness across different cryptographic and hardware components. The FHE ecosystem is inherently heterogeneous, characterized by a rich diversity of cryptographic schemes and backend implementations. This MR generates test cases by keeping the program P constant while mutating the backend execution environment β or the cryptographic scheme σ. The testing oracle asserts that the program’s output should remain invariant regardless of the underlying execution engine. Specifically, we target three distinct layers of cross-stack inconsistencies by systematically configuring the FHE framework to execute the same program P under different settings, including hardware accelerators (such as CPU vs. GPU), backend libraries (including OpenFHE vs. SEAL), and target FHE schemes (such as BFV vs. BGV). Cross-Stack Consistency Checking. As described in Sec. III-E, defining a universal oracle for all these schemes and backends is inherently challenging due to their heterogeneous nature. To address this, HERTA employs an adaptive tolerance-based oracle that adjusts its strictness based on the settings being compared. When comparing settings involving exact integer arithmetic only, such as BGV vs. BFV, or CPU vs. GPU on integer logic, the oracle enforces strict equality, ensuring F(P, Γ) = F(P, Γ′ ). However, for settings involving approximate arithmetic, such as the CKKS scheme, inherent floating-point and approximation deviations between backends are expected. In these cases, HERTA relaxes the equivalence condition to a tolerance-based check, verifying that the divergence falls within a statistically acceptable error bound ϵ. We set ϵ to 1% relative error based on analyzing benign floating-point deviations across 1,000 test cases with known correct outputs, where observed deviations consistently remained below 0.5%. As empirically validated in Sec. VI-B, this approach effectively exposes significant semantic divergences indicative of framework bugs. Putting it all Together. The three MRs are complementary and collectively provide comprehensive coverage of the FHE framework stack. MR1 stresses data representation and noise management by mutating program structure, uncovering bugs in data layout transformations and computational depth tracking. MR2 explores the vast configuration space, exposing bugs
in adaptation strategies and optimization interactions. MR3 validates cross-stack consistency, detecting semantic divergences across heterogeneous backends and schemes. Together, these MRs systematically exercise the critical components identified in Sec. III, enabling HERTA to detect both silent logic errors and explicit crashes across the entire FHE framework pipeline. In real usage, HERTA iteratively generates seed programs and applies each MR to the seeds, enabling fine-grained defect detection across diverse FHE framework implementations and configurations. V. I MPLEMENTATION AND E XPERIMENT S ETUP Implementation. We implemented HERTA primarily in Python, with approximately 3,300 lines of code (LOC). All experiments were conducted on a server equipped with an AMD Ryzen 3970X 32-Core Processor, 256GB of RAM, and an NVIDIA GeForce RTX 3090 GPU. To foster reproducibility and facilitate future research, we have open-sourced our artifact at [38]. Key hyperparameters, such as the length of seed programs, are calibrated based on a preliminary study of typical FHE programming patterns and are documented in our artifact. Target FHE Frameworks. We comprehensively reviewed current FHE frameworks and selected three representative frameworks as our evaluation subjects: Google’s HEIR, Zama’s Concrete, and IBM’s HELayers. These frameworks are maintained by major industrial companies and have achieved significant community traction. Furthermore, they cover a diverse range of compilation strategies, cryptographic schemes, and hardware backends. We tested the latest stable releases at the time of our experiments: Concrete v2.11.0, HELayers v1.5.5.3, and HEIR v0.0.2. HEIR is an MLIR-based FHE infrastructure offering a multi-level lowering pipeline. In our experiments, we primarily target its Python frontend to evaluate high-level compilation flows. Concrete specializes in the TFHE scheme and supports multiple execution backends, including CPU and GPU. Notably, it provides the most extensive support for high-level non-linear operators (such as sin, cos) via LUTs. HELayers offers various interfaces ranging from ONNX models to direct mathematical APIs. To ensure a fair and consistent evaluation across all frameworks, we specifically target the mathematical API layer, as it is the common interface supported by all three frameworks. Nevertheless, the methodology of HERTA is designed to be input-agnostic and can be extended to support other high-level interfaces (such as ONNX) in future work (further discussed in Sec. VII). VI. E VALUATION We evaluate HERTA’s effectiveness and analyze detected bugs using these research questions: RQ1: How effective and efficient is HERTA in detecting bugs across different frameworks? RQ2: How do individual components in HERTA’s testing pipeline contribute to bug detection?
TABLE I: Summary of bugs detected by HERTA. “Logic Errors” refer to cases where the FHE framework completes execution but produces incorrect results. “Crashes” include obvious manifestations like assertion failures or memory errors. Bug Type Logic Error Crash
FHE Framework
Total Bugs
Concrete (Zama) HEIR (Google) HELayers (IBM)
15 4 2
12 2 2
3 2 0
Total
21
16
5
RQ3: What root causes led to the defects of the FHE frameworks, and what lessons can be derived? RQ4: What are the potential real-world security implications of the detected bugs in FHE frameworks? A. Overall Effectiveness and Efficiency Overall Effectiveness. Table I summarizes the bugs found by HERTA during a continuous testing campaign. We detected 21 previously unknown bugs across the three frameworks. Notably, 76.2% (16/21) are logic errors, which can have critical implications and are often more subtle and challenging to detect than crashes. Unlike crashes, logic errors allow the framework to complete computation but yield incorrect results. Triggering these errors often requires intricate operation sequences. For instance, certain logic bugs in Concrete only manifest when an encrypted input is placed at a specific index within an array and subsequently reused, or when arithmetic multiplication interacts with specific non-linear operations (such as maximum) in a precise order. Such errors are subtle as they do not lead to immediate crashes. Due to the privacypreserving nature of FHE, incorrect computations may go undetected in privacy-sensitive applications and cause severe consequences such as incorrect medical diagnoses or financial model divergence. A higher concentration of bugs is observed in Concrete (15/21), despite the similar testing effort across all frameworks. This distribution is expected and can be attributed to the framework’s richer feature set and diverse optimization pipeline. Unlike other frameworks that support a limited set of arithmetic operations, Concrete supports a rich set of NumPycompatible operators, ranging from standard arithmetic to complex bitwise logic and non-linear functions like cosine and logarithm. Furthermore, it exposes a diverse set of configurations to optimize for the underlying TFHE scheme. While these features significantly enhance performance and usability, they also introduce a larger attack surface for latent bugs. In contrast, the current versions of HELayers’ mathematical layer and HEIR offer limited support for diverse non-linear functions and complex configurations. Bugs in these frameworks were primarily triggered by our diverse FHE-aware high-level programs and cross-stack consistency checks, identifying fundamental inconsistencies in how highlevel arithmetic is lowered to cryptographic backends like SEAL and OpenFHE. This distribution reflects the current
TABLE II: Efficiency statistics of HERTA. “Avg. Gen. Time” represents the average time to generate a test case. “Avg. Latency” reports time overhead per test case, which is dominated by FHE compilation and execution. “Throughput” reflects the end-to-end testing speed. Framework
Valid Rate
Avg. Gen. Time
Avg. Latency Compile & Execute
Throughput (cases/core · h)
Concrete HEIR HELayers
96.9% 100.0% 99.9%
1.87 ms 0.38 ms 1.24 ms
189.46 s 21.12 s 5.20 s
18.99 170.72 690.72
Average
98.93%
1.16 ms
71.93 s
293.48
developmental stage of frameworks like HEIR, where frontend diversity and arithmetic complexity are still evolving. The comprehensive results on Concrete demonstrate HERTA’s capability to provide continuous integration testing for all FHE frameworks as they mature and expand their operator support. To ensure validity, all detected bugs were independently triaged by two experts in FHE frameworks (also authors of this paper), who reached consensus that all 21 bugs are indeed true positives. We have responsibly reported all our findings to FHE framework developers, and several issues have been positively acknowledged with patches actively being developed [39]. Validity of Seeds. The effectiveness of MT relies heavily on the quality of the initial seed programs. If generated seeds are syntactically invalid or fail to compile due to trivial errors, subsequent mutations become futile. However, generating valid programs in the FHE context presents unique challenges beyond conventional “syntactic correctness.” FHE frameworks often impose strict validity constraints driven by performance optimizations or underlying scheme limitations. For instance, Concrete rejects intermediate values that exceed specific bitwidths, while HELayers aborts computations if accumulated noise surpasses the decryption threshold. Thus, a program can be syntactically correct yet cryptographically invalid. To balance generation efficiency with implementation complexity, we prioritized testing throughput over guaranteeing validity via expensive static analysis of cryptographic properties. Instead of employing heavyweight dataflow analysis to predict intermediate values or noise budgets, we adopted a lightweight iterative regeneration strategy. If a generated seed violates cryptographic constraints during the dry-run, HERTA discards it and attempts to regenerate a compliant variant. With a maximum of five retry iterations, our approach achieves an average validity rate of 98.93%, including those that successfully trigger framework vulnerabilities during the self-checking (Sec. IV-A). Among the small number of invalid seeds, approximately 23% were discarded due to execution timeouts, while the remainder failed due to intractable cryptographic parameter unsatisfiabilities. We excluded these computationally expensive cases for testing efficiency. The high validity rate demonstrates that our strategy practically and effectively navigates the strict FHE constraints, allowing the testing budget to focus on stressing deep logic rather than trivial validity issues. Throughput. Testing FHE frameworks faces the inherent
x = hint(x, bit_width=8) y = hint(y, bit_width=16) z = np.minimum(x, y) return z + y
t = y - 12 t = (9 == t) * 1 t = (t - 1) * 8 return t * -10
(a) Logic bug triggered by specific bit-width combination.
(b) Discrepancy bug between CPU and GPU backends.
Fig. 5: Representative bug cases found by different MRs. challenge of expensive FHE computations, which are often orders of magnitude slower than plaintext execution. Despite this challenge, HERTA maintains practical testing throughput, generating and processing an average of 293.48 test cases per core hour. As detailed in Table II, seed generation time is negligible, with primary latency stemming from compilation and execution phases. The variance in throughput across frameworks primarily reflects differences in the underlying cryptographic schemes and operator complexity—Concrete’s TFHE scheme with extensive operator support incurs higher costs than the more streamlined implementations in HEIR and HELayers. For our large-scale evaluation involving 3,000 seed programs, the entire testing campaign consumed an average computational cost of approximately 303.7 core-hours per framework. While the intrinsic characteristics of FHE operations limit absolute testing speed compared to traditional compiler fuzzing, the high bug detection count shown in Table I validates the efficiency of our design. Our crafted seed generation and targeted MR designs ensure that each executed test case is highly effective, enabling the discovery of hidden logic errors within a reasonable computational budget. Summary. In summary, our evaluation demonstrates that HERTA achieves high effectiveness (21 previously unknown bugs across three major frameworks) with practical efficiency (293 test cases per core hour on average), while maintaining a high seed validity rate (98.93%). The predominance of logic errors ( 16 21 = 76.2%) underscores the importance of our MT approach, as these silent bugs are particularly challenging to detect and can have severe real-world consequences in privacysensitive applications. B. Effectiveness of Individual Components TABLE III: Distribution of unique bugs detected by each component of HERTA. “FHE-aware Seed Gen.” denotes bugs triggered by the initial seed alone, while MRs indicate bugs revealed only after specific mutations. Component
FHE-aware Seed Gen.
MR1 Program
MR2 Config.
MR3 Backend
Bug Count
12
5
1
3
To verify the necessity of our multi-layered testing strategy, we analyze the distinct contributions of the FHE-aware seed generator and each MR. As detailed in Table III, which presents the distribution of unique bugs detected by each component, the seed generator accounts for about half of the findings. This confirms that our generation strategy, prioritizing high multiplicative depth and diverse non-linear
functions, successfully covers edge cases often overlooked by FHE framework developers, laying a robust foundation for subsequent metamorphic mutations. However, this does not diminish the value of our MRs; on the contrary, the following analysis reveals that the MRs target orthogonal classes of vulnerabilities that are unreachable by seed generation alone. MR1 is indispensable for detecting logic bugs, particularly those related to data types and complex arithmetic rewriting. While the seed generator creates valid syntax, it often defaults to standard types and structures. HERTA’s MR1 proactively mutates data types and structure layouts via semantically equivalent transformations. This capability was crucial for detecting the array-related logic bug shown in Fig. 2a and the critical memory safety violation discussed in Sec. VI-D. These vulnerabilities were exposed solely by MR1’s layout mutations, as they require specific structural perturbations that are difficult to achieve by the seed generator alone. To further illustrate MR1’s effectiveness, we present a case study regarding type mutation in Fig. 5a. This logic bug produces output drastically different from the expected result when input y is mutated to a 16-bit width. For instance, given inputs x = 1 and y = 2, the program may output random 5digit integers (such as 19796) instead of the expected 3. Our analysis of the intermediate representation of the compiled circuit showed no anomalies, indicating the flaw lies deep within the underlying backend’s handling of mixed-precision operations. Without MR1’s targeted type mutation, such backend defects are difficult to expose systematically. MR3 proves critical for verifying consistency across different computing backends and encryption schemes. Fig. 5b illustrates a simplified representative case demonstrating the synergy between our seed generator and MR3. The generated program, involving comparison operations and scalar arithmetic, executes correctly on the CPU backend. However, when MR3 switches to the GPU backend, the program yields unstable and incorrect results. Through systematic minimization, we found that the bug disappears when either the multiplicative depth is reduced or the comparison operation is removed. This suggests a flaw in how the GPU backend handles the lowering of comparison operators under deep multiplicative chains. This case highlights that the complex logic generated by our seed generator requires MR3’s cross-backend validation to expose backend-specific implementation gaps that would otherwise remain latent. While MR2 detected relatively fewer direct bugs, the nature of these discoveries is significant. The bug shown in Fig. 2b demonstrates a critical availability threat: an assertion failure triggered exclusively by a specific, valid configuration choice. Such configuration-dependent failures are particularly insidious as they represent hidden vulnerabilities that only manifest under specific optimization settings, making them harder to detect than standard crashes. Beyond confirmed bugs, MR2 also revealed numerous instances of “noise overflow runtime errors” during configuration transitions. Although we conservatively excluded these from our bug count as they may represent legitimate parameter unsatisfiability, they highlight a
TABLE IV: Root-cause families of the 21 defects discovered by HERTA. L/C denotes logic errors and crashes. R1–R5 are FHE-specific; R6 captures standard robustness defects. Family #(L/C) Root Cause Family Description R1 R2 R3 R4 R5 R6
7 (6/1) Incorrect representations for scalars. 3 (2/1) Incorrect lowering for operators. 3 (3/0) Wrong value packing or materialization. 3 (2/1) Inconsistent or overly conservative selection of cryptographic parameters. 3 (3/0) Faulty backend or scheme lowering. 2 (0/2) Programs cause aborts or memory corruptions.
significant usability gap—ensuring consistent behavior across the vast configuration space remains a challenge for current FHE frameworks. The relatively lower bug count for MR2 also reflects the current stage of FHE frameworks, which are still evolving to support diverse arithmetic operators and adaptation configurations. As these frameworks mature and expand their configuration spaces, we anticipate that MR2’s systematic exploration of configuration interactions will become increasingly valuable. To further demonstrate the necessity of our FHE-aware design, we compare HERTA with Domato [40], a representative fuzzer from Google. While this grammar-based fuzzer has proven to find thousands of critical bugs in complex software, it lacks FHE-specific generation and mutation strategies. When evaluated on Concrete with the same testing budget (see Appendix A for details), Domato finds only 3 crash-triggering inputs and no logic errors. In contrast, HERTA’s FHE-aware seed generation finds 7 logic-error inputs and 7 crash inputs, while our MRs further uncover 6 additional logic-error inputs and 18 crash inputs. These results show that FHE-aware seed generation reaches FHE-specific failure paths more effectively than generic grammar generation, and that our MRs reveal complementary failures beyond seed generation. C. Analysis of Root Causes We analyze root causes of our uncovered bugs using minimized triggers, observed failures, and available developer feedback. The 21 bugs are categorized into 6 root-cause families, as summarized in Table IV. Out of these, R1–R5 (19 bugs in total) are specific to FHE framework design and implementation, while R6 captures robustness issues that also arise in general software systems. R1: Incorrect Scalar Representation. These bugs stem from mismatches between source-level scalar semantics and FHE number representations. For instance, HEIR mishandles scalar signedness during lowering: a negative constant, when used as a multiplicand for an encrypted variable, is interpreted as unsigned during bit-width extension; another Concrete bug is triggered simply by widening one operand’s declared bitwidth, with runtime values unchanged. R2: Incorrect Lowering for Operators. This type of bug miscompiles combinations of operators under specific contexts. Concrete, for instance, triggers incorrect optimizations for programs involving multiplication and maximum operators
in a particular order. Another program with a non-default configuration, THREE_TLU_CASTED, crashes due to incorrect lowering for the min/max operators. These cases do not arise from a single operator, but rather from the interaction between operator adaptation, computation, and configuration choices, and thus are hard to detect without FHE-specific testing. R3: Errors in Data Packing and Materialization. To optimize performance, FHE frameworks may pack multiple numbers into encrypted slots. Bugs in this category arise when frameworks fail to correctly bundle this data or format the final decrypted outputs. For instance, the Concrete framework corrupts computations when embedding an encrypted number into an array and later pulling it back out. In another case, a variable yielded different results depending on whether it was stored alone or alongside others. These issues highlight flaws in how frameworks translate between user variables and internal packed formats. R4: Inaccurate Noise Management and Parameter Synthesis. A unique challenge in FHE is managing the ciphertext noise that grows with each operation (multiplicative depth). Frameworks must automatically synthesize cryptographic parameters large enough to prevent this noise from corrupting the final result. Bugs in this category happen when frameworks misjudge this “noise budget.” For example, Concrete underestimated the parameters needed for deep arithmetic and comparisons, leading to silently corrupted decrypted values. Conversely, HEIR’s noise validation was overly conservative, falsely rejecting a trivially simple and valid BGV program. R5: Cross-Backend and Scheme Inconsistencies. FHE frameworks offer developers the flexibility to switch between different cryptographic backends or schemes for performance or compatibility reasons. However, we found bugs where changing this execution stack silently breaks consistency. For instance, in Concrete, a program yielded the correct result on a CPU but produced a completely different answer when run on a GPU. Similarly, in HELayers, identical computations diverged significantly between the OpenFHE and SEAL libraries, as their output differences exceeded acceptable approximation noise. These errors emphasize that semantic equivalence across the FHE ecosystem remains fragile. R6: Runtime Robustness and Memory Safety. Unlike the previous categories that cause silent data corruption, R6 manifests as explicit crashes. We found that certain program constructs bypass safe memory management within the framework. For instance, Concrete suffered fatal heap metadata corruption when an encrypted variable was placed at the end of an array. While these bugs do not corrupt the mathematical logic, they represent critical security vulnerabilities that can crash backend servers in FHE-as-a-Service environments. Design Implications. To guide future FHE framework development, we highlight key design implications by reflecting on bugs we found. By rethinking how frameworks manage layout, noise, and backend lowering, FHE framework developers can proactively eliminate major sources of silent data corruption. Lesson 1: Explicit Representation Validation. FHE frameworks often translate program representations across multiple
layers, from source-level data types and IRs to ciphertext encodings and packed backend layouts. This complexity can allow subtle bugs in data representation and packing to silently corrupt computations, as demonstrated by R1 and R3. Frameworks should encode these representation properties as explicit IR contracts and check them after passes that change layouts or types so that developers can detect miscompilations caused by representation errors early. Lesson 2: Transparent Noise Management. Current FHE frameworks treat parameter synthesis and noise budget estimation as black boxes; while this abstraction hides complexity, it also hides critical information from users, potentially causing silent data corruption or false rejections. Instead of relying solely on brittle ahead-of-time estimations, frameworks can provide built-in diagnostics. By implementing an intermediate value inspection mode that dynamically tracks noise growth during debugging, frameworks can embed runtime assertions that warn users when computations approach theoretical thresholds to prevent the silent decryption of garbage data. Lesson 3: Strict Semantic Contracts for Backend Retargeting. Frameworks often rely on implicit backend-specific adaptation and lowering strategies for different hardware platforms or FHE libraries. This can cause semantic drift across backends and confuse users. Frameworks should embed explicit semantic contracts into their IRs, ensuring that retargeting to a GPU or a different library remains semantics-preserving, or that unsupported operations are rejected with clear diagnostics. D. Security Implications of Bugs In this section, we conduct a hazard analysis to demonstrate how the latent bugs detected by HERTA can be weaponized by adversaries to facilitate stealthy attacks on real-world privacypreserving applications. As described in Sec. III-A, FHE is increasingly adopted in high-stakes, data-sensitive environments, such as genomic privacy analysis and decentralized financial auditing. Logic bugs in FHE frameworks can silently lead to incorrect medical diagnoses or financial model divergence, misleading clinicians to mistreat patients or causing financial risks to go undetected. We illustrate three representative attack scenarios that leverage our detected bugs in FHE frameworks: Integrity Breach: Subverting Source Code Review via Framework Bugs. In privacy-sensitive applications, developers heavily rely on source-level code reviews to ensure computational integrity. However, FHE frameworks can be exploited to create a severe gap between what the code appears to do and what it actually executes. A malicious contributor to an FHE application can craft code that is mathematically sound to code reviewers but triggers a framework-level lowering bug. For example, we discovered a bug in HEIR (Fig. 6) where a signed -1 scalar is improperly lowered as a large unsigned integer (e.g., 255 for 8-bit representations) when multiplied with an encrypted variable. To an auditor, −1 × liability is mathematically equivalent to subtracting liability. However, due to the bug, the compiled encrypted computation adds an amplified liability term instead of subtracting it. This allows an adversary to manipulate the net-worth score without needing
malformed ciphertexts or decryption keys, effectively breaking computational integrity at the framework layer while passing code review. In total, we found 4 similar attack patterns, where signed scalar lowering, mixed bit-width handling, and max/min operator lowering bugs can be exploited to bypass source-level audits and cause integrity breaches in high-stakes FHE applications like encrypted scoring and risk assessment. 1 def secure_net_worth(asset_bank_a: Secret[I32], 2 asset_stock_b: Secret[I32], 3 liability_loan_c: Secret[I32]): 4 total_assets = asset_bank_a + asset_stock_b 5 deduction = -1 * liability_loan_c # Miscompiled line 6 net_worth = total_assets + deduction 7 return net_worth
Fig. 6: An example of framework-assisted backdoor attack where the compiled circuit executes an amplified addition instead of the intended subtraction. Audit Evasion: Exploiting Backend Inconsistencies. Modern FHE frameworks often expose backend retargeting as a deployment-time optimization. A developer may validate an FHE program on a reference backend during development or audit, and later switch to another backend library or hardware target for performance, compatibility, or cost. Attackers can exploit framework inconsistencies across these backends to bypass security audits. We identified a program in Concrete (Fig. 5b) that executes correctly on a CPU but produces corrupted results when retargeted to a GPU, due to a flaw in how the GPU backend lowers comparison operators under deep arithmetic chains. An adversary can introduce this specific program structure into the source code, which will pass code review and referencebackend testing. However, once deployed to the production GPU for performance, the program silently evaluates a different result, potentially leading to a manipulated financial score. We found 3 similar cases, such as discrepancies between SEAL and OpenFHE backends in HELayers. These bugs effectively render reference audits obsolete and turn performanceoriented backend retargeting into a critical security risk. Availability Breach: Valid Program Denial-of-Service. The computational intensity of FHE has driven the emergence of “FHE-as-a-Service” [33, 41], allowing users to compile and deploy their FHE-enhanced applications on managed cloud platforms [42, 43]. In these shared environments, frameworklevel memory safety bugs expose the system to severe Denialof-Service (DoS) attacks. We found 2 bugs where an adversary can submit syntactically valid FHE programs that easily pass frontend validation but trigger fatal compilation or execution errors. For instance, HERTA uncovered a bug in Concrete where a valid array initialization pattern, involving an encrypted input as the final element, corrupts heap metadata during execution, leading to a malloc: corrupted top size error and crashing the process. In a cloud environment, an attacker can repeatedly submit such programs to crash cloud worker nodes and degrade service availability for legitimate users.
Scenario Reflection. While these attack scenarios demonstrate serious security implications, we acknowledge several limitations in their practical exploitability. First, an integrity breach requires attackers to identify specific vulnerabilities and craft triggering code. Although reviewers might spot suspicious code patterns, like multiplying by negative scalars instead of subtracting, such code will become harder to distinguish from legitimate optimizations as FHE frameworks grow in complexity. Second, audit evasion relies on a mismatch between audited and production backends. While exhaustive validation of every backend and configuration could mitigate this, it is often prohibitively expensive. Since retargeting to GPUs or alternative libraries is routinely performed for throughput or cost, we believe this scenario represents a realistic risk. Third, regarding DoS attacks, cloud platforms may use rate limiting or input validation to mitigate repeated crashes. Nevertheless, the existence of these vulnerabilities, especially silent logic errors, represents a fundamental threat to the trustworthiness of privacy-preserving applications.
Limitations of Language Feature Support. We acknowledge that HERTA primarily focuses on the fundamental arithmetic layers of FHE frameworks and does not cover every high-level, domain-specific API offered by certain frameworks, such as the ONNX input interface in HELayers. However, the correctness of these high-level abstractions fundamentally relies on the reliability of the underlying cryptographic primitives. By targeting these core primitives, our MRs effectively test the foundational logic where critical failures often originate. Furthermore, given that FHE frameworks implement distinct and rapidly evolving interfaces, maintaining a comprehensive seed generator that covers all possible APIs presents a significant maintenance challenge. Nevertheless, our evaluation demonstrates that HERTA’s current strategy effectively uncovers a substantial number of bugs across multiple layers of the FHE stack. We therefore leave the development of a dynamic seed generator capable of automatically adapting to evolving FHE APIs as a promising direction for future work.
VII. D ISCUSSION
Security Assurance for Cryptographic Libraries. Securing cryptographic software demands a multi-faceted approach. DY fuzzer [49] and TLS-Attacker [48] support automated analysis and testing of cryptographic protocol implementations. TaintCrypt [50] uses taint analysis to find logical bugs in C/C++ code, while CryptoGuard [51] detects cryptographic API misuse in Java projects. CryptoLine [52] formally verifies foundational cryptography in assembly. Unlike these works that focus on traditional cryptographic libraries, HERTA addresses the unique challenges of testing FHE frameworks, which involve a complex pipeline from high-level code to low-level cryptographic operations. Testing such frameworks presents distinct challenges due to the cryptographic properties of FHE (such as noise accumulation and approximate arithmetic) and the heterogeneous nature of frameworks’ compilation and execution stacks. Vulnerability Detection in Compilers. Compilers translate high-level code into low-level machine instructions. Their complexity makes them prone to bugs that compromise software security [53]. While extensive work targets JavaScript JIT compilers [54–57], C/C++ compilers [58, 59], and deep learning compilers [60, 61], HERTA addresses the unique challenges of FHE frameworks. It introduces novel domainspecific MRs to probe FHE-intrinsic properties such as noise management, parameter selection, and approximate arithmetic. Furthermore, unlike traditional DT that relies on crosscompiler comparisons, HERTA navigates the heterogeneous configuration space within a single framework to validate semantic consistency across different backends, cryptographic schemes, and adaptation strategies.
Alternative Testing Methods. Besides MT, differential testing (DT) is commonly used in compiler testing efforts [44–46]. In DT, different compilers should produce identical outputs for the same input; discrepancies indicate bugs. However, applying DT to FHE frameworks presents unique challenges. First, the sophisticated FHE compilation and execution pipeline makes DT against different frameworks insufficient for isolating root causes, especially in complex cryptographic contexts. Second, the rapid evolution of FHE frameworks introduces specialized internal APIs and primitives that lack direct semantic equivalences in other frameworks, complicating crossdomain comparisons. Third, and most crucially, FHE frameworks often employ different approximation strategies, where valid decrypted results inherently diverge from exact values. Distinguishing between genuine bug-induced errors and acceptable approximation noise is algorithmically difficult in a DT setting. In contrast, the MT approach adopted by HERTA enables fine-grained, self-contained, and efficient testing that is better suited to the current FHE landscape. Testing vs. Verification. Although formal verification techniques provide stronger correctness guarantees by rigorously proving specific properties, the immense cryptographic complexity, the size of FHE framework codebases (often containing hundreds of thousands of lines of code [8, 9]), and the rapidly evolving nature of modern FHE frameworks make constructing comprehensive formal models prohibitively expensive. In line with other work on software reliability in complex systems [47, 48], we adopt automated testing as a pragmatic approach. It offers a scalable solution for detecting bugs in real-world FHE framework implementations, facilitating iterative improvements and rapid developer feedback. The reported issues can serve as concrete inputs for developers to debug and patch their frameworks. We leave formal verification of FHE frameworks as a complementary future direction.
VIII. R ELATED W ORK
IX. C ONCLUSION We introduce HERTA, a systematic testing framework for FHE framework correctness that leverages FHE-specific MRs. HERTA uncovered 21 previously unknown bugs in three mainstream frameworks, including critical logic and memory
safety errors. We release HERTA to support building more reliable privacy-preserving systems. R EFERENCES [1] C. Gentry, “Fully homomorphic encryption using ideal lattices,” in STOC, 2009, pp. 169–178. [2] J. H. Cheon, A. Kim, M. Kim, and Y. Song, “Homomorphic encryption for arithmetic of approximate numbers,” in Advances in Cryptology–ASIACRYPT 2017: 23rd International Conference on the Theory and Applications of Cryptology and Information Security, Hong Kong, China, December 3-7, 2017, Proceedings, Part I 23. Springer, 2017, pp. 409–437. [3] J. H. Cheon, K. Han, A. Kim, M. Kim, and Y. Song, “A full rns variant of approximate homomorphic encryption,” in Selected Areas in Cryptography–SAC 2018: 25th International Conference, Calgary, AB, Canada, August 15–17, 2018, Revised Selected Papers 25. Springer, 2019, pp. 347–368. [4] A. Wood, K. Najarian, and D. Kahrobaei, “Homomorphic encryption for machine learning in medicine and bioinformatics,” ACM Computing Surveys (CSUR), vol. 53, no. 4, pp. 1–35, 2020. [5] N. Wang, W. Zhou, J. Wang, Y. Guo, J. Fu, and J. Liu, “Secure and efficient similarity retrieval in cloud computing based on homomorphic encryption,” IEEE Transactions on Information Forensics and Security, vol. 19, pp. 2454–2469, 2024. [6] L. Bergerat, A. Boudi, Q. Bourgerie, I. Chillotti, D. Ligier, J.-B. Orfila, and S. Tap, “Parameter optimization and larger precision for (t) fhe,” Journal of Cryptology, vol. 36, no. 3, p. 28, 2023. [7] E. Aharoni, A. Adir, M. Baruch, N. Drucker, G. Ezov, A. Farkash, L. Greenberg, R. Masalha, G. Moshkowich, D. Murik et al., “Helayers: A tile tensors framework for large neural networks on encrypted data,” Proceedings on Privacy Enhancing Technologies, 2023. [8] A. Ali, J. Choi, B. Gipson, S. Gorantala, J. Kun, W. Legiest, L. Lim, A. Viand, M. Z. Demissie, and H. Zheng, “Heir: A universal compiler for homomorphic encryption,” 2025. [Online]. Available: https://arxiv.org/abs/2508.11095 [9] Zama, “Concrete: TFHE Compiler that converts python programs into FHE equivalent,” 2022, https://github.com /zama-ai/concrete. [10] “Microsoft SEAL (release 4.1),” https://github.com/Mic rosoft/SEAL, Jan. 2023, microsoft Research, Redmond, WA. [11] OpenFHE, “Openfhe.org - openfhe - open-source fully homomorphic encryption library,” https://www.openfhe. org/, 2023. [12] I. Chillotti, N. Gama, M. Georgieva, and M. Izabachène, “Tfhe: fast fully homomorphic encryption over the torus,” Journal of Cryptology, vol. 33, no. 1, pp. 34–91, 2020. [13] K. Munjal and R. Bhatia, “A systematic review of homomorphic encryption and its contributions in healthcare
industry,” Complex & Intelligent Systems, vol. 9, no. 4, pp. 3759–3786, 2023. [14] G. T. Review, “Exclusive: Duality technologies enters trade finance fraud prevention,” https://www.gtreview .com/news/fintech/exclusive-duality-technologies-enter s-trade-finance-fraud-prevention/, 2022. [15] T. Y. Chen, S. C. Cheung, and S. M. Yiu, “Metamorphic testing: a new approach for generating next test cases,” arXiv preprint arXiv:2002.12543, 2020. [16] S. Segura, G. Fraser, A. B. Sanchez, and A. Ruiz-Cortés, “A survey on metamorphic testing,” IEEE Transactions on software engineering, vol. 42, no. 9, pp. 805–824, 2016. [17] R. L. Rivest, L. Adleman, M. L. Dertouzos et al., “On data banks and privacy homomorphisms,” Foundations of secure computation, vol. 4, no. 11, pp. 169–180, 1978. [18] O. Regev, “On lattices, learning with errors, random linear codes, and cryptography,” Journal of the ACM (JACM), vol. 56, no. 6, pp. 1–40, 2009. [19] J. Fan and F. Vercauteren, “Somewhat practical fully homomorphic encryption,” Cryptology ePrint Archive, 2012. [20] Q. Lou, B. Feng, G. Charles Fox, and L. Jiang, “Glyph: Fast and accurately training deep neural networks on encrypted data,” Advances in neural information processing systems, vol. 33, pp. 9193–9202, 2020. [21] A. Guimarães, E. Borin, and D. F. Aranha, “Revisiting the functional bootstrap in tfhe,” IACR Transactions on Cryptographic Hardware and Embedded Systems, pp. 229–253, 2021. [22] R. Ran, X. Luo, W. Wang, T. Liu, G. Quan, X. Xu, C. Ding, and W. Wen, “Spencnn: orchestrating encoding and sparsity for fast homomorphically encrypted neural network inference,” in International Conference on Machine Learning. PMLR, 2023, pp. 28 718–28 728. [23] Zama, “TFHE-rs: A Pure Rust Implementation of the TFHE Scheme for Boolean and Integer Arithmetics Over Encrypted Data,” 2022, https://github.com/zama-ai/tfhe -rs. [24] C. Brynds, P. McLeod, L. Caccamise, A. Pal, D. Saiham, S. Rahman, J. S. Miguel, and D. Wu, “Cryptoracle: A modular framework to characterize fully homomorphic encryption,” arXiv preprint arXiv:2510.03565, 2025. [25] Y. Peng, D. Wu, Z. Liu, D. Xiao, Z. Ji, J. Rahmel, and S. Wang, “Testing and understanding deviation behaviors in fhe-hardened machine learning models,” in IEEE Computer Society, 2025, pp. 644–644. [26] Zama, “Concrete ML: a privacy-preserving machine learning library using fully homomorphic encryption for data scientists,” 2022, https://github.com/zama-ai/concre te-ml. [27] H. Chen, R. Gilad-Bachrach, K. Han, Z. Huang, A. Jalali, K. Laine, and K. Lauter, “Logistic regression over encrypted data from fully homomorphic encryption,” BMC medical genomics, vol. 11, no. Suppl 4, p. 81, 2018. [28] Z. Brakerski, C. Gentry, and V. Vaikuntanathan, “(lev-
eled) fully homomorphic encryption without bootstrapping,” ACM Transactions on Computation Theory (TOCT), vol. 6, no. 3, pp. 1–36, 2014. [29] E. T. Barr, M. Harman, P. McMinn, M. Shahbaz, and S. Yoo, “The oracle problem in software testing: A survey,” IEEE transactions on software engineering, vol. 41, no. 5, pp. 507–525, 2014. [30] E. Lee, J.-W. Lee, J. Lee, Y.-S. Kim, Y. Kim, J.-S. No, and W. Choi, “Low-complexity deep convolutional neural networks on fully homomorphic encryption using multiplexed parallel convolutions,” in International Conference on Machine Learning. PMLR, 2022, pp. 12 403–12 422. [31] K. Nandakumar, N. Ratha, S. Pankanti, and S. Halevi, “Towards deep neural network training on encrypted data,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, 2019, pp. 0–0. [32] S. Halevi and V. Shoup, “Design and implementation of helib: a homomorphic encryption library,” Cryptology ePrint Archive, 2020. [33] R. Yadavalli, J. Solomon, and V. Sharma, “Homomorphic encryption methods applied to cloud computing: A practical architecture for elastic, verifiable confidential compute,” Cryptology ePrint Archive, 2025. [34] N. P. Smart, “Practical and efficient fhe-based mpc,” in IMA International Conference on Cryptography and Coding. Springer, 2023, pp. 263–283. [35] C. Nie and H. Leung, “A survey of combinatorial testing,” ACM Computing Surveys (CSUR), vol. 43, no. 2, pp. 1–29, 2011. [36] D. R. Kuhn, D. R. Wallace, and A. M. Gallo, “Software fault interactions and implications for software testing,” IEEE transactions on software engineering, vol. 30, no. 6, pp. 418–421, 2004. [37] R. Kuhn, R. Kacker, Y. Lei, and J. Hunter, “Combinatorial software testing,” Computer, vol. 42, no. 8, pp. 94–96, 2009. [38] Anonymous, “Research artifacts.” 2026. [39] ——, “Confirmed bugs in HEIR,” 2026. [40] G. P. Zero, “Domato: Dom fuzzer,” https://github.com/g oogleprojectzero/domato, 2017. [41] G. AI, “Homomorphic encryption as a service,” https://www.graphapp.ai/engineering-glossary/cloudcomputing/homomorphic-encryption-as-a-service, 2025. [42] Microsoft, “Microsoft seal: Build end-toend encrypted data storage and computation services,” https://www.microsoft.com/enus/research/project/microsoft-seal/, 2023. [43] AWS, “Enable fully homomorphic encryption with amazon sagemaker endpoints for secure, real-time inferencing,” https://aws.amazon.com/blogs/machinelearning/enable-fully-homomorphic-encryption-withamazon-sagemaker-endpoints-for-secure-real-timeinferencing/, 2023. [44] G. Polito, S. Ducasse, and P. Tesone, “Interpreter-guided
differential jit compiler unit testing,” in Proceedings of the 43rd ACM SIGPLAN International Conference on Programming Language Design and Implementation, 2022, pp. 981–992. [45] S. Li and Z. Su, “Finding unstable code via compilerdriven differential testing,” in Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3, 2023, pp. 238–251. [46] Q. Wang and R. Jung, “Rustlantis: Randomized differential testing of the rust compiler,” Proceedings of the ACM on Programming Languages, vol. 8, no. OOPSLA2, pp. 1955–1981, 2024. [47] J.-P. Aumasson and Y. Romailler, “Automated testing of crypto software using differential fuzzing,” Black Hat USA, vol. 7, p. 2017, 2017. [48] J. Somorovsky, “Systematic fuzzing and testing of tls libraries,” in Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, 2016, pp. 1492–1504. [49] M. Ammann, L. Hirschi, and S. Kremer, “Dy fuzzing: Formal dolev-yao models meet cryptographic protocol fuzz testing,” in 2024 IEEE Symposium on Security and Privacy (SP), 2024, pp. 1481–1499. [50] S. Rahaman, H. Cai, O. Chowdhury, and D. Yao, “From theory to code: identifying logical flaws in cryptographic implementations in c/c++,” IEEE transactions on dependable and secure computing, vol. 19, no. 6, pp. 3790– 3803, 2021. [51] S. Rahaman, Y. Xiao, S. Afrose, F. Shaon, K. Tian, M. Frantz, M. Kantarcioglu, and D. Yao, “Cryptoguard: High precision detection of cryptographic vulnerabilities in massive-sized java projects,” in Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, 2019, pp. 2455–2472. [52] Y.-F. Fu, J. Liu, X. Shi, M.-H. Tsai, B.-Y. Wang, and B.-Y. Yang, “Signed cryptographic program verification with typed cryptoline,” in Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, 2019, pp. 1591–1606. [53] J. Xu, K. Lu, Z. Du, Z. Ding, L. Li, Q. Wu, M. Payer, and B. Mao, “Silent bugs matter: a study of compilerintroduced security bugs,” in Proceedings of the 32nd USENIX Conference on Security Symposium. USENIX Association, 2023, pp. 3655–3672. [54] J. Wang, Y. Kang, C. Wu, Y. Hu, Y. Sun, J. Ren, Y. Lai, M. Xie, C. Zhang, T. Li, and Z. Wang, “{optfuzz}: Optimization path guided fuzzing for {javascript} {jit} compilers,” in 33rd USENIX Security Symposium (USENIX Security 24), 2024, pp. 865–882. [55] H. Xu, Z. Jiang, Y. Wang, S. Fan, S. Xu, P. Xie, S. Fu, and M. Payer, “Fuzzing javascript engines with a graphbased ir,” in Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security. Association for Computing Machinery, 2024, pp. 3734– 3748.
[56] L. Wachter, J. Gremminger, C. Wressnegger, M. Payer, and F. Toffalini, “Dumpling: Fine-grained differential javascript engine fuzzing,” in Proceedings 2025 Network and Distributed System Security Symposium. Internet Society, 2025. [57] S. Groß, S. Koch, L. Bernhard, T. Holz, and M. Johns, “Fuzzilli: Fuzzing for javascript jit compiler vulnerabilities,” in Proceedings 2023 Network and Distributed System Security Symposium. Internet Society, 2023. [58] X. Yang, Y. Chen, E. Eide, and J. Regehr, “Finding and understanding bugs in c compilers,” in Proceedings of the 32nd ACM SIGPLAN conference on Programming language design and implementation, 2011, pp. 283–294. [59] V. Le, M. Afshari, and Z. Su, “Compiler validation via equivalence modulo inputs,” ACM Sigplan Notices, vol. 49, no. 6, pp. 216–226, 2014. [60] J. Liu, J. Lin, F. Ruffy, C. Tan, J. Li, A. Panda, and L. Zhang, “Nnsmith: Generating diverse and valid test cases for deep learning compilers,” in Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2, 2023, pp. 530–543. [61] X. Zhang, W. Jiang, C. Shen, Q. Li, Q. Wang, C. Lin, and X. Guan, “Deep learning library testing: Definition, methods and challenges,” ACM Computing Surveys, vol. 57, no. 7, pp. 1–37, 2025. A PPENDIX A. Comparison with a Generic Fuzzing Baseline To further isolate the benefit of HERTA’s FHE-aware design, we compare HERTA with a grammar-based fuzzer, Domato, on Concrete. Both tools use the same operator set, input domains, maximum program length, and 1K valid seed budget. Unlike HERTA, Domato does not use FHE-aware heuristics such as multiplicative-depth prioritization. TABLE V: Comparison with Domato under 1K valid seed budget on Concrete. HERTA (Seed Only) uses only the FHEaware seed generator, while HERTA (Seed+MRs) includes the generated seeds and their MR-generated variants. Values in parentheses indicate the additional failure-triggering inputs exposed by MRs beyond those found by seed only. #Crash Inputs and #Logic-Error Inputs denote the numbers of test cases that trigger crashes and logic errors, respectively. Method Domato HERTA (Seed Only) HERTA (Seed+MRs)
#Crash Inputs
#Logic-Error Inputs
3 7 25 (+18)
0 7 13 (+6)
Table V shows that generic grammar generation exercises basic frontend paths, but is less effective at reaching FHEspecific failure modes under the same valid-seed budget. In particular, Domato triggers only three crashes and no logic errors, while HERTA’s seed generation exposes both crashes and silent logic errors. HERTA’s MRs further expose
additional failure-triggering inputs, showing that HERTA’s FHE-aware seed generation and MRs provide complementary benefits over generic seed-only fuzzing.