Falcons: Pushing the Speed-Ratio Limits of Lossy Compression for Parallel Scientific Applications

arXiv:2610.41209v1 [cs.DC] 30 Oct 2026

Falcons: Pushing the Speed-Ratio Limits of Lossy Compression for Parallel Scientific Applications Bing Lu

Jinyang Liu∗

Xinyang Ma

Institute of Computing Technology, Chinese Academy of Sciences Beijing, China University of Chinese Academy of Sciences Beijing, China lubing26@mails.ucas.ac.cn

Shanghai Jiao Tong University Shanghai, China jinyang.liu@sjtu.edu.cn

Institute of Computing Technology, Chinese Academy of Sciences Beijing, China University of Chinese Academy of Sciences Beijing, China maxinyang25@mails.ucas.ac.cn

Quantong Fu

Chenwei Hou

Zedong Liu

Shanghai Jiao Tong University Shanghai, China quantongfu@sjtu.edu.cn

Shanghai Jiao Tong University Shanghai, China moonlightplague@sjtu.edu.cn

Institute of Computing Technology, Chinese Academy of Sciences Beijing, China captain.liu77@gmail.com

Wenjing Huang

Yida Gu

Xingchen Liu

Institute of Computing Technology, Chinese Academy of Sciences Beijing, China huangwenjing23@mails.ucas.ac.cn

Institute of Computing Technology, Chinese Academy of Sciences Beijing, China guyida21@mails.ucas.ac.cn

Institute of Computing Technology, Chinese Academy of Sciences Beijing, China liuxingchen232@mails.ucas.ac.cn

Zheng Wei

Anbang Wu

Xiaofeng Hou

Institute of Computing Technology, Chinese Academy of Sciences Beijing, China weizheng@ncic.ac.cn

Shanghai Jiao Tong University Shanghai, China anbang@cs.sjtu.edu.cn

Shanghai Jiao Tong University Shanghai, China xfhelen@sjtu.edu.cn

Chao Li

Guangming Tan

Minyi Guo

Shanghai Jiao Tong University Shanghai, China chaol@sjtu.edu.cn

Institute of Computing Technology, Chinese Academy of Sciences Beijing, China tgm@ict.ac.cn

Shanghai Jiao Tong University Shanghai, China myguo@sjtu.edu.cn

Dingwen Tao Institute of Computing Technology, Chinese Academy of Sciences Beijing, China taodingwen@ict.ac.cn

Abstract The extreme volume of scientific data has become a critical challenge for modern exa-scale scientific computing applications, and error-bounded lossy compression remains indispensable for scientific data management due to its high data reduction rate and controllable data fidelity. However, to ensure sufficient data reduction, existing compressors incur prohibitive computational overheads in real-world parallel

∗ Corresponding author: Jinyang Liu.

scientific computing applications. To overcome the performance bottlenecks of error-bounded lossy compression, we present Falcons, a broadly deployable error-bounded lossy compressor that simultaneously achieves high compression ratios with ultra-fast throughput, minimizing the overhead in parallel scientific applications. Falcons integrates three key designs: memory-regularized interpolation with SIMD (Single Instruction Multiple Data) vectorization, distribution-adaptive lossless coding, and applicationoriented optimizations for multiple parallelization schemes.

Table 1. Breakdown of parallel writing time (s) on the LineShine supercomputer with 64 nodes and 4096 MPI ranks. Dataset: Hurricane Isabel [23]. Error Bound: 0.1% of the value range. Time_C is for compression, and Time_W is for data writing. Only Falcons breaks the bottleneck for compression execution.

Across our evaluations on cutting-edge supercomputers, Falcons clearly outperforms state-of-the-art compressors, simultaneously achieving up to 29% compression ratio improvement and 255% throughput improvement. In case studies of real-world applications, Falcons accelerates large-scale parallel data I/O and remote transfer by up to 44.6% and 51.5%, and provides 1.55× end-to-end speedup for the full WRF (Weather Research & Forecasting) application workflow.

1

Introduction

Advances in supercomputers, large-scale scientific instruments, and AI4S models dramatically drive progress in scientific discovery. However, the explosion of data volumes in turn severely challenges the data storage and management systems. For example, although a 163843 -element NYX [2, 51, 57] cosmology simulation is computationally feasible on OLCF’s Frontier, at least 14 arrays per checkpoint and 20 snapshots per science case require nearly 10 PB of storage, making data storage and transfer a major bottleneck. Scientific data-management workflows now span a spectrum of computing facilities, ranging from personal workstations to CPU-centric leadership supercomputers (e.g., Fugaku and LineShine [55, 64]) and heterogeneous exascale systems (e.g., Aurora and El Capitan [3, 29]). Across these platforms, applications call for effective data-reduction solutions to alleviate growing bottlenecks in storage, parallel I/O, and cross-site data transfer. Error-bounded lossy compression [4, 12, 30, 33, 35, 36, 41, 42, 48, 60, 68], which guarantees a strict pointwise error bound, has been widely adopted for scientific data management [5, 7, 8, 13, 18], as it substantially reduces data volume while preserving the fidelity required for downstream analysis. While compression can also be offloaded to accelerators such as GPUs [21, 36, 65], CPU-based compression remains indispensable: many leadership systems and the workstations where scientific data are prepared and analyzed have no accelerator available for this purpose, and CPU implementations integrate directly into existing scientific software stacks and system services [17, 27, 58, 66] while remaining portable across platforms. So far, the performance of scientific lossy compressors in real-world parallel applications is bottlenecked by the dilemma of compression ratio and throughput. To achieve sufficient data reduction, lossy compression algorithms are computationally intensive and incur non-negligible performance overheads [9]. A typical use case of lossy compression in scientific applications is to accelerate data I/O [6, 8, 33, 68]. However, as indicated in Table 1, which profiles large-scale parallel data writing on the newly released LineShine Supercomputer, which has > 2 ExaFlops peak performance and leading I/O bandwidth, existing scientific compressors even slow down the I/O workflow in certain cases, because their execution time emerges as a bottleneck when reducing the data volume to an acceptable scale. Thus, scientific lossy

Compressor

Time_C Time_W Time_Total Comp_Ratio

Original I/O ZFP [36] SZ3.3 [35] SPERR [30] Falcons (ours)

/ 12.00 24.12 78.86 4.51

16.30 6.39 4.30 4.13 4.10

16.30 18.39 28.42 82.99 8.61

1.00× 6.68× 23.18× 15.19× 26.07×

compression must be comprehensively redesigned to achieve a joint optimization of compression ratio and throughput, fully supporting modern applications. Having overcome the challenges of performing effective scientific data decorrelation/encoding at minimized cost, and fine-tuning their performance in sophisticated real-world applications, we propose Falcons1 . Falcons is a prediction-based scientific error-bounded lossy compressor facilitating efficiency-boosted data interpolation design, adaptive lossless encoding modules, and parallelizationoriented optimizations. Falcons is a CPU-based scientific lossy compressor to simultaneously deliver ultra-fast throughput and high compression ratios across parallel schemes, empowering scientific data applications to maximize their performance in diverse use cases. Our contributions are summarized as follows: • We design a novel interpolation-based data prediction scheme that exposes regularized memory access and is highly vectorizable on modern CPU architectures. It can leverage AVX2 and SVE2 intrinsics to fully unlock the computational capabilities of modern x86/ARM CPUs. • We design an adaptive entropy-coding pipeline tailored to the statistical characteristics of post-prediction quantization codes to fit the scientific data inputs, significantly outperforming existing scientific lossless encoding pipelines. • We further optimize Falcons across different parallelism in real-world parallel applications. We reduce memory traffic to mitigate cache/memory bandwidth competitions and exploit fine-grained workload partition to preserve high throughput while maintaining compression effectiveness. • We conduct comprehensive evaluations for Falcons on cutting-edge supercomputers. It significantly outperforms all existing baselines, achieving both compression ratio improvement (up to 29%) and speedup (up to 255%). Falcons boosts large-scale parallel data I/O and remote transfer by 44.6% and 51.5% time reduction, and enables 1.55× end-toend application workflow speedup. The rest of this paper is organized as follows. Section 2 discusses the related work. Section 3 presents the background and formulation of our research. In Section 4 and Section 5, 1We open-source the code at https://github.com/opensdc-org/Falcons.git.

2

Usage Scenarios

we present the overview and details of the Falcons design. Section 6 demonstrates the evaluation results. Finally, we present the concluding remark and future work in Section 7.

Static Storage

Scientific Application Workflow Generator

Compressor Input

Store

Storage

Parallel I/O Rank 0 Comp.

Output

Rank 1 Comp.

Compress

Distributor

Rank 2 Comp.

Cloud Workflows

2

Related Work

2.1

Scientific Error-Bounded Lossy Compression

Transfer

······

Figure 1. The use cases of error-bounded lossy compression in scientific applications.

Various error-bounded lossy compressors have been developed for scientific data reduction to support scientific applications. Existing works can be broadly categorized into four classes: prediction-based, transform-based, dimensionreduction-based, and neural-network-based. The SZ family is a representative prediction-based framework, employing a variety of predictors, such as the mixed scheme of Regression-Lorenzo prediction in SZ2 [33] and dynamic spline interpolation in SZ3 [35, 41, 68]. MGARD [1] uses an error-guaranteed theoretical framework of multi-level decomposition. ZFP [36, 37] delivers fast throughput by using a near-orthogonal transform for data decorrelation. SPERR [30] achieves superior rate–distortion performance but low throughput by utilizing the CDF 9/7 discrete wavelet transform. Dimension-reduction-based compressors apply dimension-reduction techniques, such as high-order singular value decomposition (HoSVD) in TTHRESH [4], which achieves high compression ratios but very low throughput. Neural-network-based scientific data compressors [19, 25, 31, 39, 40] exploit learned data representations with significantly low speed, limiting their availability in real systems.

2.2

3

Research Background

3.1

The Principle and Use Cases of Error-Bounded Lossy Compression

Scientific applications routinely generate vast volumes of data for post-hoc analysis, visualization, and checkpointing [27, 58, 66]. As application scale continues to increase, data management can consume a substantial fraction of the overall execution time and a large portion of system storage capacity [26, 46]. To support data management, scientific error-bounded lossy compression that allows users to specify a pointwise error bound has been incorporated into multiple stages of data-management workflows [6–8, 26]. Specifically, with a user-specified error bound Δ, given input data 𝑋 , compressed data 𝐶, and decompressed data 𝑋 ′ , error-bounded lossy compression guarantees that max |𝑋 − 𝑋 ′ | ≤ Δ, and its compression ratio is the size of 𝑋 divided by the size of 𝐶 (so higher is better). As Figure 1 shows, when integrated into scientific applications, error-bounded lossy compression has several usages and brings benefits for scientific data management. Three representative use cases are:

Accelerating Scientific Lossy Compression in Applications

To optimize the efficiency of the scientific data management workflow with scientific lossy compression, numerous studies have been proposed. PFPL [16] targets heterogeneous HPC workflows by providing bitwise-compatible compression, enabling data to be compressed and decompressed on different devices while maintaining high throughput. SZ3_SIMD [70] introduces SIMD-based optimization of the Lorenzo data predictor [38], thereby achieving good speed but a poor compression ratio. MGARD+ [34] enhances the MGARD [1] framework with level-wise quantization and adaptive decomposition, improving compression ratio and throughput. Several studies have also explored GPUaccelerated scientific data compression [20–22, 43, 49, 62, 63, 65, 67], but they generally achieve lower compression ratios than CPU-based compressors. In summary, existing error-bounded lossy compressors have not achieved jointly optimized compression ratio, throughput, and scalability, exhibiting limitations on universal integration in parallel scientific applications. Our paper endeavors to make a breakthrough for this research gap. 3

• Static Storage Reduction: Storage reduction is a fundamental motivation for scientific data compression. To better benefit applications, compressors need to exhibit a high compression ratio. • Parallel I/O Acceleration: Scientific data compression may accelerate the data I/O of large-scale applications by performing parallelized data (de-)compression before accessing file systems. In this task, compared to the high local I/O bandwidth, compression throughput is the main bottleneck for I/O time optimization. • Cloud-Transfer Acceleration: Scientific datasets can be lossy compressed before being transferred across distributed sites to accelerate the process. As network bandwidth is relatively lower than local I/O, a joint optimization of compression ratio and throughput is needed to sufficiently accelerate the end-to-end process. In those workflows, a high compression ratio reduces the amount of data transferred and stored, whereas sufficiently high throughput is required to prevent compression from becoming part of the critical I/O path [26, 44, 45]. This requirement motivates broadly deployable compressors that can provide strong data reduction while scaling efficiently across threads, processes, and compute nodes [13, 45].

3.2

Problem Formulation and Analysis

1

The three use cases above impose two coupled requirements on scientific lossy compression: effective data reduction and sufficiently high throughput. We next quantify these requirements and formulate the resulting research target. Research Target. Our research goal is to propose an error-bounded lossy compressor that jointly maximizes compression ratio and throughput to maintain storage efficiency and end-to-end speed. Moreover, its compression ratio and throughput should match the data-processing requirement of modern applications and systems. We formulate this objective as:

Input Data

3 Adaptive Routing

Lossy Decorrelation Memory Regularized Interpolation

RunGamma

New Modules

2 Quantization

4

Existing Modules

5 FSE

Zstandard

Output Data

Context

Lossless Encoding

Figure 2. Overview of Falcons.

this bandwidth by the number of client processes yields an average baseline per-process bandwidth of 𝐵 base ≈ 482 MB/s, so here it requires 𝑇𝑐,req (𝑋, Δ, 1) ≥ 500 MB/s per core.

maximize 𝑇𝑐 (A; 𝑋, Δ, 𝑝), 𝑅(A; 𝑋, Δ, 𝑝), 𝑝 ∈ P, A

subject to 𝑅(A; 𝑋, Δ, 𝑝) ≥ 𝑅req (𝑋, Δ, 𝑝), 𝑝 ∈ P, 𝑇𝑐 (A; 𝑋, Δ, 𝑝) ≥ 𝑇𝑐,req (𝑋, Δ, 𝑝), 𝑝 ∈ P, 𝑋 − 𝑋ˆ A ≤ Δ.

4

Overview of Falcons

(1) Our proposed Falcons is a CPU-based scientific error-bounded lossy compressor, which concurrently achieves high compression ratio and ultra-fast throughput, ensuring universal availability across diverse platforms and applications. As depicted in Figure 2, Falcons follows a classical two-stage pipeline, composed of a redesigned interpolation-based data decorrelation stage that converts the input data into quantized integer errors and a distribution-adaptive lossless coding stage to pack the quantized errors. Outperforming existing works, Falcons centers on three core novel designs:

∞

Here, A denotes the compressor, 𝑋 is the original dataset, and Δ is the user-specified error bound. 𝑅(A; 𝑋, Δ, 𝑝) and 𝑇𝑐 (A; 𝑋, Δ, 𝑝) denote the achieved compression ratio and compression throughput under execution parallelism 𝑝, respectively. 𝑅req (𝑋, Δ, 𝑝) and 𝑇𝑐,req (𝑋, Δ, 𝑝) denote the corresponding minimum compression ratio and throughput requirements imposed by the target application or system. Finally, P denotes the parallelism of execution. Compression-Ratio Requirements. A higher compression ratio directly translates to a reduced storage footprint. For an original data size 𝑆 and compression ratio 𝑅, the compressed size is 𝑆/𝑅. Different scientific applications require compression ratios ranging from tens to hundreds [6, 35]. For example, seismological applications require an over 20× reduction in data footprint to control data volume [8], yielding 𝑅req (𝑋, Δ, 𝑝) ≥ 20. Earth-observation satellite systems require at least a 50× reduction to relieve bandwidth-limited downlink bottlenecks, with targets growing alongside satellite constellations [59], yielding 𝑅req (𝑋, Δ, 𝑝) ≥ 50. Throughput Requirements. For online data I/O and transfers, the compression ratio and throughput jointly determine the system performance. Let 𝑇𝑐 denote the compression throughput and 𝐵 represent the effective I/O bandwidth. The write or transfer times without (𝑇raw ) and with (𝑇comp ) 𝑆 compression can be modeled as: 𝑇raw = 𝐵𝑆 , 𝑇comp = 𝑇𝑆𝑐 + 𝑅𝐵 . Therefore, compression reduces the end-to-end execution 𝐵 time if and only if: 𝑇𝑐 > 1−1/𝑅 = 𝑇𝑐,min . On the compression ratio magnitude required by applications, 𝑇𝑐,min will be very close to the I/O bandwidth 𝐵 itself. System-Level Performance Analysis. Representative scientific workloads ideally require an aggregate parallel-throughput of 10 GB/s [6], which instantiates 𝑇𝑐,req (𝑋, Δ, 𝑝) ≥ 10 GB/s in Equation 1. Moreover, the IO500 submission for the leadership-class Aurora supercomputer reports an aggregate bandwidth of 934 GiB/s utilizing 2,080 client processes across ten nodes [24]. Normalizing

• Memory-regularized interpolation predictor. Dynamic spline interpolation has been adopted by several scientific compressors [41, 42, 68]. Unlike existing work, Falcons eliminates irregular memory accesses and exposes contiguous prediction–quantization kernels to AVX2 and SVE2 vectorization, unlocking the computational potential of interpolation on modern CPUs. • Distribution-adaptive lossless coding. Falcons features a distribution-aware design of its entropy-centric lossless encoding pipeline. It dynamically dispatches quantization code streams to adaptive lossless pipelines, including our proposed processing modules RunGamma and Context plus the FSE (Finite-State Entropy) [10] and Zstd [11], maximizing compression ratios and throughput on various data inputs and configurations. • Application-oriented parallel optimization. Falcons performs parallelization optimization to fit real-world scientific applications. In thread-independent schemes, it turns multi-pass interpolation into a low-traffic streaming dataflow, boosting cache locality and cutting memory traffic. For thread-cooperative parallelization, Falcons employs fine-grained thread scheduling to eliminate contention without degrading spatial correlation. Overall, Falcons leverages the novel software-level design and hardware microarchitecture to break the traditional trade-off between compression ratio and throughput. Those approaches allow Falcons to deliver consistent benefits for diverse requirements in parallel scientific applications. 4

5

Core Design of Falcons

used in multi predictions

To achieve significant acceleration for high-ratio scientific lossy compression in real-world use cases, Falcons employs a bottom-up design: build the hardware-friendly computing framework first, then scale it across parallel environments. Falcons first reconstructs the interpolation prediction and lossless pipeline, then introduces parallelization-oriented optimization for scalable execution.

2 1

3 0

Dim Z Interpolation

3

2 1

Strided Access

2

Strided Access

0

Dim Y Interpolation

1

0 Strided Access

Dim X Interpolation

(a) Interpolation (SZ3.3)

(The fastest Dim)

Buffer roll

5.1

Memory Regularized Interpolation

Interpolation (dynamic spline interpolation) predicts scientific data hierarchically and progressively. Starting from a sparse grid of losslessly preserved anchor points, each interpolation level predicts previously unprocessed values from available adjacent data using a stride 𝑠 ℓ , shown in Figure 3(a). To ensure consistent compression and decompression, each prediction uses reconstructed neighboring data rather than the original data. The commonly used linear and cubic splines [41, 42, 68] are:

𝑝𝑖lin =

Buffer roll ... ... ... ... ...

Prediction ... ...

Buffer roll ... ...

2n×stride

... Prediction

... Sliding Window

Dim Z Interpolation

(2n+1) ×stride

Dim Y Interpolation

Prediction

Dim X Interpolation

(b) Memory Regularized Interpolation in Falcons

Figure 3. Comparison between interpolation-based data prediction in prior work and the memory-regularized, vectorized interpolation in Falcons. Filled and hollow circles denote predictor and predictand values, respectively.

𝑥ˆ𝑖 −𝑠ℓ + 𝑥ˆ𝑖+𝑠ℓ cubic −𝑥ˆ𝑖 −3𝑠ℓ + 9𝑥ˆ𝑖 −𝑠ℓ + 9𝑥ˆ𝑖+𝑠ℓ − 𝑥ˆ𝑖+3𝑠ℓ , 𝑝𝑖 = . 2 16

buffered predictor data. For interpolation along the nonfastest-varying dimensions (e.g., Dim Z and Dim Y), it maintains multiple 1-D buffers—two for linear spline interpolation and four for cubic spline interpolation. After completing interpolation for the current predictand sequence and switching to a new one along the interpolation axis, the buffered window rolls forward, with one new buffer loaded and the others reused. For interpolation along the fastest-varying dimension (like Dim X in Figure 3 (b)), the predictand data and the predictor data interleave in the same 1-D sequence, so Falcons performs a split of them, placing all predictor data in a contiguous 1-D buffer to enable efficient slidingwindow-based interpolation. By transforming the strided memory accesses of interpolation into a contiguous pattern, Falcons facilitates vectorized SIMD operations on modern CPUs to perform a fused prediction-quantization operation and leverages architecturespecific adaptations for AVX2/SVE2 to maximize efficiency.

The prediction error is then quantized as 𝑞𝑖 = round((𝑥𝑖 − 𝑝𝑖 )/(2Δ)), and the reconstructed value is 𝑥ˆ𝑖 = 𝑝𝑖 + 2Δ𝑞𝑖 . Although this hierarchical design effectively captures spatial correlations, its conventional implementation is poorly matched to modern architectures. As illustrated in Figure 3 (a), existing interpolation-based compressors exhibit two critical issues that will harm the computational throughput. (1) Non-contiguous memory accesses: When moving across the predictand data (i.e., the data to be predicted), the accessed data index will increase by at least 2, which limits the use of hardware-level performance optimization techniques such as SIMD (Single Instruction, Multiple Data) and leads to unavoidable cache capacity misses. (2) Non-optimized reuse of predictor data: Predictor data values (i.e., the data used for prediction) are repeatedly accessed by subsequent interpolation stencils without an explicit mechanism for retaining and reusing them efficiently. Falcons integrates an interpolation-based predictor that reorganizes the interpolation computation to ensure regularized memory access, while not modifying the prediction values to maintain accuracy. As indicated in Figure 3 (b) as a 3-D example, the predictor data used for a 1-D subsequence along the fastest-varying dimension (namely, Dim X in Figure 3) form a sparse structured sub-grid. Moreover, predictions within the same subsequence are independent. Therefore, Falcons reorganizes those values into multiple aligned 1-D buffers residing in contiguous memory regions. Those buffers are along the fastest-varying dimension to minimize the build cost. Falcons further maximally reuses

5.2

Quantization-Distribution Aware Lossless Encoding

Existing interpolation-based error-bounded lossy compressors mostly apply a general-purpose Huffman encoder as their core entropy-coding module [33, 41–43, 68], which exhibits two limitations in an application-aware, performanceoriented design: (1) the algorithmic nature of Huffman coding constrains its parallel scalability; (2) the generalized pipeline does not fully exploit the compressibility of quantized errors. After empirical study, algorithmic design, and performance optimization, Falcons delivers a lossless coding 5

Table 2. Quantization-code statistics. 𝐿𝑛 denotes the average run length of code 𝑛. 𝐻 0 and 𝐻 1 denote the zero-order and first-order conditional entropy, respectively. Field

Error

𝐿0

𝐿1

𝐿 −1

𝐻0

𝐻1

temperature temperature

1E-2 1E-4

5,743.3 33.7

1.28 1.40

1.26 1.38

0.005 0.529

0.003 0.390

Table 3. Time costs and stage-wise compression ratio progression of different pipelines (data: Hurricane-TC, EB is error bound). EB

Pipeline

Run coding

Entropy coding

Huffman + Zstd [33, 35]

–

31.7×

ADT-FSE + Zstd [50]

–

527.2×

Zstd

Time (ms)

952.6×

238

942.6×

115

+2908.4%

QV QV

1E-2 1E-4

301.3 6.8

1.97 1.33

0.084 2.517

0.048 2.038

TC48 TC48

1E-2 1E-4

514.2 5.8

1.21 1.48 0.044 1.75 1.78 2.429

0.029 2.079

2.03 1.27

−→

+78.8%

1E-2

+328.5%

RLE + FSE + Zstd

159.7× −→

−→

+0.0%

684.2×

+592.5%

−→

+6.8%

RunGamma + FSE + Zstd 159.7× −→

1106.0×

Context + FSE + Zstd

–

475.6×

Huffman + Zstd [33, 35]

–

24.2×

ADT-FSE + Zstd [50]

–

−→

684.1×

18

1181.3×

17

+90.7%

−→

906.9×

99

58.0×

355

41.3×

140

31.8×

116

+139.8%

−→

+14.6%

pipeline composed of a core FSE (Finite-State Entropy) module, two newly designed preprocessors, and a lightweight distribution-aware online selection for the preprocessor.

1E-3

36.0× +370.6%

−→

6.8×

RunGamma + FSE + Zstd

6.8×

31.8×

Context + FSE + Zstd

–

56.7×

Huffman + Zstd [33, 35]

–

9.9×

ADT-FSE + Zstd [50]

–

10.0×

RLE + FSE + Zstd

1.6×

RunGamma + FSE + Zstd

1.6×

Context + FSE + Zstd

–

+711.8%

−→

−→

+0.0%

RLE + FSE + Zstd

−→

+11.0%

55.2×

−→

61.3×

85

58.9×

110

12.5×

408

10.8×

199

6.9×

461

+3.7%

−→

+25.6%

5.2.1 FSE as the Core Entropy-Coding Module. As an entropy-coding algorithm, FSE [10] is a table-based implementation of Asymmetric Numeral Systems (ANS) [14] that encodes symbols through transitions of a finite-state machine. It provides near-entropy compression with low per-symbol overhead, making it more attractive for highthroughput coding than Huffman. Prior work [50] has demonstrated its potential for encoding quantized errors, but its broader adoption remains limited by inconsistent compression ratios and throughput across datasets. This variability arises because quantization code distributions change substantially with data characteristics and error bounds, ranging from low-entropy streams dominated by long symbol runs to high-entropy streams spread across broad symbol ranges. Falcons unlocks the full potential of FSE coding with the following intra-module and inter-module optimizations: (1) Falcons partitions the quantization code sequence into multiple independent FSE streams and interleaves their state transitions to mitigate the loop-carried dependency between FSE state transitions that serializes the processing of a single coding stream. This design exposes instruction-level parallelism and improves the throughput. (2) Adapting to the quantized error patterns, Falcons leverages two different data preprocessing modules before the FSE process, ensuring the input data is friendly and efficient for FSE coding.

−→

+7.3%

1E-4

+341.1%

−→

+0.1%

6.9×

+656.3%

−→

−→ −→

+3.8%

12.1×

−→

+1.5%

15.4×

−→

12.6×

288

15.7×

126

capture the frequency skew of non-zero symbols and runlength counts. Specifically, for any non-negative run-length 𝑙, RunGamma applies Elias-gamma decomposition to transform multi-bytes 𝑙 into a single-byte class 𝑘 and a mantissa   𝑚: 𝑙 + 1 = 2𝑘 + 𝑚, 𝑘 = log2 (𝑙 + 1) , 0 ≤ 𝑚 < 2𝑘 . The class index k fits within a single byte and retains a highly skewed distribution, making it suitable for FSE encoding. The mantissa m is approximately uniform and can be directly bitpacked without entropy coding. This design is inspired by our finding that the zero-run lengths and quantization codes closely approximate a geometric distribution and are inherently suited for Elias-gamma decomposition [15, 28, 56]. For high-entropy codes with short zero-runs, Falcons includes the Context preprocessor, which models the conditional probabilities of quantization data symbols to further capture this dependency and reduce the entropy of the encoded data streams. Adjacent quantized errors in the interpolation-ordered stream are produced from spatially related data points under similar prediction conditions, so they present solid correlation. Table 2 shows that quantization codes retain strong first-order conditional spatial correlation, yielding a significantly lower first-order conditional entropy than zero-order entropy (by 14.4%-42.9%). Context first partitions a long quantization code stream into independently modeled segments to accommodate statistical variations across different interpolation levels and directional passes. Within each segment, the preceding code is mapped to a magnitude context bucket, 𝑐𝑖 = 𝑔(|𝑞𝑖 −1 |), and an independent FSE table is constructed for each bucket to model the conditional distribution 𝑃 (𝑞𝑖 | 𝑐𝑖 = 𝑐). During encoding, Context derives 𝑐𝑖 from 𝑞𝑖 −1 and encodes the current code 𝑞𝑖 using the corresponding FSE table. Each segment stores its own context tables and encoded stream, allowing the model to adapt to local distribution changes.

5.2.2 Distribution-Aware Quantization Data Preprocessing. Quantization data from interpolation-based decorrelation generally exhibit two distinct forms: frequency skew (symbol probability distribution) and spatial redundancy (contiguous symbol repetitions). As shown in Table 2, lowentropy streams contain long runs and highly skewed distributions, whereas high-entropy streams exhibit shorter runs and flatter distributions. For low-entropy quantized data, we propose the RunGamma preprocessor, which comprises mode-based RLE (Run-length encoding) and Elias-gamma decomposition. RunGamma first explicitly captures spatial mode (0 in quantization codes) runs via mode-based RLE and transforms multi-byte run lengths into a more compressible representation to efficiently 6

RunGamma+FSE

Falcons w/o L

Falcons w/ L

Time (s)

L3 miss (%)

100.00

0.000

4096

Miranda

97.14

0.086

Dataset

2048

Hurricane

95.00

0.542

8.4514

1024

SCALE

98.33

0.179

CESM

97.97

0.079

JHTDB

Dataset Accuracy (%) Speed loss (%) NYX

4.6116

F0 = 0.8

8192

Time (ms)

Context+FSE

NYX

16384

512 256

0.5

0.6

0.7

0.8

0.9

Zero-code probability F0

(a)

1.0

JHTDB

100.00

0.000

(b)

7.1075

3.6293

DRAM (GB/rank) 32.027 28.335 19.917 16.874

53.069% 19.902% 58.009% 37.908%

IPC 1.579 1.724 1.410 1.597

Figure 4. Evaluation of the adaptive encoding selection mechanism. (a) Variation of the zero-code probability and the execution times of the two encoding methods. (b) Selection accuracy and the resulting speed loss compared to the "oracle" choices.

Figure 5. Compression performance and hardware metrics at 32 MPI ranks on NYX and JHTDB datasets. L denotes Low-Traffic Execution. Within each metric, segment length is proportional to the relative improvement from Falcons w/o L to Falcons w/ L.

5.2.3 Dynamic selection of preprocessors. Table 3 provides a quantitative analysis of different lossless pipelines with one representative scientific data input under different (error-bound) configurations. The key findings are twofold: (1) Compared to existing pipelines and naive preprocessingbased pipelines, the RunGamma and Context preprocessors substantially upgrade the efficiency of lossless coding for quantized errors, while bringing non-degraded compression ratios. (2) Under various use-case configurations, the performance advantages of RunGamma and Context can diverge. To clearly distinguish the performance advantages of the two preprocessors, we thoroughly profiled them and confirmed the relationship between their performance and the top-1 (zero) symbol frequency. Figure 4 (a) shows that, with a threshold of zero-symbol frequency ≈ 0.8, the performance rank of RunGamma and Context can be determined. To this end, in the online encoding of quantization data, Falcons first profiles the zero-symbol frequency 𝐹 0 (in a sample-based routine with negligible overhead), and then dispatches the data to the better-fit preprocessor (RunGamma when 𝐹 0 ≥ 0.8). Figure 4 (b) demonstrates that this strategy achieves over 95% selection accuracy, with less than a 1% performance loss relative to oracle selection.

node processes a data snapshot independently for weakscaling parallel tasks, CPU memory bandwidth and cache size become a critical bottleneck as parallelism increases. Threads will compete for the shared L3 cache and memory channels, increasing cache evictions and memory-bandwidth pressure. To address this bottleneck, Falcons fuses its interpolation operations along multiple dimensions into a low-traffic single-pass interpolation streaming dataflow. For clarity, a three-dimensional case is provided as a representative example. At each interpolation level, Falcons pipelines the directional interpolations to keep intermediate values cache-hot. Instead of completing a volume-wide 𝑥 pass before starting the 𝑦 pass, it processes each 𝑦 line as soon as its required 𝑥 lines become available, reducing the reuse distance from a full subdomain sweep to only a few neighboring lines. The resulting 𝑥–𝑦 planes are then streamed into the 𝑧 interpolation according to their dependencies. The same principle applies to two- to four-dimensional data by pipelining the available directional passes and retaining only the small set of slices required by their interpolation dependencies. In addition, Falcons requests transparent huge pages for each rank’s quantization buffer to reduce dTLB misses under high rank density and eliminates reconstruction writebacks at the finest level when the written values will not be consumed by subsequent predictions. As shown in Figure 5, on two test cases, these optimizations substantially reduce DRAM traffic and L3-cache miss rate while increasing IPC.

5.3

Optimizing Falcons in Parallel Scientific Applications

When scientific data compression is integrated into applications and invoked in practical parallel environments, depending on the application tasks and parallelization schemes, new performance bottlenecks can arise, including but not limited to competition for memory bandwidth and cache capacity (in data-intensive weak-scaling cases) and compression-ratio degradation from reduced data access (in compute-intensive strong-scaling cases). Aware of these application-specific bottlenecks, Falcons conducts systematic optimizations to fully enhance performance and scalability in diverse parallel scientific applications.

5.3.2 Fine-grained Workload Partition for Strong Scaling Workflows. Under strong scaling, an increasing number of threads cooperatively compress a fixed-size data snapshot. Some existing compressors such as SZ3 and SPERR employ decorrelation algorithms that exploit global data redundancy. However, they straightforwardly partition input data into separate threads, which harms the performance of their decorrelation algorithms and lowers their compressionratio relative to serial execution. For example, given a data array 𝑥 0, ..., 𝑥 4, ..., 𝑥 8 , partitioning may force 𝑥 4 to be predicted using only 𝑥 0 , rather than both 𝑥 0 and 𝑥 8 , reducing prediction accuracy. In contrast, Falcons applies a fine-grained thread-level workload partition scheme, as demonstrated in Figure 6 in a

5.3.1 Reducing Data Traffic for Weak-Scaling Parallel Workflows. When each thread on the same chip or NUMA 7

Dim Z Interpolation

Dim Y Interpolation

Table 4. Scientific datasets used in the evaluation.

Dim X Interpolation

(The fastest Dim)

n-1 n-2 …

2

1

…

0

2

1

n-1 n-2 …

0

2

1

0

Threads

Dataset

Application

Dimensions

CESM-ATM NYX Miranda Hurricane ISABEL SCALE-LETKF JHTDB

Atmospheric simulation Cosmological simulation Hydrodynamics simulation Weather simulation Climate simulation Turbulence simulation

1800𝑦 × 3600𝑥 512𝑧 × 512𝑦 × 512𝑥 256𝑧 × 384𝑦 × 384𝑥 100𝑧 × 500𝑦 × 500𝑥 98𝑧 × 1200𝑦 × 1200𝑥 512𝑧 × 512𝑦 × 512𝑥

Figure 6. The fine-grained thread workload partition in Falcons. NYX

node. On Sugon-SD and LineShine, the compilers are GCC 9.3.0 and Lclang 1.0.0, respectively. Datasets. Datasets across six diverse scientific domains are adopted, which are widely used in previous studies [42, 69]. Detailed information on the datasets is provided in Table 4. The JHTDB dataset is collected from [32], and all other datasets are available at SDRBench [69]. Baselines. We compare Falcons against four state-of-theart error-bounded scientific lossy compressors, spanning both ratio-oriented and throughput-oriented designs. As explained in Section 1, GPU-based compressors are excluded because they target use scenarios different from those of CPU compressors.

JHTDB

Comp Ratio

1000 3000

800 600

2000

400

1000 1

2

4

8

16

32

200

1

2

4

8

16

32

SPERR SZ3.3 Falcons w/ F Falcons w/o F

Figure 7. The compression ratios of compressors under strongscaling parallelism at an error bound of 1E-2 on 2 representative datasets. F denotes Fine-grained workload partition.

3D example. To enable vectorization-aware regularized interpolation, the workload for each thread is contiguous along the current interpolation dimension and the fast-varying dimension. For example, for the first interpolation direction (Dim Z in Figure 6), each thread processes 2-D plane slices along Dim X (the fastest-varying dimension) and Dim Z (the current interpolation direction), and for the interpolation along the fastest-varying dimension X, each thread will process with a memory-contiguous data block along Dim X and Dim Y (not necessarily a 2-D slice). This workload partition scheme ensures a relative balance between threads, and keeps the same interpolation results as in the single-thread case. As illustrated in Figure 7, several existing compressors and Falcons without a fine-grained workload partition show a clear compression ratio drop in thread-cooperative compression, while Falcons with fine-grained strong-scaling parallelism can preserve nearly identical compression ratio to the single-thread case (the slight drop is due to the parallelized lossless encoding).

6

• ZFP [36]: A representative transform-based scientific lossy compressor with a throughput-prioritized design. • PFPL [16]: A fast scientific lossy compressor by efficiently encoding quantized input. • SPERR [30]: A high-ratio wavelet-transform-based scientific lossy compressor with superior compression quality. • SZ3.3 [35]: The most recent version of the SZ compressor, which merges the old version of SZ3 [68] and HPEZ [42]. Configurations and Metrics. For the error-bound setting, we use the widely adopted value-range-based relative error bound (denoted by 𝜖), which is essentially equivalent to the absolute error bound (denoted by Δ), following the relationship Δ = 𝜖 × value_range. Each timing experiment is repeated five times, and the reported execution time is the arithmetic mean. We perform the evaluation based on the following key metrics: • Compression/Decompression Throughput: Throughput shows the speed of compressors. It is the input data size divided by the compression/decompression time cost. • Compression Ratio (CR) and bit rate: CR is defined as the original data size divided by the compressed data size. Bit rate is inversely proportional to CR and measures the average number of bits required to store each value. A higher compression ratio or a lower bit rate indicates more efficient storage. • Peak Signal-to-Noise Ratio (PSNR): PSNR [61] quantifies the distortion between the input and decompressed data by value range and mean-square error, and a higher PSNR is better. • Performance in real-world applications: The data processing time cost in several scientific application use-case

Evaluation

In this section, we evaluate Falcons thoroughly to validate its advantages in real-world scientific applications. Its performance is profiled on 6 different scientific domains and across cutting-edge supercomputers, from fundamental metrics to comprehensive use cases. 6.1

Experimental Setup

Testbed. We conduct our experiments on two supercomputing platforms. Sugon-SD supercomputer features two Intel Xeon Platinum 8358P (x86) 2.6 GHz processors with 64 physical cores and 512 GB of memory per computing node. LineShine supercomputer [64] features LX2 304C (ARM) CPUs at 1.55 GHz with 512 GB of memory per computing 8

Table 5. Compression ratio of different compressors under singlecore execution. Improv denotes improvement over the best baseline.

studies such as parallel data I/O, remote data transfer, and end-to-end integration analysis. 6.2

Dataset

𝜖

NYX

1E-2 38.8 83.9 1E-3 13.1 27.5 1E-4 6.2 11.2

Fundamental Performance Profiling

6.2.1 Ratio and Quality. At the beginning of the evaluation, we present the compression ratio and quality analysis of Falcons together with other compressors. Table 5 presents the compression ratio of the evaluated scientific lossy compressors on all datasets under three representative error bounds. As a high-ratio scientific lossy compressor, Falcons achieves up to a 34%/155% improvement over SZ3.3 and SPERR under the same error bound, and largely outperforms ZFP and PFPL in terms of compression ratio. Figure 9 illustrates the rate-distortion (PSNR) of compressors in a 32-thread parallelism. It is also clear that Falcons achieves excellent rate-distortion, outperforming SZ3.3 and being comparable to SPERR (which is quite slower than Falcons). These results justify the effective interpolation and encoding design of Falcons, and in what follows, we can see that the high compression ratio of Falcons is achieved without a throughput trade-off.

ZFP PFPL SPERR SZ3.3 Falcons Improv 1498.6 2863.6 3596.3 +25.6% 152.9 195.0 228.3 +17.1% 24.5 27.7 30.4 +9.7%

Miranda

1E-2 88.9 104.4 1434.9 2222.2 2606.0 +17.3% 1E-3 49.9 44.4 400.4 392.3 482.2 +20.4% 1E-4 28.6 24.0 138.1 104.9 140.5 +1.7%

Hurricane ISABEL

1E-2 11.3 24.8 1E-3 6.7 11.0 1E-4 4.3 6.2

32.5 15.2 8.4

SCALELETKF

1E-2 27.1 91.9 1E-3 12.7 30.7 1E-4 7.3 13.2

1137.1 1197.1 1383.4 +15.6% 158.6 111.7 123.5 −22.1% 37.2 24.4 28.6 −23.1%

76.1 23.2 9.6

78.2 26.1 11.8

+2.8% +12.5% +22.9%

1E-2 12.6 51.6 CESM-ATM 1E-3 7.5 19.7 1E-4 4.8 9.0

257.9 49.4 15.4

338.6 53.3 15.3

388.9 56.7 16.7

+14.9% +6.4% +8.4%

1E-2 22.3 29.0 1E-3 9.8 11.9 1E-4 5.0 5.7

391.4 69.5 18.1

773.8 82.5 16.4

998.4 97.8 20.2

+29.0% +18.5% +11.6%

JHTDB

shortens the reuse distance between dependent interpolation operations. This lowers the pressure placed by co-located ranks on the shared cache and memory channels, allowing aggregate compression and decompression throughput to grow steadily through a parallelism of 32.

6.2.2 Single-Core Throughput. In Figure 8, the singlecore throughput of the evaluated scientific compressors on two CPU architectures (x86 and ARM) is demonstrated. In almost all test cases, Falcons delivers the highest throughput, and in the few remaining ones, it achieves comparable speed to PFPL, which exhibits much lower compression ratios. Specifically, compared with high-ratio compressors, Falcons achieves up to 547%/418% (to SZ3.3) and 2086%/1213% (to SPERR) compression/decompression throughput improvement. Moreover, Falcons outperforms PFPL and ZFP by up to 255%/306% and 343%/236% in compression/decompression throughput. Featuring an interpolation-based design, Falcons achieves 1.7×–6.5× speedups over SZ3.3, demonstrating the universal effectiveness of its memory-access-regularized interpolation and redesigned encoding pipeline across x86 and ARM CPU-based systems.

6.3

Real-World Application Case Study

After validating the fundamental performance metrics of Falcons, we further conduct a series of case studies, covering the most representative usage of scientific lossy compression and demonstrating how it takes significant advantages over existing compressors in parallel scientific applications. 6.3.1 Case Study I: Large-Scale Parallel Data I/O . In the first case, we directly integrate Falcons into real-world scientific application workflows with large-scale parallelism. Specifically, Falcons and other compressors are leveraged to accelerate the I/O process between large-scale parallel scientific applications and storage systems by performing errorbounded lossy compression/decompression before/after writing/reading the exascale scientific data stream to/from the storage. Figure 11 presents the time costs and compression ratios of different compressors in this task. The performance evaluation is conducted with different scientific datasets on two supercomputers, Sugon-SD and LineShine, which exhibit distinct I/O bandwidths (≈ 30 GB/s and ≈ 200 GB/s). Following the existing evaluation scheme [33, 68], each parallel rank processes a copy of the dataset. Attributed to the extremely accelerated high-ratio design, in both cases, Falcons achieves the minimized time cost for parallel I/O while reducing the storage cost to 0.21% and 0.44% of the original data size. On Sugon-SD with 1024 cores, Falcons gains ≈ 37% time reduction over the second-best PFPL, and for 4096

6.2.3 Parallel Scalability. Figure 10 evaluates the scalability of Falcons and the baselines under two parallelization schemes. Figure 10 (a) reports strong scaling on a fixed-size input, while Figure 10 (b) reports weak scaling using the increasing number of ranks with a fixed-size partition per rank. Falcons consistently achieves the highest compression and decompression throughput in both cases. Under strong scaling, ratio-oriented fine-grained scheduling in Falcons parallelizes independent interpolation lines while preserving the complete prediction context, increasing throughput with slightly degrading compression ratio, as shown in Figure 7. Scaling becomes sublinear at high thread counts because of shrinking per-thread workloads and stage-level synchronization. Under weak scaling, the low-traffic streaming dataflow in Falcons reduces repeated rank-local data traversals and 9

ZFP

Decomp Th (MB/S)

Comp Th (MB/S)

NYX

PFPL

Hurricane ISABEL

SPERR

SZ3.3

Miranda

Falcons

SCALE-LETKF

1000 500

200 1E-2

1E-3

0

1E-4

1E-2

1E-3

0

1E-4

750

500

400

0

JHTDB

750

600 500

CESM-ATM

1000

1000

500

500

250 1E-2

1E-3

0

1E-4

1500

250 1E-2

1E-3

0

1E-4

1E-2

1E-3

1000

500

500

0

1E-2

1E-3

0

1E-4

1E-2

1E-3

0

1E-4

1E-4

1E-2

1E-3

1E-4

500

500

500

250

1E-3

1000

1000

500

1E-2

1000

750

1000

0

1E-4

1E-2

1E-3

0

1E-4

1E-2

1E-3

0

1E-4

1E-2

1E-3

0

1E-4

(a) Intel Xeon Platinum 8358P (x86) on Sugon-SD ZFP

Decomp Th (MB/S)

Comp Th (MB/S)

NYX

PFPL

Hurricane ISABEL

SPERR

Falcons

SCALE-LETKF

CESM-ATM

750

1000 600

SZ3.3

Miranda

400

1E-2

1E-3

0

1E-4

1E-2

1E-3

0

1E-4

750

1000

500

400

400

250

200

200

0

0

500

200 200 0

JHTDB 600

600

400

1E-2

1E-3

1E-4

1000

250

500

1E-3

1E-4

1000

1500

500

1E-2

1E-2

1E-3

0

1E-2

1E-3

0

1E-4

1E-2

1E-3

0

1E-4

1E-2

1E-3

1E-4

1E-2

1E-3

1E-4

1000

750 500

500

500

0

1E-4

500 250

1E-2

1E-3

0

1E-4

1E-2

1E-3

0

1E-4

1E-2

1E-3

0

1E-4

(b) LX2 304C (ARM) on LineShine

Figure 8. The single-core compression and decompression throughput in MB/s on Intel and ARM CPUs under different error bounds. PFPL

Falcons

ZFP

100

90

80

80

80

100

70

60

95

40

85 1.5

0

1

2.0

2

70

2.5

3

0

SCALE-LETKF

120

100

90

60

80

90

60

3

4

4

6

100

80

60

90

Bitrate

80 1.5

1.5

3

4

0.8

3

7.5

4

5.0

2

2.5

0

1

2

0

2

Bitrate

2.5

4

40

8

Falcons

16

32

0.0

1

2

4

8

16

32

(a) Throughput for strong-scaling parallelism PFPL

90

60 2.0

4

ZFP 80

40 1.0

2

0.6

2

SZ3.3

Decompression

6

80

90

1

1

SPERR

Compression

100

80

80

0

0.4

0

JHTDB

100

40

40

CESM-ATM 100

60

95 90

2

2

PFPL

Miranda

120

Throughput (GB/s)

PSNR

SZ3.3

100

100

50

ZFP Hurricane ISABEL

Throughput (GB/s)

SPERR NYX

110

1

0

2

2

4

6

Bitrate

Figure 9. Rate Distortion (PSNR) Evaluation of Falcons and other compressors under strong-scaling parallelism (32 threads).

cores on LineShine, the corresponding reduction rate is ≈ 44.6%. Those results explicitly demonstrate that, under varying hardware specifications and parallelism configurations, the performance advantage of Falcons is consistent.

SPERR

SZ3.3

Compression 20

20

15

15

10

10

5

5

0

1

2

4

8

16

Falcons

Decompression

32

0

1

2

4

8

16

32

(b) Throughput for weak-scaling parallelism

Figure 10. The parallel compression and decompression throughput of compressors aggregated from six datasets.

lossy-compressed data chunks can be delivered much faster than the original ones. Table 6 profiles the end-to-end time (compression + transfer + decompression) of a compressionaccelerated remote data transfer between Sugon supercomputing clusters at different locations. The network bandwidth is ≈ 1 GB/s, and threads on each node compress the data stream in a strong-scaling scheme. In all tested datasets, Falcons achieves speedups over baselines, with performance

6.3.2 Case Study II: Parallel Scientific Data Transfer Accelerated with Lossy Compression. In the second usecase study, we target a scenario in which the data bandwidth is much more limited than local I/O, simulating the remote transfer of scientific data across different endpoints, which essentially supports distributed scientific analysis. Due to limited network bandwidth, scientific lossy compression can significantly accelerate the transfer process, as 10

30

CR:152.9 t:247.4 s t:103.3 s

CR:195.0 t:93.9 s

90

20

CR:13.1 t:50.6 s

60 CR:49.9 CR:392.3 t:10.3 s t:10.0 s CR:44.5 CR:482.2 t:7.1 s t:4.5 s

10

30

ZFP

60.9 67.3 27.5 144.2 376.8 118.9

25.3 30.1 11.6 56.6 51.3 51.9

12.3 19.7 7.2 33.4 27.3 28.6

R

0.05

25 0

0

500

1000

1500

West-East grid index

ER

PL

500

0

500

1000

1500

0

West-East grid index

0

500

1000

1500

0.00

West-East grid index

Compute (s) 306.20 309.98 301.97

Write (s) 173.61 721.36 8.32

Total (s) 479.81 1031.34 310.29

Speedup 1.00× 0.47× 1.55×

Figure 12. End-to-end analysis of Falcons integration into the WRF. Falcons compresses the data with the error bound of 10−3 .

7

Conclusion and Future Work

To overcome the performance bottleneck of scientific errorbounded lossy compression and advance its usability in realworld parallel scientific applications, we propose Falcons, an ultra-fast high-ratio scientific error-bounded lossy compressor that breaks the trade-off between compression ratio and throughput. By introducing a memory-access regularized interpolation predictor and co-redesigning the de-correlation and lossless encoding modules, Falcons delivers state-of-theart compression ratio and throughput in the evaluation of 6 scientific domains. In real-world application case studies, compared to baselines, Falcons gains up to 44.6% and 51.5% acceleration in large-scale parallel data I/O and remote transfer tasks. When integrated into the application as a data filter, it can reduce the end-to-end workflow time cost by 35%. To further enhance error-bounded lossy compression for real-world scientific application tasks in the future, we plan to: (1) exploit new-generation CPU hardware microarchitecture such as SME and AMX units for scientific data compression. (2) explore a more adaptive compression framework design that auto-selects the compression pipeline according to applications and hardware specifications.

PFPL SPERR SZ3.3 Falcons Speedup (%)

128.3 33.8 89.4 30.5 39.0 16.3 257.4 65.5 130.5 29.5 280.6 108.2

0.10

50

Format Data (GB) NetCDF-Classic 59.88 NetCDF-4 (Zlib) 17.86 1.10 NetCDF-4 (Falcons)

Table 6. End-to-end remote transfer time (in seconds) on Sugon using 20 nodes with 32 OpenMP threads each and an error bound of 10−3 . Transfer bandwidth is ≈ 1 GB/s. Gain denotes the percentage time reduction of Falcons over the fastest baseline.

NYX Hurricane ISABEL Miranda SCALE-LETKF CESM-ATM JHTDB

(c) Absolute difference

75

Timing breakdown

(b) LineShine, 4096 Cores, NYX

Figure 11. Large-scale parallel I/O performance with compression.

Dataset

(b) Falcons (PSNR=69.22 dB)

1000

SP

P

3. 3

PF

SZ

ns

ZF

co Fa l

p

(a) Sugon-SD, 1024 Cores, Miranda

No

Co m

R

PL

ER

PF

SP

P

3. 3

ZF

SZ

co

Co m No

ns

0 p

0

CR:27.5 t:40.1 s

CR:228.3 t:22.2 s

(a) Original

Absolute difference (m s−1 )

CR:400.4 t:31.3 s

WRF CONUS25 mass-grid horizontal slice at vertical level 24

Decompression

Wind speed (m s−1 )

Reading 120

South-North grid index

Writing

t:95.9 s

Fa l

Elapsed time (s)

Compression 40

51.5% 34.5% 37.9% 41.0% 7.5% 44.9%

gains of 51.5% in maximum and 37.3% across all datasets combined. Combining case studies I and II, we can confidently claim that Falcons achieves SOTA performance in both computing-bounded and bandwidth-bounded data compression use cases, because its compression ratio and throughput are both leading among existing compressors. 6.3.3 Case Study III: End-to-End Analysis of Falcons Integration into the WRF Workflow. At the end of the case study and our evaluation, we demonstrate a practice of deeply integrating Falcons into the WRF [58] (Weather Research & Forecasting) application model. WRF is a widely used numerical weather prediction system for atmospheric research and forecasting, and it supports output in the NetCDF classic format [53] (without data compression) and NetCDF4 [54] format with Zlib [47] compression. We utilize the HDF5 filter to transparently integrate Falcons into WRF’s NetCDF-4 I/O path, without modifying WRF’s numerical kernels. In our evaluation, the CONUS 2.5-km benchmark [52] of WRF is executed on 2048 cores of the Sugon-SD supercomputer, and Figure 12 profiles the end-to-end time costs with three different data filters. In the original WRF application, I/O time and data output size are two critical bottlenecks, and its native Zlib compression brings little help. By leveraging Falcons, the two bottlenecks can be simultaneously resolved, reducing output data size to ≈ 1.8% and accelerating end-toend execution by 1.55×. Figure 12 also visualizes a data slice of one physical field (wind speed) from the original output, Falcons lossy decompressed output, together with the error map between them. Under the error bound of 10−3 , Falcons compressed data preserves satisfactory data quality (PSNR = 69.22) and a near-random error map.

References [1] Mark Ainsworth, Ozan Tugluk, Ben Whitney, and Scott Klasky. 2018. Multilevel techniques for compression and reduction of scientific data—the univariate case. Computing and Visualization in Science 19, 5 (2018), 65–76. https://doi.org/10.1007/s00791-018-00303-9 [2] Ann S Almgren, John B Bell, Mike J Lijewski, Zarija Lukić, and Ethan Van Andel. 2013. Nyx: A massively parallel amr code for computational cosmology. The Astrophysical Journal 765, 1 (2013), 39. https://doi. org/10.1088/0004-637X/765/1/39 [3] Argonne Leadership Computing Facility. 2024. Overview of Aurora Exascale Architecture. Argonne National Laboratory. https://www.alcf.anl.gov/sites/default/files/2024-11/Overview-ofAurora-Oct-2024.pdf [4] Rafael Ballester-Ripoll, Peter Lindstrom, and Renato Pajarola. 2019. TTHRESH: Tensor compression for multidimensional visual data. IEEE transactions on visualization and computer graphics 26, 9 (2019), 2891– 2903. https://doi.org/10.1109/TVCG.2019.2904063 [5] Franck Cappello, Mario Acosta, Emmanuel Agullo, Hartwig Anzt, Jon Calhoun, Sheng Di, Luc Giraud, Thomas Grützmacher, Sian Jin, Kentaro Sano, Kento Sato, Amarjit Singh, Dingwen Tao, Jiannan Tian, Tomohiro Ueno, Robert Underwood, Frédéric Vivien, Xavier Yepes, 11

[17] Mike Folk, Gerd Heber, Quincey Koziol, Elena Pourmal, and Dana Robinson. 2011. An overview of the HDF5 technology suite and its applications. In Proceedings of the EDBT/ICDT 2011 Workshop on Array Databases (Uppsala, Sweden) (AD ’11). Association for Computing Machinery, New York, NY, USA, 36–47. doi:10.1145/1966895.1966900 [18] Pascal Grosset, Christopher M Biwer, Jesus Pulido, Arvind T Mohan, Ayan Biswas, John Patchett, Terece L Turton, David H Rogers, Daniel Livescu, and James Ahrens. 2020. Foresight: Analysis that matters for data reduction. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE, 1–15. https://doi. org/10.1109/SC41405.2020.00087 [19] Jun Han and Fan Yang. 2025. DCINR: A Divide-and-Conquer Implicit Neural Representation for Compressing Time-Varying Volumetric Data in Hours. IEEE Transactions on Visualization and Computer Graphics 31, 10 (2025), 8116–8128. doi:10.1109/TVCG.2025.3564255 [20] Yafan Huang, Sheng Di, Guanpeng Li, and Franck Cappello. 2024. cuSZp2: A GPU lossy compressor with extreme throughput and optimized compression ratio. In SC24: International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE, 1–18. https://doi.org/10.1109/SC41406.2024.00021 [21] Yafan Huang, Sheng Di, Guanpeng Li, and Franck Cappello. 2025. GPU Lossy Compression for HPC Can Be Versatile and Ultra-Fast. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC ’25). Association for Computing Machinery, New York, NY, USA, 2056–2075. doi:10.1145/ 3712285.3759817 [22] Yafan Huang, Sheng Di, Xiaodong Yu, Guanpeng Li, and Franck Cappello. 2023. cuszp: An ultra-fast gpu error-bounded lossy compression framework with optimized end-to-end performance. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. 1–13. https://doi.org/10.1145/3581784. 3607048 [23] Hurricane ISABEL simulation data. 2004. http://vis.computer.org/ vis2004contest/data.html. Online. [24] IO500 Foundation. 2023. Aurora: SC23 IO500 Submission. IO500 Submission No. 696. https://io500.org/submissions/view/696 [25] Wenqi Jia, Sian Jin, Jinzhen Wang, Wei Niu, Dingwen Tao, and Miao Yin. 2024. GWLZ: A Group-wise Learning-based Lossy Compression Framework for Scientific Data. In Proceedings of the 14th Workshop on AI and Scientific Computing at Scale using Flexible Computing Infrastructures. 34–41. https://doi.org/10.1145/3659995.3660041 [26] Sian Jin, Dingwen Tao, Houjun Tang, Sheng Di, Suren Byna, Zarija Lukic, and Franck Cappello. 2022. Accelerating Parallel Write via Deeply Integrating Predictive Lossy Compression with HDF5. In SC22: International Conference for High Performance Computing, Networking, Storage and Analysis. 1–15. doi:10.1109/SC41404.2022.00066 [27] Jennifer E Kay, Clara Deser, A Phillips, A Mai, Cecile Hannay, Gary Strand, Julie Michelle Arblaster, S. C. Bates, Gokhan Danabasoglu, James Edwards, Marika Holland, Paul Kushner, Jean-François Lamarque, David Lawrence, Keith Lindsay, Andrew Middleton, Esmond Munoz, Richard Neale, Keith Oleson, Lorenzo Polvani, and Mariana Vertenstein. 2015. The Community Earth System Model (CESM) Large Ensemble Project: A Community Resource for Studying Climate Change in the Presence of Internal Climate Variability. Bulletin of the American Meteorological Society 96, 8 (2015), 1333–1349. doi:10.1175/BAMS-D-13-00255.1 [28] Anatoliy Kuznetsov. 2009. Elias Gamma Encoding. BitMagic Library Technical Note. https://bitmagic.io/dgap-gamma Updated January 2020; accessed July 22, 2026. [29] Lawrence Livermore National Laboratory. 2024. Lawrence Livermore National Laboratory’s El Capitan Verified as World’s Fastest Supercomputer. LLNL News. https://www.llnl.gov/article/52061/lawrencelivermore-national-laboratorys-el-capitan-verified-worlds-fastestsupercomputer Accessed: 2026-07-25.

Yoshii Kazutomo, and Boyuan Zhang. 2025. Multifacets of lossy compression for scientific data in the Joint-Laboratory of Extreme Scale Computing. Future Generation Computer Systems 163 (2025), 107323. doi:10.1016/j.future.2024.05.022 [6] Franck Cappello, Allison Baker, Ebru Bozda, Martin Burtscher, Kyle Chard, Sheng Di, Paul Christopher O Grady, Peng Jiang, Shaomeng Li, Erik Lindahl, Peter Lindstrom, Magnus Lundborg, Kai Zhao, Xin Liang, Masaru Nagaso, Kento Sato, Amarjit Singh, Seung Woo Son, Dingwen Tao, Jiannan Tian, Robert Underwood, Kazutomo Yoshii, Danylo Lykov, Yuri Alexeev, and Kyle Gerard Felker. 2025. Lossy Compression of Scientific Data: Applications Constrains and Requirements. arXiv:2503.20031 [astro-ph.IM] https://arxiv.org/abs/2503.20031 [7] Franck Cappello, Sheng Di, Sihuan Li, Xin Liang, Ali Murat Gok, Dingwen Tao, Chun Hong Yoon, Xin-Chuan Wu, Yuri Alexeev, and Frederic T Chong. 2019. Use cases of lossy compression for floatingpoint data in scientific data sets. The International Journal of High Performance Computing Applications 33, 6 (2019), 1201–1220. https: //doi.org/10.1177/1094342019853336 [8] Franck Cappello, Robert Underwood, Yuri Alexeev, Alison Baker, Ebru Bozdağ, Martin Burtscher, Kyle Chard, Sheng Di, Kyle Gerard Felker, Paul Christopher O’Grady, Hanqi Guo, Yafan Huang, Peng Jiang, Sian Jin, Petter Johansson, Shaomeng Li, Xin Liang, Erik Lindahl, Peter Lindstrom, Zarija Lukić, Magnus Lundborg, Danylo Lykov, Masaru Nagaso, Kento Sato, Amarjit Singh, Seung Woo Son, Shihui Song, William Tang, Dingwen Tao, Jiannan Tian, Kazutomo Yoshii, and Kai Zhao. 2025. What to Support When You’re Compressing: The State of Practice Gaps and Opportunities for Scientific Data Compression. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC ’25). Association for Computing Machinery, New York, NY, USA, 1966–1979. doi:10.1145/ 3712285.3759856 [9] Xiang Chen, Bing Lu, Haoquan Long, Huizhang Luo, Yili Ma, Guangming Tan, Dingwen Tao, Fei Wu, and Tao Lu. 2025. Computational Burst Buffers: Accelerating HPC I/O via In-Storage Compression Offloading. IEEE Transactions on Parallel and Distributed Systems 37, 2 (2025), 518–532. [10] Yann Collet. 2013. Finite State Entropy. https://github.com/Cyan4973/ FiniteStateEntropy. Software repository. [11] Yann Collet and Murray Kucherawy. 2021. Zstandard Compression and the ’application/zstd’ Media Type. RFC 8878. doi:10.17487/RFC8878 [12] Sheng Di and Franck Cappello. 2016. Fast error-bounded lossy HPC data compression with SZ. In 2016 ieee international parallel and distributed processing symposium (ipdps). IEEE, 730–739. https: //doi.org/10.1109/IPDPS.2016.11 [13] Sheng Di, Jinyang Liu, Kai Zhao, Xin Liang, Robert Underwood, Zhaorui Zhang, Milan Shah, Yafan Huang, Jiajun Huang, Xiaodong Yu, Congrong Ren, Hanqi Guo, Grant Wilkins, Dingwen Tao, Jiannan Tian, Sian Jin, Zizhe Jian, Daoce Wang, Md Hasanur Rahman, Boyuan Zhang, Shihui Song, Jon Calhoun, Guanpeng Li, Kazutomo Yoshii, Khalid Alharthi, and Franck Cappello. 2025. A Survey on ErrorBounded Lossy Compression for Scientific Datasets. ACM Comput. Surv. 57, 11, Article 287 (June 2025), 38 pages. doi:10.1145/3733104 [14] Jarek Duda. 2013. Asymmetric numeral systems: entropy coding combining speed of huffman coding with compression rate of arithmetic coding. arXiv preprint arXiv:1311.2540 (2013). [15] P. Elias. 1975. Universal codeword sets and representations of the integers. IEEE Transactions on Information Theory 21, 2 (1975), 194– 203. doi:10.1109/TIT.1975.1055349 [16] Alex Fallin, Noushin Azami, Sheng Di, Franck Cappello, and Martin Burtscher. 2025. Fast and Effective Lossy Compression on GPUs and CPUs with Guaranteed Error Bounds. In 2025 IEEE International Parallel and Distributed Processing Symposium (IPDPS). 874–887. doi:10.1109/IPDPS64566.2025.00083

12

[30] Shaomeng Li, Peter Lindstrom, and John Clyne. 2023. Lossy Scientific Data Compression With SPERR. In 2023 IEEE International Parallel and Distributed Processing Symposium (IPDPS). 1007–1017. doi:10.1109/ IPDPS54959.2023.00104 [31] Xiao Li, Liangji Zhu, Anand Rangarajan, and Sanjay Ranka. 2025. Generative Latent Diffusion for Efficient Spatiotemporal Data Reduction. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC ’25). Association for Computing Machinery, New York, NY, USA, 1980–1991. doi:10.1145/3712285.3759836 [32] Yi Li, Eric Perlman, Minping Wan, Yunke Yang, Charles Meneveau, Randal Burns, Shiyi Chen, Alexander Szalay, and Gregory Eyink. 2008. A public turbulence database cluster and applications to study Lagrangian evolution of velocity increments in turbulence. Journal of Turbulence 9 (2008), N31. https://doi.org/10.1080/14685240802376389 [33] Xin Liang, Sheng Di, Dingwen Tao, Sihuan Li, Shaomeng Li, Hanqi Guo, Zizhong Chen, and Franck Cappello. 2018. Error-Controlled Lossy Compression Optimized for High Compression Ratios of Scientific Datasets. In 2018 IEEE International Conference on Big Data. IEEE. https://doi.org/10.1109/BigData.2018.8622520 [34] Xin Liang, Ben Whitney, Jieyang Chen, Lipeng Wan, Qing Liu, Dingwen Tao, James Kress, David Pugmire, Matthew Wolf, Norbert Podhorszki, and Scott Klasky. 2022. MGARD+: Optimizing Multilevel Methods for Error-Bounded Scientific Data Reduction. IEEE Trans. Comput. 71, 07 (July 2022), 1522–1536. doi:10.1109/TC.2021.3092201 [35] Xin Liang, Kai Zhao, Sheng Di, Sihuan Li, Robert Underwood, Ali M. Gok, Jiannan Tian, Junjing Deng, Jon C. Calhoun, Dingwen Tao, Zizhong Chen, and Franck Cappello. 2023. SZ3: A Modular Framework for Composing Prediction-Based Error-Bounded Lossy Compressors. IEEE Transactions on Big Data 9, 2 (2023), 485–498. doi:10.1109/TBDATA. 2022.3201176 [36] Peter Lindstrom. 2014. Fixed-Rate Compressed Floating-Point Arrays. IEEE Transactions on Visualization and Computer Graphics 20, 12 (2014), 2674–2683. doi:10.1109/TVCG.2014.2346458 [37] Peter Lindstrom, Jeffrey Hittinger, James Diffenderfer, Alyson Fox, Daniel Osei-Kuffuor, and Jeffrey Banks. 2025. ZFP: A compressed array representation for numerical computations. The International Journal of High Performance Computing Applications 39, 1 (2025), 104– 122. arXiv:https://doi.org/10.1177/10943420241284023 doi:10.1177/ 10943420241284023 [38] Peter Lindstrom and Martin Isenburg. 2006. Fast and Efficient Compression of Floating-Point Data. IEEE Transactions on Visualization and Computer Graphics 12, 5 (2006), 1245–1250. doi:10.1109/TVCG.2006.143 [39] Jinyang Liu, Sheng Di, Sian Jin, Kai Zhao, Xin Liang, Zizhong Chen, and Franck Cappello. 2023. Scientific Error-bounded Lossy Compression with Super-resolution Neural Networks. In 2023 IEEE International Conference on Big Data (BigData). IEEE, 229–236. https://doi.org/10. 1109/BigData59044.2023.10386682 [40] Jinyang Liu, Sheng Di, Kai Zhao, Sian Jin, Dingwen Tao, Xin Liang, Zizhong Chen, and Franck Cappello. 2021. Exploring Autoencoderbased Error-bounded Compression for Scientific Data. In 2021 IEEE International Conference on Cluster Computing (CLUSTER). IEEE, 294– 306. https://doi.org/10.1109/Cluster48925.2021.00034 [41] Jinyang Liu, Sheng Di, Kai Zhao, Xin Liang, Zizhong Chen, and Franck Cappello. 2022. Dynamic quality metric oriented error bounded lossy compression for scientific datasets. In SC22: International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE, 1–15. https://doi.org/10.1109/SC41404.2022.00067 [42] Jinyang Liu, Sheng Di, Kai Zhao, Xin Liang, Sian Jin, Zizhe Jian, Jiajun Huang, Shixun Wu, Zizhong Chen, and Franck Cappello. 2024. Highperformance effective scientific error-bounded lossy compression with auto-tuned multi-component interpolation. Proceedings of the ACM on Management of Data 2, 1 (2024), 1–27. https://doi.org/10.1145/3639259

[43] Jinyang Liu, Jiannan Tian, Shixun Wu, Sheng Di, Boyuan Zhang, Robert Underwood, Yafan Huang, Jiajun Huang, Kai Zhao, Guanpeng Li, Dingwen Tao, Zizhong Chen, and Franck Cappello. 2024. cuSZ-i: High-Ratio Scientific Lossy Compression on GPUs with Optimized Multi-Level Interpolation. In Proceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis (Atlanta, GA, USA) (SC ’24). IEEE Press, Article 13, 15 pages. doi:10.1109/SC41406.2024.00019 [44] Yuanjian Liu, Sheng Di, Kyle Chard, Ian Foster, and Franck Cappello. 2023. Optimizing Scientific Data Transfer on Globus with ErrorBounded Lossy Compression. In 2023 IEEE 43rd International Conference on Distributed Computing Systems (ICDCS). 703–713. doi:10. 1109/ICDCS57875.2023.00064 [45] Yuanjian Liu, Sheng Di, Jiajun Huang, Zhaorui Zhang, Kyle Chard, and Ian Foster. 2025. Ocelot: An Interactive, Efficient Distributed Compression-As-a-Service Platform With Optimized Data Compression Techniques. IEEE Transactions on Parallel and Distributed Systems 36, 12 (2025), 2497–2511. doi:10.1109/TPDS.2025.3568221 [46] Yuanjian Liu, Sheng Di, Kai Zhao, Sian Jin, Cheng Wang, Kyle Chard, Dingwen Tao, Ian Foster, and Franck Cappello. 2022. Optimizing Error-Bounded Lossy Compression for Scientific Data With Diverse Constraints. IEEE Transactions on Parallel and Distributed Systems 33, 12 (2022), 4440–4457. doi:10.1109/TPDS.2022.3194695 [47] Jean loup Gailly and Mark Adler. 2026. zlib Compression Library. https://zlib.net/. Accessed: 2026-08-01. [48] Bing Lu, Yida Li, Junqi Wang, Huizhang Luo, and Kenli Li. 2023. Zfp-x: Efficient embedded coding for accelerating lossy floating point compression. In 2023 IEEE International Parallel and Distributed Processing Symposium (IPDPS). IEEE, 1041–1050. https://doi.org/10.1109/ IPDPS54959.2023.00107 [49] Bing Lu, Zedong Liu, Hairui Zhao, Dejun Luo, Wenjing Huang, Yida Gu, Jinyang Liu, Guangming Tan, and Dingwen Tao. 2026. PRISM: An Efficient GPU-Based Lossy Compression Framework for Progressive Data Retrieval with Multi-Level Interpolation. In Proceedings of the 31st ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming. 164–176. https://doi.org/10.1145/3774934.3786438 [50] Tao Lu, Yu Zhong, Zibin Sun, Xiang Chen, You Zhou, Fei Wu, Ying Yang, Yunxin Huang, and Yafei Yang. 2023. ADT-FSE: A New Encoder for SZ. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (Denver, CO, USA) (SC ’23). Association for Computing Machinery, New York, NY, USA, Article 45, 13 pages. doi:10.1145/3581784.3607044 [51] Zarija Lukić, Casey W Stark, Peter Nugent, Martin White, Avery A Meiksin, and Ann Almgren. 2015. The Lyman 𝛼 forest in optically thin hydrodynamical simulations. Monthly Notices of the Royal Astronomical Society 446, 4 (2015), 3697–3724. https://doi.org/10.1093/ mnras/stu2377 [52] National Energy Research Scientific Computing Center. 2023. WRF Benchmark on NERSC Systems: CONUS 2.5-km. https://docs.nersc. gov/applications/wrf/wrf_benchmark/. Accessed: 2026-08-01. [53] Russ Rew and Glenn Davis. 1990. NetCDF: an interface for scientific data access. IEEE computer graphics and applications 10, 4 (1990), 76–82. [54] Russell K. Rew, Ethan Hartnett, and J. Caron. 2006. NetCDF-4: Software Implementing an Enhanced Data Model for the Geosciences. In 22nd International Conference on Interactive Information Processing Systems for Meteorology, Oceanography, and Hydrology, Vol. 6. 395. [55] RIKEN Center for Computational Science. 2020. About Fugaku. https://www.r-ccs.riken.jp/en/fugaku/about/. RIKEN Center for Computational Science website, accessed July 11, 2026. [56] Einar Saukas. 2021. ZX0: Data Compressor for 8-bit Computers and Low-end Platforms. Computer software and format specification. https://github.com/einar-saukas/ZX0 GitHub repository, BSD 3-Clause License; accessed July 22, 2026. 13

on gpus. In Proceedings of the 32nd International Symposium on HighPerformance Parallel and Distributed Computing. 129–142. [68] Kai Zhao, Sheng Di, Maxim Dmitriev, Thierry-Laurent D. Tonellot, Zizhong Chen, and Franck Cappello. 2021. Optimizing Error-Bounded Lossy Compression for Scientific Data by Dynamic Spline Interpolation. In 2021 IEEE 37th International Conference on Data Engineering (ICDE). 1643–1654. doi:10.1109/ICDE51399.2021.00145 [69] Kai Zhao, Sheng Di, Xin Lian, Sihuan Li, Dingwen Tao, Julie Bessac, Zizhong Chen, and Franck Cappello. 2020. SDRBench: Scientific data reduction benchmark for lossy compressors. In IEEE International Conference on Big Data. 2716–2724. https://doi.org/10.1109/BigData50022. 2020.9378449 [70] Changfeng Zou, Bigyan Ghimire, and Jon C. Calhoun. 2025. SZ3_SIMD: Accelerating Error-Bounded Lossy Compression with Architecture Independent SIMD. In 2025 IEEE International Conference on Big Data (BigData). 4845–4854. doi:10.1109/BigData66926.2025.11401598

[57] Jean Sexton, Zarija Lukic, Ann Almgren, Chris Daley, Brian Friesen, Andrew Myers, and Weiqun Zhang. 2021. Nyx: A massively parallel amr code for computational cosmology. The Journal of Open Source Software 6, 63 (2021), 3068. https://doi.org/10.21105/joss.03068 [58] William C. Skamarock, Joseph B. Klemp, Jimy Dudhia, David O. Gill, Zhiquan Liu, Judith Berner, Wei Wang, Jordan G. Powers, Michael G. Duda, Dale M. Barker, and Xiang-Yu Huang. 2019. A Description of the Advanced Research WRF Version 4. Technical Report NCAR/TN556+STR. National Center for Atmospheric Research. 145 pages. doi:10. 5065/1dfh-6p97 [59] Chuanhao Sun, Yu Zhang, Bill Tao, Deepak Vasisht, and Mahesh Marina. 2025. DeepSpace: Super Resolution Powered Efficient and Reliable Satellite Image Data Acquistion. In Proceedings of the ACM SIGCOMM 2025 Conference (São Francisco Convent, Coimbra, Portugal) (SIGCOMM ’25). Association for Computing Machinery, New York, NY, USA, 311–328. doi:10.1145/3718958.3750523 [60] Dingwen Tao, Sheng Di, Zizhong Chen, and Franck Cappello. 2017. Significantly improving lossy compression for scientific data sets based on multidimensional prediction and error-controlled quantization. In 2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS). IEEE, 1129–1139. https://doi.org/10.1109/IPDPS.2017.115 [61] Dingwen Tao, Sheng Di, Hanqi Guo, Zizhong Chen, and Franck Cappello. 2019. Z-checker: A framework for assessing lossy compression of scientific data. The International Journal of High Performance Computing Applications 33, 2 (2019), 285–303. doi:10.1177/1094342017737147 [62] Jiannan Tian, Sheng Di, Xiaodong Yu, Cody Rivera, Kai Zhao, Sian Jin, Yunhe Feng, Xin Liang, Dingwen Tao, and Franck Cappello. 2021. Optimizing Error-Bounded Lossy Compression for Scientific Data on GPUs. In 2021 IEEE International Conference on Cluster Computing (CLUSTER). 283–293. doi:10.1109/Cluster48925.2021.00047 [63] Jiannan Tian, Sheng Di, Kai Zhao, Cody Rivera, Megan Hickman Fulp, Robert Underwood, Sian Jin, Xin Liang, Jon Calhoun, Dingwen Tao, and Franck Cappello. 2020. cuSZ: An Efficient GPU-Based Error-Bounded Lossy Compression Framework for Scientific Data. In Proceedings of the ACM International Conference on Parallel Architectures and Compilation Techniques (Virtual Event, GA, USA) (PACT ’20). Association for Computing Machinery, New York, NY, USA, 3–15. doi:10.1145/3410463.3414624 [64] TOP500. 2026. LineShine Debuts at No. 1 as the TOP500 Enters a New Global Exascale Era. https://top500.org/news/lineshine-debuts-no-1top500-enters-new-global-exascale-era/. Accessed: 2026-07-11. [65] Shixun Wu, Jinwen Pan, Jinyang Liu, Jiannan Tian, Ziwei Qiu, Jiajun Huang, Kai Zhao, Xin Liang, Sheng Di, Zizhong Chen, and Franck Cappello. 2025. Boosting Scientific Error-Bounded Lossy Compression through Optimized Synergistic Lossy-Lossless Orchestration. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC ’25). Association for Computing Machinery, New York, NY, USA, 2024–2037. doi:10.1145/3712285.3759798 [66] Kai Xu, Maoxue Yu, Yuhu Chen, Jie Gao, Shuang Wang, Jiaying Song, Xiaohui Duan, Junlin Wei, Jiangfeng Yu, Hailong Liu, Jinrong Jiang, Yi Zhang, Pengfei Lin, Tianyi Wang, Pengfei Wang, Weipeng Zheng, Jingwei Xie, Jiakang Zhang, Zilu Liu, Xiaoyu Jin, Jilin Wei, Qixin Chang, Qingxia Lin, Yanzhi Zhou, Weiguo Liu, Wei Xue, Yiwen Li, Haohuan Fu, Yue Yu, Xuebin Chi, and Lixin Wu. 2025. Kilometer-Scale AI-Powered and Performance-Portable Earth System Model (AP3ESM) to Achieve Year-Scale Simulation Speed on Heterogeneous Supercomputers. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC ’25). Association for Computing Machinery, New York, NY, USA, 111–124. doi:10.1145/3712285.3771788 [67] Boyuan Zhang, Jiannan Tian, Sheng Di, Xiaodong Yu, Yunhe Feng, Xin Liang, Dingwen Tao, and Franck Cappello. 2023. Fz-gpu: A fast and high-ratio lossy compressor for scientific computing applications 14

Related documents

• Effectiveness and predictability of in-network storage cache for Scientific Workflows#1038674
• Green BOA: Determining the environmental break-even point for ML-based data compression#658917
• Green BOA: Determining the environmental break-even point for ML-based data compression#656781
• Green BOA: Determining the environmental break-even point for ML-based data compression#471251
• BlockMGARD: Accelerating Adaptive Scientific Data Reduction with Region-of-Interest Error Control on GPUs#819668