Green BOA: Determining the environmental break-even point for ML-based data compression

arXiv:2608.19994v1 [cs.LG] 20 Aug 2026

Green BOA: Determining the environmental break-even point for ML-based data compression Caterina Doglioni

Akshat Gupta

Thomas Elliott

Hanzila Hussain

caterina.doglioni@manchester.ac.uk University of Manchester Manchester, UK

akshat.gupta4@postgrad.manchester.ac.uk University of Manchester Manchester, UK

thomas.elliott@manchester.ac.uk University of Manchester Manchester, UK

hanzilahussain234@gmail.com University of Manchester Manchester, UK

Sanjiban Sengupta

Zhengkai Sun

sanjiban.sengupta@cern.ch CERN/University of Manchester Manchester, UK

zhengkai.sun@student.manchester.ac.uk University of Manchester Manchester, UK

Abstract We summarise the outcome of two summer internship projects based at the University of Manchester, focused on the break-even point in terms of environmental sustainability for ML-based data compression algorithms. Using the example of a ML-based lossless compression algorithm, we compare estimates for the carbonequivalent of the infrastructure needed for ML training and inference with the carbon-equivalent savings from reduced disk storage requirements, and discuss their break-even point. Context. Big Data experiments, such as those at the Large Hadron Collider [2] and in other astroparticle physics experiments (e.g. [13]), will be recording several Exabytes of data. This has a significant cost in terms of both budget [1] and environmental resources for data storage (see e.g. [11, 15]). R&D on data compression is ongoing, and includes machine learning (ML-) based data compression. Since this kind of compression techniques are generally more computationally intensive than standard compression algorithms such as ZSTD [4],and LZMA [12], we investigate the break-even point between the 𝐶𝑂 2 -equivalent cost of training and executing the ML-based compression algorithm, and the embodied pllus operational 𝐶𝑂 2 -equivalent of disk storage that would be displaced by storing data in its compressed form. Methods. We consider the energy use of a ML-based data compression algorithms developed at the University of Manchester [8], and estimate its energy usage for ML training and inference (compression/decompression round trip) on an Nvidia T4 GPU. We consider several country-specific scenarios for where these steps will be executed, to convert energy consumption into a 𝐶𝑂 2 -equivalent based on average energy mix. We then consider the carbon-equivalent cost of manufacturing and operating disk storage devices, in the shape of Hard Disk Drives (HDD) and tapes, for a 5 years lifetime. As examples, we consider Seagate Exos X18 HDDs [14] and tape cartridges [10] (following the methodologies in [11, 16]), hypothetically located in the UK. Inspired by [3], we define the break-even point for this project as the size of the uncompressed dataset at which the carbon cost of training and running the compression algorithm equals the embodied carbon of the extra disk storage needed for the uncompressed data. Further information on the methods can be found in the Appendix. Rather than a full analysis, this is a proof-of-principle calculation with several assumptions using a limited amount of data. Only operational costs from CPU, GPU and RAM are considered for the ML-based data compression, as we assume that the GPU will already be available and that its use for ML-based data compression

will be negligible compared to other uses within a LHC experiment. The scenario considered for the ML-based data compression comprises training on the full dataset, followed by one compression and one decompression round (on an existing temporary disk). A broader range of scenarios will be explored in future work. Since training dominates the energy consumption, this can be considered an upper bound in terms of energy consumption, as in practice BOA’s generalisation capability outlined in [8] allows for training only on a small dataset and then using this trained model for a much larger dataset. Results and conclusions. As it can be seen in Figure 1, the location of the break-even point is very sensitive to the carbon intensity of the country where the ML-based data compression is performed. As shown in previous literature, tape storage has a lower 𝐶𝑂 2 -equivalent footprint than HDDs, but at the cost of slower data access.

Figure 1: Results of the break-even analysis in terms of 𝐶𝑂 2 equivalent for ML-based data compression and subsequent displaced HDD/tape storage. We also compared ML-based data compression to standard algorithms, taking performance (compression ratio) into account: since BOA generally achieves a better compression ratio but at lower throughput with respect to standard algorithms [8], it will be more 𝐶𝑂 2 -intensive per unit of data processed than non-ML algorithms. While the team is working on improving the throughput, it will still be worth examining whether the compression gains of ML-based approaches justify the additional environmental cost in deployment at scale.

Appendix: technical configuration

where𝑒 HDD,non-use is the capacity-allocated displaced carbon in 𝑔𝐶𝑂 2𝑒/𝑘𝑊 ℎ. The operational carbon was calculated using the integer number of powered drives 𝑃 HDD 𝐶 HDD,operational (𝐷,𝑇 ) = 𝑁 HDD (𝐷) (24)(365.25)𝑇 𝐼 HDD PUE, 1000 (4) we neglect the Power Usage Effectiveness (PUE) throughout (and set it equal to unity). So the total HDD carbon cost is the sum of these two quantities. The HDD scenario used the Seagate Exos X18 18 TB drive. Its nominal capacity and average idle power were taken as 18 TB and 5.3 W, respectively [14]. The displaced 𝑘𝑔𝐶𝑂 2𝑒 value was taken as 27 𝑘𝑔𝐶𝑂 2𝑒 per drive, corresponding to a proportionality factor of 1.50 𝑔𝐶𝑂 2𝑒/𝐺𝐵. For Tape storage, cartridge-related non-use carbon was calculated using an integer number of LTO-8 cartridges, while operational carbon was calculated as proportional to the complete RAL Tape-service factor in [11]. While inconsistent with the choice made for HDDs, this choice reflected existing data availability. Similarly to HDD, the number of cartridges is    𝐷 𝑁 Tape (𝐷) = max 1, . (5) 𝐾Tape

In the following, we detail the computing hardware and software configuration used to obtain these results, as well as the setup for the estimate of HDD and tape 𝐶𝑂 2 -equivalent.

ML data compression The BOA ML compression algorithm was trained and executed on a Google Colab notebook with a runtime Nvidia T4 GPU. The model chosen was a two-layer Mamba-v1 backbone with a hidden dimension of 64, a byte vocabulary of 256 symbols, a sequence length of 10,000 bytes and a batch size of 5. Training was performed for eight epochs in FP32 precision, using a learning rate of 5 × 10−4 and a random seed of 42. The training step included the eight training epochs, validation, final test evaluation and checkpoint writing. The compression step included the execution of one compression and decompression round-trip. The file that was compressed was the 49.92 MB "bundled CMS file" (CMS_DATA_float32.bin) from [8] that can be found within the BOA GitHub repository [7]; each data record contains 24 float32 features. The results from this file were scaled to the maximum dataset size shown in Figure 1 after checking linearity of training and inference carbon costs. Carbon tracking was performed using CodeCarbon [6], following a tutorial in [9]. CodeCarbon was used in one-second powersampling intervals using machine mode to measure the energy consumption of CPU, GPU and RAM used in the ML-based compression training and inference. The same setup was used for the compression-decompression round trip using ZSTD and LZMA. We are aware that machine mode measures the energy consumption of the hardware stack of GPU, CPU and RAM (which may as well be shared when using a Google Colab notebook), but it is the only way to obtain a GPU energy estimate using CodeCarbon. Thermal Design Power (TDP) scaling for an Intel(R) Xeon(R) CPU @ 2.00GHz was used for the CPU power consumption corresponding to 8W, while the GPU power consumption used the nvidia-ml-py package. The conversion between energy use and carbon equivalent was performed using CodeCarbon’s national energy mix averaged scenarios [5].

The total carbon cost is calculated as cartridge

𝐶 Tape (𝐷,𝑇 ) = 𝑁 Tape (𝐷)𝐸 Tape,non-use +

𝐷 𝑇 𝑒 Tape,RAL, 1000

(6)

cartridge

where 𝐸 Tape,non-use is the non-use lifecycle proxy for one cartridge in 𝑔𝐶𝑂 2𝑒/𝑐𝑎𝑟𝑡𝑟𝑖𝑑𝑔𝑒, and 𝑒 Tape,RAL is the complete RAL Tape-service operational factor in 𝑔𝐶𝑂 2𝑒/𝐺𝐵𝑦𝑒𝑎𝑟 . The first term is cartridgerelated displaced (non-use) carbon, while the second represents operational emissions allocated to the stored data volume. The model used an LTO-8 cartridge with a native capacity of 12 TB. The corresponding lifecycle data report a total value of 13.72𝑘𝑔𝐶𝑂 2𝑒 per cartridge, of which 5.70 𝑘𝑔𝐶𝑂 2𝑒 is assigned to the operational phase [10, 11]. cartridge

cartridge

cartridge

𝐸 Tape,non-use = 𝐸 Tape,total − 𝐸 Tape,use .

(7)

cartridge

Here, 𝐸 Tape,total is the reported total lifecycle carbon emission of cartridge

one tape cartridge, and 𝐸 Tape,use is the corresponding use-phase contribution. Using the literature values of 13.72 kgCO2 e/cartridge and 5.70 kgCO2 e/cartridge, respectively [10, 11], gives

Storage servers The carbon cost included both embodied and operational carbon [11, 16]. For the storage of D GB over T years on a medium m, the total storage carbon cost is 𝐶𝑚 (𝐷,𝑇 ) = 𝐶𝑚,embodied (𝐷) + 𝐶𝑚,operational (𝐷,𝑇 ).

cartridge

𝐸 Tape,non-use = 13.72 − 5.70 = 8.02 kgCO2 e/cartridge.

Acknowledgments (1) This work was funded by EPSRC Studentship EP/W524347/1 (Project 2932638) via MADSIM CDT, the University of Manchester Dame Kathleen Ollerenshaw Fellowship, and is part of a project that has received funding from the European Research Council under the European Union’s Horizon 2020 research and innovation program (grant agreement 101002463). Summer student funding was also provided by the N8CIR summer internship scheme.

For HDD storage, a drive with usable capacity 𝐾𝐻 𝐷𝐷 is the basic hardware unit. The number of powered drives required to store D GB is    𝐷 𝑁 HDD (𝐷) = max 𝑁 HDD,min, , (2) 𝐾HDD where we choose 𝑁 HDD,min = 1, meaning at least one drive is assumed to remain installed and operational. Given the limited scope and total data volume of this study, the displaced carbon (non-use) for the HDD is allocated proportionally to the storage capacity 𝐶 HDD,non-use (𝐷) = 𝐷 𝑒 HDD,non-use,

(8)

References [1] 2022. ATLAS Software and Computing HL-LHC Roadmap. Technical Report. CERN, Geneva. https://cds.cern.ch/record/2802918 [2] CERN. 2017. Large Hadron Collider FAQ. Technical Report. CERN, Geneva. https://cds.cern.ch/record/2255762

(3) 2

[3] Tristan Coignion, Clément Quinton, and Romain Rouvoy. 2025. When Faster Isn’t Greener: The Hidden Costs of LLM-Based Code Optimization. In 2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE) (Seoul, Korea, Republic of). IEEE Press, 1655–1666. doi:10.1109/ASE63991.2025. 00139 [4] Yann Collet and Murray Kucherawy. 2021. Zstandard Compression and the ‘application/zstd’ Media Type. RFC 8878. IETF. doi:10.17487/RFC8878 [5] Benoît Courty et al. 2023. Global electricity carbon-intensity data for private infrastructure. Data file in the CodeCarbon source repository. https://github.c om/mlco2/codecarbon/blob/master/codecarbon/data/private_infra/global_ene rgy_mix.json Country-average factors used for regional scenarios; accessed 5 August 2026. [6] Benoît Courty et al. 2026. mlco2/codecarbon: v3.3.0. doi:10.5281/zenodo.21791294 [7] Akshat Gupta, Caterina Doglioni, and Thomas Elliott. 2026. Boa Constrictor: A Mamba-based Lossless Compressor for Scientific Data. Technical Report. Zenodo. doi:10.5281/zenodo.18481928 [8] Akshat Gupta, Caterina Doglioni, and Thomas J Elliott. 2026. BOA constrictor: a Mamba-based lossless compressor for scientific data. Machine Learning: Science and Technology 7, 3 (may 2026), 035014. doi:10.1088/2632-2153/ae64a9 [9] Melanie Hanna and Andrés Felipe Perez Murcia. 2026. Reducing your Climate Impact when Training ML Models. https://github.com/climatechange- aitutorials/tracking-ml-emissions. doi:10.5281/zenodo.21828628 [10] Brad Johns. 2021. Improving Information Technology Sustainability with Modern Tape Storage. Technical Report. Brad Johns Consulting and Fujifilm. https:

//asset.fujifilm.com/www/de/files/2023-10/97ddc3473883421cef1fb820d236dfa 2/Improving_IT_Sustainability_with_Tape_BJC_0.pdf Lifecycle assessment used for the LTO-8 cartridge proxy; accessed 5 August 2026. [11] Alison Packer and Samuel Cadellin Skipsey. 2025. Carbon costs of storage: A U.K. perspective. EPJ Web of Conferences 337 (2025), 01157. doi:10.1051/epjconf/ 202533701157 [12] Igor Pavlov. 2007. LZMA SDK (Software Development Kit). https://7-zip.org/sd k.html [13] Anna M. M. Scaife. 2020. Big telescope, big data: towards exascale with the Square Kilometre Array. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 378, 2166 (2020), 20190060. doi:10.1098/rsta.201 9.0060 [14] Seagate Technology. 2023. Exos X18 18TB Product Sustainability Report. Technical Report. Seagate Technology. https://www.seagate.com/content/dam/seagat e/assets/esg/planet/product-sustainability/images/exos-x18-sustainabilityreport/files/Exos-X18-18TB-Sustainability-Report-2023.pdf Accessed 5 August 2026. [15] Swamit Tannu and Prashant J. Nair. 2023. The Dirty Secret of SSDs: Embodied Carbon. ACM SIGEnergy Energy Informatics Review 3, 3 (Oct. 2023), 4–9. doi:10.1 145/3630614.3630616 [16] Mattias Wadenstein and Wim Vanderbauwhede. 2025. Life cycle analysis for emissions of scientific computing centres. Eur. Phys. J. C 85, 8 (2025), 913. arXiv:2506.14365 [hep-ex] doi:10.1140/epjc/s10052-025-14650-8

3

Related documents

• Green BOA: Determining the environmental break-even point for ML-based data compression#658917
• Green BOA: Determining the environmental break-even point for ML-based data compression#656781
• Rate-Adaptive One-Step Diffusion Compression for AIGC Images#1115751
• Helping Climate Agencies Share Data Securely Using Privacy-Preserving AI#990599
• Demystifying Machine Learning: Applications in African Environmental Science and Engineering#684900