A Private IPFS Data Sanctuary for Verifiable Digital Collection Objects

GLAM SCIENTIFIC PAPER

SCIENTIFIC PREPRINT v0.10 September 2026

DIGITAL COLLECTIONS / DATA SYSTEMS

A Private IPFS Data Sanctuary for Verifiable Digital Collection Objects Recovery, content-derived identity, redundancy, and federated GLAM storage in a seven-node museum prototype Joachim Snellings, Jasper Funk-Smit, Margot Belot, Eran Wolff, Bernhard Schurian, Sebastian Baldauf, Lisa Pfeiffer, Frederik Berger ELIO – Experimental Laboratory for Innovation in Object Digitalization, Museum für Naturkunde Berlin (MfN)

arXiv:2610.07970v1 [cs.DC] 6 Oct 2026

Correspondence: joachim.snellings@mfn.berlin

Background. Galleries, libraries, archives and museums (GLAM) increasingly produce scientific collection data whose digital state must

remain identifiable, recoverable, and verifiable after acquisition. A staging architecture should provide redundancy without requiring every measurement to depend on one workstation or storage path. Methods. We evaluated a private seven-node InterPlanetary File System (IPFS) deployment at the Museum für Naturkunde Berlin. Five

Raspberry Pi 5 edge peers and two archive-class peers contributed NVMe-backed repositories across two wired zones linked by a shared wireless path. Two real raw-acquisition folders were recovered by content identifier (CID), a large machine-learning model-weight folder was ingested, and sustained telemetry from two edge and two archive nodes was compared. Results. Raw acquisition A (6.20 GB) materialized in 34.0 s (182 MB/s effective logical rate) and raw acquisition B (5.19 GB) in 14.092 s (368 MB/s). A 156.1 GB model-weight folder was recursively added in approximately 300 s (520 MB/s logical ingest). Four sustained telemetry series comprised 857.3 logged node-hours and 27,616 valid block-stat probes, all reporting successful local block access. Conclusion. The prototype demonstrates a practical institution-local distributed file system (DFS) for content-addressed acquisition staging.

Museum-specific object identifiers can remain stable at the collection-object level while each imported measurement automatically receives a content-derived state identity. Replication intentionally distributes identical blocks across failure domains, while content addressing enables reuse of unchanged blocks within repositories. This combination supports verifiable digital collection objects and a federated storage model for collaborating GLAM institutions.

Keywords: GLAM; IPFS; museum digitization; content addressing; distributed file system; raw acquisition; data resilience; deduplication; provenance

Private IPFS data sanctuary for digital collection objects

v0.10

Table 1: Seven-node hardware and storage configuration.

1 Introduction Natural-history digitization produces raw image sets, computed tomography volumes, label images, meshes, machine-learning artefacts, and other scientific derivatives that must remain connected to the curated physical object from which they originate [1, 5]. Before preservation or publication, these data often pass through acquisition, quality control, reconstruction, and temporary staging. During that interval, dependence on one workstation or storage path creates a recoverability problem even when the physical specimen remains safely curated. We investigate a private InterPlanetary File System (IPFS) cluster as a local data sanctuary: an institution-controlled staging layer that combines content-derived identity, replicated availability, and restricted peer participation. IPFS addresses imported blocks through Content Identifiers (CIDs) rather than only through mutable server paths [2, 3]. Public IPFS studies show that retrieval behavior depends on provider distribution, routing, and workload [9, 12]; the present work therefore evaluates the museum prototype in its own topology instead of claiming a universal IPFS transfer rate. Museum collection workflows already assign a museum-specific unique identifier (UID) to the physical collection object. At MfN, the NURI workflow is one example of object-level identification [13]. The raw measurement folder itself, however, need not receive an additional manually assigned measurement identifier. Content-addressed import automatically gives each imported acquisition a root CID, allowing repeated measurements to remain distinguishable as separate digital states.

Class

𝑛

RAM

NVMe

Primary role

Edge Archive

5 2

16 GB ≥32 GB

2 TB 2 TB

acquisition / staging archive / recovery

sensus layer. Routing used a Distributed Hash Table (DHT); multicast Domain Name System (mDNS) discovery was enabled locally, public bootstrap peers were removed, and a private swarm.key restricted IPFS participation. An IPFS Cluster secret protected Cluster peer communication. These controls restrict membership and communication but do not provide file-level encryption at rest. Each node additionally exposed intentionally published content through a read-only HTTP gateway behind an Nginx reverse proxy, allowing browser-based retrieval without granting IPFS or Cluster administrative access. Public IPFS deployments additionally expose routing and availability considerations at the DHT layer. Sridhar et al. demonstrated that manipulation of Kademlia-based provider discovery can interfere with content retrieval and proposed corresponding detection and mitigation mechanisms [11]. The present deployment operates under a different threat model because peer participation is restricted to a private swarm. For compact comparative reporting, sustained stage/probe telemetry from two edge peers and both archive peers was retained. Edge-01 represents the favorable-WLAN edge case; Edge-04 provides a second long-duration edge series. Node-state logs report observed swarm connections and repository state. A block-stat probe queries a known block in the local IPFS blockstore and therefore measures local content access rather than peer-to-peer network latency.

Research questions

RQ1. Can a heterogeneous private DFS provide a practical staging and recovery layer for real museum acquisition data? RQ2. Can content-derived identity make failed, repeated, accepted, and derived measurement states verifiable without replacing the museum’s established object identifier? RQ3. Can the same content identity support deliberate redundancy across independent storage zones while also enabling block reuse when content is unchanged?

2.3 Scientific workloads and evidence boundary The experiment contained three application-level content operations. Recovery A and Recovery B were real raw acquisition folders from museum digitization workflows, rather than synthetic payloads. The third workload was a substantially larger folder containing machine-learning model weights. For a logical object size 𝑆 materialized in wall time 𝑡, we report

2 Methods 2.1 Seven-node prototype The prototype contained seven physical nodes. Five Raspberry Pi 5 edge systems each provided 16 GB random-access memory (RAM), 2 TB Non-Volatile Memory Express (NVMe) storage, and Gigabit Ethernet; Raspberry Pi 5 also provides dual-band Wi-Fi 5 [7]. Two archive-class systems provided at least eight central processing unit (CPU) cores, at least 32 GB RAM, and 2 TB NVMe storage. Approximately 1.0–1.5 TB per peer was configured for the IPFS repository. The nominal configured raw DFS capacity was therefore approximately 7–10.5 TB before replica overhead and operational reserve. Table 1 summarizes the configuration.

𝑅logical =

𝑆 . 𝑡

(1)

Target blockstore occupancy, provider count, and synchronized interface byte counters were not retained for the historical retrievals. Equation 1 is therefore an effective logical/materialization rate, not physical Ethernet or WLAN throughput. Publication derivatives replace operational hostnames, addresses, PeerIDs, and raw CIDs with generic labels. Private swarm/Cluster secrets and credentials are excluded.

3 Results 3.1 Recovery and ingestion Raw acquisition recovery A. The first recovery workload was a 6.20 GB raw acquisition folder. Materialization on another participating peer completed in 34.0 s, corresponding to approximately 182 MB/s effective logical rate. The operational significance is that the acquisition remained addressable through its CID without depending on the original staging path. Raw acquisition recovery B. The second raw acquisition folder contained 5.19 GB and materialized in 14.092 s, corresponding to approximately 368 MB/s effective logical rate. Together, the

2.2 Topology, private swarm, and telemetry The five edge peers shared one wired local area network (LAN) segment through eth0; the two archive peers occupied a second wired segment. Cross-zone communication used a shared wireless local area network (WLAN) path. The measured arrangement and its institutional generalization are shown in Figure 1. The deployment record specifies Kubo 0.36.0 and IPFS Cluster 1.1.4 with a Conflict-free Replicated Data Type (CRDT) con-

2

Private IPFS data sanctuary for digital collection objects

v0.10

Institutional zone A 5 edge peers acquisition / staging wired LAN

measured shared WLAN inter-zone path

Institutional zone B 2 archive peers recovery / persistence wired LAN

Collaborating GLAM A local replicas independent failure domain

campus / WAN / inter-institutional link same CID-addressed objects

Collaborating GLAM B local replicas independent failure domain

Figure 1: Measured two-zone topology and conceptual federation across independent GLAM failure domains. The lower layer is an architectural generalization, not a WAN performance measurement. Content identity remains independent of which participating zone currently serves the object.

Table 2: Historical content operations. Values are single wall-clock observations; rates are logical/materialization rates, not wire throughput. Workload Raw acquisition A Raw acquisition B Model weights

Size

Observation

6.20 GB 5.19 GB 156.1 GB

34.0 s; 182 MB/s 14.092 s; 368 MB/s ∼300 s; ∼520 MB/s

4 Discussion 4.1 From museum object to verifiable digital measurement state The physical collection object continues to carry its museumspecific UID. Content addressing adds a second, automatically generated identity at the measurement level: the imported acquisition folder receives a root CID representing that exact imported state. Figure 2 illustrates the consequence for repeat acquisition. If the first measurement is incomplete or scientifically unsuitable, it can remain retained as acquisition A. A repeat becomes acquisition B with a distinct CID, and downstream reconstruction can explicitly reference the accepted acquisition rather than a mutable folder name such as “final” or “scan2”.

two recovery observations show that a repeated or corrected acquisition can coexist with the previous measurement as a separately identifiable state. Model-weight ingestion. A 156.1 GB folder containing machine-learning model weights was recursively added in approximately 300 s, corresponding to roughly 520 MB/s logical ingest. Table 2 summarizes the three observations.

Collection object museum UID

3.2 Sustained four-node telemetry The four comparative telemetry series provide 857.3 combined logged node-hours, 10,248 stage samples, and 27,616 valid block-stat probes. All retained valid block-stat probes reported successful local block access. Because these windows were not synchronized, the sum of node-hours is descriptive coverage rather than cluster uptime. Table 3 shows the compact comparison. Archive-node block-stat latencies were lower and tighter in these logs than the two edgenode series, while Edge-01 showed a broader P90 tail despite a similar median to Edge-04.

Acquisition A raw folder / CID A failed or unsuitable retained

Reconstruction derived object references CID B

Figure 2: Repeat-acquisition lineage. The museum UID identifies the physical collection object; content addressing gives each imported measurement folder its own digital state identity. A failed acquisition can remain preserved without becoming the source for reconstruction.

Table 3: Sustained node-level telemetry used for compact comparative reporting. P50/P90 are block-stat latencies, not network ping latency. Node Edge-01 Edge-04 Archive-01 Archive-02

Window h

Stage 𝑛

Probes

P50/P90 ms

312.19 313.50 67.10 164.49

3,736 3,739 805 1,968

9,084 10,965 1,881 5,686

28.2 / 87.8 30.0 / 35.6 15.5 / 18.5 14.3 / 14.9

Acquisition B raw folder / CID B accepted

A folder CID is not merely a conventional file hash: the UnixFS root also reflects the directory representation and can depend on import/chunking parameters. For preservation, a compact manifest should therefore retain the root CID, relevant import parameters, and an independent fixity digest such as SHA-256. 4.2 Redundancy and deduplication serve different purposes Replication and deduplication are complementary rather than contradictory. Replication deliberately stores identical content in independent failure domains so that the loss of one node or zone does not remove the only available copy. Content-addressed block reuse avoids storing an additional unique block when the same block already exists in a repository. IPFS therefore provides block-level deduplication for identical content, while Cluster replication intentionally creates additional physical copies where resilience requires them [2, 10]. For related versions of an acquisition or derivative, unchanged blocks can be reused when their block identities remain un-

Evidence

RQ1 is supported at the application layer by two rawacquisition materialization observations, one large modelweight ingest, and sustained blockstore telemetry on both edge and archive hardware. The experiment demonstrates practical operation of the private DFS but does not establish physical WLAN saturation or a transfer-rate distribution.

3

Private IPFS data sanctuary for digital collection objects

v0.10

changed. The amount of reuse is workload- and chunking- data placement and erasure-coding strategies for heterogeneous dependent; this experiment did not measure a deduplication ratio. distributed storage resources [4]. Highly compressed or substantially rewritten files may share With approximately 1.0–1.5 TB configured per peer, the sevenrelatively few blocks. node prototype provides roughly 7–10.5 TB nominal raw capacity. Under balanced replication factor 𝑟, unique replicated content is 4.3 Federated GLAM storage approximately bounded by RQ3 extends the measured two-zone topology into an institutional collaboration model. Because a CID identifies content rather 𝐶raw 𝐶unique ≲ , than a particular server path, collaborating GLAM institutions 𝑟 can retain replicas of agreed digital collection objects while referring to the same content identity. The resulting shared before reserve and any deduplication benefit. content-addressed collection-data layer can be geographically or Network bandwidth scales differently. Several peers can contribute concurrent flows over a switched LAN until a shared administratively zone-independent. This does not require every institution to expose its complete network interface controller (NIC), switch fabric, uplink, storage internal repository. Each participant can retain local acquisi- device, or CPU becomes the bottleneck. A 10 GbE switch fabric tion replicas and exchange only agreed content, manifests, or or stacked 10 GbE uplink can raise aggregate LAN capacity, derivatives. Storage responsibility can therefore be distributed while an individual 1 GbE endpoint remains limited by its own incrementally across participating institutions and commodity interface. WLAN is less additive because peers share airtime hardware instead of requiring one central staging appliance. More and access-point capacity. The historical experiment did not broadly, distributed edge–cloud architectures require explicit de- independently measure the WLAN ceiling. cisions about which processing and data-management operations At larger peer populations, data availability additionally depends remain local and which are delegated across the infrastructure. on network conditions, peer responsiveness, and disseminaContinuumConductor formalizes such decentralization decisions tion strategy. PANDAS demonstrates a peer-to-peer availability for heterogeneous edge–cloud environments according to re- protocol designed to tolerate message loss, node failures, and source, responsiveness, and privacy considerations [8]. Although unresponsive participants while scaling to large peer populations developed for process mining rather than scientific collection [6]. Although its application domain differs from persistent storage, this systems perspective is relevant to future federated scientific storage, these systems-level constraints are relevant to GLAM infrastructures. The prototype motivates this poten- future multi-institutional federation. tially cost-distributable model, but no total-cost-of-ownership 5 Limitations comparison was performed. Limitations A slower campus or wide area network (WAN) link affects recovery time, not the identity of the data being recovered. If one The two raw-acquisition recovery values and the model-weight institutional zone loses its local copy, another zone retaining the ingest value are single historical observations. Target blockstore required pinned blocks can reseed replacement peers after constate, provider count, exact source/destination zone, and interface byte counters were not synchronized to those runs. The logical rates nectivity is restored. A CID alone does not guarantee persistence; therefore cannot be interpreted as physical network throughput. replica placement, pinning policy, credentials, and independent The four telemetry windows are sustained but asynchronous. Blockbackup remain operational requirements. stat probes measure local blockstore access rather than P2P retrieval. Deduplication is an architectural property discussed from the content-addressed storage model; no block-reuse ratio was measured. Cross-institutional federation and full-zone restoration are design extensions rather than experimentally demonstrated outcomes.

Interpretation

RQ2 is answered at the identity level: content addressing converts each imported measurement into an immutable-byreference state. Changing the represented acquisition produces a different content identity rather than silently changing the referenced state. RQ3 is supported architecturally by location-independent CIDs and deliberate replication, but inter-institutional recovery was not benchmarked in this experiment.

6 Conclusion This work evaluates a private seven-node IPFS data sanctuary for scientific digitization in a GLAM institution. Two real rawacquisition folders were recovered by CID at multi-gigabyte scale, a 156.1 GB model-weight folder was recursively ingested, and sustained telemetry from two edge and two archive nodes documented prolonged operation of the local content-addressed layer. The central contribution is the combination of verifiable digital measurement identity and distributed custody. The museum object retains its established UID, while each imported measurement acquires a separate content-derived CID. Failed, repeated, accepted, and derived states can therefore remain distinguishable without overwriting the scientific record. Replication places selected states across failure domains for resilience, while blocklevel content addressing enables deduplication when identical blocks recur. The same mechanism can support a federated GLAM data sanctuary in which participating institutions contribute storage capacity

A CID alone does not guarantee persistence; replica placement, pinning policy, credentials, and independent backup remain operational requirements. More generally, heterogeneous distributed storage systems must account for differences in capacity, performance, and failure characteristics between participating resources. D-Rex demonstrates a reliability-aware approach in which data placement and erasure-coding parameters are adapted to heterogeneous storage nodes [4]. 4.4 Capacity and network scaling Distributed storage systems must account not only for aggregate capacity but also for heterogeneity in node performance, storage capacity, and failure characteristics. Recent work such as D-Rex explicitly incorporates these differences into reliability-aware

4

Private IPFS data sanctuary for digital collection objects

v0.10

and recovery responsibility without changing the identity of the shared digital collection objects. This makes the storage layer a candidate substrate for affordable, distributed, and verifiable scientific collection data management, while leaving cost and inter-institutional recovery performance for future evaluation.

Sasson, J. Gregory Pauloski, Jesus Carretero, Kyle Chard, and Ian Foster. D-rex: Heterogeneity-aware reliability framework and adaptive algorithms for distributed storage. In Proceedings of the 39th ACM International Conference on Supercomputing, pages 853–867, 2025. doi: 10.1145/3721145.3730412. [5] Brandon P. Hedrick, Jason M. Heberling, Emily K. Meineke, Katelin G. Turner, Christopher J. Grassa, Daniel S. Park, James Kennedy, Julia A. Clarke, Joseph A. Cook, David C. Blackburn, Scott V. Edwards, and Charles C. Davis. Digitization and the future of natural history collections. BioScience, 70(3):243–251, 2020. doi: 10.1093/biosci/biz163. URL https://doi.org/10.1093/ biosci/biz163.

Abbreviations Term

Full form

CID CPU CRDT DFS DHT GLAM IPFS LAN mDNS NIC NURI NVMe P2P RAM UID WAN WLAN

Content Identifier Central Processing Unit Conflict-free Replicated Data Type Distributed File System Distributed Hash Table Galleries, Libraries, Archives and Museums InterPlanetary File System Local Area Network multicast Domain Name System Network Interface Controller MfN URI used in the museum object-identification workflow Non-Volatile Memory Express peer-to-peer random-access memory unique identifier Wide Area Network Wireless Local Area Network

[6] Matthieu Pigaglio, Onur Ascigil, Michał Król, Sergi Rene, Felix Lange, Kaleem Peeroo, Ramin Sadre, Vladimir Stankovic, and Etienne Rivière. Pandas: Peer-to-peer, adaptive networking allowing data availability sampling within ethereum consensus timebounds. In Proceedings of the 26th ACM International Middleware Conference, pages 167–179, 2025. doi: 10.1145/3721462.3770769. [7] Raspberry Pi Ltd. Raspberry pi 5 product specification, 2026. URL https://www.raspberrypi.com/products/ raspberry-pi-5/. Gigabit Ethernet and dual-band 802.11ac Wi-Fi 5 platform specification; accessed 2026-08-29. [8] Hendrik Reiter, Janick Edinger, Martin Kabierski, Agnes Koschmider, Olaf Landsiedel, Arvid Lepsien, Xixi Lu, Andrea Marrella, Estefania Serral, Stefan Schulte, Florian Tschorsch, Matthias Weidlich, and Wilhelm Hasselbring. Continuumconductor: Decentralized process mining on the edge-cloud continuum. arXiv preprint arXiv:2512.07280, 2025. doi: 10.48550/arXiv.2512.07280.

Data, privacy, and reproducibility Research artifact record Data: privacy-safe content-operation and four-node telemetry tables Code: analysis and publication sources in the release package Reproducibility: Kubo 0.36.0; IPFS Cluster 1.1.4; CRDT consensus; claim boundaries and privacy controls documented in the manifest Repository: <public release repository>

[9] Ruizhe Shi, Ruizhi Cheng, Bo Han, Yue Cheng, and Songqing Chen. A closer look into IPFS: Accessibility, content, and performance. Proceedings of the ACM on Measurement and Analysis of Computing Systems, 8(2):1–31, 2024. doi: 10.1145/3656015. URL https://doi.org/10.1145/3656015.

Operational IP addresses, hostnames, raw PeerIDs, private swarm keys, Cluster secrets, credentials, and unreleased CIDs are excluded from the publication package. The institution-internal raw logs remain under local control.

[10] Ruizhe Shi, Ruizhi Cheng, Yuqi Fu, Bo Han, Yue Cheng, and Songqing Chen. Centralization in the decentralized web: Challenges and opportunities in IPFS data management. In Proceedings of the ACM Web Conference 2025, pages 4068–4076. Association for Computing Machinery, 2025. doi: 10.1145/3696410.3714627. URL https://doi.org/10.1145/3696410.3714627.

Acknowledgements We thank colleagues in ELIO, institutional IT, collections, and digitization teams. We gratefully acknowledge European Union funding of the ELIO project and the infrastructure provided by the Museum für Naturkunde Berlin.

[11] Srivatsan Sridhar, Onur Ascigil, Navin V. Keizer, François Genon, Sébastien Pierre, Yiannis Psaras, Etienne Rivière, and Michał Król. [1] Margot Belot, Joël Tuberosa, Leonardo Preuss, Olha Svezhentseva, Content censorship in the interplanetary file system. In Proceedings Magdalena Claessen, Christian Bölling, Franziska Schuster, and of the Network and Distributed System Security Symposium (NDSS), Théo Léger. High-throughput information extraction of printed 2024. doi: 10.14722/ndss.2024.23153. specimen labels from large-scale digitization of entomological collections using a semi-automated pipeline. Methods in Ecology [12] Dennis Trautwein, Aravindh Raman, Gareth Tyson, Ignacio Castro, and Evolution, 17(3):790–804, 2026. doi: 10.1111/2041-210x. Will Scott, Moritz Schubotz, Bela Gipp, and Yiannis Psaras. 70235. URL https://doi.org/10.1111/2041-210x.70235. Design and evaluation of IPFS: A storage layer for the decentralized web. In Proceedings of the ACM SIGCOMM 2022 Conference, [2] Juan Benet. IPFS – content addressed, versioned, P2P file system, pages 739–752. Association for Computing Machinery, 2022. doi: 2014. URL https://arxiv.org/abs/1407.3561. 10.1145/3544216.3544232. URL https://doi.org/10.1145/ 3544216.3544232. [3] Erik Daniel and Florian Tschorsch. IPFS and friends: A qualitative comparison of next generation peer-to-peer data networks. IEEE Communications Surveys & Tutorials, 24(1):31–52, 2022. doi: [13] Justine L. Uebeler, Carla M. A. Rumpf, Valentin de Mazancourt, and Kristina von Rintelen. Workflow protocol for the digitiza10.1109/COMST.2022.3143147. URL https://doi.org/10. tion of ethanol-preserved isopods to accelerate the process of 1109/COMST.2022.3143147. species description, 2022. URL https://doi.org/10.7479/ [4] Maxime Gonthier, Dante D. Sanchez-Gallegos, Haochen Pan, padn-cx88. Bogdan Nicolae, Sicheng Zhou, Hai Duc Nguyen, Valerie Hayot-

References

5

Related documents

• A Private IPFS Data Sanctuary for Verifiable Digital Collection Objects#1217705
• CUSTOS: Toward Forensic-Ready Zero Trust at the Capture-Containment Boundary#820609
• The Collection Survey: Third Phase: Recording and Indexing Collected Data#1018998
• KI Data Repository Bevarandeplan 1.0, 2026–2028 (KI Data Repository Preservation plan 1.0, 2026–2028)#259122
• Latency-Optimal Geo-Distributed Storage over Structured Networks#972168