Using the Open Science Data Federation for data distribution: Big Bear Solar Observatory use case
arXiv:2605.15378v1 [cs.DC] 14 May 2026
SYDNEY MONTIEL, Instituto Politécnico Nacional, México ALEXSANDRA GUADARRAMA, Instituto México de Baja California, México FRANK WÜRTHWEIN, University of California San Diego, USA FABIO ANDRIJAUSKAS∗ , University of California San Diego, USA
Fig. 1. Map featuring the locations of current OSDF architectural components (https://osghtc.org/services/osdf). The growing demand for extensive data processing is now a standard in many scientific fields. Efficiently distributing data to processing sites and enabling seamless sharing has become crucial. The Open Science Data Federation (OSDF) builds on the success of the StashCache project to establish a global data distribution network. By expanding StashCache, OSDF integrates additional data origins and caches, enhancing accessibility and performance (20 origins and 30 caches), new access methods, and monitoring and accounting mechanisms. Additionally, the OSDF has become essential to the US national cyber-infrastructure landscape due to the sharing requirements of recent NSF solicitations. One use case for the OSDF is the data access to the Big Bear Solar Observatory (BBSO). Integrating the BBSO data into the OSDF provided standard and reliable data access. Moreover, the OSDF caches provide local data worldwide. Using the OSDF and the BBSO data, creating a pipeline to apply image processing techniques to all images from BBSO anywhere on the planet was possible. CCS Concepts: • Information systems → Information storage technologies; • Networks → Network performance analysis; Network measurement. Authors’ Contact Information: Sydney Montiel, [email protected], Instituto Politécnico Nacional, Gustavo A. Madero, CDMX, México; Alexsandra Guadarrama, [email protected], Instituto México de Baja California, Tijuana, Baja California, México; Frank Würthwein, [email protected], University of California San Diego, San Diego, USA; Fabio Andrijauskas, [email protected], University of California San Diego, San Diego, USA. PEARC’25, Columbus, Ohio 2025. ACM ISBN 978-x-xxxx-xxxx-x/YYYY/MM https://doi.org/10.1145/nnnnnnn.nnnnnnn , Vol. 1, No. 1, Article . Publication date: May 2025.
2
Andrijauskas et al.
Additional Key Words and Phrases: OSDF, data processing, caches, solar image processing
1
Introduction
Data sharing has become one of the most complex challenges researchers face in a technological age. It must be stored, managed, processed, and analyzed to generate scientific advances. Therefore, current research must satisfy processing speed and memory requirements. Furthermore, such data is shared by geographically distributed users, which increases network traffic due to access paths [4]. Taking this problem as a starting point, the development of the Open Science Data Federation (OSDF) emerged. Based on the Open Science Grid (OSG) project, StashCache, the OSDF is a data access framework that provides infrastructure and tools for accessing data globally, following the notion of "Any Data, Anytime, Anywhere," making data access more efficient for users. It supports large amounts of data from several independent experiments and funded projects by the National Science Foundation (NSF). This paper aims to show an OSDF application that can access data from the Big Bear Solar Observatory (BBSO) and efficiently perform image processing for the user. 2
Open Science Data Federation
OSDF builds upon the OSG project, StashCache [6, 8], creating a robust research environment in collaboration with OSG. They provide users with high computational throughput and efficient data access [7]. The OSDF environment continually enhances its tools and capacity to support growth and demand while providing reliable data delivery to compute sites. Additionally, it is essential to empower staff to identify and resolve potential issues early. To facilitate data access for computational workflows running on distributed computing infrastructure, the OSG leverages the Open Science Data Federation (OSDF) [8]. At the core of this data delivery system are "origins," "caches," and "redirectors," all implemented using the XRootD [2, 5] software framework, which enables low-latency and scalable data access. Figure 1 shows all the OSDF host locations. The Pelican Project created a new layer for the OSDF [1], providing centralized services for registration, cache selection, and monitoring. Origins serve as the backend storage for project data. Within OSDF, an origin is an XRootD/Pelican configuration that provides access to storage through a data transfer node that mounts a project directory. Multiple origins form a hierarchical structure that connects to a redirector, which then communicates with the cache network. Applications typically access OSDF through the nearest cache to their computing site, with proximity determined using GeoIP [8]. A containerized approach on a federated Kubernetes infrastructure is ideal for deploying and managing OSDF software and services, whether at origins or caches. This federated model enables effective monitoring of data access and troubleshooting of issues. 3
BBSO data processing
The Sun generates energy by fusing hydrogen atoms into helium. Various events can occur throughout this process, including solar storms, which are particularly interesting due to their potential impact on telecommunications. These storms can disrupt GPS signals, alter or destroy satellite trajectories, and cause electricity transmission issues. For instance, geomagnetically induced currents from solar storms can overload power grids. However, such catastrophic consequences can be mitigated if solar storms are detected in time, allowing for preventive measures such as shutting down satellites or reducing power flow in transmission lines. Solar storms exhibit distinct characteristics, one of which is the emergence of filaments on the Sun’s surface. These filaments, visible in H𝛼 images, consist of dense plasma held in place by magnetic fields. They appear darker than the surrounding solar surface due to their lower , Vol. 1, No. 1, Article . Publication date: May 2025.
Using the Open Science Data Federation for data distribution: Big Bear Solar Observatory use case
3
temperature. Figure 3 illustrates a solar storm event featuring a filament, where the dark ribbons on the Sun represent these structures [3]. Figure 2 illustrates the flow of solar image data from the Big Bear Solar Observatory through the Open Science Data Federation (OSDF) to users and back. The process begins with the observatory collecting solar images in FITS format. These images are then gathered through a synchronization mechanism and transferred to OSDF, an intermediary data storage and distribution system. Once in OSDF, the data is transmitted via Internet/Internet2 to users, who access and process the images. The image processing stage enhances the solar images, as shown in Figure 2. After processing, the refined data is returned to the origin, completing the cycle. This workflow ensures efficient handling, distribution, and enhancement of solar observational data.
Fig. 2. Data flowchart of the gathered images from the BBSO to perform processing.
An example of a result produced by the diffusion filter is depicted in Figure 2. We can observe that the image is more homogeneous than the original images, but its borders are well-defined. The other steps are threshold calculation, filament extraction, and labeling. The OSDF was used to get all the required images to run this detection.
Fig. 3. Sun image processed by the diffusion filter with a manually detached filament [3].
, Vol. 1, No. 1, Article . Publication date: May 2025.
4
4
Andrijauskas et al.
Conclusion
The Open Science Data Framework is crucial for sharing scientific data across the United States, featuring caches and origins located worldwide. This capability is vital for high-throughput and high-performance computing. With the addition of over 10 new caches and origins, we can deliver more data to a larger number of users. The BBSO use case demonstrates that it is possible to utilize the OSDF to establish an efficient processing pipeline. References [1] 2024. Pelican Platform. https://pelicanplatform.org/. [Accessed 4-Jun-2024]. [2] 2024. XRootD. https://xrootd.slac.stanford.edu [Accessed 20-Apr-2024]. [3] Fabio Andrijauskas and André Leon Sampaio Gradvohl. 2012. Solar filaments detection using parallel programming in hybrid architectures. In Proceedings of the 2012 Workshop on High-Performance Computing for Astronomy Date (Delft, The Netherlands) (Astro-HPC ’12). Association for Computing Machinery, New York, NY, USA, 41–48. doi:10.1145/ 2286976.2286987 [4] Ziyue Deng, Alex Sim, Kesheng Wu, Chin Guok, Damian Hazen, Inder Monga, Fabio Andrijauskas, Frank Würthwein, and Derek Weitzel. 2023. Analyzing Transatlantic Network Traffic over Scientific Data Caches. In Proceedings of the 2023 on Systems and Network Telemetry and Analytics (HPDC ’23). ACM, 19–22. doi:10.1145/3589012.3594897 [5] Alvise Dorigo, Peter Elmer, Fabrizio Furano, and Andrew Hanushevsky. 2005. XROOTD-A Highly scalable architecture for data access. 348–353 pages. [6] E Fajardo, A Tadel, M Tadel, B Steer, T Martin, and F Würthwein. 2018. A federated Xrootd cache. Journal of Physics: Conference Series 1085, 3 (sep 2018), 032025. doi:10.1088/1742-6596/1085/3/032025 [7] David Schultz, Igor Sfiligoi, Benedikt Riedel, Fabio Andrijauskas, Derek Weitzel, and Frank Würthwein. 2023. IceCube experience using XRootD-based Origins with GPU workflows in PNRP. arXiv:2308.07999 [physics.comp-ph] [8] Derek Weitzel, Marian Zvada, Ilija Vukotic, Rob Gardner, Brian Bockelman, Mats Rynge, Edgar Fajardo Hernandez, Brian Lin, and Mátyás Selmeci. 2019. StashCache: A Distributed Caching Federation for the Open Science Grid. In Proceedings of the Practice and Experience in Advanced Research Computing on Rise of the Machines (Learning) (Chicago, IL, USA) (PEARC ’19). Association for Computing Machinery, New York, NY, USA, Article 58, 7 pages. doi:10.1145/3332186.3332212
Acknowledgments This work was supported in part by National Science Foundation (NSF) awards #1836650, CNS1730158, ACI-1540112, ACI-1541349, OAC-1826967, OAC-2030508, OAC-2112167, CNS-2100237, CNS-2120019, PHY-2323298, the University of California Office of the President, and the University of California San Diego’s California Institute for Telecommunications and Information Technology/Qualcomm Institute. Thanks to CENIC for the 100Gbps networks.
, Vol. 1, No. 1, Article . Publication date: May 2025.