Toward an Architectural Blueprint to Observe Sustainability in and by Software Systems
arXiv:2604.09278v1 [cs.SE] 10 Apr 2026
Klervie Toczé1 , Andrei Dragomir1 , Vincenzo Stoico, and Patricia Lago Vrije Universiteit Amsterdam, Amsterdam, The Netherlands Email: [email protected], [email protected], {v.stoico, p.lago}@vu.nl 1 Klervie Toczé and Andrei Dragomir contributed equally. Abstract—Enabling observability in software systems brings many benefits. It can, for example, ease the identification of issues or the implementation of improvements. It is especially critical to be able to observe sustainability-related dimensions of systems to know and mitigate their impact. However, adding observability to a system, especially related to software sustainability, requires technical knowledge that may not be available in every project that would benefit from it. In this work, we propose an architectural blueprint along with its deployment code that can be used to facilitate the addition of observability in software systems. As a special case, it includes measuring the energy consumption of software. This toolkit provides support in defining which components are necessary for a given use case and for structuring their deployment. Moreover, we exemplify the addition of observability in two different use cases. Index Terms—Observability, Monitoring, Dashboard
I. I NTRODUCTION Software systems benefit from being monitored to decide whether their performance is satisfactory or not, and how they can be improved. Therefore, an important property of software systems is observability. Observability states how much stakeholders of a software system can understand the internals of the application using exposed outputs [1]. Additionally, software systems can be used to enable observability in other types of systems, such as farms, manufacturing plants, and transportation networks, by collecting, processing, and analyzing data to monitor their behavior and improve performance, reliability, and sustainability [2]. Although sustainability has become a central topic in software engineering discussions, there remains a notable gap: few approaches bridge sustainability metrics and observability, especially when it comes to distributed software systems, as well as in monitoring systems beyond software, including physical and industrial domains. Existing performance observability literature focuses almost exclusively on traditional indicators such as latency, throughput, and resource utilization. There is a need to bridge this gap while also collecting sustainabilityrelated measurements that provide software stakeholders with the necessary data to support it. To expose such metrics, software engineers need to implement a system to monitor, collect, and effectively communicate these metrics [2]. However, enabling observability means deploying a comprehensive set of tools, ranging from tools to collect the required metrics to tools for visualizing them [3]. There are
two challenges here: 1) knowing which tool is relevant for each part and how each of them work, and 2) knowing how to connect all these tools. This knowledge may not exist in every project team using software for monitoring and acquiring it can be costly, especially if the core of the project is not about software. It is thus needed to provide guidance and tools to ease the deployment of observability stacks, especially addressing the second challenge so that the project team can focus their efforts on the project-specific deployment needs. If this deployment is made easier, a wider range of systems could benefit from observability. We propose an architectural blueprint to guide software practitioners (SPs) in the design and implementation of a software stack enabling domain-agnostic observability of sustainability-metrics. The resulting observability stack is meant for SPs with intermediate technical knowledge who lack the experience of deploying a full stack, but are able to select the individual components and manage them once they are in place. Why not just directly use AI for this? While generative AI is increasingly applied to deployment tasks, its effectiveness depends on well-documented components. Subtle configuration differences across services are difficult for AI to capture when generating manifests from scratch. Once the general structure is in place, which is what this works is focusing on, AI may be used to do the tailoring of components. To summarize, we contribute: (i) An architectural blueprint enabling observability in software systems together with the corresponding code repository to realize an observability stack, including energy-consumption metrics [4], (ii) Two real sustainability-focused use cases illustrating the result when using the blueprint in practice, (iii) A discussion of existing challenges and potential solutions. II. R ELATED W ORK Observability tools, and dashboards in particular, are commonly reported to be used in scientific literature for various use cases. However, current literature stop at merely stating that a dashboard tool is used, without providing further details on its implementation (e.g., [5], [6], [7], [8]). This hinders reproducibility and re-use. Beyond individual dashboards, a wide range of openly available observability tools exist for monitoring resource usage and energy consumption in software systems (e.g., [9],
[10], [11]). While many of these tools are mature and well engineered, they are often presented as comprehensive solutions that are oriented towards Software Life Cycle Assessment (SLCA) or as tightly integrated platforms. As a result, SPs, especially those with limited prior expertise, are frequently left without clear methodological guidance on how to design and implement a monitoring environment tailored to their own application rather than adopting a fixed end to end solution. To address this gap, our blueprint and accompanying examples provide a structured methodology for assembling a modular observability stack using readily available, well documented, and highly configurable tools. As an illustrative example, CEEMS [12] offers a resource manager agnostic energy and emissions monitoring stack that can be naturally integrated within the type of monitoring environment supported by our blueprint. To lower the technical barrier to create observability tools (OTs) such as dashboards, tools have been created to ease for the novice OT creator. For example, Elias and Bezerianos [13] propose and evaluate a graphical tool for Business Intelligence dashboards, including recommendations for data mapping and visual templates. Bellini et al. [14] also use a graphical way to create dashboards for smart cities, to provide a solution with no or low coding as the OT creator may not have coding skills. Another approach is used by Clements [15], who created a package for automatically creating R dashboards from Excel spreadsheets. These works emphasize the need to provide tools adapted to the technical knowledge of the OT creator, to make dashboard creation accessible to a wider range of OT creators. Our work differs in two ways: we want to ease creating the full observability stack (and not the dashboard only), and we target intermediate OT creators, namely those with some technical experience. III. A RCHITECTURAL B LUEPRINT Collection Layer (Profilers/Exporters)
Aggregation & Storage Layer (Data Normalization & Database)
«Profiler» Resource Usage (CPU, Mem, Disk, Energy, etc.)
Aggregator (Normalizes Raw Data)
Time-Series Data
«Profiler» Application Usage (Traces, Usage Patterns, etc.) «Exporter» Application Interface (User-defined Data)
Metrics & Metadata
Regular Database (Long-term data & condensed temporal data)
Normalized Data
Time-Series Database (High-frequency data)
Compressed Data
Visualization Layer Dashboard (User-defined/template graphs and overviews ) Alerting APIs
Raw Data Derived Data Alert Reason
Analytics Engine (Anomaly Detection & Correlation)
Processing Layer (Statistics, Compression) Data Compression (Distill data & extract trends)
Figure 1: Architectural Blueprint Figure 1 describes the main components and their relationships within our architecture. This figure serves as a blueprint that SPs can reuse and adapt to different systems and use cases. The blueprint emerged from our experience rather than
a systematic design process, and the depicted elements are grounded in both literature and our use cases (see Section IV). The blueprint is organized in layers, each fulfilling a distinct role: collection, aggregation and storage, processing, and visualization. The components follow a pipe-and-filters pattern, with data managed and passed sequentially until reaching the user-facing interface. Figure 1 should be read starting from the top left. The data flow begins with the Collection layer, which profiles metrics of interest in a (software) system. This includes resource and energy usage as well as application-specific data such as traces, usage monitoring, and patterns directly generated by end users. Metrics can be captured at various granularities, from machine to function level, while usergenerated data may range from interaction logs to domainspecific information (e.g., purchase statistics or feature usage). Collected data is transmitted in raw form, typically timestamped values, to the next layer via push or pull interfaces. The Aggregation & Storage layer consolidates this data. The Aggregator gathers and normalizes heterogeneous inputs into a common representation. Normalization ensures comparability across profilers, e.g., aligning CPU time units or structuring user interaction logs. Push-based storage systems may embed aggregation, while other cases require an independent module to collect and organize user- or system-level data. Data is stored in two databases: a time-series database for high-frequency metrics such as resource usage, and a regular database for non time-series data, condensed statistics, and user or system metadata. This distinction enables both efficient temporal queries and contextual long-term storage. The Processing layer retrieves data through the Analytics Engine, which supports all kinds of data analyzes. This includes, e.g., anomaly detection, correlation, and root-cause reasoning by combining time-series with metadata. It also performs compression, distilling raw time-series into compact summaries to optimize storage. This establishes a continuous loop with the aggregation & storage layer: gather, store, distill, and clear. Here, both system- and user-level data are processed, preparing tailored outputs for different stakeholders. The Visualization layer provides the user-facing interface. The Dashboard integrates tracing, correlation, and monitoring views, supporting visualization of, e.g., collected data and aggregated metrics. Alerts are surfaced at this layer as they constitute the primary point of end-user interaction, delivered through webhooks or equivalent mechanisms. Different audiences may receive distinct perspectives: administrators may access holistic system overviews, whereas individual users see only their own data and analytics. A. From Blueprint to Implementation We support the transition from blueprint to implementation through a code repository with templates for modular components and deployment instructions [4]. This toolkit targets project members familiar with the components but needing guidance to deploy them as an integrated observability stack.
SPs start by identifying the blueprint components relevant to their specific use case across the different layers (e.g., collection, storage, visualization). After selecting these components, they can customize the corresponding configuration files in the repository to match the desired setup. Environment variables, including database and admin credentials, must be defined in a .env file, for which an example template is provided in the repository. Deployment can then be carried out either by launching individual services using their dedicated deployment files or by merging multiple service definitions into a single deployment.yml. The latter approach is recommended, as it simplifies customization and ensures all required services are started together. After deployment, SPs may further tailor configurations, such as scrape targets or data sources, depending on the services involved. This process supports both systemwide observability views (for administrators) and user-specific analytics. For detailed instructions, examples, and deployment templates, please refer to [4]. B. Challenges The implementation of the proposed architectural blueprint requires consideration of several key aspects. Not all the layers described in the blueprint are needed. While it is designed to capture diverse software characteristics, only the collection and the visualization layers are mandatory, as they provide the minimum needed to profile an application and offer insights. For example, SPs may focus only on energy usage variation under workload. Implementing an observability stack can pose challenges related to analysis granularity, data sensitivity, storage, transmission, and visualization. Observer effect: A complex observability stack may require running several profiles in parallel to collect different types of data about the software system under observation. This process, which takes place in the Collection layer, consumes additional computing resources, potentially introducing overhead that can influence and bias the metrics of interest. Nõu et al. [16] investigate how to trace impacts performance in microservices and serverless setups. They show that tracing with OpenTelemetry and Elastic APM can slow down the throughput by up to 80%, and latency can increase by 175%. The authors point out further challenges with the setup/initialization of the tracing system, data mole generated by fine-grained profiling, and data transmission. Possible mitigation strategies involve balancing the trade-off between overhead and data granularity, batching data collection, and offloading processing to external machines. The latter case can introduce burden on the network, especially if real-time data is needed. [17] tackle the problem by providing a system to visualize multiple end-to-end microservice requests, thus reducing data size when reaching the Visualization layer. Security: It is a concern affecting all layers of the observability stack. Collected data may contain sensitive information about users or the system that should not be exposed. Aghili et al. [18] discuss several cases where security threats arise in software logs, including the accidental disclosure of personal
information and quasi-identifiers. This threat can be partially mitigated through anonymization or sanitization. Aggregation vs. Granularity in Monitoring: In software performance and energy analysis, aggregated data (such as average latency or average energy per request) are commonly stored and displayed. Aggregation helps reduce storage requirements and simplifies real-time monitoring. This approach can obscure important variations and hide information needed to diagnose unexpected issues. The Visualization layer may report only the average latency for a software system undergoing 1000 request/minute, for example 200 ms, a value that satisfies extra-functional requirements. However, examining disaggregated data (e.g., per-request), may reveal a different latency per request showing a bottleneck during processing. Within that same minute, 950 requests could have completed in 100 ms, while 50 requests suddenly spiked to 3 seconds due to a temporary failure. Yu et al. [19] in the context of root-cause analysis, which is usually done in Processing layer. Fault propagation due to a root cause service can cause surge in latency, which can influence the performance of multiple services. Root cause analysis can be facilitated by automatically processing disaggregated data, since aggregation may hide the actual source of the fault. However, storing every kind of data can overwhelm the database leading to huge memory consumption, slow queries, and high costs. Dashboard Design: The Visualization layer plays a vital role in enabling OT users to extract meaningful insights from collected data and to detect anomalies within software systems. However, visualization also brings challenges related to interpretation and configurability. Indeed, without proper guidance, OT users may struggle to interpret dashboards effectively. Therefore, clear documentation and comprehensive guidelines are essential to help OT users create and understand visual dashboards. Allowing OT users to build custom visualizations tailored to their specific metrics of interest can also be highly beneficial, particularly for non-IT OT users. This process can be supported through the provision of predefined dashboard templates for specific use cases, e.g., a template focusing solely on sustainability metrics. This feature is well-known and supported by tools for interactive and customizable dashboard creation like Grafana. IV. U SE C ASES We illustrate the use of our toolkit in two different scenarios: one focused on monitoring urban agriculture initiatives as part of a EU project, and another within our research group’s High-Performance Computing (HPC) lab, where it is used to track energy consumption and resource utilization of various software workloads. These workloads range from distributed applications to single processes, and the collected data serve as a basis for research and analysis. A. Urban Agriculture Monitoring Our first use case is the Feed4Food project [20]. In this project, an observability tool including data gathering and dashboard functionalities is supporting three different urban
Collection Layer
Aggregation & Storage Layer Collection Layer
«Profiler» Scaphandre «Profiler» Node-exporter «Exporter» Django Visualization Layer «Custom Interface» Django «Open Source Service» Grafana
Metrics
User Data
«Profiler» Scaphandre
«Short-term DB» Prometheus
«Profiler» Node-exporter
«Long-term DB» PostgreSQL
Processing Layer Derived Data
«Time-series DB» Prometheus
«Profiler» CAdvisor
Raw Data
Raw Data
Aggregation & Storage Layer Metrics
«Analytics Script» KPI Computing
Visualization Layer «Open Source Service» Grafana «External WebHook» Slack Messaging
Figure 2: The Feed4Food stack
Raw Data
Derived Data Service Failures
Raw Data Processing Layer «Grafana Feature» Data Transformation «Prometheus Feature» Target Monitoring
Figure 3: The GreenLab distributed monitoring stack agriculture setups called Living Labs (LL) that are developing urban gardens with a common goal (sustainable, inclusive and healthy urban food environments) but in different contexts (e.g., size, location, and type of LL gardeners). Most of the data collected in this project comes from humans (e.g., harvest, irrigation, event attendance) and some can be collected by software measurement tools (e.g., CPU and energy use from the machine running the server), hence requiring the use of two different databases. Regarding data processing and visualization, there is a need to provide several visualization alternatives: 1) the LL gardeners want to see direct information about their own piece of land (e.g., using the dashboard as a light bookkeeping tool), 2) the LL managers want to see how the LL performs with regards to defined targets, 3) the project partners want to learn and extract guidelines for future LLs based on the performance of all three LLs. Therefore, there is a need for three different types of visualizations: 1) a straightforward view on the data collected, 2) a view of key performance indicators derived from the data collected, and 3) a highly flexible visualization option enabling researchers to “play around” with the data as needed. In addition, different analyzes need to be implemented to be able to get the data that should be visualized in 2) and 3). The specifics of the current architecture are illustrated in Figure 2. The specificity of this use case is that part of the system observed (the gardens) is not a software system and that data for this part has to be manually gathered. The use case code is available on GitHub [21]. B. Distributed Energy and Resource Profiling Our second use case is situated in the GreenLab, our HPC laboratory [22]. There, the observability tool supports monitoring and oversight of software experiments on computing clusters. The setup is particularly valuable for administrators, as it provides an overview of cluster-wide energy consumption and resource utilization. For researchers, the tool is most beneficial when studying (large-scale) distributed applications, where aggregated metrics and dashboard views capture meaningful system behavior. In contrast, for experiments focused on single processes, finer-grained instrumentation is required, as the stack’s aggregation and visualization features introduce unnecessary overhead for such narrow-scope tasks.
For data collection, most metrics are collected automatically by exporters running on the compute nodes, including CPU, memory, and power consumption. This ensures reproducibility and minimizes manual intervention during experiments. For visualization, Grafana dashboards provide ready-to-use templates giving researchers an immediate starting point for experiment monitoring. Real-time views are available during execution, while aggregated dashboards summarize nodelevel utilization and efficiency for administrators. In addition, Grafana’s interface allows researchers to export selected data segments , enabling them to perform custom analyzes and integrate the results into their workflows. The specifics of the current architecture are illustrated in Figure 3. The specificity of this use case is that the collection and the aggregation and storage layers are distributed. The code for this use case can also be found on GitHub [23]. V. C ONCLUSION AND F UTURE W ORK The proposed architectural blueprint and associated code base toolkit provides support and examples for researchers and practitioners who would like to add sustainability-oriented observability to their (software) systems but are not proficient in how to build such observability stacks. In addition, the two use cases described propose comprehensive open-source examples of how the blueprint can be used in practice. While our current work introduces a reusable observability stack and demonstrates its applicability in two use cases, several directions remain open for further work. Systematic validation studies are required to assess the effectiveness of our blueprint across contexts, evaluating usability and adaptability for researchers, administrators, non-IT users, as well as its support for decision-making driven by sustainability metrics. Additionally, the modular nature of the stack highlights opportunities for creating a curated ecosystem of domain-specific collector and visualization modules, tailored to domains like agriculture, education, or HPC, to reduce deployment time and foster reproducibility. At the same time, challenges remain in ensuring accurate and efficient sustainability-oriented observability: metric isolation, profiling overhead in distributed environments, and clear documentation of profiler limitations should be addressed to improve reliability and adoption.
ACKNOWLEDGMENT The authors would like to thank Kostas Vilkelis for his contribution to the initial design and implementation of the Feed4Food use case. Klervie Toczé and Patricia Lago were supported by the Feed4Food project, Grant Agreement n° 101069506, which has been funded by: Research and Innovation Foundation (Cyprus), Dutch Research Council (Netherlands) General Secretariat for Research and Innovation (Greece), UE Fiscdi (Romania) under the Driving Urban Transitions Partnership, co-funded by the European Union. R EFERENCES [1] S. Newman, Building microservices: designing fine-grained systems. O’Reilly Media, 2021. [2] F. A. Almalki, B. O. Soufiene, S. H. Alsamhi, and H. Sakli, “A low-cost platform for environmental smart farming monitoring system based on IoT and UAVs,” Sustainability, vol. 13, no. 11, p. 5908, 2021. [3] B. Mayer and R. Weinreich, “A dashboard for microservice monitoring and management,” in 2017 IEEE International Conference on Software Architecture Workshops (ICSAW). IEEE, 2017, pp. 66–69. [4] Software and Sustainability Group. (2025) Boilerplate for the Architectural Blueprint. [Online]. Available: https://github.com/ andrei-calin-dragomir/software-monitoring [5] S. Wolfert and G. Isakhanyan, “Sustainable agriculture by the internet of things – a practitioner’s approach to monitor sustainability progress,” Computers and Electronics in Agriculture, vol. 200, 2022. [6] E. Freitas and V. Cabrera, “Dairy victory platform: A novel benchmarking platform to empower economic decisions on dairy farms,” JDS Communications, vol. 6, no. 1, 2025. [7] A. Scipioni, A. Mazzi, M. Mason, and A. Manzardo, “The Dashboard of Sustainability to measure the local urban sustainable development: The case study of Padua Municipality,” Ecological Indicators, vol. 9, no. 2, 2009. [8] J. Vannieuwenhuyze, “Dashboards for input-evaluation of policy programs: Lessons learned from an antwerp dashboard for garden streets,” ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. VI-4/W2-2020, 2020. [9] A. Tarara, D. Mateas, and G.-D. Hoffmann. (2026) Green Metrics Tool. [Online]. Available: https://github.com/green-coding-solutions/ green-metrics-tool [10] Green Frame project contributors. (2024) Green Frame. [Online]. Available: https://github.com/marmelab/greenframe-cli [11] Cloud Carbon Footprint project contributors. (2024) Cloud Carbon Footprint. [Online]. Available: https://github.com/cloud-carbon-footprint/ cloud-carbon-footprint [12] M. Paipuri, “CEEMS: A resource manager agnostic energy and emissions monitoring stack,” in SC24-W: Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2024, pp. 1862–1866. [13] M. Elias and A. Bezerianos, “Exploration views: Understanding dashboard creation and customization for visualization novices,” in HumanComputer Interaction – INTERACT 2011. Springer Berlin Heidelberg, 2011, pp. 274–291. [14] P. Bellini, M. Fanfani, P. Nesi, and G. Pantaleo, “Snap4city dashboard manager: A tool for creating and distributing complex and interactive dashboards with no or low coding,” SoftwareX, vol. 26, p. 101729, 2024. [15] L. Clements, “ExcelToShiny: An R Package for Automating Shiny Dashboard Creation from Excel Templates,” Journal of Open Source Software, vol. 10, no. 111, p. 7876, 2025. [16] A. Nõu, S. Talluri, A. Iosup, and D. Bonetta, “Investigating performance overhead of distributed tracing in microservices and serverless systems,” in Companion of the 16th ACM/SPEC International Conference on Performance Engineering, 2025, pp. 162–166. [17] L. Traini, J. Leone, G. Stilo, and A. Di Marco, “VAMP: Visual analytics for microservices performance,” in Proceedings of the 39th ACM/SIGAPP Symposium on Applied Computing, 2024, pp. 1209–1218. [18] R. Aghili, H. Li, and F. Khomh, “An empirical study of sensitive information in logs,” 2024. [Online]. Available: https://arxiv.org/abs/ 2409.11313v1
[19] G. Yu, Z. Huang, and P. Chen, “Tracerank: Abnormal service localization with dis-aggregated end-to-end tracing data in cloud native systems,” Journal of Software: Evolution and Process, vol. 35, no. 10, p. e2413, 2023. [20] Feed4Food project. (2024) Feed4Food - Promoting Sustainable, Inclusive and Healthy City Food Systems. [Online]. Available: https://feed4food.eu/ [21] K. Toczé and K. Vilkelis. (2025) Feed4Food. [Online]. Available: https://github.com/Ilydocus/feed4food [22] I. Malavolta, V. Stoico, and P. Lago, “Ten years of teaching empirical software engineering in the context of energy-efficient software,” in Handbook on Teaching Empirical Software Engineering. Springer, 2024, pp. 209–253. [23] A. Dragomir. (2025) PeekStack. [Online]. Available: https://github.com/ andrei-calin-dragomir/PeekStack