A novel NFR-based conceptual quality framework for modern API industry.

A novel NFR-based conceptual quality framework for modern API industry - PMC Skip to main content An official website of the United States government Here's how you know Here's how you know Search Log in Primary site navigation Search Log in Search PMC Full-Text Archive Search in PMC Journal List User Guide As a library, NLM provides access to scientific literature. Inclusion in an NLM database does not imply endorsement of, or agreement with, the contents by NLM or the National Institutes of Health. Learn more: PMC Disclaimer | PMC Copyright Notice Sci Rep . 2026 Apr 10;16:16955. doi: 10.1038/s41598-026-48021-4 A novel NFR-based conceptual quality framework for modern API industry Aumir Shabbir 1, 2 , Aziz Deraman 1, ✉ , Mohamad Nor Bin Hassan 1 , Kamal Uddin Sarker 3 , Shahid Kamal 4, ✉ Author information Article notes Copyright and License information 1 Faculty of Computer Science and Mathematics, Universiti Malaysia Terengganu, Kuala Nerus, Terengganu 21030 Malaysia 2 Department of Business and Management Studies, Gulf College, Al Mabaila, Muscat, 133 Oman 3 Department of Computer Science and Engineering, Uttara University, Dhaka, 1230 Bangladesh 4 Center for Advanced Analytics, CoE for Artificial Intelligence, Multimedia University, Parsiaran Multimedia, Cyberjaya, Selangor 63100 Malaysia ✉ Corresponding author. Received 2025 Dec 11; Accepted 2026 Apr 6; Collection date 2026. © The Author(s) 2026 Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ . PMC Copyright notice PMCID: PMC13230622  PMID: 41963525 Abstract The modern software ecosystems employ application programming interface (API) to allow interoperability and integration across various platforms. Non-functional requirements (NFRs) as quality attributes are as vital as functional requirements (FRs) in assuring software quality and user satisfaction. API quality management is a challenge to software industry owing to poor NFRs handling during API development, resulting in cost increase, project failure and dissatisfied customers. The existing standards failed in producing a comprehensive quality framework for APIs. OpenAPI Specification (OAS) focuses on functional aspects, whereas ISO/IEC 25010:2023 emphases on traditional software quality. Hence, this study proposes a novel non-functional requirement quality framework for APIs (NFRQF-API). In compliance with ISO/IEC 25010, the framework includes key NFRs and their influencing factors to facilitate the development of high-quality APIs. Explanatory sequential mixed-methods design was applied in this study, involving a survey tool. Data were collected using a questionnaire comprising eight Likert-scale items and two open-ended items. Descriptive statistics and reliability testing were applied for quantitative data analysis, while thematic analysis was applied to evaluate the qualitative data, which supports in achieving a comprehensive expert-based justification for the conceptual feasibility and applicability of framework. Based on the experts’ survey, analyses results demonstrated the potential effectiveness of the proposed framework in managing the Core, Critical, and Contextual NFRs in the entire API development lifecycle. This study bridges the gap between existing quality models and API quality assurance challenges. The structured expert-refined framework facilitates the enhancement of API quality management and offers KPI-based conceptual quality standards for the industry to design high-quality APIs by effectively handling critical NFR issues. Keywords: Application programming interface, Non-functional requirements, Non-functional requirement quality framework for APIs, Explanatory sequential mixed-methods design, ISO/IEC 25010:2023 Subject terms: Engineering, Mathematics and computing Introduction Application programming interface (API) is prevalent in the current era of software engineering. In a digital environment, APIs support web apps, mobile platforms, and enterprise solutions 1 . In modern software ecosystems, APIs facilitate modularity, integration, efficiency and collaboration 2 – 4 . Through the use of high-quality APIs, resiliency, maintainability, observability as well as security can be attained, resulting in trust and adoption potential among organizations and developers. Nonetheless, despite their vital role and extensive usage, API often fail owing to overlooked Non-Functional requirements like maintainability 5 , 6 , resiliency 7 , 8 and observability 9 , while failure of API can result in interrupted digital services. In APIs, observability and resiliency are among the key NFRs as they facilitate the handling of intricate issues related to quality 8 , 9 . As APIs behave differently from conventional software applications; therefore, dedicated API quality management is essential in modern distributed environments 10 . For instance, the observability attributes involve the monitoring of logs and continuous system during real-time operations 9 , 11 , while the resiliency attributes involve the handling of failures that occur during API operation, without causing disruption 12 . Notably, auto recovery from application failure is a recent feature that makes the system considerably more resilient 13 , while decreasing the downtime 7 , 8 . OpenAPI Specification 3.1 and ISO/IEC 25010, which are among the existing standards for software systems, have drawbacks. For instance, OpenAPI 3.1 only focuses on operations and schemas, not on NFRs and KPI (Key Performance Indicators) metrics 14 , resulting in gaps for NFRs in API. On the other hand, ISO/IEC 25010, which has a broad scope, provides an outline for a high-level product quality model and characteristics 15 . Somehow, it often neglects the specific NFRs that are integral to APIs 16 . APIs, although important, are still ineffectively managed in terms of quality. In fact, the existing quality models mostly focus on the handling of functional aspects. On the other hand, various aspects of NFRs, including performance, security, scalability, observability, and resiliency, are often inadequately addressed or even neglected altogether 17 . Clearly, the factors of NFRs have not been adequately considered during API development 16 , while a comprehensive quality framework to support high-quality API development is yet to exist 5 , 18 , 19 . Accordingly, this study addresses a critical gap in existing software quality standards, where the effective management of API-specific NFRs is not explicitly defined. Although ISO/IEC 25010 provides a comprehensive foundation for the quality of traditional software, while it does not adequately address the unique characteristics and operational requirements of modern APIs. Hence the current study offers an expert-refined Non-Functional Requirement Quality Framework for APIs (NFRQF-API) aligned with ISO/IEC 25010:2023, which provides a structured and systematic approach for managing API quality and its influencing factors throughout the API development lifecycle. Consequently, to both academic research and modern API domains, this study contributes to the following: The development of a structured, ISO/IEC 25010-aligned conceptual quality framework that integrates the NFRs along with their influencing factors across Strategic, Operational and Quality Assurance (QA) levels, supported by continuous validation through a feedback loop. The integration of a KPI-based mechanism to support the conceptual operationalization of NFRs, which provides indicative guidance for monitoring and assessing API quality within the proposed framework. The expert-based refinement of the framework using Likert scale results and thematic feedback to enhance its clarity, structure, and practical relevance. In the next section (Sect.  2), this study provides a review on the research background and then highlights the gaps in existing models and standards. In the section that follows (Sect.  3), this study details the methodology applied in the research, focusing on the questionnaire design, the employed criteria in the selection of experts, and the methods used in analyzing the data. In Sect.  4, the study results are discussed, while in Sect.  5, this study delves into the threats to validity and limitations. The last section, namely Sect.  6, concludes the study by highlighting its key contributions. Research background Challenges in API development and management APIs are an integral element in modern software ecosystems and the internet of things (IoT), where the main function of APIs is to connect applications, systems, and platforms. In API development and documentation, security, performance, and reliability are among the NFRs to be taken into account 20 , considering that APIs operate the external access to back-end systems 21 . It is thus crucial to systematically addressed these NFRs to achieve dependable, secure and high-performing APIs 22 . Furthermore, clear and inclusive API documentation would result in increased API adoption and smooth integration 23 . Role of NFRs in API quality NFRs, which are as important as functional requirements (FRs), specify the quality attributes that are vital for software applications to assure software quality and user satisfaction 20 . The attributes that are considered vital to software applications include reliability, security, maintainability, performance, functional suitability, scalability, and usability 24 . Failure in fulfilling NFRs can result in undesirable consequences such as increased costs, customer dissatisfaction, and even project failure 25 , 26 . Eliciting, documenting and integrating NFRs are no easy tasks. Somehow, owing to the positive impact of NFRs on the conceptual models and on the project outcomes, they should be treated as main requirements 27 . Significant impact of NFRs on project cost, duration, and business success has been empirically proven, especially when NFRs were perceived as important as FRs 25 , 26 . Observability and Resiliency have emerged as major NFRs, with observability being highly influential in assuring APIs’ dependability and performance 28 , 29 . Observability is primarily associated with the critical post-deployment operations, especially the logging and continuous monitoring of real-time APIs 30 – 32 . In this regard, efficient and robust API is one that has inclusive logging and real-time monitoring 32 – 34 . Meanwhile, the attribute of resiliency allows fault tolerance, automatic smooth recovery and smooth degradation in APIs 35 . Specifically, fault tolerance assures that the services ensue amidst system failure 8 , 36 . Resiliency also allows the system to automatically recover to its normal operations 7 , 37 . Clearly, in modern software systems, the incorporation of NFRs (e.g., performance, security, maintainability, observability, and resiliency) will assure robust, dependable and quality APIs. Somehow, it is challenging for organizations to effectively manage NFRs 38 . Limitations in current standards and models ISO/IEC 25010:2023 is a commonly used standard in software product quality evaluation. It outlines nine quality characteristics, namely, Functional Suitability, Performance Efficiency, Compatibility, Interaction Capability, Reliability, Security, Maintainability, Flexibility, and Safety. Notably, this standard merely provides a general framework to manage the quality of conventional software systems and services 15 , 39 , and there are no predefined metrics and API-specific guidance 16 . Correspondingly, OpenAPI Specification is commonly used in describing the structure, endpoints, and data models of REST APIs. The aim of this standard is to support users in their understanding of and interaction with API operations 40 , 41 . Additionally, OAS, which was developed mainly to document functional aspects, doesn’t have the built-in support to manage NFRs (e.g., performance and reliability) or security issues 14 , 21 , 42 . Numerous models and approaches for managing NFRs in APIs have been proposed, for instance, continuous integration (CI) environments 43 and automated tools 44 . Both these approaches address the aspects of security, performance, and reliability. On the other hand, other aspects, which are also crucial, such as maintainability, usability, and scalability, have been ignored. Meanwhile, a few other available domain-specific frameworks are focusing only the precise domains like machine learning systems or product lines, and this limits their general pertinence to broader contexts of API 45 , 46 . In their study, Sherif, Helmy, and Galal-Edeen 47 proposed the MANoR framework to manage NFRs in Agile Software Development. However, the framework was mainly focusing on Scrum and offered no specific guidance for API. Correspondingly, the studies of Viviani 48 and Kumar 49 highlighted the usefulness of NFRs during software development, owing to the frequent disregard over the aspects of usability, security, privacy, and reliability. However, the authors did not offer a structured quality framework for APIs. Similarly, Pargaonkar 50 , Koci 51 , and Lamothe 52 presented some insights on NFR management in their studies. However, their insights did not comply with quality standards like ISO/IEC 25010. Agile, Scrum, Spiral, and DevOps are among the different development methods available today, but their main focus are on the functional aspects, resulting in the neglect on the vital aspect of NFRs during API designing 53 . Markedly, the quality of an application is majorly determined by NFRs. Consequently, maintaining high software quality within their development environment is essential and requires constant attention 54 . Additionally, Table 1 mentioned below presents a systematic comparison of existing studies in accordance with our research aim. Table 1. A systematic comparison of existing literature. Title with reference Contribution NFRs addressed API addressed ISO/IEC 25010 alignment Methodology adopted Limitations and future work 3 “State of API Practice: A Case Study” Empirical case study of API development and management practices to identify challenges and gaps. Yes Yes No Descriptive qualitative case study based on interviews with 4 API experts. Limited sample size affects generalizability No analysis of NFR frameworks Broader empirical API research required 10 “Prioritizing NFRs for API Quality: An Industry Approach” Identified and prioritized most critical API-related NFRs using ISO/IEC 25010. Yes Yes Yes Likert-scale based structured survey for experts’ validation Focuses only on NFR prioritization No comprehensive API framework proposed Recommend developing industry-validated API quality framework 14 “OpenAPI-Based Framework for Testing REST API NFRs” Provided a framework that uses OpenAPI descriptions to automatically test and monitor NFRs. Yes Yes No Demonstration-based evaluation such as, tool prototype tested on real REST APIs Limited NFR scope and runtime focus No ISO 25010 mapping or validation Expand NFR coverage and visualization methods 23 “API Documentation: What Developers Want?” Use of API documentation and common challenges during API implementation. Yes Yes No No statistical / empirical validation Limited NFR coverage No ISO 25010 mapping included Recommend automated methods implementation 34 “eBPF-Based Observability for Cloud-Native Microservices” Proposed an eBPF-based observability framework that captures metrics, logs, and distributed traces across microservices. Yes Yes No Statistical validation through a structured quantitative-based survey Limited NFR coverage No ISO 25010 mapping included Extend observability across cloud environments 38 “API Management Quality and ISO/IEC SQuaRE: Practitioner Perspective” Provided the most critical quality attributes based on the practitioner’s perceptions. Yes Yes Yes Pilot exploratory survey Limited NFR coverage Limited sample size for validation Suggested using larger samples and a quality framework to address broader range of NFRs 42 “Automated Specification-Based Testing of REST APIs” Proposed an automated test case generation approach for REST APIs based on OpenAPI. Yes Yes No Validation through case study Limited NFR coverage No ISO 25010 mapping or validation Expand NFR scope and specifications 47 “Proposed framework to manage NFRs in agile” Proposed MANoR, a full framework for managing NFRs in Agile. Yes No No Likert-scale Survey-based evaluation Not API-focused, lacks ISO/IEC 25010 mapping Evaluation limited to perception-based approach Suggested applying real cases and developing NFR-based framework 55 “API Research Review: Past Insights and Future Direction” Classification of themes, methodologies, theories, and gaps in API research. Yes Yes Yes No statistical / empirical validation. Limited NFR coverage No empirical validation Suggested to develop and validate comprehensive API quality framework 56 “Minimising conflicts among run-time NFRs within DevOps” Proposed a data-driven framework to minimize conflicts among run-time NFRs in DevOps. Yes No No Experimental evaluation using run-time measurements and scenarios. Focus only on runtime NFRs No DevOps validation or ISO mapping Extending the framework with design-time NFRs 57 “Using Behavior-Driven Development (BDD) for NFRs” Provided a BDD framework that can be applied to elicit and test NFRs. Yes No Yes A real industrial case study Limited NFR coverage, not API-focused Validation limited to single system Extend the framework with wide range of ISO NFRs and practitioner surveys 58 “Prioritizing NFRs in Agile Using Multi-Criteria Analysis” Proposed an AHP-based prioritization model for NFRs in agile projects. Yes No No Likert scale-based survey Not API-focused or ISO-mapped Limited validation Recommending quantitative empirical agile study 59 “API Management Practices and Capabilities: A Systematic Review” Identified the practices and capabilities of API Management and proposes a consolidated definition of API Management. Yes Yes No No empirical validation. Limited NFR coverage No ISO 25010 mapping included Recommended to develop API framework covering wide range of NFRs 60 “Model for Detecting NFR Conflicts and Dependencies” Proposed a model (MAICD) that identifies, stores, and detects conflicts and dependencies among NFRs Yes No No Validation through one case study Single case, no practitioner validation Limited NFR coverage, no ISO mapping Expand automation and multi-project validation 61 “API development model for digital twins” Proposes a development model for Digital Twin APIs, addressing API-related challenges. Yes Yes Yes No statistical / empirical validation. NFRs discussed without quantitative measurement No structured or validated NFR framework Develop user-friendly API framework and Digital Twin interfaces 62 “Benchmarking web API quality-revisited” Web API quality, various NFRs and their impact on application performance and user experience. Yes Yes No Long-term empirical benchmarking Limited NFR coverage, no ISO mapping No statistical or empirical validation Recommending an industry-validated API quality framework 63 “Conceptual Framework for Decomposing NFRs into QoS Attributes” The study decomposes business process of NFRs into web service QoS attributes and addresses the alignment challenge. Yes No No No empirical validation Lack of validation and API focus No ISO 25010 mapping included Explore automated NFR–QoS framework integration Open in a new tab Table 1 mentioned above summarizes prior studies specifically focus on API quality management. Since the behaviour of API is different from traditional software, few additional non-API focused studies are intentionally included and discussed in the literature review to highlight gaps regarding the limited addressing of API-specific NFRs. While APIs are integral components of software applications, however, studies on traditional software applications do not explicitly address API-specific quality aspects. Similarly, a detailed review of seventeen articles presented in Table 1 revealed that the six articles focused on traditional software applications rather than the APIs. Likewise, twelve articles are not aligned with any of the existing quality standards, such as ISO/IEC 25010 whereas, seven articles have shown limited or no validation in their studies. Additionally, six articles recommended that an NFR based quality framework is required explicitly for managing the NFR-based challenges in APIs. Besides, benchmarking in this study is conducted qualitatively to compare conceptual aspects, such as NFR coverage, API coverage, alignment with existing quality standards (ISO/IEC25010) as well as identified limitations with existing studies and frameworks. However, quantitative performance benchmarking against the frameworks and approaches mentioned in the literature would require empirical deployment, which is beyond the scope of this conceptual study. The following Table 2 presents a summarized literature review on NFRs-based studies in accordance with ISO/IEC25010. As shown, there are eleven NFRs namely, Functional Suitability (FS), Performance Efficiency (PE), Compatibility (CY), Interaction Capability (IC), Reliability (RL), Security (SE), Maintainability (MY), Flexibility (FY), Safety (SA), Observability (OY) and Resiliency (RS). Notably, this study is an expansion of our previous work 10 , where 17 studies are summarized in the below mentioned Table 2 . However, 12 articles have been recently identified which were not included in the prior study, whereas 5 are retained to ensure continuity and comparability. Table 2. A Summarized Review of addressed NFR. References List of addressed NFRs FS PE CY IC RL SE MY FY SA OY RS 3 ⊠ ⊠ ⊠ □ ⊠ ⊠ ☑ ⊠ ⊠ ⊠ ⊠ 10 ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ 14 ⊠ ☑ ⊠ ⊠ □ ⊠ ⊠ ⊠ ⊠ ⊠ ⊠ 23 ⊠ ⊠ ⊠ □ ⊠ ⊠ ⊠ ⊠ ⊠ ⊠ ⊠ 34 ⊠ ☑ ⊠ ⊠ ⊠ ⊠ ⊠ ⊠ ⊠ ☑ ⊠ 38 ☑ ☑ ☑ □ ☑ ☑ ☑ □ □ ⊠ ⊠ 42 ☑ ⊠ ⊠ ⊠ ⊠ ⊠ ⊠ ⊠ ⊠ ⊠ ⊠ 47 ☑ ☑ ⊠ ⊠ ☑ ☑ ☑ ☑ ⊠ □ □ 55 ⊠ ☑ ⊠ □ ⊠ ☑ ⊠ ⊠ ⊠ ⊠ ⊠ 56 ⊠ ☑ ⊠ □ ⊠ ⊠ ⊠ ⊠ ⊠ ⊠ ⊠ 57 ⊠ ☑ ⊠ ⊠ ⊠ ⊠ ⊠ ⊠ ⊠ ⊠ ⊠ 58 ⊠ ☑ ⊠ □ ⊠ ☑ ⊠ ⊠ ⊠ ⊠ ⊠ 59 ⊠ ☑ ⊠ □ ⊠ ☑ ☑ ⊠ ⊠ ⊠ ⊠ 60 ⊠ ☑ ⊠ □ ☑ ☑ ☑ ⊠ ⊠ ⊠ ⊠ 61 ☑ ☑ ⊠ ⊠ ☑ ☑ ☑ □ ⊠ ⊠ □ 62 ⊠ ☑ ⊠ ⊠ ⊠ ☑ ⊠ ⊠ ⊠ ⊠ ⊠ 63 ⊠ ☑ ☑ □ ☑ ☑ ☑ ⊠ ⊠ ⊠ ⊠ NFRQF-API ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ Open in a new tab ☑ = Fully Covered, □ = Partially Covered, ⊠ = Not Covered. Table 2 shows that most articles were focusing on PE, SE, MY, as well as FS and RL, but to some extent in their studies. On the other hand, other NFRs, such as CY, SA and particularly the two emerging NFRs, OY and RS, have not been adequately explored. Conversely, the proposed framework, NFRQF-API, covers all the eleven NFRs. In other words, the proposed framework soundly aligns with ISO/IEC25010. Hence, a comprehensive review of existing literature evident in Tables 1 and 2 highlights a consistent need for a practical and systematic framework to address a wide range of NFRs in API development, ensuring their quality and alignment with current quality standards. Hence, this article proposes a conceptual Non-Functional Requirement Quality Framework for APIs to develop high-quality APIs that bridges the earlier-mentioned gaps. Methodology Research design This study adopted an Explanatory Sequential Mixed-Methods Design, as shown in Fig. 1 below, following the guidelines of Creswell and Plano Clark (2017) 64 , who recommended collecting quantitative data first, followed by qualitative data, to evaluate the NFRQF-API. The approach combined the quantitative analysis of software industry expert ratings on the framework and the qualitative insights of the experts on the framework into its practical applicability. This enhanced the understanding of the potential effectiveness and relevance of the framework in the industry of API development. Fig. 1. Open in a new tab A structure of research methodology. Survey design and validation This study employed a survey questionnaire as the data collection instrument. The questionnaire included demographic questions to evaluate the experts’ relevancy, in addition to ten questions to determine the clarity and the practical applicability of the proposed framework (eight Likert-scale questions and two open-ended questions). In order to assure accuracy, clarity, and appropriateness, the first draft of the questionnaire was sent to six experts who were selected based on eligibility criteria (see Appendix A : Expert Selection and Survey Validation Criteria) belonging to the academic research and software industry domains, where both the face validity and content validity 65 , 66 of the questionnaire were evaluated. Furthermore, we also verified their expertise using publicly available profiles, such as institutional webpages, publication histories, and professional records. The face validity and content validity of the questionnaire were ascertained, as detailed below: Face validity The six experts (as mentioned below in Table 3 ) evaluated the face validity 65 of the questionnaire. In particular, the experts evaluated the clarity, layout, overlap, and consistency of the items in the questionnaire. Based on their feedback, one overlapping item was removed, the term “Developer” was changed to “API Developer,” the country options were changed to continent options, and all demographic items were standardized to have five answer options. Table 3. Feedback Summary of Industry Experts on Face Validity. Expert Focus area Action taken Status Expert 1 Item overlap Removed the overlapping question Resolved Expert 2 Option consistency Standardized five options per demographic item Resolved Expert 3 Role label clarity Changed “Developer” to “API Developer” Resolved Expert 4 Region question length Replaced countries with continents Resolved Expert 5 Wording clarity Edited instructions and item wording Resolved Expert 6 Layout and flow Adjusted spacing, headings, and section order Resolved Open in a new tab The following Table 3 presents summarized feedback from the experts on the questionnaire’s face validity. The above-mentioned Table 3 presents the detailed feedback summary of the experts regarding face validity of the questionnaire. Although each expert comment is listed separately in Table 3 , whereas several suggestions addressed the same face-validity concerns, particularly clarity of questionnaire wording, overlapping items, and consistency in layout and flow. Even though, similar feedback was received from multiple experts but is presented individually to ensure transparency of responses. Content validity The content validity of the questionnaire was evaluated by six experts in the industry 65 , 66 . Based on the feedback, two items were added, specifically on the training required and the practical applicability of the framework. One Likert item was revised, with the inclusion of clear applicability of KPI measurement, while one overlapping item was removed. To improve clarity, the open-ended questions were revised, and simple and understandable vocabulary was used for all items. Table 4 below provides a summary of the feedback on the content validity of the questionnaire from the experts. Table 4. Feedback Summary of Industry Experts on Content Validity. Expert Focus area Action taken Status Expert 1 Training requirement Ask whether training is required to implement the framework Added training-related Likert item Expert 2 Practical applicability Ask about the practicality of applying the framework in industry Added practicality-related Likert item Expert 3 Item overlap Remove overlapping questions Merged duplicates into one item Expert 4 KPI clarity and measurability Ask whether KPIs are clear and measurable Amended existing Likert item to “The KPI metrics in the framework are clear and measurable.” Expert 5 Open-ended clarity Rephrase open-ended questions for precision Revised both items in clear language Expert 6 Language clarity Improve readability and consistency Simplified language, standardized terms and grammar. Open in a new tab After revising the questionnaire based on the feedback, the Item-level Content Validity Index (I-CVI) and Scale-level Content Validity Index (S-CVI Ave ) 65 , 66 were computed to ensure the relevance and appropriateness of the questionnaire items before the final distribution, as mentioned in Table 5 below. Table 5. Item-level CVI and Scale-level CVI Ave . Item no. Questionnaire items Expert rating 3 or 4 I-CVI 1. The classification of Non-Functional Requirements (Core, Critical, Contextual) and their mapped factors is relevant to current API quality challenges. 5 0.83 2. The Strategic Level provides clear direction for planning and managing API quality. 5 0.83 3. The operational level (Plan, Develop, Test, Deploy, Retire) addresses NFRs comprehensively and practically. 5 0.83 4. The Quality Assurance Level ensures systematic management of NFRs and their KPIs. 6 1.00 5. The integration of feedback loops across Strategic, Operational, QA levels, and the software industry effectively supports continuous improvement of API quality standards. 5 0.83 6. The KPI metrics for each NFR are clear, measurable, and enhance the framework’s practical applicability. 6 1.00 7. The framework is practical and can be easily implemented in the software industry to support the development of high-quality APIs 6 1.00 8. Training is important for effectively implementing the framework in the API domain. 6 1.00 9. Are the classified NFRs and their factors in the framework appropriate for addressing API quality challenges? If not, what changes or additions would you recommend? 5 0.83 10. What suggestions do you have to further enhance clarity, usability, or practical implementation of the framework in API development? 6 1.00 S-CVI Ave (Average: I-CVIs) 0.92 Open in a new tab Number of Experts (N) = 6. Number of Items (M) = 10. Number of experts rated the item as 3 or 4 (A i ) A i = 5 and N = 6 A i = 6 and N = 6 Based on the expert rating mentioned in Table 5 above, I-CVI i and S-CVI Ave were assessed on the revised questionnaire. Six experts rated each item on a relevant scale of 4 points, where 1 denotes Not relevant, 4 denotes Highly relevant, and the ratings of 2–3 were counted as relevant. All the questionnaire items scored I-CVI i ≥ 0.83. Such value was deemed acceptable for six experts ( N = 6) 65 , 66 . Hence, the items were retained. The scale scored S-CVI Ave ≥ 0.90, demonstrating strong content validity 65 , 66 . Data collection As previously indicated, data were collected from software industry experts through a structured questionnaire, where these experts were recruited from various professional platforms, such as LinkedIn, institutional webpages, and publication records, and invited to participate in the survey via LinkedIn professional groups or direct email invitations. The informed consent was obtained from the experts before their participation, the consent statement was included at the beginning of the online survey. The questionnaire comprising eight Likert-scale items and two open-ended items where the items on Likert scale comprises of 5 levels of agreement as follows: 5 to denote Strongly Agree, 4 to denote Agree, 3 to denote Neutral, 2 to denote Disagree, and 1 to denote Strongly Disagree. For the two open-ended items, they were for ascertaining the efficacy of the framework in the industry, as perceived by the experts. The experts comprised API developers, Quality Assurance Experts, and Software Architects. They were chosen due to their capability in evaluating the potential effectiveness of the proposed framework (NFRQF-API). The aim of the questionnaire was to assess the key aspects, such as the clarity and flow of the structure of the framework, NFR coverage, and also the practical pertinence of the framework to the API industry. As API quality engineering is a one of the highly specialized domains therefore, overall, 63 responses were collected. However, based on the strict guidelines (see Appendix A ) recommended by Palinkas et al. 67 , 49 of the collected responses were considered usable. The samples were excluded based on the limited industrial experience of participants with less than 3 years, who have no practical knowledge of development, as well as a lack of knowledge about NFRs and quality standards. Where on the other hand, this sample size exceeds the typical threshold of existing API experts validation studies, where the samples remain under 30 3 , 38 . Thus, the achieved sample size is sufficiently rich for the validation of framework. Data analysis IBM SPSS Statistics (Version 21.0) was employed for data analysis 68 . The software is available at https://www.ibm.com/products/spss-statistics . Cronbach’s alpha based on the guidelines of Malapane and Ndlovu 69 , Tavakol and Dennick 70 was applied in determining the internal consistency of the eight Likert-scale items, to validate the proposed framework. Cronbach’s alpha of quantitative data from 49 valid responses was computed and the resulting Cronbach’s alpha coefficient for the scale was α = 0.831, denoting good internal consistency 69 , 70 . Notably, the eight Likert-scale items were treated as a single scale, because these items jointly measure the perceptions of the respondents on the clarity, usability, comprehensiveness, and practical applicability of the proposed framework. Furthermore, the option “alpha if item deleted” in SPSS affirmed that no individual item decreased the overall reliability. The reliability evaluation did not include open-ended questions. Subsequently, the descriptive statistics using IBM SPSS Statistics were run on the responses of the eight Likert-scale items to compute the mean and standard deviation values. The guidelines from Boone and Boone 71 and George & Mallery 72 were followed. On the other hand, thematic analysis was performed on the responses of the respondents on the two open-ended questions. This was to obtain clarity and deeper understanding from the feedback in case of further refinement in framework. The analysis involved six phases proposed by Nowell et al. 73 and Braun and Clarke 74 . The phases were: Familiarization with the data (Phase 1), Generating initial codes (Phase 2), Searching for themes (Phase 3), Reviewing the themes (Phase 4), Defining and naming the themes (Phase 5), and Producing the report (Phase 6). Results Quantitative results By following the guidelines of “Boone & Boone, 2012” 71 and “Miller and Salkind, 2002” 75 , an interpretation based on the mean and standard deviation was included in Table 6 . Table 6. Mean and Standard Deviation. Sr. no Survey items Mean Standard deviation Interpretation 1. The classification of Non-Functional Requirements (Core, Critical, Contextual) and their mapped factors is relevant to current API quality challenges. 4.39 0.67 High Importance, High Consensus 2. The Strategic Level provides clear direction for planning and managing API quality. 4.20 0.61 High Importance, High Consensus 3. The Operational Level (Plan, Develop, Test, Deploy, Retire) addresses NFRs comprehensively and practically. 4.29 0.79 High Importance, High Consensus 4. The Quality Assurance Level ensures systematic management of NFRs and their KPIs. 4.47 0.65 High Importance, High Consensus 5. The integration of feedback loops across Strategic, Operational, QA levels, and the software industry effectively supports continuous improvement of API quality standards. 4.47 0.77 High Importance, High Consensus 6. The KPI metrics for each NFR are clear, measurable, and enhance the framework’s practical applicability. 4.10 0.82 Moderate Importance, Moderate Consensus 7. The framework is practical and can be easily implemented in the software industry to support the development of high-quality APIs. 4.39 0.64 High Importance, High Consensus 8. Training is important for effectively implementing the framework in the API domain. 4.47 0.58 High Importance, High Consensus Open in a new tab Results of descriptive data analysis, in the form of mean and standard deviation, can be viewed in Table 6 . As shown, the proposed framework achieved desirable feedback from the industry experts (the study respondents). In particular, the mean values of items on classification of NFRs, strategic direction, operational coverage, and quality assurance mechanisms were greater than 4.20, with low standard deviations. The values demonstrated high importance and strong accord among respondents (industry experts). Furthermore, the integration of feedback loops and the practical applicability of the framework received high ratings. This further supported the relevance of the proposed framework to API quality management. Item on KPI clarity and measurability received positive score, but of moderate importance and consensus (mean value of 4.10 and a standard deviation of 0.82). The obtained results generally demonstrated high significance, practicality, and feasibility of the proposed framework in the software industry. Results also showed training as a crucial factor for successful adoption. Besides, the survey items were designed to assess the relevance and clarity of the framework based on expert agreement, which may limit the extent of divergent responses. Hence, the potential response bias is acknowledged, whereas this limitation is further discussed in Sect.  5 (Threats to Validity and Limitations). Qualitative results Qualitative data from the responses on open-ended questions were analyzed through thematic analysis. The analysis involved six phases (see Fig. 2 ) based on Braun and Clarke 74 and Nowell et al. 73 . The approach was appropriate for this study as it supported a systematic and rigorous identification of patterns in expert responses. The aspects that were identified were clarity, usability, and practical relevance of the NFRQF-API framework proposed in this study. Hence, the frequent aspects were highlighted by the experts regarding the frameworks’ clarity as well as need for clearer KPI integration, such as “Label the levels to clearly specify the flow direction” , “Appropriate labeling of arrow directions can enhance readability , ” and “Reconsider the arrow directions and overall framework flow” . In addition, experts emphasized the importance of conceptual KPI-related guidance, such as “the framework may seem more practical in future deployments if KPI measurement tools are included” , “commonly used measurement tools would significantly improve practical applicability” , and “KPIs should be clearly defined at the Quality Assurance level.” One expert further suggested that metrics such as “ API uptime could be monitored using commonly available monitoring platforms , mentioned as illustrative examples (e.g. , Datadog or similar tools)” . Fig. 2. Open in a new tab A detailed process of thematic analysis. Thematic analysis process Based on the qualitative findings, the six phases of thematic analysis process are presented below: Phase 1: familiarization with the data Initially, the responses from all the respondents (the experts) were repeatedly read. This was to gain a thorough understanding of their insight. From the reading, several key points were identified. The identified key points were: “improve the clarity of data flow labels,” “enhance diagram readability,” “provide the relevant arrow labelling,” “use of measurement tools for KPIs” and “Add KPI testing tools.” Phase 2: generating initial codes Subsequently, the researcher systematically coded all the responses, where the significant aspects of the data were highlighted. The codes were assigned as follows: “framework flow,” “diagram readability,” “arrow labelling,” “KPI measurement tools,” and “KPI testing tools.” Phase 3: searching for themes Following this, connected codes were placed together into broader categories, while codes on labelling and readability were placed together into a candidate theme labelled “Visual clarity and flow.” The “KPI testing tools” code was combined into the candidate theme called “KPI measurement tools.” Phase 4: reviewing the themes In order to assure internal consistency, the defined themes were prudently reviewed. Codes that were overlapping were merged together, while codes with insignificant contribution but were contributing to broader themes, were added into its respective theme. In this context, the “framework flow” code was placed under “Visual clarity and flow”. Phase 5: defining and naming the themes Thereafter, the two last themes were clearly defined, where the themes were given simple and profound names, to generate clear understanding. “Visual clarity and flow,” which was the first theme, displays the suggestions made by the experts in improving the readability of the framework diagram. Among the suggestions were to include labels and clarify the directions of the arrows to further enhance the coherency of framework. “KPIs measurement tools” as the second theme, shows the suggestions of the experts to link KPIs to relevant practical monitoring and testing tools. Through such names and definitions, it could be assured that each theme focuses on a certain aspect of the experts’ feedback. Phase 6: producing the report In this phase, which is the ending phase, the themes were finalized by providing with comprehensive descriptions. This was to show how the feedback from experts guided the researcher in refining the proposed framework. The “Visual clarity and flow” theme was labelled. Also, clear direction was provided to facilitate the understanding of the diagram. For the “KPIs measurement tools” theme, the KPI sample tools were added into the QA layer, as per the suggestion of the experts. The above-mentioned Fig. 2 illustrates the six-phase thematic analysis process applied in this study. Through this six-phase approach, actionable refinements could be made, resulting in enhanced clarity, usability, and practical implementation of the NFRQF-API framework. Details of the refinements are displayed in the revised framework (see expert-refined framework in Appendix C ). Expert-based refinement through integrated data analysis Results of both quantitative and qualitative data analyses were combined to generate an in-depth understanding of the potential effectiveness and perceived practical significance of the proposed NFRQF-API framework. In particular, quantitative data demonstrated patterns of agreement or disagreement of the respondents, and these patterns of agreement or disagreement were justified by the qualitative data to confirm that why certain attributes were rated highly or poorly as well as to achieve comprehensive justifications based on the efficacy and industry implementation of framework. Although the qualitative data were limited to two open-ended questions, which were designed to support the quantitative findings by capturing experts’ feedback through open-ended responses rather than conducting follow-up interviews or focus group discussions. Later, the thematic coding was applied systematically to identify recurring patterns and refine the framework. In this study, the use of both quantitative and qualitative approaches allowed the validation of the general clarity and practical significance of the proposed framework and the identification of areas to be improved based on the feedback obtained from the experts. Accordingly, Table 7 provides a summary of the feedback provided by the experts during the validation process. Meanwhile, the pre-validated framework, along with the diagram, is included in Appendix B . Table 7. Feedback summary on framework validation. Expert feedback Action taken Final remarks Visual clarity and flow : A few of the experts observed that the framework diagram requires more clarity in the direction of arrows and labels. Diagram layout simplified, arrow labels revised for consistency, and captions added to clarify relationships between levels. Successfully addressed KPI measurement tools : Experts have requested to add a few examples of commonly used KPI measurement tools, especially at the QA level, such as GitHub Insights, Datadog, Postman, and OWASP ZAP. The proposed KPI measurement tools were incorporated, and the quality assurance level has also been updated. Successfully addressed Language clarity : Minor edits are suggested to ensure consistency in terminology and improve readability across the framework’s levels. Revised wording throughout the framework to ensure consistent terminology and simplify phrasing for improved readability. Successfully addressed Open in a new tab The revised guidelines based on the experts’ feedback, such as visual clarity and flow of arrow directions, addition of KPI measurement tools. Moreover, the illustrative KPIs are provided to demonstrate the operational practice of NFRs, such as Code Change Rate (CCR) for maintainability, Mean Time to Recovery (MTTR) for resiliency, and Trace Completion Rate (TCR) for observability 76 , 77 . To further illustrate practical monitoring mechanisms, the commonly used supporting tools (for example, GitHub analytics features, observability platforms such as Datadog, and performance monitoring utilities) are referenced exclusively for conceptual illustration, without employing empirical validation. In line with the conceptual scope of the framework, the KPIs are provided as indicative examples and are intended to be adapted to specific organizational and technological contexts. Furthermore, KPI related refinements including the language clarity across the levels of framework have been evident in the expert-refined version (see Appendix C ). Expert-refined framework (NFRQF-API) The comprehensive quality guidelines supporting each NFRs are presented in Appendix C (Expert-refined version). However, the expert-refined NFRQF-API framework is visually presented in the following Fig. 3 . The framework comprises three main levels, namely, Strategic level, Operational level, and Quality Assurance level. The expert-refined version combines the feedback from the experts and is directly used by the Strategic level for quality guidelines definition and by the Quality Assurance level for ensuring KPI alignment across the API development lifecycle. Thus, the proposed framework as expert-refined version, supports API quality management practices by structuring the identification and addressing of NFRs along-with their influencing factors across the development lifecycle. Consequently, through the alignment of NFRs with conceptually defined KPIs, the framework provides a systematic base for monitoring key performance aspects during API design. Although the framework has not been empirically applied in a real-world environment, its detailed conceptual quality guidelines are presented in Appendix C to support future practical implementation and empirical validation. Compared to the pre-validated version, the expert-refined version of the framework has better structure, visual clarity, and KPI integration as detailed below: Fig. 3. Open in a new tab A diagrammatic view of NFRQF-API (Expert-refined version). Strategic level For this level, input is obtained from the NFRs, where the NFRs are classed into the following categories: Core (MY, RS) 78 – 80 , Critical (FS, PE, CY, RL, SE, OY) 78 , 79 , 81 and Contextual (IC, FY, SA) 79 , 82 , 83 . These classifications were recommended by the experts and also in studies like Shabbir in 2025 10 . Not only that, NFR classification becomes the base for planning and implementation to assure early and judicious preparation for the Core and Critical NFRs during APIs development. Quality standards are clearly outlined in accordance with KPI evaluations. Meanwhile, roles are assigned within the team for the management and review of quality-related activities. Next, the developed guidance and quality standards at this level are incorporated into the operational level during API design. Consistent feedback from the Quality Assurance Level and industry experts (internal QA validation and external industry feedback) improves the quality standards of the strategic level. Operational level This level aligns with the lifecycle of API development that includes the phases of Planning, Developing, Testing, Deploying, and Retiring. During operational level, the guidance and quality standards developed at the strategic level are implemented, with the primary purpose of systematically addressing NFRs, the inducing factors and their KPIs all through the development phases of API. This would assure the development of APIs of high quality aligned with the framework. Quality assurance level The purpose of this level is to assure that NFRs as well as their inducing factors are addressed appropriately all through the lifecycle of API development. Quality assurance also provides verification on the appropriate implementation of KPIs at the Operational Level. Furthermore, through its immediate feedback, quality assurance facilitates corrective actions. Additionally, to assure that the newly-developed framework-aligned APIs fully meet the expected quality standards to support output validation. As part of long-term quality planning, QA findings are reported to the strategic level in quality performance reviews and feedback. Illustrative application of the framework (NFRQF-API) The expert-refined version of NFRQF-API is demonstrated across the API development lifecycle to conceptually present its applicability in real-world API contexts. As NFRQF-API is structured into three major levels, where the strategic level is responsible for early planning, defining quality guidelines based on the selected KPIs and assigning relevant team roles prior to the development. While during the Operational level, these guidelines are conceptually applied at each phase of API lifecycle, including planning, developing, testing, deployment, until the retirement by following the common CI/CD practices. Later, the Quality Assurance level verifies the consistent implementation of these guidelines and KPIs to ensure that targeted NFRs are effectively addressed. Hence, to strengthen the practical clarity of the proposed framework, Table 8 presents the selected NFRs with illustrative KPIs, including their definitions, indicative measurement approaches, and assessment criteria. These KPIs along with supporting tools are incorporated at a conceptual level to illustrate how quality attributes can be systematically assessed within the NFRQF-API framework. The provided examples focus on key structural and runtime aspects of APIs, while the comprehensive NFR guidelines are provided in the Appendix C . Table 8. KPI-based Representation of selected NFRs within the NFRQF-API Framework. NFR Illustrative KPI Definition Indicative measurement approach (with Supporting Tools) Indicative assessment Maintainability (MY) Code change rate (CCR) Percentage of code changes relative to total codebase Code changes ÷ Total codebase × 100 (Tool: GitHub Insight) CCR < 10% indicates high maintainability Resiliency (RS) Mean time to recovery (MTTR) < 1 h. Time requires to restore API functionality after failure Total downtime ÷ Number of incidents (Tool: Datadog) MTTR < 1 h (context dependent) Observability (OY) Trace completion rate (TCR) Percentage of requests successfully traced across the system Completed traces ÷ Total requests × 100 (Tool: Datadog) TCR > 90% indicates strong observability Open in a new tab The inclusion of selected KPI-based NFRs in Table 8 provides structured guidance for assessing API quality within the proposed framework. While the KPIs are presented as illustrative examples, however, they enhance the practical applicability of the framework without varying its conceptual scope. Subsequently, maintainability as a core NFR, is supported through the use of Code Change Rate (CCR) as an illustrative KPI to provide structural stability and adaptability. At the strategic level, maintainability requirements are defined and aligned with modular design principles. At the operational level, development practices aim to design the APIs with clear and modular code that ensures the CCR remains below 10%. While the quality assurance level verifies the systematically addressing of maintainability factors along with the KPI throughout the API design phases. Similarly, resiliency is represented through Mean Time to Recovery (MTTR) as an illustrative KPI to support in restoring the API functionality after failure. The strategic level defines the resiliency requirements to assure system robustness. At the operational level, recovery mechanisms and error-handling strategies are incorporated across the API lifecycle, where the quality assurance level verifies that the defined resiliency factors and associated KPIs are systematically addressed throughout the API design phases. Besides, Trace Completion Rate (TCR) as an illustrative KPI is incorporated in observability which support the monitoring and traceability strategies across the system. At the strategic level, observability requirements are defined to ensure visibility of API operations, where the tracing and logging mechanisms are integrated across the API development lifecycle at the operational level. Meanwhile, the quality assurance level verifies the consistent addressing of defined observability factors and associated KPIs throughout the API design phases to ensure the completeness and reliability of system traces. Overall, the measurement approaches presented for each KPI are illustrative and provide conceptual guidance for systematically assessing the NFRs within the proposed framework. Therefore, the NFRQF-API is a refined framework where KPIs and quality standards are consistently improved through constant feedback from the Strategic, Operational, and QA levels. This illustrative application demonstrates the conceptual feasibility of the expert-refined framework, while empirical validation in real-world environments is proposed as a direction for future work. Figure 3 accordingly displays the diagram of the expert-refined framework. Discussion and implications Results demonstrated strong accord among the study respondents (the experts in the industry) on the overall structure and practical significance of the NFRQF-API framework. ISO/IEC 25010, as one of the available standards, presents quality guidelines for general software quality. Somehow, it does not provide API-specific guidance, and it also ineffectual in dealing with emergent API related attributes such as observability and resiliency. Meanwhile, the OpenAPI Specification (OAS), which is an API quality standard, merely focuses on functional aspects and interoperability, but it does not offer inclusive NFR management support, where NFRQF-API proposed in this study addresses these issues directly. Specifically, it classifies NFRs into Core, Critical, and Contextual categories, with measurable KPIs and continuous feedback loops embedded to it. This study employed an explanatory sequential mixed-methods design. It facilitated this study in showing how quantitative validation and afterwards qualitative analysis, can strongly prove the adaptability of the proposed framework across various contexts of API development. However, one of the 5-point Likert scale based-items related to the implication of KPI metrics in frameworks’ practicality achieved a positive mean score of 4.10 but showed “Moderate consensus”. Later the thematic analysis of the open-ended question revealed the full agreement of experts on the clarity and relevance of the NFRQF-API with the suggestion of inclusion of KPI measurement tools which were incorporated in the expert-refined version of the framework. Hence the item with “Moderate consensus” accompanying by open-ended suggestions successfully addressed in the expert-refined version of NFRQF-API. Overall, results of quantitative analysis affirmed the general aptness of the framework, while results of qualitative analysis facilitated the study in identifying the ways to improve the framework, specifically, by improving the visual clarity and by outlining the KPI measurement tools. The proposed framework was refined after modifications and its effectiveness for potential industry adoption was improved. In theoretical implication, the proposed framework in this study expands the ISO/IEC 25010 with the inclusion of two emergent NFRs into the API domain. It also addresses the limitations in the existing standards, like in OAS that only addresses the API-specific FRs, where OAS was selected as a reference specification because it is widely adopted as a standardized model for API description. The NFRQF-API framework is intentionally designed as a technology-agnostic and architecture-independent. Subsequently, to maintain the technology-agnostic nature of the framework, the referenced KPI tools are provided as illustrative examples and are not mandatory for the framework, however, they are intended to be adapted to specific organizational and technological contexts. From a practical perspective, the expert-refined framework classes eleven NFRs into three categories, namely Core, Critical, and Contextual categories. Such classification supports a more comprehensive approach to define and implement the conceptual quality guidelines at the Strategic, Operational, and Quality Assurance levels and assures the framework to address diverse API architectures (such as, centralized versus distributed, monolithic versus microservice-based), API technologies (like, REST, GraphQL, gRPC, WebSocket), as well as various API types (private, public, composite). In addition, while Service Level Objectives (SLOs) and Service Level Agreements (SLAs) are typically defined at the operational stage, the proposed framework supports their early alignment by structuring NFR identification and KPI integration during the design phase. This supports clearer and more consistent SLO and SLA definition after real-time deployment of the framework. Consequently, the framework provides a solid bridge between scholarly research and industry practice. Threats to validity and limitations Even though the NFRQF-API has been refined based on the experts’ feedback, however, this study has several limitations. Notably, the proposed framework could not be fully deployed for real-world testing. Still, this study provides an opportunity for testing the conceptually validated framework in real-world API projects and assessing the adaptableness of the framework across various contexts of software engineering. Accordingly, future research may focus on empirically defining and validating SLO and SLA targets through real-world deployment of the framework. Similarly, the KPI definitions and measurement approaches presented in this study are illustrative and provide the conceptual guidance for operationalizing NFRs within the proposed framework. Additionally, the study faces potential threats to validity as the validation methodology relies on experts-based perception rather than empirical experimentation. Furthermore, the survey may not capture all aspects of API quality due to the limited extent of divergent responses, so the findings may have limited generalizability beyond the selected experts. These limitations are acknowledged, and future studies are encouraged to empirically validate the framework by employing more discriminative items after implementing it in real-world API environments. Conclusion This study presents an expert-refined Non-Functional Requirement Quality Framework for APIs (NFRQF-API) to significantly manage API development practices. The framework aims to address the most critical NFRs and their influencing factors systematically. Likewise, it provides clear quality standards, defined KPIs, and constant feedback mechanisms through three levels namely Strategic level, Operational level, and Quality Assurance level. Additionally, the internal QA validation and external industry feedback are the two mechanisms that strengthen the foundation of NFRQF-API by providing the consistent refinement of its quality standards. The Explanatory Sequential Mixed-Methods Design was the approach employed in this study, involving the evaluation of quantitative data from eight Likert-scale items to ascertain the clarity, structure, feasibility and applicability of the framework. Whereas, the evaluation of qualitative data from two open-ended questions was used to determine the adequateness of the classified NFRs and their inducing factors in addressing API quality challenges. The experts also offered some suggestions to improve the clarity, usability, and applicability of the framework. Quantitative and qualitative analysis results reflect positive experts’ perceptions of the framework’s clarity and structural soundness within the API domain. Additionally, the feedback by the experts allowed the researcher to refine the framework, especially through the inclusion of measurable KPI indicators in addition to the improvement in visual flow of the framework. Besides, the launching of expert-refined NFRQF-API framework in this study, bridged the gap in the extant literature. In particular, the framework combines a robust theoretical base with practical guidance, which offers validated conceptual quality guidelines to facilitate the software industry in the management of NFR-related issues as well as support the researchers in their scholarly works on APIs. Hence, this comprehensive study and the industry feedback have resulted in NFRQF-API framework with the ability to effectively guide both researchers and industry experts in improving API quality management practices. Acknowledgements We express our sincere gratitude to the Universiti Malaysia Terengganu, the Ministry of Higher Education Malaysia, and Gulf College, as well as the Ministry of Higher Education, Research and Innovation - Sultanate of Oman, for their essential research support. We would also appreciate the Multimedia University, Malaysia for providing the funding support through their Article Page Charge Sponsorship Scheme. Appendices Appendix A Expert selection and survey validation criteria. Appendix B Pre-validated framework. Appendix C Expert-refined framework. Author contributions We, the undersigned, declare that this manuscript, “A Novel NFR-Based Quality Framework for the Modern API Industry”, is original, has not been published previously, and is not under consideration for publication elsewhere. We confirm that all listed authors have read and approved the manuscript and that no other individuals who meet the criteria for authorship are not listed. We also confirm that the order of authors has been agreed upon by all of us. We understand that the Corresponding Authors are the sole contact for the editorial process and is responsible for communicating with the other authors regarding the overall progress of the submission, from submission to publication. 1. Aumir Shabbir: Writing—review & editing, Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Validation, Visualization, Writing—original draft. 2. Aziz Deraman: Writing—review & editing, Conceptualization, Methodology, Project administration, Supervision, Validation. 3. Mohamad Nor Bin Hassan: Supervision, Writing—review & editing, Validation. 4. Kamal Uddin Sarker: Supervision, Writing—review & editing, Validation. 5. Shahid Kamal: Writing—review & editing, Conceptualization, Methodology, Project administration, Supervision, Validation. Data availability The data generated and analyzed during the current research is available on reasonable request to the corresponding author. Declarations Competing interests The authors declare no competing interests. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Contributor Information Aziz Deraman, Email: a.d@umt.edu.my. Shahid Kamal, Email: shahid.kamal@mmu.edu.my. References 1. Godinho, A., Rosado, J., Sá, F. & Cardoso, F. Method for evaluating the performance of web-based apis. In International Conference on Smart Objects and Technologies for Social Good , pp. 30–48 10.1007/978-3-031-52524-7_3 (Springer, 2023). 2. Lercher, A., Glock, J., Macho, C. & Pinzger, M. Microservice api evolution in practice: a study on strategies and challenges. J. Syst. Softw. 215 , 112110. 10.1016/j.jss.2024.112110 (2024). [ Google Scholar ] 3. Raatikainen, M. et al. State of the practice in application programming interfaces (APIs): A case study. In European Conference on Software Architecture pp. 191–206. 10.1007/978-3-030-86044-8_14 (Springer, 2021). 4. Andreo, S. & Bosch, J. API management challenges in ecosystems. In Eur. Commission, Louxembourg, Louxembourg, UK, Tech. Rep. JRC118082 , pp. 86–93. 10.1007/978-3-030-33742-1_8 (2019). 5. Andreo, S. & Bosch, J. API Management challenges in ecosystems. In Software Business: 10th International Conference, ICSOB Jyväskylä, Finland, November 18–20, 2019, Proceedings 10 . pp. 86–93. 10.1007/978-3-030-33742-1_8 (Springer, 2019). 6. Ahmad, M., Geewax, J. J., Macvean, A., Karger, D. & Ma, K. L. API Governance at scale. In Proceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice . pp. 430–440. 10.1145/3639477.3639713 (ACM, 2024). 7. Chu, S., Koe, J., Garlan, D. & Kang, E. Integrating graceful degradation and recovery through requirement-driven adaptation. 2024 IEEE/ACM 19th Symp. Softw. Eng. Adapt. Self-Managing Syst. , pp. 122–132, (2024). 10.1145/3643915.3644090 8. Rajput, P. & Sikka, G. Multi-agent architecture for fault recovery in self-healing systems. J. Ambient Intell. Humaniz. Comput. 12 , 2849–2866. 10.1007/s12652-020-02443-8 (2020). [ Google Scholar ] 9. Scrocca, M., Tommasini, R., Margara, A., Della Valle, E. & Sakr, S. The Kaiju project: enabling event-driven observability. Proc. 14th ACM Int. Conf. Distrib. Event-based Syst. . 10.1145/3401025.3401740 (2020). 10. Shabbir, A., Deraman, A., Bin Hassan, M. N., Sarker, K. U. & Kamal, S. Prioritizing non-functional requirements and influencing factors for API quality framework: An industry approach. Int. J. Adv. Comput. Sci. Appl. 16 (9). 10.14569/IJACSA.2025.0160934 (2025). 11. Cinque, M., Della Corte, R. & Pecchia, A. Microservices monitoring with event logs and black box execution tracing. IEEE Trans. Serv. Comput. 15 , 294–307. 10.1109/TSC.2019.2940009 (2019). [ Google Scholar ] 12. Zhang, J. et al. DBFT: A byzantine fault tolerance protocol with graceful performance degradation. IEEE Trans. Dependable Secur. Comput. 19 , 3387–3400. 10.1109/TDSC.2021.3095544 (2021). [ Google Scholar ] 13. Cho, H., Sung, J. H., Kang, H. J., Jang, J. & Shin, D. Quantifying cyber resilience: a framework based on availability metrics and AUC-based normalization. Electronics 14 (12), 2465. 10.3390/electronics14122465 (2025). [ Google Scholar ] 14. Bucaille, S., Cánovas, J. L., Izquierdo, H., Ed-Douibi & Cabot, J. An OpenAPI-based testing framework to monitor non-functional properties of REST APIs. In International Conference on Web Engineering . pp. 533–537. 10.1007/978-3-030-50578-3_39 (Springer, 2020). 15. ISO, I. S. O. & IEC 25010:. Online Browsing Platform (OBP). https://www.iso.org/obp/ui/#iso:std:iso-iec:25010:ed-2:v1:en (2023). 16. Shabbir, A., Deraman, A., Nor Bin Hassan, M., Sarker, K. U. & Kamal, S. Enhancing API quality: A comprehensive review of non-functional requirements for quality-centric framework development. KSII Trans. INTERNET Inf. Syst. 20 (2), 23 10.3837/tiis.2026.02.011 (2026). 17. Zou, J., Xu, L., Yang, M., Zhang, X. & Yang, D. Towards comprehending the non-functional requirements through developers’ eyes: An exploration of stack overflow using topic analysis. Inf. Softw. Technol. 84 , 19–32. 10.1016/j.infsof.2016.12.003 (2017). [ Google Scholar ] 18. Murphy, L., Kery, M. B., Alliyu, O., Macvean, A. & Myers, B. A. API designers in the field: Design practices and challenges for creating usable APIs. In IEEE Symposium on Visual Languages and Human-Centric computing (vl/hcc) . pp. 249–258. 10.1109/VLHCC.2018.8506523 (IEEE, 2018). 19. Ehsan, A., Abuhaliqa, M. A. M. E., Catal, C. & Mishra, D. RESTful API testing methodologies: Rationale, challenges, and solution directions. Appl. Sci. 12 (9), 4369. 10.3390/app12094369 (2022). [ Google Scholar ] 20. dos Santos, E. & Casas, S. Unveiling quality in API management: A systematic mapping study. In L Latin American Computer Conference (CLEI) pp. 1–10 10.1109/CLEI64178.2024.10700447 (IEEE, 2024). 21. Sadi, M. & Yu, E. WEBAPIK: a body of structured knowledge on designing web APIs. Requir Eng. 28 , 441–479. 10.1007/s00766-023-00401-2 (2023). [ Google Scholar ] 22. Tanveer, F., Iradat, F., Iqbal, W. & Ahmad, A. Towards secure APIs: A survey on RESTful API vulnerability detection. Comput. Mater. Contin . 84 (3), 4223. 10.32604/cmc.2025.067536 (2025). [ Google Scholar ] 23. Meng, M., Steinhardt, S. & Schubert, A. Application programming interface documentation: What do software developers want? J. Tech. Writ. Commun. 48 (3), 295–330. 10.1177/0047281617721853 (2018). [ Google Scholar ] 24. Singjai, A. Deriving empirically grounded NFR specifications from practitioner discourse: A validated methodology applied to trustworthy APIs in the AI Era. Information 17 (3), 304. 10.3390/info17030304 (2026). [ Google Scholar ] 25. Ali, A., Khalil, I., Ahmad, I., Parveen, I. & Zaman, U. K. U. Role of Non-functional Requirements in projects’ success, 2nd Int. Conf. Digit. Futur. Transform. Technol. 1–7, 10.1109/ICoDT255437.2022.9787463 (2022). 26. Gupta, V., Chopra, R. & Chauhan, D. Nonfunctional requirements in mobile application development. 10.1201/9781003160922 (2021). 27. Khan, F., Jan, S. R., Tahir, M., Khan, S. & Ullah, F. Survey: Dealing non-functional requirements at architecture level. 9 , 7–13 10.21015/vtse.v9i2.410 (2016). 28. Albuquerque, C. & Correia, F. F. Tracing and metrics design patterns for monitoring cloud-native applications. arXiv (2025). 29. Ahmed, H. et al. Observability in microservices: An in-depth exploration of frameworks, challenges. IEEE Access 13 , 72011–72039 10.1109/ACCESS.2025.3562125 (2025). 30. Usman, M., Ferlin, S., Brunstrom, A. & Taheri, J. A survey on observability of distributed edge & container-based microservices. IEEE Access. 10 , 86904–86919. 10.1109/ACCESS.2022.3193102 (2022). [ Google Scholar ] 31. Gorton, I., Fong-Jones, L. & Larsson, A. Observability Q&A. IEEE Softw. 41 , 50–54. 10.1109/MS.2023.3330234 (2024). [ Google Scholar ] 32. Kratzke, N. & Observability, C. N. The many-faceted benefits of structured and Unified logging - A multi-case study. Futur Internet . 14 , 274. 10.3390/fi14100274 (2022). [ Google Scholar ] 33. Lima, S., Correia, J., Araújo, F. & Cardoso, J. Improving observability in event sourcing systems. J. Syst. Softw. 181 , 111015. 10.1016/j.jss.2021.111015 (2021). [ Google Scholar ] 34. Sharma, B. & Nadig, D. eBPF-enhanced complete observability solution for cloud-native microservices. ICC 2024 - IEEE Int. Conf. Commun. 1980–1985. 10.1109/ICC51166.2024.10622329 (2024). 35. Becker, K. & Voss, S. Analyzing graceful degradation for mixed critical fault-tolerant real-time systems. IEEE 18th Int. Symp. Real-Time Distrib. Comput. . pp. 110–118 10.1109/ISORC.2015.10 (2015). 36. Rouholamini, S. R., Mirabi, M., Farazkish, R. & Sahafi, A. Proactive self-healing techniques for cloud computing: A systematic review. Concurr Comput. Pract. Exp. (2024). 37. Mohammed, B., Kiran, M., Maiyama, K., Kamala, M. & Awan, I. Failover strategy for fault tolerance in cloud computing environment. Softw. Pract. Exp. 47 , 1243–1274. 10.1002/spe.2491 (2017). [ Google Scholar ] 38. Reboucas Dos Santos, E. & Casas, S. Api management quality and iso/iec square: a practitioners’ perspective. (2024). 39. Stefani, A. & Vassiliadis, B. ISO25000-related metrics for evaluating the quality of complex information systems. J. Comput. Commun. 10.4236/jcc.2023.119002 (2023). [ Google Scholar ] 40. Muhamad, W. Designing semantic web service based on OAS 3.0 through relational database. In 2020 International Conference on Information Technology Systems and Innovation (ICITSI) . pp. 306–311. 10.1109/ICITSI50517.2020.9264930 (IEEE, 2020). 41. Espinoza-Arias, P., Garijo, D. & Corcho, O. Mapping the web ontology language to the openapi specification. In International Conference on Conceptual Modeling . pp. 117–127. 10.1007/978-3-030-65847-2_11 (Springer, 2020). 42. Banias, O., Florea, D., Gyalai, R. & Curiac, D. Automated specification-based testing of REST APIs. Sens. (Basel) . 21 10.3390/s21165375 (2021). [ DOI ] [ PMC free article ] [ PubMed ] 43. Yu, L., Alégroth, E., Chatzipetrou, P. & Gorschek, T. Utilising CI environment for efficient and effective testing of NFRs. Inf. Softw. Technol. 117 10.1016/j.infsof.2019.106199 (2020). 44. Yu, L., Alégroth, E., Chatzipetrou, P. & Gorschek, T. Automated NFR testing in continuous integration environments: a multi-case study of Nordic companies. Empir. Softw. Eng. 28 , 1–29. 10.1007/s10664-023-10356-1 (2023). [ Google Scholar ] 45. Habibullah, K., Gay, G. & Horkoff, J. A framework for managing quality requirements for machine learning-based software systems. pp. 3–20 10.1007/978-3-031-70245-7_1 (2024). 46. Jadoon, G. Preserving non-functional requirements in goal models using meta-models of the software product lines. Proc. 28th ACM Int. Syst. Softw. Prod. Line Conf. 10.1145/3646548.3676541 (2024). [ Google Scholar ] 47. Sherif, E., Helmy, W. & Galal-Edeen, G. H. Proposed framework to manage non-functional requirements in agile. IEEE access. 11 , 53995–54005. 10.1109/ACCESS.2023.3281195 (2023). [ Google Scholar ] 48. Viviani, L., Guerra, E., Melegati, J. & Wang, X. An empirical study about the instability and uncertainty of non-functional requirements. In International Conference on Agile Software Development . pp. 77–93. 10.1007/978-3-031-33976-9_6 (Springer, 2023). 49. Kumar, D., Kumar, A. & Singh, L. Non-functional requirements elicitation in agile base models. Webology 19 (1), 1992–2018. 10.14704/WEB/V19I1/WEB19135 (2022). [ Google Scholar ] 50. Pargaonkar, S. A comprehensive review of performance testing methodologies and best practices: software quality engineering. Int. J. Sci. Res. 12 , 2008–2014. 10.21275/SR23822111402 (2023). [ Google Scholar ] 51. Koçi, R., Franch, X., Jovanovic, P. & Abelló, A. Web api evolution patterns: A usage-driven approach. J. Syst. Softw. 198 , 111609. 10.1016/j.jss.2023.111609 (2023). [ Google Scholar ] 52. Lamothe, M., Guéhéneuc, Y. G. & Shang, W. A systematic review of API evolution literature. ACM Comput. Surv. 54 , 1–36. 10.1145/3470133 (2021). [ Google Scholar ] 53. Erme, H. Challenges of quality assurance of software developed by a subcontractor: a case study. (2024). 54. De, B. Api architecture trends in 2023. In API Management: An Architect’s Guide to Developing and Managing APIs for Your Organization . 385–411. 10.1007/979-8-8688-0054-2_16 (Springer, 2023). 55. Ofoeda, J., Boateng, R. & Effah, J. Application Programming Interface (API) Research. Int. J. Enterp. Inf. Syst. 15 (3), 76–95. 10.4018/IJEIS.2019070105 (Jul. 2019). 56. Das, S., Deb, N., Chaki, N. & Cortesi, A. Minimising conflicts among run-time non‐functional requirements within DevOps. Syst. Eng. 27 (1), 177–198. 10.1002/sys.21715 (2024). [ Google Scholar ] 57. Santos, S., Pimentel, T., Rocha, F. G. & Soares, M. Using behavior-driven development (BDD) for non-functional requirements. 10.20944/preprints202406.1061.v1 (2024). 58. Muhammad, A., Siddique, A., Mubasher, M., Aldweesh, A. & Naveed, Q. N. Prioritizing non-functional requirements in agile process using multi criteria decision making analysis. IEEE Access. 11 , 24631–24654. 10.1109/ACCESS.2023.3253771 (2023). [ Google Scholar ] 59. Mathijssen, M., Overeem, M. & Jansen, S. Identification of practices and capabilities in API management: a systematic literature review. arXiv .10481, 2020. (2006). 60. Martinez, G. G., Del Carpio, Á. F. & Gómez, L. N. A model for detecting conflicts and dependencies in non-functional requirements using scenarios and use cases. In XLV Latin American Computing Conference (CLEI) . pp. 1–8. 10.1109/CLEI47609.2019.235051 (IEEE, 2019). 61. Scheibmeir, J. & Malaiya, Y. An API development model for digital twins. In IEEE 19th International Conference on Software Quality, Reliability and Security Companion (QRS-C) . pp. 518–519. 10.1109/QRS-C.2019.00103 (IEEE, 2019). 62. Bermbach, D. & Wittern, E. Benchmarking web api quality-revisited. J. Web Eng. 19 , 5–6. 10.13052/jwe1540-9589.19563 (2020). [ Google Scholar ] 63. Castro, C. F. et al. Towards a conceptual framework for decomposing non-functional requirements of business process into quality of service attributes. In ICEIS -Proceedings of the 21st International Conference on Enterprise Information Systems . pp. 481–492 10.5220/0007656504810492 (SciTePress, 2019). 64. Plano Clark, V. L. Mixed methods research. J. Posit. Psychol. 12 (3), 305–306. 10.1080/17439760.2016.1262619 (2017). [ Google Scholar ] 65. Saw, Z. K. et al. Forward-backward translation, content validity, face validity, construct validity, criterion validity, test-retest reliability, and internal consistency of a questionnaire on patient acceptance of orthodontic retainer. PLoS One . 20 (1), e0314853. 10.1371/journal.pone.0314853 (2025). [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 66. Polit, D. F. & Beck, C. T. The content validity index: are you sure you know what’s being reported? Critique and recommendations. Res. Nurs. Health . 29 (5), 489–497. 10.1002/nur.20147 (2006). [ DOI ] [ PubMed ] [ Google Scholar ] 67. Palinkas, L. A. et al. Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Adm. policy Ment Heal Ment Heal Serv. Res. 42 (5), 533–544. 10.1007/s10488-013-0528-y (2015). [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 68. IBM Corp. IBM Statistical Package for the Social Sciences (IBM SPSS), Version 21.0. https://www.ibm.com/products/spss-statistics 69. Malapane, T. A. & Ndlovu, N. K. Assessing the reliability of Likert scale statements in an e-commerce quantitative study: A Cronbach alpha analysis using SPSS Statistics. In systems and information engineering design symposium (sieds) pp. 90–95. 10.1109/SIEDS61124.2024.10534753 (IEEE, 2024). 70. Tavakol, M. & Dennick, R. Making sense of Cronbach’s alpha. Int. J. Med. Educ. 2 , 53. 10.5116/ijme.4dfb.8dfd (2011). [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 71. Boone, H. N. Jr & Boone, D. A. Analyzing likert data. J. Ext. 50 (2), 48. 10.34068/joe.50.02.48 (2012). [ Google Scholar ] 72. George, D. & Mallery, P. IBM SPSS statistics 29 step by step: A simple guide and reference 10.4324/9781032622156 (Routledge, 2024). 73. Nowell, L. S., Norris, J. M., White, D. E. & Moules, N. J. Thematic analysis: Striving to meet the trustworthiness criteria. Int. J. Qual. methods . 16 (1), 1609406917733847. 10.1177/1609406917733847 (2017). [ Google Scholar ] 74. Braun, V. & Clarke, V. Using thematic analysis in psychology. Qual. Res. Psychol. 3 (2), 77–101. 10.1191/1478088706qp063oa (2006). [ Google Scholar ] 75. Miller, D. C. & Salkind, N. J. Handbook of research design and social measurement 10.4135/9781412984386 (Sage, 2002). 76. Figueira, J. & Coutinho, C. Developing self-adaptive microservices. Procedia Comput. Sci. 232 , 264–273. 10.1016/j.procs.2024.01.026 (2024). [ Google Scholar ] 77. Souza, A. Observability and monitoring. In Tech Leadership Playbook: Building and Sustaining High-Impact Technology Teams . 171–191. 10.1007/979-8-8688-0543-1_7 (Springer, 2024). 78. Bogart, C., Kästner, C., Herbsleb, J. & Thung, F. How to break an API: cost negotiation and community values in three software ecosystems. In Proceedings of the 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering . pp. 109–120. 10.1145/2950290.2950325 (2016). 79. dos Santos, E., Casas, S., An API management software quality metamodel based on square and GQM. Eng. e soluç oes Ciência e Tecnol. para o Desenvolv. Hum. , pp. 249–266 10.22533/at.ed.30225210317 (2025). 80. Dragoni, N. et al. Microservices: yesterday, today, and tomorrow. Present ulterior Softw. Eng. , pp. 195–216, 10.1007/978-3-319-67425-4_12 (2017). 81. Alshuqayran, N., Ali, N. & Evans, R. A systematic mapping study in microservice architecture. In IEEE 9th international conference on service-oriented computing and applications (SOCA) . pp. 44–51 10.1109/SOCA.2016.15 (IEEE, 2016). 82. Rauf, I., Troubitsyna, E. & Porres, I. A systematic mapping study of API usability evaluation methods. Comput. Sci. Rev. 33 , 49–68. 10.1016/j.cosrev.2019.05.001 (2019). [ Google Scholar ] 83. Mosqueira-Rey, E., Alonso-Ríos, D., Moret-Bonillo, V., Fernández-Varela, I. & Álvarez-Estévez, D. A systematic approach to API usability: Taxonomy-derived criteria and a case study. Inf. Softw. Technol. 97 , 46–63. 10.1016/j.infsof.2017.12.010 (2018). [ Google Scholar ] Associated Data This section collects any data citations, data availability statements, or supplementary materials included in this article. Data Availability Statement The data generated and analyzed during the current research is available on reasonable request to the corresponding author. Articles from Scientific Reports are provided here courtesy of Nature Publishing Group ACTIONS View on publisher site PDF (2.7 MB) Cite Collections Permalink RESOURCES Similar articles Cited by other articles Links to NCBI Databases Follow NCBI NCBI on X (formerly known as Twitter) NCBI on Facebook NCBI on LinkedIn NCBI on GitHub NCBI RSS feed Connect with NLM NLM on X (formerly known as Twitter) NLM on Facebook NLM on YouTube National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894 Web Policies FOIA HHS Vulnerability Disclosure Help Accessibility Careers NLM NIH HHS USA.gov Back to Top

Related documents

• CatRO: A Category-Theoretic Formalization of Staged Concept Formation for Symbol Grounding#204211
• Homo Cyfranicus: Ontological Reductions and Semantic Sovereignty within the Closed System#695091
• A Domain-Driven Design Simulator for Business Logic-Rich Microservice Systems#155259
• Meaning Preserving Error Correction - A Symbolic Framework for Semantic Communication#167078
• SpiralOS®: Volume V – Emergent Interface - World, Witness, and Spiral Infrastructure#704937