Construction of a public health emergency information system framework: A case study of Zhuhai city, China.

Construction of a public health emergency information system framework: A case study of Zhuhai city, China - PMC Skip to main content An official website of the United States government Here's how you know Here's how you know Search Log in Primary site navigation Search Log in Search PMC Full-Text Archive Search in PMC Journal List User Guide As a library, NLM provides access to scientific literature. Inclusion in an NLM database does not imply endorsement of, or agreement with, the contents by NLM or the National Institutes of Health. Learn more: PMC Disclaimer | PMC Copyright Notice PLoS One . 2026 Jun 5;21(6):e0350917. doi: 10.1371/journal.pone.0350917 Construction of a public health emergency information system framework: A case study of Zhuhai city, China Sicheng Huang 1 , Xuebao Zhang 1 , Long Chen 1 , Xihe Ni 1 , Ying Fan 1 , Chaomin Zhao 1 , Junfeng Xiao 2 , Feng Ruan 3, * Editor: Pasyodun Koralage Buddhika Mahesh 4 Author information Article notes Copyright and License information 1 Department of Emergency Management, Zhuhai Center for Disease Control and Prevention, Zhuhai, Guangdong, China 2 Institute of Public Health and Research, Zhuhai Center for Disease Control and Prevention, Zhuhai, Guangdong, China 3 Center Leadership Office, Zhuhai Center for Disease Control and Prevention, Zhuhai, Guangdong, China 4 Ministry of Health, Sri Lanka, SRI LANKA ✉ * E-mail: zhcdcyjb@163.com Competing Interests: The authors declare no competing interests. Roles Sicheng Huang : Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Writing – original draft, Writing – review & editing Xuebao Zhang : Investigation, Resources Long Chen : Investigation Xihe Ni : Investigation Ying Fan : Investigation Chaomin Zhao : Investigation Junfeng Xiao : Software Feng Ruan : Writing – review & editing Pasyodun Koralage Buddhika Mahesh : Editor Received 2026 Jan 10; Accepted 2026 May 20; Collection date 2026. © 2026 Huang et al This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. PMC Copyright notice PMCID: PMC13240892  PMID: 42247382 Abstract Background A public health emergency information system serves as a critical tool for collecting and analyzing data from sudden public health events, thereby providing a scientific basis for governmental decision-making. However, research on the systematic construction of such information system frameworks within China’s public health infrastructure is lacking. Objective Taking Zhuhai city as a case study, this study aims to construct a comprehensive public health emergency information system framework applicable to public health departments at the municipal, county, and street/township levels. Methods First, through a literature review and expert group discussion, the preliminary framework of system indicators is determined. Second, through two rounds of the Delphi method, 41 experts are invited to qualitatively select the system framework indicators, with the aim of obtaining consensus among experts. Finally, the system is improved through application, feedback, and redesign. Results After two rounds of consultation, the final system at the city and county levels consists of 5 first-level indicator modules and 21 second-level indicator modules, whereas the system at the city, county, and street/township levels consists of 4 first-level indicator modules and 17 second-level indicator modules. Most of the indicators in the “emergency preparedness” and “emergency response” modules are considered important and should be retained as they can play a role in collecting and analysing information on infectious disease outbreaks through practical applications. Conclusion The public health emergency information system constructed in this study can be applied to public health departments such as disease prevention and control centres. Promotion can improve the efficiency of handling infectious disease outbreaks and provide a scientific basis for decision-making analysis. Background A public health emergency information system (PHEIS) is a technology and research method that uses epidemiology, statistics, and computer science to collect real-time and reliable data during public health emergencies, providing the best solution for epidemic control [ 1 ]. It is mainly divided into two parts; the first part consists of information data collection, cleaning, preprocessing, while the second part consists of management, and data analysis and interpretation [ 2 ]. Since the COVID-19 epidemic, the role of PHEISs has become increasingly prominent. When an epidemic breaks out, timely and accurate data collection is crucial for commanders to make decisions; however, it is often difficult to obtain such data. With the increasing nature and complexity of emergency management for infectious diseases, the processes of data collection, management, and reporting face challenges. In recent years, research on information collection and management systems in the field of public health has focused mainly on chronic disease monitoring in hospitals [ 3 ] and community clinical practice [ 4 ]; this approach has certain limitations in regard to promoting these methods in the field of epidemic response. In addition, there have been studies that have focused on information collection and management systems for certain infectious disease outbreaks, such as the African Ebola epidemic information system [ 5 ], the application of Brazil’s national health information system in collecting Zika epidemic information [ 2 ], and the application of Liberia’s comprehensive disease surveillance system in collecting Ebola epidemic information [ 6 ]. However, these studies have focused mainly on a specific infectious disease and thus have limitations in their application to other infectious disease outbreaks and public health emergencies. In terms of research methods, most of the abovementioned studies have used literature reviews [ 2 ], expert experience, and interviews [ 6 ] to introduce the construction process of information systems, thus demonstrating a lack of scientific research methods. Alternatively, such studies have focused on the introduction of the construction process of mathematical statistical models [ 7 ] and computer programming fields [ 1 , 8 ], thereby demonstrating a lack of elaboration in the field of epidemiology. In summary, there is currently a lack of research on PHEISs in the field of infectious disease outbreaks. At present, many studies on the construction of PHEISs, both domestically and internationally, have been based on official policy documents and relevant literature for system framework construction [ 6 ]; most of these studies have focused on only national-, provincial-, and city-level system construction. In recent years, with the establishment of street/township-level disease control centres in many cities in China, frontline disposal work has gradually shifted to include district- and street-level public health departments. Therefore, it is necessary to build PHEISs at the city, district, and township/street levels so that the systems can achieve vertical commands from top to bottom in daily operation. Zhuhai is a prefecture-level city located in Guangdong Province, China; it had a permanent population of 2.4941 million at the end of 2023. After the outbreak of COVID-19 in 2020, Zhuhai established five district centres for disease control and prevention. In 2024, Zhuhai city established 21 additional street/township-level disease control centres, and front-line disposal work began to gradually shift to district- and street-level disease control departments. Therefore, this study aims to simultaneously construct PHEISs at the Zhuhai city and district levels, as well as at the city, district, and street/township levels, to meet current needs. Most of the research on the construction of PHEISs, both at home and abroad, uses official policy documents and relevant literature to construct systematic frameworks [ 3 ] that collaborate closely with stakeholders (government, frontline public health professionals, engineers) during the system design phase for joint development. However, this method has a strong level of subjectivity in the qualitative selection stage of the indicator framework. To address the shortcomings of this method, in this study, we combine this stage with the Delphi method for construction. The Delphi method can be applied to establish various evaluation index systems and determine specific indicators. Through several rounds of feedback, we fully utilize and absorb the experience and knowledge of the experts, gradually converging their opinions. The process of combining the expert experience method and the Delphi method is currently widely used in the construction of health information systems [ 9 ]. Method Design We aimed to combine the opinions and needs of numerous stakeholders to provide an optimal solution. In terms of participating in system design, stakeholders within the system (leaders, emergency management experts, and onsite disposal personnel of the city and district centres for disease control and prevention) were invited to conduct research and develop a preliminary framework, which includes components and a list of elements for system design [ 9 ]. Ethics approval and informed consent This study was approved by the Medical Research Ethics Committee of the Zhuhai Center for Disease Control and Prevention in Guangdong Province, China. All the research procedures complied with the ethical standards of the institutional research committee. Owing to the extremely low risk of this study and the absence of a routine written consent procedure, the Institutional Review Board (IRB) granted a waiver of written informed consent for the subjects. The study invitation letter detailed the purpose and process of the study, and experts’ completion and submission of an online questionnaire is considered their voluntary informed consent to participate. This study was conducted in strict accordance with the Declaration of Helsinki. Modified e-Delphi approach This study employed a modified e-Delphi approach. The design and reporting of the study were guided by the core principles of the Conducting and REporting DElphi Studies (CREDES) guidelines [ 10 ]. In contrast to the classical Delphi method—which typically commences with an open-ended qualitative round to generate an initial item pool—a modified Delphi method uses a structured questionnaire derived from a systematic literature review as the basis for the first round of consultation. All the consultation procedures were conducted electronically via email using Excel-based survey forms, and no face-to-face meetings were held. Experts were permitted to reevaluate and revise their initial ratings within one week of submission. During this study, we invited 41 experts in the field of public health in the selected city to not only screen and revise indicators through two rounds of the Delphi method [ 11 ] but also determine indicators and weights. Expert eligibility criteria and sampling strategy To ensure the professionalism and authority of the expert consultation and considering the public health emergency response context of this study, the following inclusion criteria were established: (1) Professional title and education: Experts were required to hold at least an intermediate professional title in a relevant field or possess a postgraduate degree or higher. (2) Years of experience: A minimum of two years of cumulative professional experience in public health, emergency management, or related fields was needed. (3) Role and responsibilities: Eligible participants included frontline response personnel and managerial staff from the emergency response departments of centers for disease control and prevention at various administrative levels. Individuals were excluded if they failed to meet any of the above criteria, did not respond to the invitation within the specified timeframe (21 days), or submitted an incomplete first-round questionnaire that remained incomplete after a reminder was sent. A purposive sampling strategy was employed for expert selection. Through routine professional contacts and daily work interactions, the research team identified individuals who served as responders and managers for public health emergency events at various levels of centers for disease control and prevention in advance. These individuals were then evaluated against the predefined eligibility criteria, and those meeting all the inclusion requirements were purposefully selected and invited to participate as core experts. Expert recruitment process First, we constructed a preliminary indicator framework. The data sources of this study included the framework of domestic and international infectious disease epidemic information collection systems, the framework of Guangdong Province’s health emergency information system, government documents such as the emergency response plan and technical scheme for public health emergencies in Zhuhai city, and national health emergency work standards for disease prevention and control institutions. After this round of modifications, a preliminary indicator framework was developed. There were two rounds of Delphi consultation, and we invited 41 experts within Zhuhai city (frontline emergency response workers, health emergency management personnel, and health decision-makers at the city and district levels) to participate in our research. Forty-one of them agreed to participate. To ensure a diversity of perspectives, the selection of source institutions followed the principles of hierarchical and geographic representativeness, encompassing (1) the Zhuhai Municipal Center for Disease Control and Prevention, (2) all the district-level centers for disease control and prevention in Zhuhai; and (3) selected street/township-level centers for disease control and prevention. Following the designation of these institutions, individuals from relevant departments responsible for public health emergency response and management were invited to participate. All the experts were contacted via formal email invitations. The invitation letter provided detailed information regarding the study background and the Delphi procedure. Before the formal invitation was sent, potential participants were verbally informed of the voluntary nature of participation, with explicit assurance that nonparticipation would entail no adverse consequences and that no administrative mandates were employed to compel involvement. The purpose of this recruitment process was to facilitate expert evaluation of the modules comprising the preliminary framework, thereby determining whether specific components and elements within each module warranted further development. In the first round of consultation, we evaluated the expert authority coefficient (Cr). The Cr is determined in two dimensions, namely, the familiarity coefficient (Cs) and the judgement coefficient (Ca). The former represents the expert’s familiarity with the survey, whereas the latter reflects the evidence of the expert’s judgement [ 12 , 13 ], as shown in Table 1 . The Cr can be calculated using the formula Cr = (Cs + Ca)/2. Usually, a Cr value higher than 0.7 is considered an acceptable indicator of reliability [ 14 ]. Table 1. Judgement basis and familiarity for consultation from experts. Judgement basis (Cs) Degree Familiarity (Ca) Value Great deal Moderate Little Theoretical analysis 0.30 0.20 0.10 Very familiar 0.90 Practical experience 0.50 0.40 0.30 More familiar 0.70 Referring to the literature 0.10 0.10 0.10 Average 0.50 Intuition 0.10 0.10 0.10 Less familiar 0.30 Unfamiliar 0.10 Open in a new tab In addition, the experts rated the importance and feasibility of the first- and second-level indicators. According to a Likert-scale format [ 15 ], scores were divided into five levels ranging from high to low (5 = very important, 1 = very unimportant). After the data were collected in the first round, the mean importance score, feasibility score, and coefficient of variation (CV) for both the first- and second-level indicators were calculated, and the indices that did not meet the mean value of importance and feasibility assignment higher than 3.5 and whose coefficient of variation lower than 0.25 were eliminated [ 16 ]. If CV > 0.25, then the research team conducted a comprehensive analysis of importance and feasibility to determine whether to retain or delete the corresponding project [ 14 ]. Moreover, public suggestions were adopted, and at least two experts’ suggestions were selected as new indicators. Prior to the initiation of the second round of consultation, the research team provided all the experts with a structured feedback report summarizing the results of the first round. The report consisted of two components: (1) Statistical summary of item screening outcomes. The report explicitly listed the indicators that had been removed following the first round, accompanied by the rationale for their deletion—specifically, failure to satisfy the predefined retention criteria, namely, a mean importance score below 3.5 or a CV exceeding 0.25. (2) New indicators derived from expert suggestions. The research team systematically collated and thematically synthesized all the textual suggestions submitted by the experts through open-ended questions during the first round. Any content that was independently proposed by at least two experts in identical or highly similar form was incorporated as a new indicator into the second-round questionnaire. Throughout the process of organizing and presenting these expert suggestions, any information that could reveal individual expert identities—such as institutional affiliation, specific job title, or distinctive phrasing—was either removed or generalized to ensure strict adherence to the principle of anonymity required by the Delphi method. In the second round, the revised indicators were distributed. The distribution objects consisted of the experts who provided feedback in the first round, and the evaluation content included the average importance score, feasibility score, and coefficient of variation of the first- and second-level indices. The indices that did not meet the mean value of importance and feasibility assignment higher than 3.5 and a coefficient of variation lower than 0.25 were eliminated [ 16 ]. Decision rules for item retention and exception handling If an indicator had a CV > 0.25 but simultaneously satisfied both of the following conditions—(1) the research team determined that the item represented a conceptually indispensable core element of the construct, and (2) the mean score was ≥3.5—it could be retained as an exception, accompanied by an explicit justification, to mitigate subjective bias [ 16 ]. Testing and redesign After completing the opinion summary and developing the system indicator framework, all of the abovementioned requirements were fed back to engineers for development. We tested and applied the usability and operability of the system, regularly collected opinions from users, discussed any problems encountered, and then modified and improved the system on the basis of valuable information according to the needs of all parties, ultimately achieving the goal of maximizing the system’s effectiveness. Results We used two rounds of the Delphi method to revise the preliminary framework of the information system indicators. Basic information This study distributed 41 questionnaires in two rounds of the Delphi survey and collected 31 questionnaires, for an effective response rate of 75.6%. In this study, 31 experts at the city and district levels (100%) had at least intermediate or higher professional titles or graduate degrees or above. Fifteen people (48.4%) had a postgraduate degree or above, and 7 people (22.6%) held a vice senior professional title or above. The experts consisted of frontline personnel or emergency management experts with an average of 8.6 (2–19) years of experience in public health work (see Table 1 ). These descriptive statistics indicate that the basic advice of the experts should be helpful. The questionnaire response rate was relatively high, with a positive coefficient of 75.6% for the two rounds of experts. The familiarity level of all the experts was 0.51, the judgement coefficient was 0.93, and the Cr was 0.72 (>0.7) [ 17 ], indicating that the expert consultation results were accurate and reliable. See Tables 2 and 3 . Table 2. Population statistics of Delphi survey experts. Classification Number(N) Percentage(%) Gender Male 12 38.7 Female 19 61.3 Age (years) 21–30 10 32.3 31–40 17 54.8 41–50 4 12.9 51–60 0 0 Education Undergraduate 16 51.6 Postgraduate 14 45.2 Doctor 1 3.2 Title Junior 5 16.1 Intermediate 19 61.3 Deputy Senior 6 19.4 Senior 1 3.2 Professional field Emergency response 21 67.7 Contingency management 10 32.3 Work experience (years) 0–10 22 71.0 11–20 9 29.0 21–31 0 0 Open in a new tab Table 3. Results of two rounds of expert discussions (PHEIS at the city and district levels). First-level indicator Second-level indicator Importance Feasibility Comprehensive Score (mean ± standard deviation) CV Score (mean ± standard deviation) CV Score (mean ± standard deviation) CV 1. Emergency preparedness 1.1 Emergency duty (information) 4.58 ± 0.85 0.19 4.45 ± 0.81 0.18 4.52 ± 0.64 0.14 1.2 Institutional management 4.39 ± 0.99 0.23 4.23 ± 0.99 0.23 4.31 ± 0.86 0.20 1.3 Team management 4.48 ± 0.89 0.20 4.32 ± 1.05 0.24 4.40 ± 0.89 0.20 1.4 Material management 4.39 ± 1.02 0.23 3.87 ± 1.28 0.33* 4.13 ± 0.96 0.23 1.5 Knowledge materials 4.68 ± 0.60 0.13 3.81 ± 1.19 0.31* 4.24 ± 0.66 0.15 1.6 Form management 4.55 ± 0.77 0.17 4.10 ± 1.11 0.27* 4.32 ± 0.74 0.17 1.7 Scheduling management 4.58 ± 0.81 0.18 4.48 ± 0.81 0.18 4.53 ± 0.62 0.14 2. Monitoring and early warning 2.1 Daily monitoring 4.61 ± 0.80 0.17 4.19 ± 0.83 0.20 4.40 ± 0.65 0.15 2.2 Information monitoring 4.65 ± 0.66 0.14 3.94 ± 1.09 0.28* 4.29 ± 0.75 0.17 3. Emergency response 3.1 Event registration 4.61 ± 0.88 0.19 4.26 ± 1.12 0.26* 4.44 ± 0.91 0.21 3.2 Task management 4.71 ± 0.74 0.16 4.29 ± 1.01 0.23 4.50 ± 0.71 0.16 3.3 Disposal feedback 4.71 ± 0.64 0.14 4.29 ± 0.97 0.23 4.50 ± 0.71 0.16 3.4 Event correction 4.55 ± 0.85 0.19 4.39 ± 0.99 0.23 4.47 ± 0.86 0.19 3.5 Sudden reporting 4.71 ± 0.78 0.17 4.39 ± 0.84 0.19 4.55 ± 0.71 0.16 3.6 Event report 4.81 ± 0.40 0.08 4.06 ± 1.21 0.30* 4.44 ± 0.67 0.15 3.7 Investigation summary 4.45 ± 0.77 0.17 4.39 ± 0.84 0.19 4.42 ± 0.74 0.17 3.8 Response termination 4.55 ± 0.89 0.20 4.45 ± 0.85 0.19 4.50 ± 0.80 0.18 4. Event and reserve management 4.1 Event query and management 4.65 ± 0.71 0.15 3.90 ± 1.04 0.27* 4.27 ± 0.77 0.18 4.2 Knowledge data statistics and query 4.65 ± 0.80 0.17 3.61 ± 1.28 0.36* 4.13 ± 0.87 0.21 5. System management 5.1 Organizational structure 4.13 ± 1.15 0.28* 3.71 ± 1.24 0.34* 3.92 ± 0.95 0.24 5.2 Module configuration 4.19 ± 1.14 0.27* 3.65 ± 1.20 0.33* 3.92 ± 0.92 0.24 Open in a new tab * Retained based on conceptual importance; see Methods section for exception criteria. Results of two rounds of expert consultation The initial two sets of PHEISs had the same primary indicator framework (S1–S3 Tables in S1 File ). In the first round of the survey, the average score for the importance of first-level indicators in the PHEIS framework was 4.31 (3.58–4.74), with a CV of 0.22 (0.14–0.39). The average feasibility score was 3.67 (3.29–3.92), with a CV of 0.29 (0.20–0.35). According to the calculation results, the indicator “1. Daily office” did not meet the requirements (feasibility score less than 3.50). Although the feasibility score of “5. Decision Analysis” was less than 3.50; considering its importance, this indicator was temporarily retained. In the second round of the survey, the average score for the importance of first-level indicators in the PHEIS framework at the city and district levels was 4.51 (4.13–4.90), with a CV of 0.16 (0.06–0.26). The average feasibility score was 3.89 (3.39–4.26), with a CV of 0.25 (0.16–0.29). The average score for the importance of the first-level indicators in the PHEIS framework at the city, district, and town levels was 4.29 (3.90–4.74), with a CV of 0.25 (0.16–0.33); the average feasibility score was 3.70 (3.55–4.00), with a CV of 0.30 (0.26–0.33). While the feasibility score of “5. Reserve Management” was less than 3.50, owing to the existence of second-level indicators that had not yet been removed, the 2 first-level indicators of “event management” and “resource management” were merged. In the PHEIS framework at the city and district levels, in the first round of consultation, the average importance score of 50 second-level indicators was 4.36 (3.58–4.84), with a CV of 0.21 (0.09–0.36), and the the average feasibility score was 3.62 (2.76–4.37), with a CV of 0.33 (0.18–0.48). A total of 26 indicators that did not meet the feasibility assignment mean higher than 3.5 were excluded, and 3 second-level indicators were deleted because of the deletion of first-level indicators; thus resulted in a total of 2 first-level indicators and 29 second-level indicators being removed. In the second round of consultation, the average importance score of the 21 second-level indicators was 4.55 (4.13–4.81), with a CV of 0.18 (0.08–0.28), and the average feasibility score was 4.13 (3.61–4.48), with a CV of 0.25 (0.18–0.36); no indicators needed to be removed. In the PHEIS framework at the city, district, and town levels, the average importance score of 33 second-level indicators was 4.21 (3.42–4.68), with a CV of 0.26 (0.15–0.43) and the average feasibility score was 3.71 (3.00–4.13), with a CV of 0.34 (0.25–0.48). One indicator that did not meet the importance assignment mean higher than 3.5 and 11 indicators that did not meet the feasibility assignment mean higher than 3.5 were excluded. In addition, 3 second-level indicators were deleted because of the deletion of first-level indicators; this resulted in a total of 2 first-level indicators and 14 second-level indicators being removed. In the second round of consultation, the average importance score of the 19 second-level indicators was 4.35 (4.10–4.61), with a CV of 0.24 (0.18–0.31), and the average feasibility score was 3.96 (3.39–4.39), with a CV of 0.30 (0.22–0.40); this indicated that the expert opinions tended to be consistent. Two indicators that did not meet the feasibility assignment mean higher than 3.5 were excluded, and 1 first-level indicator was deleted due to the removal of second-level indicators; the 2 first-level indicators with similar functions of “event management” and “reserve management” were merged. On the basis of two rounds of expert opinions, a city- and district-level PHEIS framework consisting of 5 first-level indicators and 21 second-level indicators was ultimately formed. The main indicators are as follows: (1) emergency preparedness: arrangement of on-duty personnel, material reserves, and management of knowledge materials; (2) monitoring and early warning: daily monitoring of infectious diseases and epidemic monitoring [ 18 ]; (3) emergency response: the entire process ranging from epidemic registration, on-site verification and disposal, incident reporting to case closure; (4) event and reserve management: management and retrieval of event information, knowledge materials, and other related content; (5) system management: setting up and assigning permissions for backend system accounts; see Table 3 . In addition, city-, district-, and street/township-level PHEIS framework indicators consisting of 4 first-level indicators and 17 second-level indicators were formed; see Table 4 . Table 4. Results of two rounds of expert discussions (PHEIS at the city, district, and town/street levels). First-level indicator Second-level indicator Importance Feasibility Comprehensive Score (mean ± standard deviation) CV Score (mean ± standard deviation) CV Score (mean ± standard deviation) CV 1. Emergency preparedness 1.1 Emergency duty (information) 4.52 ± 0.89 0.20 4.39 ± 0.99 0.23 4.45 ± 0.90 0.20 1.2 Institutional management 4.23 ± 1.18 0.28* 3.97 ± 1.30 0.33* 4.10 ± 1.02 0.25 1.3 Team management 4.39 ± 0.99 0.23 4.06 ± 1.24 0.30* 4.23 ± 1.04 0.25 1.4 Knowledge materials 4.61 ± 0.84 0.18 3.87 ± 1.31 0.34* 4.24 ± 0.91 0.21 1.5 Scheduling management 4.29 ± 1.10 0.26* 4.03 ± 1.17 0.29* 4.16 ± 0.97 0.23 2. Monitoring and early warning 2.1 Daily monitoring 4.32 ± 1.14 0.26* 4.00 ± 1.13 0.28* 4.16 ± 1.06 0.25 2.2 Information monitoring 4.35 ± 1.02 0.23 3.55 ± 1.41 0.40* 3.95 ± 1.08 0.27* 3. Emergency response 3.1 Event registration 4.29 ± 1.10 0.26* 4.19 ± 1.05 0.25 4.24 ± 0.98 0.23 3.2 Task management 4.39 ± 0.99 0.23 4.10 ± 1.16 0.28* 4.24 ± 0.96 0.23 3.3 Disposal feedback 4.45 ± 0.99 0.22 4.06 ± 1.12 0.28* 4.26 ± 0.98 0.23 3.4 Event correction 4.39 ± 1.09 0.25 4.19 ± 1.14 0.27* 4.29 ± 1.03 0.24 3.5 Sudden reporting 4.52 ± 1.00 0.22 4.26 ± 1.06 0.25 4.39 ± 0.93 0.21 3.6 Event report 4.68 ± 0.83 0.18 3.94 ± 1.26 0.32* 4.31 ± 0.91 0.21 3.7 Investigation summary 4.32 ± 0.98 0.23 4.19 ± 1.05 0.25 4.26 ± 0.96 0.23 3.8 Response termination 4.45 ± 0.93 0.21 4.29 ± 0.94 0.22 4.37 ± 0.88 0.20 4. Event and reserve management 4.1 Event query and management 4.19 ± 1.14 0.27* 3.68 ± 1.08 0.29* 3.94 ± 0.88 0.22 4.2 Knowledge data statistics and query 4.13 ± 1.23 0.30* 3.55 ± 1.36 0.38* 3.84 ± 1.05 0.27* Open in a new tab * Retained based on conceptual importance; see Methods section for exception criteria. Concentration and coordination of expert opinions In this study, we calculated the concentration and coordination of expert opinions [ 19 ] on various indicators (the average score and coefficient of variation of second-level indicators). After two rounds of expert consultation, the average score for the importance rating of the second-level indicators of PHEIS at the city and district levels included in the project increased from 4.47 (3.89 to 4.84) to 4.55 (4.13 to 4.81), and the CV decreased from 0.09–0.35 to 0.08–0.28. The comprehensive score of feasibility evaluation increased from 3.99 (3.55 to 4.37) to 4.13 (3.61 to 4.48), and the CV decreased from 0.18–0.42 to 0.18–0.36. The average comprehensive score increased from 4.23 (3.72 to 4.55)-4.34 (3.92 to 4.55), and the CV decreased from 0.10–0.36 to 0.14–0.24. See Table 3 . For the second-level indicators of the city-, district-, and town-level systems, the average importance score of the included projects decreased from 4.40 (4.03 to 4.68) to 4.38 (4.13 to 4.68), and the CV increased from 0.15–0.30 to 0.18–0.30. The comprehensive score of feasibility evaluation increased from 3.90 (3.50 to 4.13) to 4.02 (3.55 to 4.39), and the CV decreased from 0.25–0.40 to 0.22–0.40. The average comprehensive score increased from 4.15 (3.87 to 4.36)-4.20 (3.84 to 4.45), and the CV decreased from 0.18–0.29 to 0.20–0.27. The concentration of expert opinions was found to be relatively high; see Table 4 . The Kendall harmony coefficient and coefficient of variation were used to measure the level of coordination between expert opinions [ 20 ]. The Kendall harmony coefficient ranges from 0 to 1, with higher values indicating a higher degree of coordination. In the PHEIS framework, the results revealed that after two rounds of expert consultation, the Kendall harmony coefficient decreased from 0.140–0.218 to 0.061–0.165, p < 0.01, and the difference was statistically significant; see Table 5 . Table 5. Coordination degrees of expert opinions in two rounds. Importance Feasibility Comprehensive score W chi-square p W chi-square p W chi-square p The first-round PHEIS at the municipal and district levels 0.140 259.943 <0.00 0.218 405.851 <0.00 0.184 342.563 <0.00 The first-round PHEIS at the city, district, and town street levels 0.187 227.669 <0.00 0.173 210.727 <0.00 0.170 206.616 <0.00 Second-round PHEIS at the municipal and district levels 0.072 44.915 <0.00 0.134 82.863 <0.00 0.127 78.518 <0.00 Second-round PHEIS at the city, district, and town street levels 0.061 33.970 0.01 0.150 83.490 <0.00 0.165 92.131 <0.00 Open in a new tab Testing and redesign After the development was completed, we continued to improve the system while using it by collecting feedback from users. From January 1 to November 30, 2024, during the process of use and improvement, we utilized information systems to handle 536 outbreaks in Zhuhai city. The emergency system provided significant assistance for onsite disposal through functions such as reporting and registration and emergency response. Discussion The first-level indicator with the highest expert ratings is “emergency response”, with comprehensive scores of 4.58 and 4.37 for the city/district-level system and the city/district/town-level system, respectively. Emergency response includes the entire process of investigating infectious disease outbreaks, and through information technology, the efficiency of onsite response by emergency team members has improved. This module has the highest utilization rate in daily use; therefore, its importance and operability are recognized by experts. The comprehensive scores of “emergency preparedness” in the two systems are 4.34 and 4.16, making it the second most commonly used module. Emergency preparedness is the foundation of health emergency work, including the construction of personnel teams, the reserve of various emergency materials, the preparation of knowledge materials, the development of emergency plans, and holding regular training and drills on a daily basis. Only when the abovementioned preparations are fully implemented can resources be immediately mobilized for disposal in the event of an epidemic. “Monitoring and early warning” is the third most commonly used module in the system. This function is reflected mainly to the collection of information related to sudden public health emergencies through multiple channels, such as hospitals’ online reporting of legally recognized infectious diseases, school doctors’ reporting of student symptom monitoring data, media information monitoring, etc., to discover relevant information that may halt infectious disease outbreaks in a timely manner and intervene in the early disposal of the epidemic. In the second round of expert consultation, the mean comprehensive scores increased, and the CVs decreased; however, the Kendall coefficient of concordance (W) decreased from 0.140–0.218 in the first round to 0.061–0.165 in the second round. This phenomenon may be explained by the following two considerations: (1) Indicator screening and reduced discrimination. The first-round consultation included 50 and 33 second-level indicators, respectively, and expert ratings exhibited considerable variability across items, enabling Kendall’s W to effectively capture the consistency of rank ordering among panel members. Following the first-round screening, only 21 and 17 highly consistent second-level indicators were retained in the second round, and the experts assigned high scores of 4 or 5 to the majority of these items. When nearly all ratings are concentrated within a narrow high-score band, the discriminatory capacity of Kendall’s W—a nonparametric statistic designed to measure rank-order agreement—is substantially diminished. This decline thus reflects the high degree of consensus already achieved on the retained items rather than an increase in expert disagreement. (2) Expert attrition and panel heterogeneity. Ten experts withdrew during the second round of consultation (attrition rate: 24.4%). Such attrition may have altered the composition of the panel. Heterogeneity among experts in terms of professional background or institutional affiliation may also have influenced the W statistic. In summary, the observed decrease in Kendall’s W in the present study is more attributable to the statistical characteristics of reduced discrimination following item screening and the inherent nature of the rating data than to a substantial deterioration in the degree of expert coordination. Several published Delphi studies have reported similar phenomena [ 20 , 21 ], and Kendall’s W values ranging from 0.1–0.3 with P < 0.05 are accepted in the methodological literature as indicating an acceptable level of concordance. Parts of the preliminary framework developed in this study were ultimately deleted, i.e., risk assessment [ 22 ], emergency training, decision analysis, and daily office work; this was mainly due to the low daily usage rate of these modules, their incomplete functional development, and experts’ belief in their weak operability and low expectations for module improvement. However, these features still need to be explored in the future. Some experts argued that there is a high turnover of personnel in grassroots disease control centres and that information systems should be streamlined as much as possible; otherwise, excessive training time costs may result, which is not conducive to the promotion and popularization of the system. In the “emergency response” module, we aim to modify the system with the goal of convenience, making it user friendly and easy to use [ 23 ] and avoiding making fundamental changes to existing work procedures, thereby stimulating the enthusiasm of frontline public health personnel and emergency management personnel to use the system. In terms of investigation reports, intelligent report generation is currently widely used in medical imaging [ 24 ]. This study also achieved the intelligent generation of survey reports through web crawlers (which automatically capture key fields of survey data), greatly reducing the workload of report writing. In terms of food poisoning investigations, a QR code scanning [ 25 ] function has been developed for individual cases. Cases can scan the code on their own mobile phones and enter information under the guidance of the investigation personnel, thereby accelerating the investigation process. At present, decision analysis functions are relatively lacking. Owing to the difficulty of achieving qualitative and quantitative decision indicators in PHEISs, decision-making still requires experts to combine experience to make decisions. In addition, the lack of timely and accurate onsite data collection during the use of the PHEIS resulted in a lack of effective data support for this module, making it unable to function properly. The original plan of this study was to apply commonly used data analysis models for decision-makers, such as ARIMA product seasonal models [ 26 ] and machine learning techniques [ 27 ], in daily monitoring and early warning, pathogen prediction, and epidemic evolution prediction. However, owing to the low usage rate of this module and the low recognition level of experts towards it, after two rounds of consultation, the first- and second-level indicators related to this function were deleted. Advantages and limitations This study has the following advantages. First, it involves public health professionals and engineers in the development process of information systems, thereby reducing the possibility of significant changes after system construction, saving system development costs, and improving system availability. Second, it uses the Delphi method, which overcomes the shortcomings of the expert experience method, such as an indicator selection that is too arbitrary, a lack of brainstorming [ 9 ], or too much emphasis on the use of user interests by development units. However, this method also has the disadvantage of being labour intensive, requiring health professionals to fully participate in the development of the information system within one year. Third, the integration and data sharing of multiple systems have been completed, integrating different data into the public health emergency information system to enhance monitoring and early warning functions [ 18 , 28 ]. Based on the preliminary investigation, databases with which the PHEIS plans to connect include the following: 1. epidemic warning information, including basic information about schools, suspected case information, etc., which is provided by the school symptom monitoring system [ 29 ], with the aim of decreasing the need for public health personnel to conduct repeated onsite investigations to obtain basic information about individual cases; 2. laboratory system information, in which sample testing information is generated from onsite case investigation information, saving the input operation of sample delivery forms; subsequently, such information will be obtained by capturing cases and collecting sample testing information from other sites, which means that cumbersome steps such as reporting results and printing and archiving test reports can be omitted. In this study, mean scores and the CVs were used as the consensus criteria [ 16 , 18 ]. Although this approach has precedents in related fields, median-based measures, interquartile range (IQR), and percentage agreement are more commonly recommended in the Delphi methodology literature. Future studies may consider reporting multiple consensus metrics concurrently to increase the robustness of the findings. Although indicator selection was guided primarily by prespecified statistical thresholds (CV ≤ 0.25 and mean score ≥3.5), a limited number of indicators were retained despite exceeding the CV threshold on the basis of their conceptual importance and content validity. This introduces a degree of subjectivity into the selection process. Nevertheless, in Delphi studies where rigid adherence to statistical cut-offs would result in the exclusion of conceptually indispensable items, exception-based retention—particularly when cross-validated against other quantitative metrics—remains a methodologically acceptable compromise. To mitigate this limitation, all exceptions were documented with explicit justification and were reviewed and confirmed by two independent members of the research team. This study was conducted mainly in Zhuhai, Guangdong Province. However, Zhuhai is a developed coastal area in China, and its common types of infectious disease outbreaks may differ from those of other regions in the country, which limits the applicability and generalizability of the current findings. Owing to the recent establishment of district-level and street/township-level disease control centres in Zhuhai, some experts invited to participate in this Delphi study did not possess extensive work experience. Conclusion This study used the Delphi method to scientifically construct a set of PHEISs at the city, district, and town levels in Zhuhai, which can greatly improve emergency responses, data collection, and command decision-making for infectious disease outbreaks. This is the first study to focus on the construction of an information system for infectious disease outbreaks, and its construction method, process, and results have significant implications for the construction of PHEISs in various provinces and cities in China. This study elaborates on the PHEIS construction process and combines it with the Delphi method to compensate for the shortcomings of the expert experience method. Owing to the short usage time of the newly constructed system, we analyzed the data during the usage process on the basis of the system’s usage situation, further demonstrating and improving the practicality of the system. On the basis of the findings of this Delphi consensus study, we propose the following actionable recommendations: Capacity building at the grassroots level. Given the recent establishment of street/township-level centers for disease control and prevention in Zhuhai and the anticipated nationwide expansion of similar institutions, targeted training programs should be developed to ensure that staff at all administrative levels—municipal, district, and street/township—possess the requisite skills to operate the system effectively. Training curricula should emphasize streamlined workflows to minimize the onboarding burden in settings with high staff turnover. Periodic evaluation and iterative refinement. The indicator framework should be subjected to regular review cycles to accommodate evolving public health threats, emerging infectious diseases, and feedback from end-users. The high retention rates and favorable consensus scores observed in the “emergency preparedness” and “emergency response” modules in this study underscore their operational importance among frontline public health professionals and their central role in the daily functioning of the system. As system maturity and data infrastructure improve, indicators excluded during the current consensus process due to feasibility concerns should be re-evaluated. Integration with existing health information systems. To maximize the utility of the public health emergency information system and reduce redundant data entry, we strongly recommend that the framework be integrated with existing surveillance platforms, including hospital-based notifiable disease reporting systems, school symptom monitoring networks, and laboratory information management systems. Such integration will enhance the timeliness and completeness of data captured within the public health emergency information system and support more robust monitoring and early warning functions. The framework presented herein offers a pragmatic, evidence-based pathway toward a more resilient and responsive public health emergency management system. Further research is warranted to evaluate the real-world impact of this PHEIS framework on key performance metrics, including outbreak response times, data completeness, and decision-making efficiency. Additionally, validation studies conducted in other geographic regions would help establish the generalizability and adaptability of the framework beyond the specific conditions of Zhuhai city. Supporting information S1 File. Results of the first round of Delphi consultation. S1 Table. First-level indicators: Importance scores. S2 Table . Second-level indicators (city and district systems): Importance scores. S3 Table. Second-level indicators (city, district, and town/street-level systems): Importance scores. S4 Table. First-level indicators: Feasibility scores. S5 Table. Second-level indicators (city and district systems): Feasibility scores. S6 Table. Second-level indicators (city, district, and town/street-level systems): Feasibility scores. S7 Table. Second-level indicators (city and district systems): Comprehensive scores. S8 Table. Second-level indicators (city, district, and town/street-level systems): Comprehensive scores. (XLSX) pone.0350917.s001.xlsx (67.9KB, xlsx) S2 File. Results of the second round of Delphi consultation. S9 Table. First-level indicators (city and district systems): Importance scores. S10 Table. First-level indicators (city, district, and town/street-level systems): Importance scores. S11 Table. Second-level indicators (city and district systems): Importance scores. S12 Table. Second-level indicators (city, district, and town/street-level systems): Importance scores. S13 Table. First-level indicators (city and district systems): Feasibility scores. S14 Table. First-level indicators (city, district, and town/street-level systems): Feasibility scores. S15 Table. Second-level indicators (city and district systems): Feasibility scores. S16 Table. Second-level indicators (city, district, and town/street-level systems): Feasibility scores. S17 Table. Second-level indicators (city and district systems): Comprehensive scores. S18 Table. Second-level indicators (city, district, and town/street-level systems): Comprehensive scores. (XLSX) pone.0350917.s002.xlsx (39.8KB, xlsx) Acknowledgments The authors wish to extend their sincere gratitude to the study participants for their valuable contributions to this research. Appreciation is also expressed to both current and former researchers and staff members. Special thanks are due to Feng Ruan, Xuebao Zhang, Long Chen, Xihe Ni, Ying Fan, Chaomin Zhao, and Junfeng Xiao for their assistance with the research and administrative support. Data Availability All data are in the manuscript and/or Supporting information files. Funding Statement This work was supported by Hygiene and Health Bureau Fund Project of Zhuhai (2220009000231). References 1. Zaid M, Akram MW, Rizvi A, Rizvi SKJ. Blockchain based integrity assurance framework for COVID-19 information management & decision making at National Command Operation Center, Pakistan. Concurrency Comput Pract Exp. 2023;35(8). [ Google Scholar ] 2. Pavão ALB, Barcellos C, Pedroso M, Boccolini C, Romero D. The role of Brazilian National Health Information Systems in assessing the impact of Zika virus outbreak. Rev Soc Bras Med Trop. 2017;50(4):450–7. doi: 10.1590/0037-8682-0506-2016 [ DOI ] [ PubMed ] [ Google Scholar ] 3. Wang J, Bao B, Shen P, Kong G, Yang Y, Sun X, et al. Using electronic health record data to establish a chronic kidney disease surveillance system in China: protocol for the China Kidney Disease Network (CK-NET)-Yinzhou Study. BMJ Open. 2019;9(8):e030102. doi: 10.1136/bmjopen-2019-030102 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 4. Park H, Kang SH, Lee YS, Lee IS, Hwangbo Y, Cho K. Global trends of regional health information systems and suggested strategic utilization of their medical information. Healthc Inform Res. 2021; 27(3):175–81. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 5. Durski KN, Singaravelu S, Naidoo D, Djingarey MH, Fall IS, Yahaya AA, et al. Design thinking during a health emergency: building a national data collection and reporting system. BMC Public Health. 2020;20(1):1896. doi: 10.1186/s12889-020-10006-x [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 6. Nyenswah TG, Skrip L, Stone M, Schue JL, Peters DH, Brieger WR. Documenting the development, adoption and pre-ebola implementation of Liberia’s integrated disease surveillance and response (IDSR) strategy. BMC Public Health. 2023;23(1):2093. doi: 10.1186/s12889-023-17006-7 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 7. Zha H, Liu K, Tang T, Yin Y-H, Dou B, Jiang L, et al. Acceptance of clinical decision support system to prevent venous thromboembolism among nurses: an extension of the UTAUT model. BMC Med Inform Decis Mak. 2022;22(1):221. doi: 10.1186/s12911-022-01958-8 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 8. Çelik Ertuğrul D, Toygar Ö, Foroutan N. A rule-based decision support system for aiding iron deficiency management. Health Informatics J. 2021;27(4):14604582211066054. doi: 10.1177/14604582211066054 [ DOI ] [ PubMed ] [ Google Scholar ] 9. Meidani Z, Moravveji A, Gohari S, Ghaffarian H, Zare S, Vaseghi F, et al. Development and testing requirements for an integrated maternal and child health information system in Iran: a design thinking case study. Methods Inf Med. 2022;61(S 02):e64–72. doi: 10.1055/a-1860-8618 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 10. Jünger S, Payne SA, Brine J, Radbruch L, Brearley SG. Guidance on Conducting and REporting DElphi Studies (CREDES) in palliative care: recommendations based on a methodological systematic review. Palliat Med. 2017;31(8):684–706. doi: 10.1177/0269216317690685 [ DOI ] [ PubMed ] [ Google Scholar ] 11. Huang S, Lin Z, Lin X, Li L, Ruan F, Mei W, et al. Establishment of a no-notice drill mode evaluation system for public health emergencies. PLoS One. 2022;17(4):e0266093. doi: 10.1371/journal.pone.0266093 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 12. Wu C, Zhang Y, Wu J, Zhang L, Du J, Li L, et al. Construction and application on the training course of information literacy for clinical nurses. BMC Med Educ. 2023;23(1):614. doi: 10.1186/s12909-023-04505-9 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 13. Construction of an index system of core competence assessment for otolaryngology nurse specialists in China A Delphi study. [ DOI ] [ PubMed ] 14. Sun W, Dong X, Yu G, Yang Y, He B, Wei Y, et al. Behavioral assessment scale of consciousness for nonhuman primates: a Delphi study. Sci Prog. 2023;106(3):368504231200995. doi: 10.1177/00368504231200995 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 15. Luo R, Zhang C, Liu Y. Health risk assessment indicators for the left-behind elderly in rural China: a Delphi study. Int J Environ Res Public Health. 2020;17(1):340. doi: 10.3390/ijerph17010340 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 16. Zeng X, Gu Y, Zhang J, Liu H, Yang D, Ni X, et al. Development of a perioperative enteral nutrition program for gastric cancer surgery. Nutr Cancer. 2023;75(9):1752–67. doi: 10.1080/01635581.2023.2237217 [ DOI ] [ PubMed ] [ Google Scholar ] 17. Ma Z-Z, Zhang H-B, Niu M-E, Ding Y-H, Zhou Y, Yang Q, et al. Construction of pelvic floor muscle rehabilitation training program for patients undergoing laparoscopic radical prostatectomy. Transl Cancer Res. 2022;11(2):392–402. doi: 10.21037/tcr-21-2738 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 18. Wang H, Ye H, Liu L. Constructing big data prevention and control model for public health emergencies in China: a grounded theory study. Front Public Health. 2023;11:1112547. doi: 10.3389/fpubh.2023.1112547 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 19. Chen L, Zhang Y, Li C, Li Q, He L. Construction of haemodialysis nursing-sensitive quality indicators based on Donabedian theory: a Delphi method study. Nurs Open. 2023;10(2):807–16. doi: 10.1002/nop2.1349 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 20. Meng Y, Liu L, Zheng Y, Li L, Liu X, Qin J. Traditional Chinese medicine constitution identification based on objective facial and tongue features: a Delphi study and a diagnostic nomogram for blood stasis constitution. Evid Based Complement Alternat Med. 2022;2022:6950529. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 21. Wang Y, Wang T, Wang A, Chen S, Jiao L, Shi J, et al. Identifying the competencies of China’s paediatric residents: a modified Delphi method study. BMJ Open. 2021;11(2):e041741. doi: 10.1136/bmjopen-2020-041741 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 22. Wang Y, Wang L, Zhao X, Zhang J, Ma W, Zhao H, et al. A semi-quantitative risk assessment and management strategies on COVID-19 infection to outpatient health care workers in the post-pandemic period. Risk Manag Healthc Policy. 2021;14:815–25. doi: 10.2147/RMHP.S293198 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 23. Bakibinga P, Kamande E, Kisia L, Omuya M, Matanda DJ, Kyobutungi C. Challenges and prospects for implementation of community health volunteers’ digital health solutions in Kenya: a qualitative study. BMC Health Serv Res. 2020;20(1):888. doi: 10.1186/s12913-020-05711-7 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 24. Pang T, Li P, Zhao L. A survey on automatic generation of medical imaging reports based on deep learning. Biomed Eng Online. 2023;22(1):48. doi: 10.1186/s12938-023-01113-y [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 25. Wong C-C, Chong L-Y, Chong S-C, Law C-Y. QR food ordering system with data analytics. JIWE. 2023;2(2):249–72. doi: 10.33093/jiwe.2023.2.2.18 [ DOI ] [ Google Scholar ] 26. Wang M, Pan J, Li X, Li M, Liu Z, Zhao Q, et al. ARIMA and ARIMA-ERNN models for prediction of pertussis incidence in mainland China from 2004 to 2021. BMC Public Health. 2022;22(1):1447. doi: 10.1186/s12889-022-13872-9 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 27. Cabrera M, Leake J, Naranjo-Torres J, Valero N, Cabrera JC, Rodríguez-Morales AJ. Dengue prediction in Latin America using machine learning and the one health perspective: a literature review. Trop Med Infect Dis. 2022;7(10):322. doi: 10.3390/tropicalmed7100322 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 28. Bahr T, Klein N, Fehr A, Fischer H-T, Hanefeld J, Iskandarani D, et al. Providing strategic information during a public health emergency: lessons learned during the COVID-19 pandemic in Germany. Evid Policy. 2023;19(4):606–12. doi: 10.1332/174426421x16880476494516 [ DOI ] [ Google Scholar ] 29. Donaldson AL, Hardstaff JL, Harris JP, Vivancos R, O’Brien SJ. School-based surveillance of acute infectious disease in children: a systematic review. BMC Infect Dis. 2021;21(1):744. doi: 10.1186/s12879-021-06444-6 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] PLoS One. doi: 10.1371/journal.pone.0350917.r001 Decision Letter 0 Pasyodun Koralage Buddhika Mahesh Author information Copyright and License information Roles Pasyodun Koralage Buddhika Mahesh : Academic Editor © 2026 Pasyodun Koralage Buddhika Mahesh This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. PMC Copyright notice 8 Apr 2026 -->PONE-D-26-01579-->-->Construction of a Public Health Emergency Information System Framework: A Case Study of Zhuhai City, China-->-->PLOS One Dear Dr. Ruan, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.-->--> Please submit your revised manuscript by May 23 2026 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org . When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript:--> A letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols . Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols . As the corresponding author, your ORCID iD is verified in the submission system and will appear in the published article. PLOS supports the use of ORCID, and we encourage all coauthors to register for an ORCID iD and use it as well. Please encourage your coauthors to verify their ORCID iD within the submission system before final acceptance, as unverified ORCID iDs will not appear in the published article. Only the individual author can complete the verification step; PLOS staff cannot verify ORCID iDs on behalf of authors. We look forward to receiving your revised manuscript. Kind regards, Pasyodun Koralage Buddhika Mahesh Academic Editor PLOS One Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. PLOS requires an ORCID iD for the corresponding author in Editorial Manager on papers submitted after December 6th, 2016. Please ensure that you have an ORCID iD and that it is validated in Editorial Manager. To do this, go to ‘Update my Information’ (in the upper left-hand corner of the main menu), and click on the Fetch/Validate link next to the ORCID field. This will take you to the ORCID site and allow you to create a new iD or authenticate a pre-existing iD in Editorial Manager. 3. Please include your full ethics statement in the ‘Methods’ section of your manuscript file. In your statement, please include the full name of the IRB or ethics committee who approved or waived your study, as well as whether or not you obtained informed written or verbal consent. If consent was waived for your study, please include this information in your statement as well. 4. Please provide a complete Data Availability Statement in the submission form, ensuring you include all necessary access information or a reason for why you are unable to make your data freely accessible. If your research concerns only data provided within your submission, please write "All data are in the manuscript and/or supporting information files" as your Data Availability Statement. 5. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information . 6. We note that there is identifying data in the Supporting Information file <Supplementary material.zip>. Due to the inclusion of these potentially identifying data, we have removed this file from your file inventory. Prior to sharing human research participant data, authors should consult with an ethics committee to ensure data are shared in accordance with participant consent and all applicable local laws. Data sharing should never compromise participant privacy. It is therefore not appropriate to publicly share personally identifiable data on human research participants. The following are examples of data that should not be shared: -Name, initials, physical address -Ages more specific than whole numbers -Internet protocol (IP) address -Specific dates (birth dates, death dates, examination dates, etc.) -Contact information such as phone number or email address -Location data -ID numbers that seem specific (long numbers, include initials, titled “Hospital ID”) rather than random (small numbers in numerical order) Data that are not directly identifying may also be inappropriate to share, as in combination they can become identifying. For example, data collected from a small group of participants, vulnerable populations, or private groups should not be shared if they involve indirect identifiers (such as sex, ethnicity, location, etc.) that may risk the identification of study participants. Additional guidance on preparing raw data for publication can be found in our Data Policy ( https://journals.plos.org/plosone/s/data-availability#loc-human-research-participant-data-and-other-sensitive-data ) and in the following article: http://www.bmj.com/content/340/bmj.c181.long . Please remove or anonymize all personal information (<specific identifying information in file to be removed>), ensure that the data shared are in accordance with participant consent, and re-upload a fully anonymized data set. Please note that spreadsheet columns with personal information must be removed and not hidden as all hidden columns will appear in the published file. 7. We are unable to open your Supporting Information file [Supplementary material.zip]. Please kindly revise as necessary and re-upload. 8. If the reviewer comments include a recommendation to cite specific previously published works, please review and evaluate these publications to determine whether they are relevant and should be cited. There is no requirement to cite these works unless the editor has indicated otherwise. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions --> Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. --> Reviewer #1: Partly Reviewer #2: Yes ********** -->2. Has the statistical analysis been performed appropriately and rigorously? --> Reviewer #1: Yes Reviewer #2: Yes ********** -->3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.--> Reviewer #1: Yes Reviewer #2: Yes ********** -->4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.--> Reviewer #1: Yes Reviewer #2: Yes ********** -->5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)--> Reviewer #1: This manuscript addresses an important and relevant topic: the development of a public health emergency information system (PHEIS) framework using a Delphi approach. The multi-level design and incorporation of real-world application are notable strengths, and the study has potential practical value for public health systems. However, several methodological and reporting concerns limit confidence in the findings. Addressing the issues below will substantially improve the rigor, transparency, and interpretability of the study. Major Comments 1. Lack of clarity on expert selection and recruitment Comment: Although expert characteristics are presented (Table 2), the manuscript does not clearly describe the criteria used to select experts, nor the sampling strategy or recruitment process. This limits transparency and raises concerns about the validity and representativeness of the Delphi panel. Action required by authors: • Clearly define eligibility criteria for experts (e.g., qualifications, years of experience, roles in public health emergency systems). • Specify the sampling strategy (e.g., purposive, convenience, nomination-based). • Describe the recruitment process (e.g., email invitation, institutional selection, voluntary participation). • Indicate whether any screening or inclusion/exclusion criteria were applied. • Add these details to the Methods section (Delphi method subsection). • Acknowledge potential selection bias and limited generalizability in the Discussion under limitations. 2. Inadequate justification for consensus criteria Comment: The study uses mean scores (≥3.5) and coefficient of variation (<0.25) as thresholds for indicator retention. These cut-offs are not justified, and there are no universally accepted standards supporting this approach. In Delphi studies, consensus is more commonly assessed using median-based measures, interquartile ranges (IQR), or percentage agreement. Action required by authors: • Provide explicit justification with appropriate references for using mean and CV thresholds; OR • Revise the analysis to include more widely accepted Delphi consensus measures, such as: o Median and IQR o Percentage agreement among experts • If retaining the current method: o Clearly justify why mean and CV were selected over standard approaches o Discuss the limitations of this method in the Discussion section • Ensure the chosen method aligns, where possible, with established reporting standards such as CREDES guideline. 3. Insufficient description of the Delphi process Comment: The Delphi methodology is insufficiently described, particularly regarding the iterative process between rounds and the rationale for conducting only two rounds. This affects reproducibility and methodological transparency. Action required by authors: • Expand the Methods section to include: o How feedback was provided between rounds (e.g., summary statistics, anonymized expert comments) o Whether participants were allowed to revise their responses o The mode of administration (e.g., online survey, email-based; specify if this was an e-Delphi approach) • Provide a clear justification for limiting the process to two rounds (e.g., consensus achieved, response stability, feasibility constraints). • Clarify whether this was a classical Delphi, modified Delphi, or e-Delphi approach. 4. Inconsistency in applying predefined inclusion criteria Comment: There appears to be inconsistency in applying the predefined thresholds, as some retained indicators exceed the stated coefficient of variation criteria (Tables 3 and 4). This undermines methodological consistency. Action required by authors: • Reassess all indicators in Tables 3 and 4 to ensure alignment with predefined inclusion criteria. • Either: o Apply the criteria consistently across all indicators, OR o Clearly explain exceptions and decision-making rationale (e.g., conceptual importance overriding statistical thresholds). • If exceptions are retained: o Explicitly state this in the Methods as part of the decision rules o Acknowledge this as a source of subjectivity in the Discussion under limitations. • Ensure full consistency between Methods and Results sections. 5. Misinterpretation of Kendall’s coefficient of concordance (W) Comment: Kendall’s W decreases across rounds (Table 5), indicating reduced agreement among experts. However, the manuscript interprets this as an improvement in consensus. This interpretation is inconsistent with standard Delphi methodology, where agreement is expected to increase with successive rounds. Action required by authors: • Correct the interpretation of Kendall’s W in both the Results and Discussion sections. • Clearly state that a decrease in W reflects reduced agreement, not improved consensus. • Provide a plausible explanation for this finding (e.g., indicator reduction, panel heterogeneity, attrition between rounds). • Reframe conclusions regarding consensus accordingly. • Explicitly acknowledge this issue in the Discussion under limitations. 6. Overall methodological alignment with reporting standards Comment: Overall, substantial revisions are required to improve methodological transparency and align the study with accepted standards such as the CREDES guideline. Action required by authors: • Revise the manuscript to ensure alignment with established Delphi reporting standards, particularly CREDES. • Ensure comprehensive reporting of: o Expert selection and characteristics o Consensus definition and justification o Iterative feedback process o Stability and agreement across rounds • Consider adding a statement in the Methods section indicating adherence (or partial adherence) to CREDES. • Improve overall clarity, transparency, and reproducibility of the methodology. Reviewer #2: Congratulations on your valuable attempt in constructing a public health information system using the Delphi technique! This is a timely and much-needed contribution towards ensuring timely responses during public health emergencies. Please find some comments for further improvement of this manuscript mentioned below: Ethics statement (page 4 of the PDF document) • In the ethics statement, it is stated that “Verbal assent was obtained from all participants prior to the study”. It is suggested to clarify what this means, as assent is usually taken from minors/children. Abstract • It is suggested to separate the Background from the objectives in the abstract. Please mention the objective clearly. • Please be mindful of the tense. Some sentences were written in the present tense Manuscript • It is good if the initial indicator framework could be shown (page 5, line 8) • Please avoid starting sentences with abbreviations (Cr) (page 6, line 1) • Please provide references/ rationale for your cut-offs (page 6- line 2, page 7- lines 2&3) • It is suggested to move the sentence on response rate (line 20, page 7) to the beginning of the Basic information section, before describing sample characteristics. Conclusion • The ending seemed a bit abrupt. Please end the manuscript with a strong recommendation. Thank you ********** -->6. PLOS authors have the option to publish the peer review history of their article ( what does this mean? ). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy .--> Reviewer #1: Yes: I.O.K.K.Nanayakkara Reviewer #2: No ********** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] To ensure your figures meet our technical requirements, please review our figure guidelines: https://journals.plos.org/plosone/s/figures You may also use PLOS’s free figure tool, NAAS, to help you prepare publication quality figures: https://journals.plos.org/plosone/s/figures#loc-tools-for-figure-preparation . NAAS will assess whether your figures meet our technical requirements by comparing each figure against our figure specifications. PLoS One. 2026 Jun 5;21(6):e0350917. doi: 10.1371/journal.pone.0350917.r002 Author response to Decision Letter 1 Article notes Copyright and License information Collection date 2026. PMC Copyright notice 27 Apr 2026 We have carefully revised the manuscript in accordance with the comments and suggestions provided by the editor and both reviewers. Detailed point-by-point responses to all comments are provided in the accompanying document titled "Response to Reviewers." Attachment Submitted filename: Response to Reviewers.docx pone.0350917.s003.docx (31KB, docx) PLoS One. doi: 10.1371/journal.pone.0350917.r003 Decision Letter 1 Pasyodun Koralage Buddhika Mahesh Author information Copyright and License information Roles Pasyodun Koralage Buddhika Mahesh : Academic Editor © 2026 Pasyodun Koralage Buddhika Mahesh This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. PMC Copyright notice 21 May 2026 Construction of a Public Health Emergency Information System Framework: A Case Study of Zhuhai City, China PONE-D-26-01579R1 Dear Dr. Ruan, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. For questions related to billing, please contact billing support . If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Pasyodun Koralage Buddhika Mahesh Academic Editor PLOS One Additional Editor Comments (optional): Reviewers' comments: Reviewer's Responses to Questions --> Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.--> Reviewer #1: All comments have been addressed Reviewer #2: All comments have been addressed ********** -->2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. --> Reviewer #1: Yes Reviewer #2: (No Response) ********** -->3. Has the statistical analysis been performed appropriately and rigorously? --> Reviewer #1: Yes Reviewer #2: (No Response) ********** -->4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.--> Reviewer #1: Yes Reviewer #2: (No Response) ********** -->5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.--> Reviewer #1: Yes Reviewer #2: (No Response) ********** -->6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)--> Reviewer #1: The authors have substantially addressed the previous comments. The revised manuscript now provides clearer details on expert selection, recruitment, the modified e-Delphi process, feedback between rounds, decision rules for indicator retention, and alignment with CREDES guidance. The issue regarding consensus criteria has been reasonably addressed through justification and discussion of limitations. The interpretation of Kendall’s W has also been improved.Overall, I consider the manuscript much improved and suitable for acceptance after minor editorial revision. Reviewer #2: Dear authors, I would like to congratulate you all once again! Thank you for taking my comments into consideration. ********** -->7. PLOS authors have the option to publish the peer review history of their article ( what does this mean? ). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy .--> Reviewer #1: Yes: I.O.K.K.Nanayakkara Reviewer #2: Yes: Herath Mudiyanselage Chathurika Dulmini Herath ********** PLoS One. doi: 10.1371/journal.pone.0350917.r004 Acceptance letter Pasyodun Koralage Buddhika Mahesh Author information Copyright and License information Roles Pasyodun Koralage Buddhika Mahesh : Academic Editor © 2026 Pasyodun Koralage Buddhika Mahesh This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. PMC Copyright notice PONE-D-26-01579R1 PLOS One Dear Dr. Ruan, I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS One. Congratulations! Your manuscript is now being handed over to our production team. At this stage, our production department will prepare your paper for publication. This includes ensuring the following: * All references, tables, and figures are properly cited * All relevant supporting information is included in the manuscript submission, * There are no issues that prevent the paper from being properly typeset You will receive further instructions from the production team, including instructions on how to review your proof when it is ready. Please keep in mind that we are working through a large volume of accepted articles, so please give us a few days to review your paper and let you know the next and final steps. Lastly, if your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. You will receive an invoice from PLOS for your publication fee after your manuscript has reached the completed accept phase. If you receive an email requesting payment before acceptance or for any other service, this may be a phishing scheme. Learn how to identify phishing emails and protect your accounts at https://explore.plos.org/phishing . If we can help with anything else, please email us at customercare@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Pasyodun Koralage Buddhika Mahesh Academic Editor PLOS One Associated Data This section collects any data citations, data availability statements, or supplementary materials included in this article. Supplementary Materials S1 File. Results of the first round of Delphi consultation. S1 Table. First-level indicators: Importance scores. S2 Table . Second-level indicators (city and district systems): Importance scores. S3 Table. Second-level indicators (city, district, and town/street-level systems): Importance scores. S4 Table. First-level indicators: Feasibility scores. S5 Table. Second-level indicators (city and district systems): Feasibility scores. S6 Table. Second-level indicators (city, district, and town/street-level systems): Feasibility scores. S7 Table. Second-level indicators (city and district systems): Comprehensive scores. S8 Table. Second-level indicators (city, district, and town/street-level systems): Comprehensive scores. (XLSX) pone.0350917.s001.xlsx (67.9KB, xlsx) S2 File. Results of the second round of Delphi consultation. S9 Table. First-level indicators (city and district systems): Importance scores. S10 Table. First-level indicators (city, district, and town/street-level systems): Importance scores. S11 Table. Second-level indicators (city and district systems): Importance scores. S12 Table. Second-level indicators (city, district, and town/street-level systems): Importance scores. S13 Table. First-level indicators (city and district systems): Feasibility scores. S14 Table. First-level indicators (city, district, and town/street-level systems): Feasibility scores. S15 Table. Second-level indicators (city and district systems): Feasibility scores. S16 Table. Second-level indicators (city, district, and town/street-level systems): Feasibility scores. S17 Table. Second-level indicators (city and district systems): Comprehensive scores. S18 Table. Second-level indicators (city, district, and town/street-level systems): Comprehensive scores. (XLSX) pone.0350917.s002.xlsx (39.8KB, xlsx) Attachment Submitted filename: Response to Reviewers.docx pone.0350917.s003.docx (31KB, docx) Data Availability Statement All data are in the manuscript and/or Supporting information files. Articles from PLOS One are provided here courtesy of PLOS ACTIONS View on publisher site PDF (416.3 KB) Cite Collections Permalink RESOURCES Similar articles Cited by other articles Links to NCBI Databases Follow NCBI NCBI on X (formerly known as Twitter) NCBI on Facebook NCBI on LinkedIn NCBI on GitHub NCBI RSS feed Connect with NLM NLM on X (formerly known as Twitter) NLM on Facebook NLM on YouTube National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894 Web Policies FOIA HHS Vulnerability Disclosure Help Accessibility Careers NLM NIH HHS USA.gov Back to Top

Related documents

• Construction of a public health emergency information system framework: A case study of Zhuhai city, China#263321
• The effect of integration of medical and preventive services on the prevention and control of public health emergencies in China.#354662
• Risk perceptions, government role, and public protective behavior in the face of environmental health hazards.#323266
• Degree-of-worry and Illness Perception in Patients Suffering From Acute Illness in the Emergency Department#850071
• Stakeholder perspectives on the use of enhanced mobile phone capabilities for public health surveillance for non-communicable disease risk factors: A qualitative study#216688