Testing the Effectiveness of a Text-Message-Based Home Monitoring Program and Oxygen Monitoring for Patients with COVID-19—The COVID Watch Study

Testing the Effectiveness of a Text-Message-Based Home Monitoring Program and Oxygen Monitoring for Patients with COVID-19—The COVID Watch Study - NCBI Bookshelf An official website of the United States government Here's how you know The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site. The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely. Log in Show account info Close Account Logged in as: username Dashboard Publications Account settings Log out Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation Bookshelf Search database Books All Databases Assembly Biocollections BioProject BioSample Books ClinVar Conserved Domains dbVar Gene Genome GEO DataSets GEO Profiles GTR Identical Protein Groups MedGen MeSH NLM Catalog Nucleotide OMIM PMC Protein Protein Clusters Protein Family Models PubChem BioAssay PubChem Compound PubChem Substance PubMed SNP SRA Structure Taxonomy ToolKit ToolKitAll ToolKitBookgh Search term Search Browse Titles Advanced Help Disclaimer NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health. Testing the Effectiveness of a Text-Message-Based Home Monitoring Program and Oxygen Monitoring for Patients with COVID-19—The COVID Watch Study M. Kit Delgado , MD, MS, Anna U. Morgan , MD, MSc, MSHP, David A. Asch , MD, MBA, Ruiying Xiong , MS, Austin S. Kilaru , MD, MSHP, Kathleen C. Lee , MD, David Do , MD, Ari B. Friedman , MD, PhD, Zachary F. Meisel , MD, MPH, MSHP, Christopher K. Snider , MPH, Doreen Lam , BA, Andrew Parambath , BA, Christian Wood , BA, Chidinma M. Wilson , BA, BS, Michael Perez , BS, BA, Deena L. Chisholm , MPH, Sheila Kelly , MPH, Christina J. O'Malley , MHA, Nancy Mannion , DNP, RN, CEN, FAEN, Ann Marie Huffenberger , DBA, RN, NEA-BC, Susan McGinley , CRNP, Mohan Balachandran , MA, MS, Neda Khan , BS, MPH, MSHP, Medha Ghosh , MS, Julianne Reilly , BS, Lauren Hahn , MBA, Michaela Ward , MSEd, Maria N. Nelson , MA, Zoe Barbati , BA, Jessica E. Hemmons , MS, Dina Abdel-Rahman , BS, Jeffrey P. Ebert , PhD, Judy A. Shea , PhD, Nandita Mitra , PhD, and Krisda H. Chaiyachati , MD. Author Information and Affiliations Authors M. Kit Delgado , MD, MS, 1-3 Anna U. Morgan , MD, MSc, MSHP, 3-5 David A. Asch , MD, MBA, 3-6 Ruiying Xiong , MS, 1,3,4 Austin S. Kilaru , MD, MSHP, 3,4 Kathleen C. Lee , MD, 1,6 David Do , MD, 6,7 Ari B. Friedman , MD, PhD, 1,3 Zachary F. Meisel , MD, MPH, MSHP, 1,3 Christopher K. Snider , MPH, 6 Doreen Lam , BA, 6 Andrew Parambath , BA, 6 Christian Wood , BA, 1 Chidinma M. Wilson , BA, BS, 1 Michael Perez , BS, BA, 6 Deena L. Chisholm , MPH, 1 Sheila Kelly , MPH, 4 Christina J. O'Malley , MHA, 6 Nancy Mannion , DNP, RN, CEN, FAEN, 8 Ann Marie Huffenberger , DBA, RN, NEA-BC, 8 Susan McGinley , CRNP, 8 Mohan Balachandran , MA, MS, 6 Neda Khan , BS, MPH, MSHP, 3,4,6,8 Medha Ghosh , MS, 1 Julianne Reilly , BS, 1 Lauren Hahn , MBA, 6 Michaela Ward , MSEd, 9 Maria N. Nelson , MA, 9 Zoe Barbati , BA, 9 Jessica E. Hemmons , MS, 1 Dina Abdel-Rahman , BS, 1 Jeffrey P. Ebert , PhD, 1 Judy A. Shea , PhD, 4 Nandita Mitra , PhD, 2,3 and Krisda H. Chaiyachati , MD 3,4 . Affiliations 1 Center for Emergency Care Policy and Research, Department of Emergency Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia 2 Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia 3 Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia 4 Division of General Internal Medicine, Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia 5 Wharton School, University of Pennsylvania, Philadelphia 6 Center for Health Care Innovation, University of Pennsylvania Health System, Philadelphia 7 Department of Neurology, Perelman School of Medicine, University of Pennsylvania, Philadelphia 8 Center for Connected Care, University of Pennsylvania Health System, Philadelphia 9 Mixed Methods Laboratory, Department of Family Medicine and Community Health, Perelman School of Medicine, University of Pennsylvania, Philadelphia Washington (DC): Patient-Centered Outcomes Research Institute (PCORI) ; 2024 Feb . Copyright and Permissions Copyright © 2024. University of Pennsylvania, Perelman School of Medicine. All Rights Reserved. This book is distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License which permits noncommercial use and distribution provided the original author(s) and source are credited. (See https://creativecommons.org/licenses/by-nc-nd/4.0/ Structured Abstract Background: Nearly 90% of patients with COVID-19 are asked to self-isolate and monitor their symptoms at home. Remote monitoring has the potential to facilitate emergency department (ED) and hospital-level care for patients who need it while supporting access to care for patients who can safely remain at home. In March 2020, the University of Pennsylvania Health System (Penn Medicine) launched COVID Watch, an automated text message-based remote monitoring program with 24/7 clinical support. Patients enrolled in COVID Watch received text message check-ins twice a day for 2 weeks. When a patient reported new or worsening dyspnea (difficulty breathing), an alert was sent to the clinical support team, who called the patient within 1 hour and either directed them to the ED, scheduled an urgent telemedicine appointment, or advised them to continue to self-monitor at home. Objectives: The present research (ClinicalTrials.gov identifier NCT04581863 ) examined whether patients who were enrolled in COVID Watch experienced better health outcomes compared with usual care (aim 1), tested whether augmenting COVID Watch with at-home monitoring of oxygen saturation as measured by pulse oximetry (Sp o 2 ) improved patient outcomes (aim 2), and elucidated patient and stakeholder ideas for improving COVID Watch and similar monitoring programs (aim 3). Methods: Aim 1 was accomplished with a retrospective analysis that included all community-dwelling adults who tested positive for COVID-19 at Penn Medicine as outpatients during the 8 months after the launch of COVID Watch. Those adults who enrolled in COVID Watch were compared with those who remained in usual care on the primary outcome, death at 30 days, using propensity score weighted models. Subgroup analyses examined the outcomes among racial and ethnic subgroups. Aim 2 was accomplished with a pragmatic randomized trial taking place after the aim 1 study. Patients with confirmed COVID-19 who were enrolled in COVID Watch as outpatients or upon discharge from the ED or hospital were randomly assigned to either the standard COVID Watch program or COVID Watch with home Sp o 2 monitoring. The latter group received a pulse oximeter and reported their Sp o 2 during the twice-daily check-ins. In addition to the standard alert sent to telemedicine clinicians when a patient reported new or worsening dyspnea, an alert was triggered if a patient reported an Sp o 2 drop of at least 3 percentage points to a level below 95%, or any reading below 90%. The primary outcome was days alive and out of hospital (DAOH) at 30 days, and subgroup analyses assessed the outcomes of Black and Hispanic patients. Aim 3 was executed using semistructured interviews with patients who had been enrolled in COVID Watch, clinicians whose patients had been enrolled in COVID Watch, and administrators in the health system. Questions were open ended and included follow-up probes. Participant responses were coded for emergent themes with respect to sentiments about COVID Watch, feedback for improving COVID Watch, and lessons for future remote patient monitoring programs. Results: In the aim 1 study, 3488 outpatients were enrolled in COVID Watch and 4377 in the usual care control group. Participants were compared using propensity score weighted models. At 30 days, COVID Watch patients had an odds ratio for death of 0.32 (95% CI, 0.12-0.72), with 1.8 fewer deaths per 1000 patients (95% CI, 0.5-3.1; P = .005). Black and Hispanic participants were more likely to be enrolled in COVID Watch, and all major racial and ethnic groups experienced similar reductions in mortality when enrolled in COVID Watch. The aim 2 randomized controlled trial found no difference in DAOH between COVID Watch with home Sp o 2 monitoring (n = 611) and standard COVID Watch (n = 606) overall (29.4 vs 29.5 DAOH; difference, −0.1 [95% CI, −0.4 to 0.2]; P = .58), within racial and ethnic subgroups, and within enrollment subgroups (outpatient or upon ED or hospital discharge). For aim 3, 47 patients and 38 clinicians and administrators were interviewed. Many patients found the presence of COVID Watch to be comforting, although some found the frequency of its check-ins to be irritating, and others wished for more human contact. Clinicians also took comfort knowing that their patients were being checked in on and had access to 24/7 clinical support, and they thought that the program reduced follow-up burden. Interviewees pointed out ways to improve the enrollment process and make the program more accessible to patients regardless of their language, literacy, or cell phone access. Conclusions: Enrollment in the automated remote monitoring program COVID Watch was associated with a 68% relative reduction in the mortality of outpatients with COVID-19. COVID Watch had high utilization rates among Black and Hispanic patients, and these subgroups experienced better health outcomes when enrolled in the program compared with usual care. Providing at-home Sp o 2 monitoring did not yield incremental benefits over the standard COVID Watch program. This suggests that subjectively reported difficulty breathing by itself is an adequate metric for determining when COVID-19 outpatient care should be escalated. Patients and clinicians generally found COVID Watch to be valuable and gave consistent suggestions for actionable improvements to make the program more widely accessible. Limitations: Aim 1 was an observational study and shares the limitations inherent to such designs. For the set of potential confounding variables that were observed, however, patients enrolled in COVID Watch were at higher risk for poor outcomes. In addition, robustness analyses indicated that the magnitude of unobserved confounding would need to be very large to reverse the study findings. This degree of unmeasured confounding is unlikely given the robust set of variables accounted for in the analysis. The aim 2 randomized controlled trial only compared COVID Watch plus home Sp o 2 monitoring to the standard COVID Watch program; therefore, the null results obtained cannot speak to the potential benefits of home Sp o 2 monitoring by itself. Aim 3 results must be interpreted with an appreciation for the context in which the interviews were conducted: a large academic institution, in the first year of the pandemic—a time of uncertainty and anxiety when surges were common and vaccinations were not readily available. Background In the first year of the COVID-19 pandemic, most (80%-90%) patients with COVID-19 were asked to self-isolate at home because they did not qualify for home care or inpatient admission. 1 , 2 Alternatives to in-person care were urgently needed to (1) manage an unpredictable clinical course, (2) identify and intercept patients rapidly deteriorating at home, (3) prevent viral spread during in-person visits, and (4) triage symptoms to minimize surges to emergency departments (EDs). 3-9 In addition, fingertip pulse oximeters (pulse oximetry) were proposed to improve in-home early detection of respiratory deteriorations; however, pulse oximeters were untested for this use, and the operational infrastructure to support large-scale monitoring was limited. 10-12 Although telemedicine was widely adopted during the pandemic as an alternative to conventional outpatient care, worse COVID-19 outcomes observed among Black and Latinx patients may have been due, in part, to worse telemedicine access for these socially and medically disadvantaged groups. 13-20 To improve in-home monitoring and health care access for patients with suspected or confirmed COVID-19, our team in the University of Pennsylvania Health System (Penn Medicine) developed COVID Watch, a free-of-charge, 24/7, automated, text message-based home monitoring program across Penn Medicine's large geographic catchment area. 21 We were able to quickly launch COVID Watch because of Penn Medicine's prior investments in platforms for innovation and telemedicine, including Way to Health, OnDemand, and Agent. Our first meeting to develop the program was on March 11, 2020, and our first patient was invited on March 23. Rapid cycle innovation enabled us to refine the program. The automated component of COVID Watch used existing health system resources, and its follow-up call component was staffed by physicians and nurses whose normal activities were displaced by the pandemic. The present research was designed to answer 3 core questions: (1) to what extent has COVID Watch affected days alive and out of hospital (DAOH) for patients with COVID-19, (2) how does the addition of pulse oximetry affect clinical outcomes for patients with COVID-19, and (3) how has COVID Watch been received and implemented by key stakeholders, including patients and health care professionals (HCPs)? All 3 questions were assessed across Penn Medicine's diverse population, with a targeted focus on outcomes among Black and Latinx patients. Notably, 61% of patients with COVID-19 in COVID Watch self-reported as Black or Latinx. Significance Our research was designed to investigate the impact of COVID Watch's automated remote monitoring on COVID-19 outcomes and implementation facilitators and barriers to guide other health systems wanting to offer similar services to patients with COVID-19. Understanding the effect of COVID Watch on Black and Latinx patients was of central importance because communities of color saw higher rates of COVID-19 morbidity and mortality nationally 22-24 and locally in Pennsylvania and New Jersey. 25 Although symptom monitoring apps and programs were commercially available, 26-31 COVID Watch was uniquely embedded in its host health care system, Penn Medicine. Patients were enrolled through the electronic health record (EHR) system, escalations were addressed by Penn Medicine nurses, and patients were referred to Penn Medicine-based telemedicine, primary care, and social services when appropriate. COVID Watch is a scalable operational platform for monitoring patients with home pulse oximetry readings. Whether remote monitoring with or without pulse oximetry helps patients with COVID-19 stay safe at home is an urgent, patient-centered question. 10 , 32 , 33 Finally, health systems can better prepare for future COVID-19 surges by adopting evidence-based strategies to monitor more patients with COVID-19 at home. 34 , 35 Overview Our overall objective was to evaluate the clinical effectiveness and implementation outcomes of COVID Watch, an automated, text message-based, remote monitoring intervention—with and without pulse oximetry—for home-dwelling adults diagnosed with COVID-19, particularly among Black and Latinx patients. Intervention COVID Watch ( https://chti.upenn.edu/covid-watch ) is powered by Penn Medicine's NIH-funded research and operational texting platform, Way to Health ( www.waytohealth.org ), which is integrated with the Penn Medicine EHR system. Because hypoxemia is the primary driver of admission and fatality among patients with COVID-19, COVID Watch's escalation algorithm focused on shortness of breath, an approximation for concerning hypoxemia. 9 , 36 , 37 Given the rapid and unpredictable decompensation of patients with COVID-19, COVID Watch patients received clinical support 24 hours a day, 7 days a week. 4 , 38-40 COVID Watch sent them twice-daily, scheduled text messages to assess shortness of breath using an algorithm to determine whether patients needed urgent escalation to a team of dedicated nurses within 1 hour. These nurses were supported by an on-call team of clinicians who could conduct urgent phone or video assessments. At each level, patients' clinical needs were managed with self-care advice, prescriptions, or a referral to the ED. The key features of this intervention can be replicated on any automated text messaging platform (eg, Twillio, Mosio), and EHR integration is not necessary; therefore, it is widely scalable. At the time our research was proposed, more than 80% of COVID Watch patients had been enrolled by outpatient HCPs conducting telephone and video visits; less than 10% had been enrolled at ED discharge and less than 10% at inpatient discharge. The program was available at all 6 Penn Medicine hospitals and EDs and all 530 affiliated ambulatory settings through the EHR system. Any clinical staff member (eg, nurse, physician, medical assistant) logged in to the EHR system was able to enroll patients. Patients received the intervention only if they opted in and could stop at any time. Patients were scheduled to be in COVID Watch for 14 days, with the option to extend to 21 days if interested. A Spanish-language version of COVID Watch was made available to patients on May 18, 2020, with nurses using translation lines to communicate with patients. Aim 1. Conduct a Retrospective Quantitative Evaluation by Using Secondary Data (Obtained From the EHR System and the National Death Index) to Evaluate Clinical Outcomes for COVID Watch Patients, Particularly Black and Latinx Patients A challenge for patients with COVID-19, and the clinicians who care for them, is balancing the tension between the need to escalate care if COVID-19 symptoms worsen and the desire to avoid unnecessary in-person outpatient visits, trips to the ED, and hospitalizations. 2 , 41 Clinicians can assess patients over the phone or by video, but there can be delays in these consultations, and patients without established primary care or insurance may have difficulty accessing these services. 15 , 42 , 43 This aim sought to quickly “generate actionable findings to inform clinical and public health responses, decision-making, and planning” so that there was a window to change the trajectory of the COVID-19 pandemic. Given a diverse population of patients who had already received care in COVID Watch, a retrospective analysis of data already collected, with a quasiexperimental control group afforded by a staggered rollout, was the best available way to quickly generate evidence on the potential benefits of this intervention. Study Population and Setting This study capitalized on patients treated in real-world settings and used existing EHRs to produce rapid, actionable findings. The inclusion criteria were broad, enabling generalizable findings and allowing for key subgroup analyses, particularly among Black and Latinx patients. To generate evidence rapidly, all outpatients who tested positive for COVID-19 (primary study cohort) in the Penn Medicine system before the grant's start date were included. We restricted the population to outpatients who tested positive for COVID-19 before any need for acute care. This was primarily because we hypothesized most of the potential health benefit of COVID Watch would be due to escalating patients to initial acute care sooner. Also, although outpatient enrollment in COVID Watch was more random and variable, we anticipated an increased likelihood of selection bias of enrolling patients in COVID Watch upon discharge because of standardized discharge processes. Therefore, these postacute discharge populations were not included in our aim 1 observational comparative effectiveness analysis. Outpatients enrolled in COVID Watch were compared with patients who were eligible to be enrolled in COVID Watch but were not enrolled. Penn Medicine is a large academic medical center that cares for patients across southeast Pennsylvania, southern New Jersey, and Delaware, with a racially and ethnically diverse population residing in urban and rural areas. As part of usual care, from the beginning of the COVID-19 pandemic in February 2020, Penn Medicine regularly disseminated health system guidelines for the outpatient and inpatient management of COVID-19 by the Penn Medicine COVID-19 task force by email and posted clinical guidelines on the internal health system's Microsoft SharePoint-supported websites. This information included directing patients with acute shortness of breath to the ED for evaluation and treatment and for the initiation of time-sensitive acute therapies such as dexamethasone and remdesivir as the evidence became available. COVID Watch was advertised for use in the health system. There were no health system-wide guidelines regarding contacting patients with COVID-19. COVID Watch was not included in health system guidelines as standard of care until December 2020, after the conclusion of the aim 1 study. Outcomes Our primary outcome was mortality in the 30 days after the patient's positive COVID-19 test. 44-47 Our secondary outcomes were accessibility and timeliness of outpatient care, need for serious illness care, need for intubation and critical care, outcomes of hospitalization including death and discharge to hospice or long-term care, and timing of escalation to hospital care. As a proxy for patient anxiety and emotional distress related to managing symptoms of COVID-19 at home, we also analyzed rates of patient communications by telephone or electronic communication. Aim 2. Conduct a Pragmatic Randomized Controlled Trial to Determine the Incremental Benefit of Providing Home Fingertip Pulse Oximetry for Patients Enrolled in COVID Watch Conceptual Model Preventing or aggressively treating respiratory failure is essential for patients infected with COVID-19. An estimated 10% to 20% of patients with COVID-19 develop severe illness necessitating hospitalization, of whom 30% to 40% develop critical illness necessitating support in an intensive care unit. 48 Among the critically ill, respiratory failure is common and results from acute hypoxemic respiratory failure due to acute respiratory distress syndrome. 48 This syndrome is a clinical manifestation of pulmonary tissue inflammation, preventing the exchange of oxygen between inhaled air and blood circulating in the lungs (hypoxemia). Increasing the concentration of oxygen in the air that patients breathe (eg, supplemental oxygen) and reducing the degree of pulmonary inflammation can support patients with COVID-19 by reducing the severity of hypoxemia and providing sufficient oxygen supply to vital organs. Oxygen saturation as measured by pulse oximetry (Sp o 2 ) can provide an objective metric by which to identify patients who have experienced declines in their respiratory function. 49 Our conceptual model ( Figure 1 ) was based on emerging evidence that early medical interventions could improve clinical outcomes for COVID-19, resulting in less severe clinical sequelae and reduced mortality. For patients with COVID-19 who needed supplemental oxygen, preliminary data from a large multicenter randomized controlled trial indicated that patients have markedly lower mortality when given dexamethasone. 50 This corticosteroid probably reduces pulmonary inflammation and prevents the development of severe acute respiratory distress syndrome. Additional evidence suggested that remdesivir could speed recovery and reduce hospital days among patients with COVID-19. 51 Finally, there were promising data suggesting that simple changes in ED care and hospital practices may improve outcomes. For example, awake proning, which involves positioning an awake, nonintubated patient at regular intervals onto their abdomen with their face down, can enable greater recruitment of lung tissue for oxygen exchange and reduce the need for subsequent intubation and mechanical ventilation. 52 , 53 Early interventions such as these can reduce the number of days patients spend in the hospital, prevent complications such as ventilator-associated pneumonia, and reduce mortality related to COVID-19. Figure 1 Conceptual Model of Oxygen Saturation and Pulmonary Inflammation Over Time in Patients With COVID-19 Who Progress to Critical Illness and Hypothesized Effects of Earlier Intervention Through Remote Monitoring and Pulse Oximetry. Respiratory decline must be identified early for patients to benefit from early medical interventions. Respiratory decline can be (1) signaled when a patient reports the sensation of “shortness of breath” or “difficulty breathing” (ie, dyspnea) or (2) identified when the concentration of oxygen in a patient's blood or Sp o 2 is objectively measured. Current models of care (usual care), even with expanded telemedicine access, rely on patients contacting HCPs—a reactive process. This reactive process can delay care because most primary care practices use an intermediary (eg, a staff member or message inbox) to screen communications for busy clinicians. Many practices do not respond to patients' concerns efficiently at night or on weekends. Even if a patient has concerns about their breathing, they may be reluctant to contact their HCP. The usual care model is even harder for patients with limited primary care or telemedicine access and those who face language barriers—problems disproportionately affecting Black and Latinx communities. In contrast, COVID Watch is proactive, requesting patients to report symptoms twice a day, and patients can trigger a phone call to a nurse within an hour by simply texting. In addition, COVID Watch is available free of charge, with a Spanish-language option. COVID Watch plus pulse oximetry mirrors COVID Watch but may enable earlier detection of respiratory decline using an objective measure of Sp o 2 . To avoid overburdening EDs, respiratory decline must be identified accurately. Both unnecessary escalations of care to the ED for patients with mild COVID-19 and delayed detection of respiratory decline in patients with severe COVID-19 may occur. For usual care and COVID Watch, clinicians rely on the subjective sensation of dyspnea. Dyspnea is concerning enough that patients are often redirected to an urgent care center or ED where their Sp o 2 can be measured. For patients found to have an Sp o 2 less than 94%, referral to the ED for additional clinical evaluation and supportive therapy would be justified. Referring patients with mild dyspnea who turn out to have a normal Sp o 2 ( Figure 2 ) may be a burden to patients and their families and strain resources at EDs and hospitals. Figure 2 Mild COVID-19 Phenotype at Risk of Overtriage in the Emergency Department. During times of rapid rises in community infection rates, or surges, efficient resource allocations are imperative for supporting patients who are critically ill. At the same time, patients still benefit from enhanced connections with HCPs to reassure them that they are not experiencing respiratory decline. COVID Watch, supplemented by pulse oximetry, could provide the responding team of clinicians with objective data so they can accurately assess a patient's respiratory state. In an alternative scenario, patients may have low Sp o 2 without the sensation of dyspnea, a phenomenon unique to patients with COVID-19 commonly known as “silent” or “happy” hypoxemia ( Figure 3 ). 32 Figure 3 Severe COVID-19 “Silent Hypoxemia” Phenotype at Risk of Delayed Presentation. These patients receive medical interventions late because they are unaware they are hypoxemic. Early in the pandemic, there were reports of patients with COVID-19 arriving in EDs with profoundly low Sp o 2 levels but little or no dyspnea. 10 This information led to proposals in the news media and resuscitation medicine forums calling for the use of home pulse oximetry to detect silent hypoxemia. 54 , 55 To make this technology widely available, the Vermont Department of Public Health implemented a program that mailed pulse oximeters to patients with newly diagnosed COVID-19 24 to 48 hours after contact tracing interviews. 56 As reported in Science , however, at the time the benefits of such a program were unclear: No one, however, has studied whether early detection of hypoxia might head off bad outcomes. Some physicians believe pulse oximeters are best used with a doctor's guidance, perhaps through telemedicine. With many COVID-19 patients frightened to visit a hospital and arriving only when their symptoms have dangerously advanced, doctors also wonder whether home monitoring could hasten treatment—and whether, for some, that could make all the difference. 32 Our study was designed to provide the first rigorous quantitative evidence on the value of home pulse oximetry. It was motivated, in part, by anecdotal reports from survivors of COVID-19 who found value in having an accessible and objective measure of their oxygen levels ( Table 1 ). 57 Table 1 Qualitative Evidence on the Value of Home Pulse Oximetry in Improving Outcomes for Patients With COVID-19 . Aim 3. Use a Mixed-Methods Approach to Quantify Uptake, Trajectories, and Predictors of COVID Watch Activity and Query Health System Leaders, Clinicians, and Patients to Understand Contextual Factors Related to COVID Watch Referral, Program Engagement, and Health Care Access Understanding facilitators and barriers to implementation will guide other health systems or public health agencies wanting to adopt a similar automated remote monitoring service for their patients with COVID-19. This aim will also enhance our contextual understanding of how Black and Latinx patients are referred and engaged with this program and their perceptions of health care access due to this program. Study Design We conducted a mixed-methods evaluation of both COVID Watch (aim 1) and COVID Watch plus pulse oximetry (aim 2). Using the COVID Watch database, we could continually quantify uptake, trajectories, and correlates of use (eg, geographic area infections) among patients enrolled. We designed interviews based on the Consolidated Framework for Implementation Research. 58-60 Interviews with health system leaders and clinicians were intended to provide insights to contextual factors (eg, local practice leadership, patient panel characteristics) perceived to influence COVID Watch referrals. Embedded in the interview was a chart-stimulated recall exercise to facilitate the reconstruction of decision-making. 61 It was expected that the influence of pulse oximeters would be negligible because leaders and clinicians were blinded to who received the device. Interviews with patients were designed to explore barriers and facilitators to enrolling in the program and the perceived effects of COVID Watch with or without pulse oximetry on health care access. The semistructured interview guide was built around the implementation outcomes of acceptability, adoption, and appropriateness, and it explored patients' contextual experiences. 62 , 63 Both patients who (1) enrolled and (2) were referred but did not enroll were recruited to complete telephone interviews within 48 hours of the referral. Because most COVID-19 test results were available within 12 hours, the referral and timing of a positive result were similar. Black and Latinx patients were sampled to make up 50% of all patient participants, and English- and Spanish-language options were available for surveys and interviews. Participation of Patients and Other Stakeholders Composition of Stakeholders The stakeholders for this project were chosen because of their personal experience with COVID-19 or caring for a loved one with COVID-19 or because their expertise in epidemiology and health equity made them essential members of our team ( Table 2 ). One of the patient stakeholders was also an emergency medicine physician who had experience treating patients with COVID-19 and managing their own diagnosis. Another patient stakeholder was an essential worker at a community health clinic who contracted COVID-19 at the height of the pandemic. The researchers worked with 2 community members who were part of the Black and Latinx communities and helped care for loved ones and community members with COVID-19. Finally, this project involved a stakeholder whose research focused on health disparities and who published peer-reviewed journal articles about racial and ethnic disparities in the outcomes of patients with COVID-19. Table 2 Description of the Patient and Stakeholder Investigators. Engagement in Research Many of the stakeholders worked in their community as clinicians, researchers, advocates, social workers, or care coordinators trying to gain access to care for their community members. They found this study fit within their current scope of work or as an extension to their current work. Stakeholders and patients stated they hoped to gain relief for their community and better access to care for communities that are often underserved. Stakeholders engaged with the researchers by email communications and formally gathered on virtual platforms during this project. They participated in several aspects of study development for all 3 aims: Stakeholders met virtually with the researchers in September 2020 to provide insight on protocol development, the choice of outcomes for the aim 1 study, and the design and implementation of the aim 2 intervention. The insights, suggestions, and feedback from this initial meeting greatly informed further revision of the aim 2 study design and its patient-facing materials. Stakeholders actively edited text copy in patient-facing materials in both English and Spanish. Stakeholders were among the first pilot testers for the aim 2 intervention in late October 2020. With their feedback, changes were made to the script, including the ordering, timing, and wording of the questions. In December 2020, stakeholders were engaged to provide suggestions and feedback on the preliminary analysis of aim 1 and the operationalization of aim 3. Stakeholders were engaged by email and met in June 2021 to provide insights on developing the aim 3 study interview guide. The insights, suggestions, and feedback from the meeting and email correspondence greatly informed further revision of the aim 3 study materials. Stakeholders were among the first pilot testers for the aim 3 study. With their feedback, changes were made to the script, including the ordering, timing, and wording of the questions. Notable Impacts The stakeholders were effective in helping the researchers develop and test the aim 2 texting script and instructional materials. The patient stakeholders provided valuable feedback about the frequency of messages and best times to send the text messages to patients enrolled in the program. They were also supportive in developing patient-centered language for the instructional pamphlet given to patients who received a pulse oximeter. Finally, the Spanish-language version of COVID Watch was developed with a professional Spanish-language translator. We then adjusted the language to be more patient centered based on pilot testing with our Spanish-speaking translator. With their insights on the best times to send messages, the stakeholders helped the researchers develop a patient-centered operational timeline for the frequency and time of day text messages were sent as part of the intervention. Furthermore, the patient stakeholders provided valuable information on the best ways to engage with patients for the aim 3 qualitative trial to ensure that we could meet our enrollment goals (eg, helping the researchers with the pitch, advising on the best times to call). Finally, the stakeholders were integral to formulating the interview questions in a patient-centered manner so that the researchers could best elicit the feedback that would be most helpful for the study team. Research Findings Our patient stakeholders thought that the study findings for aim 2—namely, that pulse oximeters given to patients for at-home use were not necessarily more useful than the text platform itself—were helpful because they could relay to their loved ones or organizations that a text program would be valuable without the added burden or expense of a pulse oximeter. Impact on the Research The presence of patient stakeholders on the research team provided a real-time connection to the population that the team was trying to help. The researchers were able to work with the patient stakeholders as “boots on the ground” and rely on their input for how to best increase engagement with patients, particularly Black and Latinx and Spanish-speaking populations, during this very uneasy and stressful time in our society. The engagement with the stakeholders had an impact on study participants in that they received messaging tailored to their needs, which hopefully enhanced their engagement with the program and potentially helped them during their experience with COVID-19. Penn Medicine as an institution benefited because the interventions were designed with the patient experience as a top priority, which enhances patient trust and further engagement with the institution. Collaboration With Patient Stakeholders Engaging with patient stakeholders was critical for effective implementation of the clinical trial and the qualitative interviews. Thinking about an intervention as a researcher or clinician can create abstraction or distance between the study design and the impact on the actual patient population. Having patient stakeholders actively engage in the development of the study materials and provide feedback on how the intervention could be improved helped the study team with enrollment and retention of study participants. No one understands what it is like to be a patient more than the patients themselves, and having this input at the design table created a more equitable and patient-centered research trial. Communication and Formal Meetings Online communication was the best tool for our research team. Because this trial took place during the height of the pandemic, it was important for the researchers and stakeholders to be able to engage remotely. The standard approach was to set up a meeting and provide the stakeholders with study materials to review beforehand that would then be discussed during the meeting (on Zoom or another platform). After we received and incorporated their feedback, the stakeholders became our pilot testers, an experience they told us was personally rewarding. They reported feeling valued by being able to provide their perspective and have their feedback incorporated into the study design. The research team did not attempt any in-person meetings with the stakeholders because of the pandemic. Providing a remote option would have been welcomed even in the absence of the pandemic. The patient stakeholders lived busy lives with many responsibilities and varied schedules, so it was important for the researchers to be able to provide a remote option for them. Aim 1 Conduct a retrospective quantitative evaluation by using secondary data (obtained from the EHR system and the National Death Index) to evaluate clinical outcomes for COVID Watch patients, particularly Black and Latinx patients . Justification for This Aim With each surge of COVID-19 cases during the pandemic, hospitals in several regions experienced severe capacity strain. 64 Nearly 90% of patients with COVID-19 were asked to self-isolate and monitor their symptoms at home. 1 , 2 , 65 , 66 Outpatient remote monitoring of patients with COVID-19 was needed because (1) patients with COVID-19 can decline rapidly and unpredictably and (2) EDs and hospitals were overwhelmed during surge periods of high community incidence rates and prevalence. 67 , 68 Without active monitoring, patients with COVID-19 who are worsening at home may delay seeking care or face delays when they do seek care. 69-71 Remote monitoring has the potential to facilitate ED and hospital-level care for patients who need it while supporting access to care for patients who can safely remain at home. 8 , 21 , 29 Toward these goals, Penn Medicine developed COVID Watch, an automated text message-based remote monitoring program with 24/7 clinical support. 21 The aim 1 study compared outcomes for patients enrolled in COVID Watch with patients who were eligible to enroll but received usual care. Our hypothesis was that COVID Watch enrollment would be associated with reduced mortality. Methods Overview This was a retrospective cohort study that followed the Strengthening the Reporting of Observational Studies in Epidemiology guideline for reporting observational studies. It was approved by the IRB at the University of Pennsylvania. Setting Penn Medicine is an academic health system serving large portions of southeast Pennsylvania, New Jersey, and Delaware, with 6 hospitals and hundreds of outpatient practices. Participants We included all community-dwelling adults who tested positive for COVID-19 at Penn Medicine as outpatients between March 23 (the start of COVID Watch) and November 30, 2020, and determined whether they were enrolled in COVID Watch vs usual care. We excluded patients who did not meet the eligibility criteria for COVID Watch: Those who were under 18 years old, actively enrolled in home health services or hospice, or currently in long-term care (eg, skilled nursing facility, long-term acute care, or acute rehabilitation) were ineligible. To enroll in COVID Watch, patients needed to have a cell phone that could receive text messages. To further reduce potential for bias, we excluded those who were tested in a location where COVID Watch enrollment had not yet begun, were previously in long-term care or hospice, or had a documented “do not resuscitate” or “do not intubate” code status before COVID-19 test collection. If patients had multiple positive COVID-19 tests, the first positive test was chosen as the index test. We excluded patients enrolled in COVID Watch more than 7 days before or after the date of their index positive COVID-19 test collection to avoid attribution of outcomes to COVID Watch enrollment for other episodes of care (eg, repeat COVID-19 testing) ( Figure 4 ). Figure 4 Study Sample Flowchart and Patients Excluded. Interventions and Comparators or Controls Community-dwelling adults (>18 years old) with presumed or confirmed COVID-19 could be enrolled in COVID Watch through an EHR order. Health care workers enrolling the patient in COVID Watch through an electronic order did not have to be physicians; they included nurses, social workers, and care coordinators involved in the patient's care in the health system. Enrollment often took place during or after an HCP had a conversation with the patient. Patients were not required to have established care with a Penn Medicine HCP; for example, a patient may have only received COVID-19 testing at Penn Medicine. The design of COVID Watch is detailed in Figures 5 through 7 . The first patient was enrolled on March 23, 2020. Figure 5 COVID Watch Text Messaging Algorithm. Figure 6 COVID Watch Clinical Escalation Algorithm. The enrollment order resulted in an instantaneous text message sent to the patient introducing them to the program. If the patient texted back confirming they wanted to engage with the program, they began receiving twice-daily automated text message check-ins asking 1 question: “How are you feeling compared to 12 hours ago: better, same, or worse?” Patients replying “worse” received a follow-up question: “Is it harder than usual for you to breathe: yes or no?” A “yes” response generated an alert for a telemedicine clinician to contact the patient by phone within 1 hour. Patients were instructed that outside the twice-daily check-ins, they could text “worse” at any time to trigger a clinician callback. During the daytime, nurses were the primary responders, with support from nurse practitioners and physicians. At night, escalations were directly relayed to on-call nurse practitioners and physicians. COVID Watch's clinical team received regular training and guidelines for managing COVID-19 cases. After an escalation, clinicians could provide reassurance, advise how best to manage symptoms at home, prescribe medications, or redirect patients to the ED. The program continued for 14 days from enrollment, at which point patients were offered the option to remain enrolled for 7 more days. A Spanish-language version of COVID Watch was made available on May 18, 2020. Aim 1 Outcomes Outcomes were ascertained within 30 days of the COVID-19 test collection date. We used an intent-to-treat approach in which any patient enrolled in COVID Watch was included in the COVID Watch group even if they did not respond to any text messages. The primary outcome was any-site mortality. We separately analyzed deaths that occurred in hospital vs out of hospital. Secondary outcomes included rates of total ED encounters (discharges and hospitalizations), hospitalizations (inpatient admissions and observation), and outpatient encounters (in-person office visits and telemedicine, including video or telephone visits). We tabulated a composite outcome of DAOH, which factors in death and the ability to remain out of the hospital by accounting for ED visits and, if hospitalized, the length of stay. 44-47 As a secondary analysis, we extended the follow-up window to 60 days for mortality and health care utilization measures. Among patients who needed acute care, we tabulated the time from the collection of the positive test to ED presentation, the ED vital signs, the length of stay if admitted to the hospital, and whether the patient needed intubation and mechanical ventilation. Finally, we tabulated process outcomes among those enrolled in COVID Watch, including how many text message check-ins they responded to, the proportion who triggered an escalation to the on-call nurse, and the triage recommendations of the on-call nurse. Covariates We collected patients' COVID-19 test data, age, sex, race, ethnicity, primary insurance, county of residence, and household income derived from zip code median values. We included comorbid conditions known to be associated with severe COVID-19 or with treatment adherence. 4 , 72-76 We captured whether patients had a listed primary HCP and baseline health care utilization, including the frequency of encounters with Penn Medicine in the prior year (ED visits, hospitalizations, and outpatient encounters). Data Collection and Sources We derived patient-level sociodemographic and clinical characteristics from Penn Medicine's EHR system. We derived patient mortality, clinical encounter details (ED visits, hospitalizations, outpatient office visits, and telemedicine), and documented details (eg, encounter dates and in-hospital vs out-of-hospital mortality) from Penn Medicine's EHR system and a regional health information exchange containing data from 53 surrounding hospitals in Pennsylvania, New Jersey, and Delaware. 77 Analytical and Statistical Approaches All extracted variables were checked for outliers and missingness. Only 21 patients out of 7865 who met inclusion criteria (0.27%) were missing any covariate data and were excluded from the primary analysis. To account for imbalances on covariates between comparison groups in the outpatient cohort, we estimated propensity scores (the probability of enrollment in COVID Watch) using logistic regression. After using inverse probability of treatment weighting, we found that all covariates achieved balance between patients enrolled in COVID Watch and usual care, with standardized mean differences less than 0.1. 78 Outcome models weighted by inverse probability of treatment weights were either logistic for binary outcomes or linear for continuous outcomes. We conducted a priori subgroup analyses of outcomes by race and ethnicity. To check the robustness of our findings, we assessed the sensitivity of our mortality results to potential unmeasured confounding by using the Rosenbaum Γ approach. We also tabulated any deaths that occurred in patients with missing covariates (n = 21). We also conducted a per-protocol analysis, excluding any patients enrolled in COVID Watch who did not respond to any text messages including the enrollment invitation text; for this analysis, new propensity scores were estimated for this select sample. Finally, we tabulated baseline characteristics and diagnoses of patients who died. All analyses were conducted with R, version 3.6.0, software (R Foundation for Statistical Computing). All analyses used 2-sided statistical tests, and P < .05 was considered statistically significant. Additional Methodological Details Selection of outcomes and follow-up period Our IRB protocol was submitted in July 2020 and prespecified the main outcomes reported in Tables 6 and 7 later in this report. After our PCORI grant was funded (August 18, 2020) we met with our patient and stakeholder advisory panel on August 25, 2020. Stakeholders identified survival as the outcome most important to them, and thus we selected 30-day mortality as our primary outcome. Stakeholder engagement in driving the research question was important to our team of investigators and required of this grant. Table 6 Mortality Outcomes. Table 7 Health Care Utilization Outcomes. The journal review process for our aim 1 manuscript gave us additional time to ascertain follow-up outcomes. At the request of the editors and peer reviewers, we added 60-day outcomes as a secondary analysis. Outcomes presented in this manuscript were pulled in July 2021, allowing 7 months for out-of-hospital death notifications to make it into the EHR. National Death Index data were not yet available at the time of manuscript submission for this aim. Calculation of DAOH Traditional calculation of DAOH factors includes inpatient hospital days and mortality over a given period of follow-up. 47 , 79 Given the nature of our intervention to send patients to the ED when needed and avoid ED visits when not needed, however, we sought to capture trips to the ED as an event in which the patient is not “out of hospital.” Our adjusted DAOH algorithm was as follows: Start of the 30-day period (day 0): – For patients tested in outpatient setting, date of COVID-19 test collection – For patients tested in the hospital (ED and inpatient discharges), the date of discharge – Day 1: the calendar date after the index event. Events counted between day 1 and day 30: ED days Inpatient days Days after death Any subsequent ED or hospital contact counts for the entire day of that contact: – Do not double-count a day because it has an ED event and a hospital event Propensity score modeling and outcome estimation It is common practice to specify a fully parametric model (eg, logistic regression) to obtain predictions such as the propensity score. With a large number of covariates, however, machine learning approaches such as random forests and neural nets are often preferred because of their flexibility to better discover complex relationships, including interactions, higher-order terms, and nonlinear functions. The challenge is to decide which machine learning approach to use. We therefore began our propensity score modeling by using an ensemble machine learning approach called SuperLearner, which weights both parametric and nonparametric models to obtain optimal predictions through cross-validation. 80 , 81 The advantage to ensemble machine learning approaches such as SuperLearner is that the analyst does not have to choose 1 algorithm over others. SuperLearner combines multiple algorithms (learners) into a single algorithm and provides a prediction function. SuperLearner uses cross-validation to obtain an optimal prediction by minimizing the loss function (such as the best cross-validated mean squared error). SuperLearner allows a wide range of parametric, semiparametric, and nonparametric algorithms, models, learners, and wrappers, including generalized linear models, bayesian additive regression trees, random forests, boosting, bagging, neural nets, ridge regression, least absolute shrinkage and selection operator, and support vector machines. In our analysis, we specified logistic regression, bayesian additive regression trees, random forests, support vector machines, and caret models simultaneously in the model fitting process. We found that logistic regression performed the best and received almost full weight compared with the other algorithms. Therefore, we presented our results based on propensity score weights estimated from logistic regression (which are nearly identical to our SuperLearner results). Propensity score distributions were evaluated for nonoverlap to ensure that there were no serious positivity violations. Propensity scores and weights obtained with SuperLearner are presented in Figures 8 and 9 . Figure 8 Overlap of Propensity Scores for Enrollment in COVID Watch, by Treatment Group. Figure 9 Change in Standardized Mean Differences Before and After Propensity Score Weighting. In our final models, we used only inverse probability weights and did not further adjust for covariates. Our reasoning for this approach was threefold. First, we used propensity score weighting to achieve balance between the treatment groups, and this was achieved. Second, we believed that any adjustment of covariates would lead to dangerous extrapolations of the model to covariate spaces that were not supported by the data. Third, we targeted a marginal estimand because of its relevance to policymaking rather than a conditional treatment effect that would have resulted from adjusting for further covariates. As a sensitivity analysis, we did further adjust for key covariates that could affect the outcome but found nearly identical results. For example, for difference in death rate at 30 days: Primary analysis with propensity score weighting: −0.19% (95% CI, −0.32% to −0.06%); P = .005 Primary analysis with propensity score weighting plus regression adjustment for age categories, sex, place of residence, month, income: −0.18% (95% CI, −0.31% to −0.05%); P = .005 For difference in death rate at 60 days: Primary analysis with propensity score weighting: −0.25% (95% CI, −0.41% to −0.10%); P = .002 Primary analysis with propensity score weighting plus regression adjustment for age categories, sex, place of residence, month, income: −0.25% (95% CI, −0.40% to −0.06%); P = .002 Changes to the Original Study Protocol Our IRB protocol prespecified the main outcomes reported in Tables 6 and 7 later in this report. The protocol stated we would use a mixed-effects model with random intercepts to account for clustering by testing site. In practice, however, it was logistically infeasible to universally measure testing site in the EHR data. Some tests were ordered remotely, others were done in pop-up testing tents, and these data were not well captured in the EHR. Therefore, we used county of residence as a variable in the propensity score model. Participants were well balanced across counties. Because of the many counties, we summarized the counties by state of residence in Tables 3 and 4 and in Figure 9 . As described earlier, after considering the views of our patient and stakeholder advisory panel (see “ Acknowledgments ”), we selected 30-day mortality as our primary outcome. We also added 60-day outcomes as a secondary analysis in response to peer review feedback. Table 3 Unadjusted Baseline Characteristics. Table 4 Propensity Score Weighted Baseline Characteristics. Results Baseline Characteristics After ineligible patients were excluded ( Figure 4 ), 7865 outpatients were available for analysis, 3488 in COVID Watch and 4377 in usual care. Patients in COVID Watch were enrolled a mean (SD) of 1.8 (2.3) days after the date of COVID-19 test collection. Before propensity score weighting, patients enrolled in COVID Watch were similar in age to those who received usual care but were less likely to be male (37.1% vs 42.5%) and more likely to identify as non-Hispanic Black (47.9% vs 32.1%), have public insurance (29.3% vs 26.6%), not have a primary care physician (PCP) (28.3% vs 24.5%), have a higher mean body mass index (33.6 vs 29.7), have certain comorbidities (hypertension, diabetes, asthma), have a lower median household income ($51 491 vs $60 638), have higher rates of ED and office visits in the past year, and become eligible for the study in the earlier months of the pandemic ( Table 3 ). Covariates were well balanced after propensity score weighting ( Table 4 , Figure 9 ). Outcomes Of the 3488 patients enrolled in COVID Watch, 3028 (86.8%) engaged by responding to at least 1 text (mean, 23 check-in responses). Of patients who engaged, 434 (14.3%) triggered an escalation to a nurse, with a mean response time of 24 minutes ( Table 5 ). Table 5 COVID Watch Program Metrics. At 30 days, 3 out of 3488 (0.09%) of those enrolled in COVID Watch died compared with 12 out of 4377 (0.27%) who received usual care ( Table 6 ). Of deaths, 0 in COVID Watch occurred outside the hospital vs 6 (37.5%) of those who received usual care. Among in-hospital deaths, 2 out of the 3 in COVID Watch occurred in hospitals outside Penn Medicine compared with 5 out of 6 in the usual care group. At 60 days, there were 2 additional deaths among those enrolled in COVID Watch and 4 additional deaths among those who received usual care. After propensity score weighting and modeling, at 30 days those enrolled in COVID Watch had an odds ratio for overall mortality of 0.32 (95% CI, 0.12-0.72), with a difference of −1.8 deaths per 1000 patients (95% CI, −3.1 to −0.5; P = .005). At 60 days, COVID Watch outcomes remained consistently better, with a difference of −2.5 deaths per 1000 patients (95% CI, −4.0 to −0.9; P = .002). Furthermore, COVID Watch was associated with equitable treatment benefits, with White, Black, and Hispanic subgroups all experiencing lower mortality by 60 days ( Table 6 ). Within 30 days COVID Watch patients had a total of 489 ED encounters (121 per 1000); 384 (78.5%) occurred within Penn Medicine and 105 (21.5%) outside the health system. Of the 489 ED encounters by COVID Watch patients, only 62 (12.7%) were by patients who never engaged with COVID Watch. Usual care patients had a total of 252 ED encounters (50 per 1000), with 161 (63.9%) occurring within Penn Medicine and 91 (36.1%) outside the health system. After propensity score weighting, we found COVID Watch patients had a higher mean number of ED encounters per patient (adjusted difference, 0.06 [95% CI, 0.04-0.07]; P < .001) and mean number of hospitalizations per patient (adjusted difference, 0.03 [95% CI, 0.01-0.04]; P < .001). COVID Watch patients had a similar number of office visits but a greater mean number of telemedicine encounters per patient (adjusted difference, 0.31 [95% CI, 0.27-0.34]; P < .001) than usual care patients within 30 days after their positive COVID-19 test sample was collected. When ED and hospital use were factored in with mortality, the mean number of DAOH within 30 days was slightly lower among COVID Watch patients (adjusted absolute difference, −0.16 [95% CI, −0.26 to −0.06]; P < .001) but similar at 60 days (adjusted absolute difference, −0.13 [95% CI, −0.29 to 0.3]; P = .102) ( Table 7 ) as deaths accrued and health care use diminished. Among patients who presented to any hospital (within Penn Medicine or outside) for the first time in the 30 days after their date of COVID-19 test collection, COVID Watch patients presented to the ED sooner (6.6 days vs 8.9 days), with a propensity score weighted difference of −1.9 days (95% CI, −2.9 to −0.9; P < .001). Among the subset of patients who presented to the ED within Penn Medicine ( Table 8 ), COVID Watch enrollees compared with usual care patients presented sooner (6.1 vs 9.0 days) (propensity score weighted difference, −2.9 days [95% CI, −4.1 to −1.7]; P < .001). During their ED evaluation, there were no statistically significant differences in vital signs or in need for intubation and ventilation. Compared with the 10.8% of usual care patients who were treated with dexamethasone in Penn Medicine hospitals, however, the 11.3% of COVID Watch patients who were treated with dexamethasone received it sooner (propensity score weighted difference, −3.0 days [95% CI, −5.6 to −0.4]; P = .024). Table 8 Timing and Severity of Initial Presentations to Health System EDs Within 30 Days . Sensitivity Analyses To assess the sensitivity of mortality results to potential unmeasured confounding, we conducted a sensitivity analysis based on the Rosenbaum Γ approach by using the R package rbounds (version 2.1). The Rosenbaum bounds analysis demonstrated there would need to be 1.8 times greater odds of differential assignment to COVID Watch attributable to unobserved factors, a substantial amount of unmeasured confounding needed to reverse the statistically significant findings ( Table 9 ). Given the large number of important covariates we have accounted for in our analysis, it is unlikely that such an impactful covariate was not included; therefore, we think our results are robust to potential unmeasured confounding. Table 9 Sensitivity Analysis for Unobserved Confounding . There were also no deaths among the 21 patients excluded from the outpatient cohort due to missing covariate data ( Table 10 ). We assessed the effect of reincluding patients with do not resuscitate (DNR) or do not intubate (DNI) directives in place before COVID-19 testing and patients with any prior evidence of acute rehab, long-term care facility, or skilled nursing facility services, even if these patients had no documentation of actively using these services at the time of COVID-19 testing, given that these were not exclusion criteria for COVID Watch enrollment. Reincluding patients in the sample who had DNR or DNI orders (COVID Watch, n = 3; usual care, n = 19) or who had any prior evidence of long-term care in the past year (COVID Watch, n = 24; usual care, n = 64) only strengthened the primary finding that patients enrolled in COVID Watch had significantly lower mortality rates than those in usual care ( Table 10 ). Table 10 Sensitivity Analysis for Deaths Within 30 Days Among Those Excluded From the Primary Analysis. We also found that 2 of the 5 deaths in COVID Watch occurred among the 13.2% of the patients who never engaged with it. The per-protocol analysis of patients who engaged with COVID Watch indicated even stronger treatment effects: an odds ratio for death of 0.25 (95% CI, 0.10-0.55), with a difference of −2.8 deaths per 1000 patients (95% CI, −4.3 to −1.3; P = .001) ( Table 11 ). Finally, baseline characteristics of patients who died were varied but not statistically significant from each other across treatment groups ( Table 12 ), and coded diagnoses of in-hospital deaths were consistent with COVID-19 being the primary cause of death ( Table 13 ). Table 11 Per-Protocol Sensitivity Analyses of Outcomes, by COVID Watch Engagement. Table 12 Baseline Characteristics of Patients Who Died Within 60 Days. Table 13 Coded Diagnoses of In-Hospital Deaths. Discussion This study has 4 main findings. First, the mortality rate for community-dwelling adults with COVID-19 was significantly lower among those in COVID Watch compared with usual care, even after adjustment for differences in patients' clinical and sociodemographic characteristics. Second, more than one-third of the deaths in the usual care group occurred outside the hospital compared with none among those treated with COVID Watch. Third, patients in COVID Watch were more likely to present to the hospital, and they presented earlier. Fourth, all major racial and ethnic subgroups experienced lower mortality rates when enrolled in COVID Watch. These findings imply COVID Watch is associated with a 64% relative reduction in the risk of death and that 1 life was saved for every 400 patients enrolled—or about 1 every 4 days during peak enrollment weeks. Although remote monitoring programs for patients infected with COVID-19 outside hospital settings have been previously described, 82 , 83 including 1 study from Kaiser Permanente that reported unadjusted morality rates of 2.3% in usual care and 1.3% with remote monitoring, 84 we believe ours to be the first risk-adjusted study to demonstrate improved survival. Public health messaging strongly encouraged staying home to promote social distancing and decrease hospital strain during the height of the pandemic. 85 , 86 Those messages, however, were accompanied by decreases in patients with emergent conditions presenting to the ED and increased out-of-hospital deaths. 69 , 70 , 87-95 In this study, 37.5% of deaths among patients who received usual care occurred outside the hospital compared with none among patients in COVID Watch, consistent with the interpretation that COVID Watch exerted its effect by increasing vigilance over those at home and efficiently sorting them into those who would benefit from the ED and those who would not. 69 , 88 Further evidence supports this mechanistic explanation. Patients in COVID Watch were more likely to present to the hospital and presented earlier, probably increasing their ability to benefit from the care they received. For example, dexamethasone reduces mortality and length of stay for patients with COVID-19, 96 and the benefit may be larger if the drug is administered earlier in the disease course. 97-101 We found that among those who received dexamethasone in Penn Medicine hospitals, patients in COVID Watch received the medication 3 days earlier on average. We also found that the treatment effects associated with COVID Watch were stronger among those who engaged with the remote monitoring service. The constellation of these findings is consistent with the view that COVID Watch operates as an early warning and referral system for community-dwelling patients. 8 The combination of technology-based automated remote monitoring 102 and clinician support may be necessary for the observed clinical effect. Because COVID Watch was automated, only 2 to 4 staff members were needed to oversee over 1000 patients at a time, far fewer than personnel-intensive calling systems. 82 , 83 Because it relied on symptom self-report, it did not require dedicated temperature sensors or pulse oximetry. 21 , 49 , 83 , 103 , 104 The use of additional equipment in the home varied substantially across remote patient monitoring programs, and its incremental value was unknown—a knowledge gap our aim 2 trial sought to bridge. 105 Future research is needed to determine whether this type of monitoring service could be adapted to other acute conditions (eg, pneumonia, cellulitis) and chronic conditions (eg, asthma, diabetes) in which automated text check-ins and low-barrier access to rapid clinical assessment and ED triage could improve outcomes. In our study, non-Hispanic Black patients were more likely to be enrolled in COVID Watch than usual care, and Hispanic patients were about as likely to be in either group. White, Black, and Hispanic populations also had significantly lower mortality when enrolled in COVID Watch, and the overall mortality rates were lower relative to reports nationally. 84 , 95 , 106 , 107 These findings suggest no substantial racial or ethnic barriers to program enrollment or differences in its effectiveness, and they indicate that this type of remote monitoring service has the potential to reduce racial disparities in regions in which Black and Hispanic patients have lower access to care and higher mortality rates. Strengths and Limitations This study had limitations. First, we could observe deaths in our hospitals and in hospitals outside our health system from a health information exchange linkage, but we might have had incomplete ascertainment of out-of-hospital deaths. COVID Watch patients were highly engaged with Penn Medicine, however, replying to a mean of 23 text message check-ins through the program, suggesting their deaths would be more likely to be ascertained. Second, we could not capture hospital use outside the geographic area of the health information exchange. Fortunately, 99.4% of patients in our study sample had residential zip codes within the geographic region covered by the health information exchange, decreasing potential for incomplete capture of hospital use. Third, we could not ascertain why patients were enrolled in COVID Watch or whether patients were verbally offered COVID Watch and declined it. Although 97% of people living in the United States owned cell phones capable of receiving text messages in 2020, 108 we could not determine whether all patients in the control group owned a cell phone that could receive text messages or the ability to engage with text messages. We also could not capture social needs, the nature and timing of symptoms before the testing date, and other unobserved confounders that may have affected outcomes. Higher rates of characteristics associated with worse outcomes from COVID-19 were observed in the COVID Watch group, however, including the lack of a PCP, Black race, residing in a lower-income zip code, greater body mass index, higher rates of high-risk comorbid conditions, and higher proportion treated early in the pandemic, 67 , 76 , 109-111 all suggesting higher expected mortality among the COVID Watch group. Furthermore, our sensitivity analyses indicated there would need to be 1.8 times greater odds of differential assignment to COVID Watch than the control group that was attributable to unobserved factors. Given the large number of important covariates we have accounted for in our analysis, it is unlikely that such an impactful covariate was not included. Fourth, outcomes measured reflect care received at a single health system, a select set of hospitals in a specific region of the United States, which may limit generalizability of our findings. This study, however, included populations with a diverse set of comorbidities and sociodemographic characteristics. Relatedly, we were unable to measure clinical status and treatments provided during hospital encounters outside our health system. Given the differential use of hospitals outside our health system, clinical treatment rates observed within our health system should not be extrapolated to the subset treated in hospitals outside our health system. This study also had strengths. It reflected what was to our knowledge the largest and most comprehensive sample and evaluation of a remote monitoring service for COVID-19 in the United States. There was careful adjustment for differences in patient characteristics. The effect size was large and could be explained by plausible mechanisms supported by secondary analyses. Conclusion Enrollment in an automated text messaging service among community-dwelling adults newly diagnosed with COVID-19 in outpatient settings was associated with reduced mortality. This reduction may have been driven by higher rates and earlier presentation to the ED by those benefiting from early interventions. These results reveal a model for outpatient health system management of COVID-19 and possibly other conditions where the early detection of clinical declines is critical. Aim 2 Conduct a pragmatic randomized controlled trial to determine the incremental benefit of providing home fingertip pulse oximetry for patients enrolled in COVID Watch. Justification for This Aim By the beginning of 2021, the United States was averaging more than 761 000 new cases of COVID-19 per day, 64 with widespread reports of severe health care capacity strain. 112 As our aim 1 study found, remote outpatient monitoring for dyspnea can efficiently help large populations of patients with COVID-19 remain safely at home while facilitating timely hospital access when needed. 21 , 113 , 114 Reports of silent hypoxia—infected patients with no dyspnea despite profound hypoxemia—raised concerns, however, that self-reported dyspnea may not be sufficiently sensitive as an indicator of respiratory distress in all patients with COVID-19. 32 , 115-119 For this reason, home pulse oximetry was recommended in the United States and internationally to improve early detection and management of respiratory decline. 120-122 The United Kingdom's National Health Service offered home pulse oximeters to all patients with COVID-19 who were symptomatic and more than 65 years of age or symptomatic and at high risk for poor outcomes (eg, concomitant diabetes or asthma diagnoses). 123 Distributing pulse oximeters made clinical sense because peripheral Sp o 2 less than 92% independently predicts mortality in patients hospitalized with COVID-19, 124-126 and pulse oximetry offers an objective method to identify infected patients who may need hospitalization for supplemental oxygen, advanced pharmacotherapies (eg, corticosteroids, interleukin-6 receptor blockers), or mechanical respiratory support. 49 , 96 , 120 Despite the conceptual appeal, there had been no randomized evaluations of adding pulse oximetry to remote monitoring of subjective dyspnea on clinical outcomes. We conducted a pragmatic randomized trial to determine whether the addition of home pulse oximetry monitoring provided clinical benefit to the management of COVID-19 infection in home-dwelling patients compared to remote monitoring of subjective dyspnea alone. Methods Overview This trial was conducted between November 29, 2020, and February 5, 2021, within Penn Medicine, a 6-hospital health system spanning southeastern Pennsylvania and New Jersey. Penn Medicine had previously developed COVID Watch, an automated, text message-based program to monitor patients with suspected or confirmed COVID-19 at home, escalating their care to telemedicine clinicians based on patients' report of dyspnea alone. 21 Among patients with confirmed COVID-19 infection, COVID Watch was associated with a 68% mortality reduction, with 1.8 fewer deaths per 1000 patients. 113 , 114 Invitation to participate in COVID Watch became standard at Penn Medicine, and 30 000 patients had been enrolled by fall 2022. During this trial, patients with confirmed or suspected COVID-19 in the outpatient setting or upon discharge from ED or hospital could be enrolled in the COVID Watch program through the EHR or as part of standard medical care. Patients who were receiving home health services, were in long-term care, or were in an alternative remote monitoring program were not eligible for the COVID Watch clinical program. At the time of the trial, patients who tested positive for COVID-19 were also provided an additional opportunity to enroll in COVID Watch through an automated text-message results notification process. During business hours while research coordinators were available, patients who were started in the COVID Watch clinical program as part of standard medical care were randomly assigned 1:1 to receive the standard version of COVID Watch, in which patients were queried for dyspnea twice a day for 2 weeks, or to COVID Watch supplemented with Sp o 2 monitoring with a home pulse oximeter (COVID Watch plus pulse oximetry). Randomization was automated by Way to Health, the web-based text messaging platform that provided the COVID Watch service, 21 using simple randomization via a random number generator. Given the potential for benefit from pulse oximetry and the need to deliver equipment as soon as possible after enrollment into the program, automatic random assignment for all patients enrolled in the program was seen as the most pragmatic approach rather than risking potential delays by manually confirming eligibility with research staff. The primary hypothesis was that provision of pulse oximetry would lead to more DAOH 45 , 47 , 113 , 127 over a 30-day period than COVID Watch alone. This could occur through reduced mortality, shorter hospital length of stay, or fewer ED encounters within 30 days of receipt of COVID Watch. The study was approved by the IRB at the University of Pennsylvania with a waiver of HIPAA authorization and consent and registered at ClinicalTrials.gov ( NCT04581863 ). Setting This trial was conducted within Penn Medicine, a 6-hospital health system spanning southeastern Pennsylvania and New Jersey. Penn Medicine is an academic health system serving large portions of southeast Pennsylvania, New Jersey, and Delaware, with 6 hospitals and hundreds of outpatient practices. Participants During the study period, all home-dwelling patients 18 years of age or older who were enrolled in COVID Watch during normal business hours were eligible and automatically enrolled in the trial to receive standard COVID Watch or COVID Watch plus pulse oximetry. Patients who should have been excluded from COVID Watch (because they were already receiving home health services, skilled nursing, inpatient rehabilitation, or an alternative remote monitoring program for patients) were uninvited from COVID Watch within 48 hours based on standard operational procedures. These patients as well as those with an invalid date of birth were excluded from the trial's analytic cohorts. Interventions and Comparators or Controls Control group Patients randomly assigned to the control group received an invitation to the standard COVID Watch (see Figure 7 ). If patients accepted the invitation, they then received twice-daily automated text messages in English or Spanish, asking, “How are you feeling compared to 12 hours ago: better, same, or worse?” Patients replying “worse” were asked, “Is it harder than usual for you to breathe: yes or no?” Patients who responded “yes” generated an alert to a team of telemedicine clinicians (nurses, advanced practice HCPs, and physicians) available 24/7 who were required to contact the patient within 1 hour. Patients could also text the word “worse” at any time to connect with a clinician, separate from the scheduled twice-daily check-ins. Clinicians called each patient who escalated a minimum of 3 times, every 5 minutes, leaving messages for nonresponders when possible. Clinicians followed standardized assessments and guidelines ( Figure 10 ) and used interpretation services when needed. Patients were enrolled for 14 days and were given the option to extend to 21 days or end earlier. Figure 7 COVID Watch Patient Instructions. Figure 10 Standard COVID Watch Clinical Management Protocol. Intervention group Patients randomly assigned to the intervention group received an invitation to standard COVID Watch monitoring and a free pulse oximeter with step-by-step user instructions ( Figures 11 and 12 ), which arrived by tracked expedited delivery within 2 days. Figure 11 COVID Watch Plus Pulse Oximetry User Instructions (English). Figure 12 COVID Watch Plus Pulse Oximetry User Instructions (Spanish). To confirm receipt of the pulse oximeter, patients were instructed to text the keyword “ready,” at which point the patient began receiving twice-daily check-in messages about dyspnea and Sp o 2 levels. Before device arrival, patients received standard COVID Watch check-in messages. If intervention patients did not text “ready,” a study team member called within 2 business days to remind the patient. Those with (1) new or worsening dyspnea, (2) a decrease in Sp o 2 by 3 or more percentage points to a level below 95%, or (3) a reading below an absolute value of 90% triggered an alert to the COVID Watch telemedicine clinicians ( Figure 13 ). Figure 13 COVID Watch Plus Pulse Oximetry Program Text Message Protocol. For intervention group patients, clinicians referenced the standard COVID Watch protocols supplemented by Sp o 2 -specific guidelines ( Figure 14 ). Figure 14 COVID Watch Plus Pulse Oximetry Clinical Management Protocol. Aim 2 Outcomes The prespecified primary outcome was the number of DAOH for patients with COVID-19 during the 30 days after study enrollment. Prespecified secondary outcomes included health care use telephone encounters, telemedicine visits, outpatient office visits, ED visits, hospitalizations, intubation, and mortality, derived from the health system's EHR system. To capture ED visits, inpatient hospitalizations, and mortality occurring outside Penn Medicine, participant records were linked to the regional health information exchange for 53 surrounding hospitals in Pennsylvania, New Jersey, and Delaware, 77 as was done for aim 1. 73 , 113 Sample Size Calculations and Power Although patients could be enrolled in COVID Watch at the clinician's discretion based on high clinical suspicion or COVID-19 tests conducted outside the health system or at home, the prespecified primary analytic sample was defined as the subset of randomly assigned patients who were confirmed to be COVID-19 positive through polymerase chain reaction testing within the Penn Medicine system. Assuming an SD of DAOH of 2.5, we estimated that an enrollment of 1078 participants with COVID-19 would provide 90% power to detect a difference of 0.5 DAOH between groups in the overall sample and 1.0 DAOH between groups among Black patients, assuming DAOH would be 28.8 in the control group. We sought to enroll at least 408 patients in the Black subgroup. The study team that assessed outcomes was blinded to assignment of interventions, and the entire study and clinical teams were blinded to outcomes until data collection was complete. Enrollment was stopped once it was determined that the minimum number of patients positive for COVID-19 in the overall sample and in the Black subgroup had been enrolled. Time Frame for Aim 2 This trial was conducted between November 29, 2020, and February 5, 2021. Data Collection and Sources Patients in both groups received weekly text message-based surveys while they were enrolled, asking about their health confidence and anxiety in managing COVID-19 at home as well as program helpfulness derived from previously validated instruments, 128 , 129 adapted to the context of COVID Watch and modified by a community and patient advisory committee. In addition, patients who escalated to a clinician were asked how they would have managed their care had they not been in COVID Watch. On program completion, all patients were texted a net promoter score (NPS) question asking them how likely (0-10) they were to recommend the program to a friend or colleague. 130 , 131 We calculated the NPS separately for the control and intervention groups by subtracting the percentage of patients who gave a rating of 0 to 6 from the percentage who gave a 9 or 10. Below are the text message survey questions administered on enrollment day 0, day 7, and day 14: Health confidence: – “How confident were you that you could manage your COVID symptoms at home? (1 = not at all confident to 5 = very confident)” Program helpfulness: – “How helpful has THIS program been for managing your COVID symptoms at home? (1 = not at all helpful to 5 = very helpful)” Anxiety: – “In the past 24 hours, how worried have you been about your COVID symptoms? (1 = very worried to 5 = not at all worried)” A text message survey question was administered after an escalation to a clinician. Self-management: – “If you were not in this program, what would you have done about your COVID symptoms today? Gone to the ER Contacted my doctor Stayed at home” Other Analytical and Statistical Approaches Primary analyses used an intention-to-treat approach. We also performed a per-protocol analysis including only patients who actively participated in their allocated protocol (eg, responded to at least 1 text message check-in). For both analyses, we compared mean DAOH between study groups using 2-sample t tests. We compared continuous variables using 2-sample t tests and categorical variables using χ 2 tests. Adjustment for covariates using regression was not deemed necessary because all sociodemographic and clinical characteristics were balanced between groups. We also compared continuous secondary outcomes, including mean number of outpatient visits, ED encounters, and hospitalizations, using t tests. We compared categorical secondary outcomes using the Fisher exact test. We completed a prespecified subgroup analysis by comparing outcomes for Black and Hispanic patients with those for White patients. Exploratory subgroup analyses were completed for age and enrollment context (inpatient, outpatient, and ED). All analyses were conducted in R, version 3.6.0. Changes to the Original Study Protocol Our aim 2 analysis plan stated we would use a mixed-effects model with random intercepts to adjust for residual imbalances and account for clustering by testing site. In practice, however, it was logistically infeasible to universally measure testing site in the EHR data. Some tests were ordered remotely, others were done in pop-up testing tents, and these data were not well captured in the EHR. Therefore, we used county of residence as a proxy variable for testing location. Adjustment for covariates using regression was not deemed necessary, however, because all sociodemographic and clinical characteristics, including county of residence, were balanced between groups. Results Patient Characteristics A total of 2097 patients were randomly assigned: 1056 to intervention (COVID Watch plus pulse oximetry) and 1041 to remain in the control group (COVID Watch). Of all randomly assigned patients, 1217 (58%) were confirmed to be COVID-19 positive by polymerase chain reaction test: 611 in the intervention group and 606 in the control group ( Figure 15 ). Study groups were balanced by sociodemographic and clinical characteristics ( Table 14 ). Figure 15 Enrollment and Random Assignment of Patients. Table 14 Characteristics of the Patients at Baseline. Main Outcomes In the prespecified intention-to-treat analysis, there was no significant difference in mean DAOH between patients with COVID-19 in the intervention and control groups (29.4 vs 29.5 days; P = .58; difference, −0.1 [95% CI, −0.4 to 0.2]) ( Table 15 ). There was no significant difference in 30-day telemedicine visits, outpatient office visits, ED visits, hospitalizations, intubation, or mortality. Primary and secondary health care utilization outcomes were also similar across all study patients. Those randomly assigned to the intervention had significantly more telephone encounters (3.3 vs 2.4, P < .001; difference, 0.9 [95% CI, 0.5-1.3]). Table 15 Prespecified Outcomes for Patients With COVID-19 Within 30 Days. The per-protocol analysis showed no significant difference in primary and secondary outcomes, with the exception of telephone encounters (3.4 vs 2.5, P < .001; difference, 0.8 [95% CI, 0.4-1.3]) ( Table 16 ). Table 16 30-Day Outcomes for Patients With COVID-19 (Per-Protocol Analysis). For patients who returned to the ED, there was no significant difference in the mean number of days to presentation (9.1 vs 10.0, P = .63; difference, −0.9 [95% CI, −4.3 to 2.6]) ( Table 17 ). Patients in the intervention group, however, were significantly more likely to present with a lower oxygen saturation (94.9 vs 96.9, P = .02; difference, −2.0 [95% CI, −3.9 to −0.2]) and need supplemental oxygen in the ED (19.4% vs 2.8%, P = .04; difference, 16.6% [95% CI, 2.4%-34.2%]). Table 17 Clinical Characteristics of Initial Presentations of Patients Positive for COVID-19 Who Returned to Emergency Departments Within 30 Days. A prespecified subgroup analysis was completed for patients based on race and ethnicity. It revealed no statistically significant differences in DAOH in Black and Hispanic patients compared with non-Hispanic White patients ( Table 18 ). Additional exploratory subgroup analyses were completed for patients based on age ( Table 19 ) and enrollment context ( Table 20 ). In patients 60 to 69 years of age, DAOH was lower in the intervention group than in the control group (28.2 vs 29.5; difference, −1.3 [95% CI, −2.4 to −0.1]) ( Table 21 ). Table 18 Health Care Utilization 30-Day Outcomes of Patients Positive for COVID-19, by Race . Table 19 Health Care Utilization 30-Day Outcomes of Patients Positive for COVID-19, by Age Group . Table 20 Health Care Utilization 30-Day Outcomes of Patients Positive for COVID-19, by Enrollment Context . Table 21 Clinical Characteristics of Initial Presentations of Patients Positive for COVID-19 Who Returned to Emergency Departments Within 30 Days. Patient-Reported Outcomes Patients assigned to the intervention group reported that the program was more helpful for managing their symptoms at home on a scale of 1 to 5 (5, very helpful) than those in control group on day 7 (4.4 vs 4.1; P = .02; difference, 0.3 [95% CI, 0.1-0.5]), although there was no difference in this outcome at day 14. Patient-reported confidence and anxiety did not differ between the 2 groups at any time point. When patients were asked how they would have managed their symptoms without the program, groups were similar in terms of stating they would have gone to the ED, called their PCP, or stayed at home (see Table 16 ). The intervention protocol received a higher NPS (65 vs 45), although both exceeded the means in measured industries. 131 Enrollment and Process Metrics Patients in the intervention group used considerably more clinical resources, including longer enrollment in the program (12.2 vs 11.1 mean days; difference, 1.1 [95% CI, 0.2-2.1]; P = .02), a greater likelihood of extending their enrollment (26.7% vs 11.9%; difference, 14.8% [95% CI, 10.4%-19.2%]; P < .001), more texts per day of enrollment (3.4 vs 2.3; difference, 1.1 [95% CI, 0.9-1.3]; P < .001), and a higher proportion of escalations to clinician support (39.6% vs 17.8%; difference, 21.8% [95% CI, 16.8-26.7%]; P < .001) ( Table 22 ). Table 22 General Enrollment and Program Metrics. Of those who triggered an escalation, a higher proportion in the intervention group were told to continue to self-monitor (73.8% vs 58.1%; difference, 15.7% [95% CI, 0.1%-24.3%]; P < .001), whereas a lower proportion were scheduled for an urgent telemedicine appointment (6.0% vs 14.5%; difference, −8.5% [95% CI, −15.9% to −3.2%]; P = .002). Discussion This trial had 2 main findings. First, among patients with COVID-19 managed at home, adding home pulse oximetry to monitoring for subjective dyspnea provided no advantage in DAOH within 30 days of enrollment. These results remained the same across subgroups defined by race, age, and clinical characteristics. Second, patients managed with pulse oximetry used more clinical resources. The combination of these findings reveals home pulse oximetry may be ineffective and inefficient in supporting the management of patients with COVID-19 in outpatient settings. There are multiple potential explanations for these findings. First, the existing COVID Watch program's monitoring of subjective dyspnea, previously shown to have a benefit in reducing mortality for outpatients with COVID-19, 113 may have been sufficiently sensitive to capture patients with worsening respiratory illness, suggesting that pulse oximetry offered limited additional clinical value. This study did not test the impact of home Sp o 2 monitoring against the absence of remote monitoring or providing home pulse oximeters without providing on-call, remote clinical support. A second and related explanation is that pulse oximetry may have increased hospital utilization for some patients with minimal subjective dyspnea but objectively low Sp o 2 levels while helping other patients with more moderate subjective dyspnea but normal Sp o 2 levels avoid hospitalization, resulting in our observed null findings between trial groups. Indeed, our findings show that patients with home Sp o 2 monitoring were more likely to be instructed to self-monitor at home and yet were also more likely to return to the ED with lower Sp o 2 and were more likely to need supplemental oxygenation. Strengths and Limitations This study had limitations. One is that we did not compare outcomes relative to usual care because COVID Watch text message monitoring has become the standard of care in our health system. Our aim 1 research demonstrated its superiority over care without COVID Watch, and the COVID Watch program required little additional variable cost. 113 A second limitation is that although most patients who were assigned to the pulse oximeter intervention initiated the Sp o 2 monitoring procedures (77.7%), not everyone did. We did not find a significant difference in outcomes in the per-protocol analysis, however, thereby limiting the intervention sample to those who initiated Sp o 2 monitoring. A third limitation is that, in part because of device shipping times, mean days from COVID Watch initiation to Sp o 2 monitoring initiation was 2.4 days. It is possible that the most clinically beneficial period for Sp o 2 monitoring occurs in the first few days after COVID Watch initiation. Patients who returned to the ED, however, did so at a mean of 7 days after enrollment, suggesting that any delays in pulse oximeter delivery or initiating Sp o 2 monitoring procedures was unlikely to have affected clinical outcomes. This study also had strengths, including its pragmatic randomized design, its large sample size, and results that were robust and consistent across subgroup analyses, without differences in engagement or clinical outcomes by race, ethnicity, or age. Conclusion Despite the theoretical advantages and widespread recommendation of home pulse oximetry monitoring for home-dwelling patients with COVID-19, adding objective Sp o 2 measurement to a text messaging-facilitated telehealth program, COVID Watch, provided no clinical benefit while consuming more clinical resources. Notably, we did not compare remote monitoring plus pulse oximetry with usual care and therefore cannot draw conclusions on the value of pulse oximetry relative to usual care. Based on prior findings indicating that enrollment in COVID Watch was associated with lower mortality than usual care, we conclude that text messaging-facilitated telehealth remote monitoring is an effective and scalable approach for managing COVID-19 at home. Aim 3 Use a mixed-methods approach to quantify uptake, trajectories, and predictors of COVID Watch activity and query health system leaders, clinicians, and patients to understand contextual factors related to COVID Watch referral, program engagement, and health care access . Justification for This Aim Patients with COVID-19 can experience rapid and unpredictable clinical deterioration. This concern was heightened in the early months of the pandemic, when the clinical course of COVID-19 was unknown. To manage the large volumes of encounters, particularly during high community case counts, several health systems developed remote patient monitoring programs to support home-dwelling patients with COVID-19. 104 , 113 , 122 , 132-137 Our 6-hospital health system with over 500 outpatient practices enrolled adult patients with test-confirmed COVID-19 or symptoms of COVID-19 in COVID Watch, which resulted in lower patient mortality compared with matched control patients not enrolled in the program. 113 , 138 Although the program was clinically effective, and quantitative measures such as NPS indicated that patients on the whole liked the program and found it helpful, we wanted to ask patients and clinicians probing questions about their experience with the program and how it could be improved. We chose a qualitative approach because it is patient centered; provides a deeper understanding of the context, complexities, and nuance of individual experience with COVID Watch; and allows exploration of why people liked or disliked aspects of the program and how it did or did not meet their needs. Interviewees' perspectives could lead to improvements in this program or in remote engagement programs more generally. 139-143 This study investigated the perspectives of patients and HCP groups who interacted with COVID Watch. Its aim was to understand patients' and clinicians' experiences interacting with the COVID Watch program, how the program could be improved, and lessons from COVID Watch that could be extended to the design and implementation of future remote patient monitoring programs. This study was approved by the IRB at the University of Pennsylvania. Methods Overview We conducted semistructured interviews with 3 groups of people who interacted with COVID Watch: (1) patients who had been enrolled in COVID Watch within the prior 90 days, (2) primary care and ED clinicians who enrolled or had their patients enrolled in COVID Watch, and (3) administrators in primary care or the ED. Setting and Participant Enrollment Patient recruitment and sampling strategy Patients were recruited by phone between February and June 2021, a time span that allowed us to include patients who had recently participated in the randomized controlled trial of fingertip pulse oximetry. 138 Patients in COVID Watch were purposively sampled across 2 patient-level strata to ensure a diversity of patient perspectives: (1) home pulse oximetry status (device mailed to them or not) and (2) level of engagement in the program (high vs low). High engagement was defined as responding to text message prompts on at least 10 out of 14 days of enrollment, with low engagement being fewer than 10 responses. This cutoff was based on the median level of engagement. We monitored and recruited patients to attempt balance across racial, ethnic, and language subgroups. Patients providing verbal informed consent were compensated $50 for their time. Clinician and administrator recruitment and sampling strategy Clinicians (eg, physicians, nurse practitioners, and physician assistants) and administrators (eg, medical directors, physician leads, and nonclinical practice managers) were recruited by email between July and November 2021. Clinicians and administrators were purposively sampled across 2 health system-level strata in hopes of gaining diverse HCP perspectives of COVID Watch: (1) those who worked primarily in an ED setting vs primary care setting and (2) those who enrolled a high vs low number of patients in COVID Watch. For both settings, high-enrolling clinicians were defined as those who enrolled at least 15 patients, and low enrollers were defined as those who enrolled fewer than 10 patients. These cutoffs were above and below the median level of enrollment, respectively. Administrators were recruited from the clinical sites of the HCPs. Snowballing techniques were used to identify additional clinicians or staff who might have been influential in encouraging clinicians or staff members to enroll patients in COVID Watch. Clinicians and administrators provided verbal informed consent and were compensated $50 for their time. Interviews were conducted before the publication of COVID Watch's evaluation. 113 , 138 Interview Guide Development We created 3 semistructured, open-ended interview guides ( Table 23 ; Appendix ) for each cohort: patients, HCPs, and administrators. Guides were created by core members of the research team and reviewed by the larger team. They were pilot-tested with at least 2 participants in each cohort. Questions were open ended and included follow-up probes to allow participants to expand upon answers. After the interview, participants self-reported sociodemographics. Table 23 Patient and Clinician Interview Question Examples According to Themes in Participant Responses. Aim 3 Outcomes Themes were categorized into 3 categories: (1) sentiments about COVID Watch, (2) feedback for improving COVID Watch, and (3) lessons learned from COVID Watch that have implications for future remote patient monitoring programs (clinicians only). Sample Size Calculations and Power Although researchers have developed methods to forecast the number of interviews needed to reach thematic saturation, these methods depend on assumptions about a number of factors (eg, sample heterogeneity) and remain an inexact science. We anticipated needing 20 ± 10 interviews per group (patients, clinicians, and administrators), but we planned to continue interviews until saturation was achieved. 144 Time Frame for Aim 3 Cross-sectional 1-on-1 telephone interviews were conducted from February to June 2021 with patients and from July to November 2021 with clinicians. Data Collection and Sources Researchers conducted audio-recorded phone interviews in English and Spanish. Verbal informed consent was obtained before all interviews. English and Spanish audio recordings were transcribed by Datagain Services, with the Spanish audio transcribed into English. Transcripts were then entered into NVivo, version 1.5, software (QSR International) for coding and analysis. Analytical and Statistical Approaches We partnered with the Mixed Methods Research Lab at the University of Pennsylvania to carry out the qualitative analysis. Separately for patients, clinicians, and administrators, line-by-line reading of early interview transcripts was used to identify recurrent themes and develop an initial codebook by modified content analysis. 145 , 146 The codebook was applied to all transcripts, and the research team summarized findings in biweekly team meetings that guided iterative interpretation of the data. The achieved interrater reliability was ĸ = 0.81 across co-coded transcripts. 147 Patient interviews lasted an average of 53 minutes (range, 37-74 minutes), and clinician and administrator interviews lasted an average of 36 minutes (range, 23-54 minutes). All study protocols and instruments were deemed exempt by the University of Pennsylvania's IRB. Changes to the Original Study Protocol No changes were made to the original study protocol. Results Participant Characteristics In total, 85 interviews were completed. Forty-seven patients were interviewed; patients were on average 50 years old, and most were female, non-Hispanic, and English speaking ( Table 24 ). Table 24 Patient Characteristics. Because most administrators were also clinicians and no major thematic differences were identified between clinicians and administrators, both groups are referred to as “clinicians” henceforth. The sample of 38 clinicians was primarily female, White, and non-Hispanic. Most were physicians and had been in practice for 11 years or more ( Table 25 ). Table 25 Clinician and Administrator Characteristics. Across all cohorts, themes aligned into 3 categories: (1) sentiments about COVID Watch, (2) feedback for improving COVID Watch, and (3) lessons learned from COVID Watch that have implications for future remote patient monitoring programs (clinicians only). There were no notable thematic differences by level of patient engagement or clinician enrollment volume (high vs low); therefore, themes were aggregated across these strata. Quotes to illustrate each theme are presented in Table 26 for patients and Table 27 for clinicians. Table 26 Summary of Patient Themes and Illustrative Interview Excerpts. Table 27 Summary of Clinician Themes and Illustrative Interview Excerpts. Sentiments about COVID Watch Patient Perspectives Comforting Most participants described text messages as a comforting reminder that HCPs were monitoring their well-being. COVID Watch was viewed as a positive alternative to being admitted and monitored in a hospital inpatient setting. Even among patients who never needed the support of a clinician, being able to contact a clinician or escalate their care while they had COVID-19 provided peace of mind. Participants also appreciated that the regular text messages helped them monitor their symptoms over time and track when their symptoms were improving. Irritating Some participants thought that the text messages were excessive, intrusive, or annoying. These feelings were common for patients at the ends of the illness spectrum, either with mild to no symptoms, or conversely so ill they did not have the energy to respond to messages in a timely manner. Those who had a low response rate to the twice-daily messages were often participants who thought the text messages were unhelpful. Insufficient Some patients who were worried about their COVID-19 symptoms or who were fearful about the risk of severe illness expressed a desire for real-time support from human clinicians instead of the automated, routine text messages provided by COVID Watch. Some participants wanted to report additional symptoms beyond feeling short of breath. Clinician Perspectives Comforting The ED clinicians often described taking comfort in knowing that COVID Watch could monitor patients discharged home. Many described it as a valuable safety net for patients, especially when COVID-19 was a new illness. This feeling of a safety net was particularly true in clinicians' discussions of patients with significant social needs (eg, without a primary care HCPs, socially isolated) and patients who did not meet admission criteria but who they worried might decompensate at home after being discharged from the ED. The ED and primary care clinicians believed that COVID Watch also gave their patients a sense of comfort. They knew the program would monitor them and provided an alternative to the ED as the sole source of COVID-19 care. Some clinicians saw the provision of pulse oximetry devices to use at home as an important way to give HCPs and patients additional data points about the severity of a patient's illness. For example, 1 ED clinician described being able to trust a patient's report of dyspnea more if the patient used a pulse oximeter at home. The ED clinicians did not believe that the ability to enroll patients in the program influenced their decisions to admit vs discharge. Increased access to care for patients COVID Watch's accessibility was seen as a key benefit to the program. Patients had quick and easy access to a clinician if needed; patients were not alone in their health decision-making; the program alleviated patient fear of the unknown; and the program was free of charge. In addition, a few clinicians reported COVID Watch increased access for their Spanish-speaking patients, which was described as a key need at some sites. Reduced follow-up burden Many primary care clinicians found COVID Watch to be an important tool for managing follow-up care when the volume of patient needs was high. Feedback for Improving COVID Watch Patient Perspectives Improve the enrollment process Some patients did not recall when or how they were enrolled in COVID Watch. Patients who knew they were enrolled tended to describe more positive feelings about starting the program. A misconception about the program was that some patients thought it was their own doctor who had enrolled them in COVID Watch and was personally monitoring their symptoms. Clarify the monitoring and escalation process Patients found the subjective nature of the daily text message (eg, ”Are your symptoms the same, better, or worse than 12 hours ago”) to be challenging. Some patients desired more specific symptom monitoring (eg, cough or fever) or quantitative measures such as a 0 to 10 number scale for their dyspnea. Similarly, patients with a pulse oximetry device were positive about their experience because the objective measure instilled confidence about their clinical trajectory. Some patients expressed a preference for phone calls over text messages, referencing the difficulty that older patients can have with texting or not having phones that are equipped for text messaging. Other patients thought phone calls would be preferable because they would give clinicians more clarity about how patients were feeling. Spanish-speaking patients more often thought COVID Watch was not able to fully meet their needs. Some Spanish speakers expressed how Spanish-speaking cultures tended to be more phone call-oriented, and so an option to choose the modality of the messages may provide a better cultural fit for some. Clinician Perspectives Improve the enrollment process Clinicians tended to describe the process of enrolling in COVID Watch as easy, but there was a desire to make enrollment even easier. Some believed it was tedious to, for example, go into the patient's exam room to ensure patients received the program's initial text message or to ensure the patient's phone number was correct. In addition, some clinicians thought that their own familiarity with the enrollment process waned if they had not enrolled a patient recently. Provide solutions for patients with limited device access or hesitancy Clinicians highlighted barriers related to patients' access to COVID Watch and hesitancy to use their phones for engaging in care. Participation required the ability to use a cell phone with text messaging. Access to the necessary technology was particularly challenging for older adults (especially those who lived alone) and patients experiencing homelessness. Address low-literacy and language preferences among patients A key access-related barrier clinicians discussed was that the program required comfort in reading and writing in English or Spanish; other languages should be considered. Other enhancements for accessibility included offering an option for patients to use a landline, offering access to a central hotline phone number those patients could call, or distributing cell phones for patients to use. Finally, some clinicians recommended that patients be given the ability to enroll themselves. Create a feedback loop for clinicians The ED and primary care clinicians discussed a desire to know the clinical course of their patients after enrollment in COVID Watch and were interested in knowing which patients did not escalate, which escalated to a COVID Watch nurse, or which had unenrolled from the program early. This would be a mechanism to inform clinicians about the quality of their care and a reminder that the COVID Watch program was still enrolling patients. Lessons for the Future of Remote Patient Monitoring Programs From Clinicians Enhanced data collection Clinicians thought that remote patient monitoring programs would be an important part of practicing medicine in the future, although data collected should have concrete benchmarks. Clinicians were hesitant about COVID Watch's subjective self-reports and thought that more objective measures should be used in future programs, COVID-19-related or otherwise. Both ED and primary care clinicians thought remote patient monitoring programs should provide patients with the appropriate health data collection tools, such as a home pulse oximeter or blood pressure cuff, to collect and report data back to their health care team. Some also suggested greater integration with existing EHR systems, directly embedding remotely recorded results into the medical record. A guide for patients The ED clinicians thought that remote patient monitoring could reduce the number of ED visits by giving patients more accurate, objective data about when not to come to the ED. By using objective data and clear cutoffs, patients could be clearly guided to seek the right level of care. In addition, the ability to provide reassurance to patients with a remote monitoring program might help patients being discharged home from the ED. Extend remote patient monitoring to other conditions Many clinicians also thought remote patient monitoring would be particularly valuable for certain chronic and acute conditions. They saw targeted data collection to be practical and effective for monitoring and treating conditions such as congestive heart failure, diabetes, asthma, weight management, and postsurgical recovery. To evolve for other use cases, however, clinicians emphasized the desire for high-quality collected data. Discussion Overall, although patients and clinicians found COVID Watch to be comforting and beneficial, improvements to the design and implementation of the program will be important for the program's future and will have implications for the design of future remote patient monitoring programs. Our findings have generated 3 key insights for future remote patient monitoring programs to manage COVID-19 or other clinical conditions. First, remote patient monitoring programs should not be static, 1-time builds or implementations. Although these programs may have digitized algorithms, they are human-facing programs that should evolve as technology advances, patient and clinician expectations of technology evolve, and standards for managing targeted disease conditions change. Second, health systems must acknowledge the human resources needed to support remote patient monitoring programs, even if automation is embedded in the program. Although automation can increase efficiency for some patients, successful programs will need to marry technology with options for human interaction because tech-first approaches may not always be welcomed. Some patients in our study indicated the desire to connect with a human clinician or wanted to avoid the automated text message system more generally. Using default pathways (eg, text messages) that are automated, complemented by alternative pathways (eg, interactive voice recordings or human-to-human phone calls) that are customized to the user's needs, may be 1 solution for greater engagement while not overburdening clinicians. Finally, future programs must be designed with equity as a primary principle, recognizing that patients who have the most limited access to care may need additional design considerations. For example, programs should be offered in multiple languages. In our study both patients and clinicians expressed concerns that vulnerable populations may have been excluded because more direct human-to-human connections (eg, telephone calls) were not made available. Balancing inclusivity and patients' desires for human-to-human connections with the efficiency gains of automated processes will be important for future remote patient monitoring programs. Strengths and Limitations This study had limitations. Our analysis concerned 1 large academic institution, limiting generalizability. These interviews took place early in the pandemic—in the first year—and therefore reflected the stress that both patients and clinicians felt when faced with an unprecedented crisis. Finally, patient and clinician experiences with COVID Watch and COVID-19 evolved during the study period. For instance, automated and opt-out enrollment were implemented as part of COVID Watch in the fall of 2020. In addition, surges of infection and the increased availability of vaccination and effective treatments might have influenced participants' responses over time. Strengths of this research include the breadth and size of our participant samples. Although the study took place at a single institution, this institution includes 6 hospitals and over 500 outpatient practices across a wide geographic area. We were therefore able to sample participants from multiple hospitals, encompassing urban and suburban settings across the large catchment area. In addition, the number of patients and clinicians interviewed was large compared with the typical qualitative study. Conclusion The success of remote patient monitoring programs hinges on having a user-centered design to enhance experiences for both patients and clinicians 148 and intentionally designing for traditionally marginalized groups who have not historically been considered as early adopters of new technology-based care programs. 149-151 Future remote monitoring programs should learn from our lessons to help a diverse group of patients engage and benefit. Discussion Summary of Results We found that the remote patient monitoring program COVID Watch was associated with a 68% relative reduction in mortality at 30 days, an advantage that continued to hold at 60 days. A randomized controlled trial found that adding home pulse oximetry to COVID Watch as an objective measure of blood oxygen saturation did not improve outcomes but did place additional demands on both patients and clinicians. Qualitative interviews revealed concerns about COVID Watch's accessibility for certain groups (eg, older adults, patients who do not read English or Spanish), but in general both patients and clinicians reported that the program made them feel more comfortable with managing COVID-19 at home. Results in Context This research took place in a large, well-resourced academic health system in the mid-Atlantic. For much of the study period, COVID-19 case counts were high, and vaccines and effective medications were not widely available. At the time, remote monitoring of patients with COVID-19 offered the best hope for simultaneously reducing ED and hospital strain, allowing patients to remain at home and escalating cases that probably warranted immediate medical care. Unlike some remote monitoring programs, COVID Watch was embedded directly in our health system, enabling fast, 24/7 clinician responses to patients' worsening symptoms. Potential to Affect Health Care Decision-Making On one hand, results such as ours could encourage health systems to invest more resources in remote monitoring programs that enable a small staff of trained clinicians to effectively manage a large patient population. This model may be especially attractive for infectious diseases where self-isolating at home has broader population health benefits. On the other hand, the null results of our randomized controlled trial could give health systems and payers pause before distributing costly medical devices for at-home monitoring; more evidence that these devices yield incremental benefits over lower-cost subjective symptom reporting may be needed. A potential concern was that knowing that outpatient monitoring by COVID Watch would be available would increase the likelihood that HCPs would discharge patients whose symptoms warranted inpatient care. At least according to clinicians' interview responses, their decisions to admit or discharge patients were unaffected by the knowledge that discharged patients would receive outpatient monitoring. Lessons Learned During the design phase, we learned much from our patient stakeholder group about the wording of messages and survey items and about the selection of outcomes of greatest importance to the patient population we serve. The interviews we conducted after the studies revealed that we could go further to improve the program's accessibility and user-friendliness. Most notably, to increase the program's reach, more language options should be made available, along with an audio-based call option. Contrary to what we hypothesized, adding pulse oximetry to get an objective measure of Sp o 2 did not lead to an increase in DAOH. This finding is at odds with what some patients and clinicians told us (before the findings of the trial were made public), which is that having an objective indicator of patient decline is essential for making determinations about the need for medical care. This may be true for some subsets of the population and some conditions; however, for the health system's COVID-19 patient population overall, the trial data, not intuition, should guide future decisions about how best to monitor for patient decline. Generalizability Both studies included diverse samples that should permit the generalization of our findings to White, Black, and Hispanic or Latinx populations. Our health system's population is largely urban and suburban, limiting conclusions that can be drawn about program effectiveness with rural populations. To the extent that remote monitoring programs such as COVID Watch increase access for patients who need to travel long distances for health care services, rural populations should also see a benefit. It should be noted that although prompting patients to use a home pulse oximeter twice a day may not provide an incremental benefit over simply prompting them to report subjective dyspnea, home pulse oximetry may be beneficial in other contexts. In some health systems and for some patients, remote monitoring of subjective dyspnea may not be feasible. In addition, wearable technology that monitors Sp o 2 continuously could be more advantageous than relying on patient reports. Subgroup Analyses and Heterogeneity of Treatment Effects Both our retrospective study of COVID Watch effectiveness (relative to usual care) and our randomized controlled trial testing the effects of home pulse oximetry (relative to COVID Watch alone) included large, diverse samples that gave us the power to conduct subgroup analyses. In the retrospective study, we found an association between COVID Watch enrollment and decreased mortality across all racial and ethnic groups examined. In the randomized controlled trial, we found no added benefit of home pulse oximetry across all subgroups. Study Limitations Our retrospective study was observational, preventing us from drawing strong conclusions about causality. The potential confounding variables we observed, however, suggested that patients enrolled in COVID Watch were at higher risk for poor outcomes yet experienced lower mortality. Our randomized controlled trial only compared COVID Watch plus home pulse oximetry with the standard COVID Watch program, meaning that the null results obtained say nothing about the potential benefits of home pulse oximetry by itself. Future Research Future research could test ways to optimize COVID Watch based on feedback from patients and clinicians, increase the accessibility of the program for populations who would otherwise find it difficult or impossible to use, and scale it beyond Penn Medicine to learn more about barriers and facilitators to implementation in a wider variety of settings. A big question for the future is the extent to which the success of a COVID Watch-like program will generalize to outpatient management of other conditions. At a high level, ideal candidates for this kind of remote monitoring are chronic or acute conditions that can usually be managed at home but sometimes require urgent care when symptoms worsen. Clinicians we interviewed specifically named congestive heart failure, diabetes, asthma, weight management, and postsurgical recovery as potential targets. The remote monitoring program would need to be carefully adapted for each condition, and we recommend testing subjectively reported symptoms vs objectively measured data and weighing the benefits and costs of each. Conclusions The present research aimed to (1) examine whether patients enrolled in the remote monitoring program COVID Watch experienced better health outcomes compared with usual care, (2) test whether augmenting COVID Watch with at-home Sp o 2 monitoring would improve patient outcomes, and (3) collect and interpret patient and stakeholder ideas for improving COVID Watch and similar monitoring programs. Our retrospective study found that enrollment in COVID Watch was associated with a 68% relative reduction in mortality at 30 days. This reduction in mortality appears to be driven by timely medical care: Patients in COVID Watch were more likely to present to the hospital, and they did so earlier. During the study period, no one enrolled in COVID Watch died outside the hospital. COVID Watch was also associated with equitable treatment benefits. The COVID Watch group had a higher proportion of Black patients and a similar proportion of Hispanic patients compared with usual care, and at 60 days White, Black, and Hispanic patient subgroups all experienced reduced mortality. Our randomized controlled trial found no differences in DAOH, or in mortality by itself, between patients enrolled in COVID Watch augmented with home pulse oximetry and those in the standard COVID Watch program. This was true across racial and ethnic subgroups. These null effects, coupled with the success of COVID Watch relative to usual care, suggest that asking patients to report difficulty breathing may be adequate for determinations of when COVID-19 outpatient care should be escalated. At least in health systems where remote monitoring of subjective dyspnea already exists, adding home pulse oximetry—with its financial costs and hassles—appears to be unjustified. Qualitative interviews with patients, clinicians, and administrators revealed agreement that COVID Watch was valuable and provided peace of mind for patient and clinician alike. Clinicians liked that the program reduced some of their follow-up burden at a time of immense health system strain. Interviewees suggested ways to improve the enrollment process and make the program more accessible to patients regardless of their language, literacy, or cell phone access. COVID Watch is a scalable program that our research indicates simultaneously reduces health system strain, increases equitable access to care, and improves patient outcomes. Its basic model of automated check-ins with escalation to telehealth support is well suited for other conditions that can usually be managed at home but sometimes warrant urgent care when symptoms worsen. References 1. Garg S, Kim L, Whitaker M, et al. Hospitalization rates and characteristics of patients hospitalized with laboratory-confirmed coronavirus disease 2019—COVID-NET, 14 States, March 1-30, 2020. MMWR Morb Mortal Wkly Rep. 2020;69(15):458-464. doi:10.15585/mmwr.mm6915e3 [ PMC free article : PMC7755063 ] [ PubMed : 32298251 ] [ CrossRef ] 2. Centers for Disease Control and Prevention. What to do if you are sick: coronavirus patient guidelines. Accessed May 16, 2020. https://www ​.cdc.gov/coronavirus ​/2019-ncov ​/if-you-are-sick/steps-when-sick.html 3. Centers for Disease Control and Prevention. Outpatient and ambulatory care settings: responding to community transmission of COVID-19 in the United States. Accessed May 16, 2020. https://www ​.cdc.gov/coronavirus ​/2019-ncov ​/hcp/ambulatory-care-settings.html 4. Liang W, Liang H, Ou L, et al. Development and validation of a clinical risk score to predict the occurrence of critical illness in hospitalized patients with COVID-19. JAMA Intern Med. 2020;180(8):1081-1089. doi:10.1001/jamainternmed.2020.2033 [ PMC free article : PMC7218676 ] [ PubMed : 32396163 ] [ CrossRef ] 5. American College of Emergency Physicians. National strategic plan for emergency department management of outbreaks of COVID-19. Accessed May 16, 2020. https://www ​.acep.org ​/globalassets/sites/acep ​/media/by-medical-focus ​/covid-19-national-strategic-plan_0320.pdf 6. Chopra V, Toner E, Waldhorn R, Washer L. How should U.S. hospitals prepare for coronavirus disease 2019 (COVID-19)? Ann Intern Med. 2020;172(9):621-622. doi:10.7326/M20-0907 [ PMC free article : PMC7081177 ] [ PubMed : 32160273 ] [ CrossRef ] 7. Mehrotra A, Ray K, Brockmeyer DM, Barnett ML, Bender JA. Rapidly converting to “virtual practices”: outpatient care in the era of Covid-19. NEJM Catalyst. 2020. doi:10.1056/CAT.20.0091 [ CrossRef ] 8. Hollander JE, Carr BG. Virtually perfect? Telemedicine for Covid-19. N Engl J Med. 2020;382(18):1679-1681. doi:10.1056/NEJMp2003539 [ PubMed : 32160451 ] [ CrossRef ] 9. Zhou F, Yu T, Du R, et al. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study. Lancet. 2020;395(10229):1054-1062. doi:10.1016/S0140-6736(20)30566-3 [ PMC free article : PMC7270627 ] [ PubMed : 32171076 ] [ CrossRef ] 10. Levitan R. The infection that's silently killing coronavirus patients. New York Times. April 20, 2020. Accessed May 16, 2020. https://nytimes ​.com/2020 ​/04/20/opinion/sunday ​/coronavirus-testing-pneumonia.html 11. American Lung Association. Pulse oximeter not a substitute for talking to healthcare provider, watching for early COVID-19 symptoms. https://www ​.lung.org ​/media/press-releases ​/pulse-oximeter-covid-19 12. Gantz S. What's a pulse oximeter and do you need one to monitor for coronavirus? Philadelphia Inquirer. Updated May 7, 2020. https://www ​.inquirer ​.com/health/coronavirus ​/coronavirus-covid-19-pulse-oximeter-20200507.html 13. Yancy CW. COVID-19 and African Americans. JAMA. 2020;323(19):1891-1892. doi:10.1001/jama.2020.6548 [ PubMed : 32293639 ] [ CrossRef ] 14. Chowkwanyun M, Reed AL Jr. Racial health disparities and Covid-19—caution and context. N Engl J Med. 2020;383:201-203. doi:10.1056/NEJMp2012910 [ PubMed : 32374952 ] [ CrossRef ] 15. Nouri S, Khoong E, Lyles CR, Karliner L. Addressing equity in telemedicine for chronic disease management during the Covid-19 pandemic. NEJM Catalyst. 2020. doi:10.1056/CAT.20.0123 [ CrossRef ] 16. Wang Z, Tang K. Combating COVID-19: health equity matters. Nat Med. 2020;26(4):458. doi:10.1038/s41591-020-0823-6 [ PubMed : 32284617 ] [ CrossRef ] 17. Velasquez D, Mehrotra, A. Ensuring the growth of telehealth during COVID-19 does not exacerbate disparities in care. Health Affairs. May 8, 2020. doi:10.1377/forefront.20200505.591306 [ CrossRef ] 18. Pew Research Center. Demographics of mobile device ownership and adoption in the United States. Accessed May 16, 2020. http://www ​.pewinternet ​.org/fact-sheet/mobile/ 19. Pew Research Center. Demographics of internet and home broadband usage in the United States. Accessed May 10, 2020. http://www ​.pewinternet ​.org/fact-sheet/internet-broadband/ 20. Agency for Healthcare Research and Quality. National health quality and disparities report. Accessed May 10, 2020. https://www ​.ahrq.gov ​/research/findings/nhqrdr/nhqdr18/index ​.html [ PubMed : 35389601 ] 21. Morgan AU, Balachandran M, Do D, et al. Remote monitoring of patients with Covid-19: design, implementation, and outcomes of the first 3,000 patients in COVID Watch. NEJM Catalyst. 2020;1(4). doi:10.1056/CAT.20.0342 [ CrossRef ] 22. Dyer O. Covid-19: Black people and other minorities are hardest hit in US. BMJ. 2020;369:m1483. doi:10.1136/bmj.m1483 [ PubMed : 32291262 ] [ CrossRef ] 23. Kirby T. Evidence mounts on the disproportionate effect of COVID-19 on ethnic minorities. Lancet Respir Med. 2020;8(6):547-548. doi:10.1016/S2213-2600(20)30228-9 [ PMC free article : PMC7211498 ] [ PubMed : 32401711 ] [ CrossRef ] 24. Centers for Disease Control and Prevention. COVID-19 in racial and ethnic minority groups. Accessed May 18, 2020. https://www ​.cdc.gov/coronavirus ​/2019-ncov ​/need-extra-precautions ​/racial-ethnic-minorities.html 25. The COVID Tracking Project. The COVID Racial Data Tracker. Accessed May 18, 2020. https: ​//covidtracking.com/race 26. HGE Health. HGE COVID care. Accessed May 16, 2020. https://www ​.hgehealth.com/covid-19 27. Partners Health Care. Help scientists learn more information about COVID-19 by using our app! Accessed May 18, 2020. https://rally ​.partners ​.org/study/covid19app?code ​=PHSBroad_20200513_11 28. Annis T, Pleasants S, Hultman G, et al. Rapid implementation of a COVID-19 remote patient monitoring program. J Am Med Inform Assoc. 2020;27(8):1326-1330. doi:10.1093/jamia/ocaa097 [ PMC free article : PMC7239139 ] [ PubMed : 32392280 ] [ CrossRef ] 29. Watson AR, Wah R, Thamman R. The value of remote monitoring for the COVID-19 pandemic. Telemed J E Health. 2020;26(9):1110-1112. doi:10.1089/tmj.2020.0134 [ PubMed : 32384251 ] [ CrossRef ] 30. Ross C. Hospitals turn to remote monitoring tools to free up beds for the sickest coronavirus patients. Updated March 25, 2020. https://www ​.statnews ​.com/2020/03/25/coronavirus-hospitals-weigh-remote-patient-monitoring-tools/ 31. Menni C, Valdes AM, Freidin MB, et al. Real-time tracking of self-reported symptoms to predict potential COVID-19. Nat Med. 2020;26(7):1037-1040. doi:10.1038/s41591-020-0916-2 [ PMC free article : PMC7751267 ] [ PubMed : 32393804 ] [ CrossRef ] 32. Couzin-Frankel J. The mystery of the pandemic's “happy hypoxia.” Science. 2020;368(6490):455-456. doi:10.1126/science.368.6490.455 [ PubMed : 32355007 ] [ CrossRef ] 33. The role of home pulse oximeters in treating COVID-19. Transcript. Weekend Edition Saturday. National Public Radio. May 2, 2020. Accessed May 18, 2020. https://www ​.npr.org/2020 ​/05/02/849535986 ​/the-role-of-home-pulse-oximeters-in-treating-covid-19 34. Fontanarosa PB, Bauchner H. COVID-19—looking beyond tomorrow for health care and society. JAMA. 2020;323(19):1907-1908. doi:10.1001/jama.2020.6582 [ PubMed : 32301955 ] [ CrossRef ] 35. Swerdlow DL, Finelli L. Preparation for possible sustained transmission of 2019 novel coronavirus: lessons from previous epidemics. JAMA. 2020;323(12):1129-1130. doi:10.1001/jama.2020.1960 [ PubMed : 32207807 ] [ CrossRef ] 36. Bhatraju PK, Ghassemieh BJ, Nichols M, et al. Covid-19 in critically ill patients in the Seattle region—case series. N Engl J Med. 2020;382:2012-2022. doi:10.1056/NEJMoa2004500 [ PMC free article : PMC7143164 ] [ PubMed : 32227758 ] [ CrossRef ] 37. Xie J, Covassin N, Fan Z, et al. Association between hypoxemia and mortality in patients with COVID-19. Mayo Clin Proc. 2020;95(6):1138-1147. doi:10.1016/j.mayocp.2020.04.006 [ PMC free article : PMC7151468 ] [ PubMed : 32376101 ] [ CrossRef ] 38. Wang D, Yin Y, Hu C, et al. Clinical course and outcome of 107 patients infected with the novel coronavirus, SARS-CoV-2, discharged from two hospitals in Wuhan, China. Crit Care. 2020;24(1):188. doi:10.1186/s13054-020-02895-6 [ PMC free article : PMC7192564 ] [ PubMed : 32354360 ] [ CrossRef ] 39. Bonow RO, Fonarow GC, O'Gara PT, Yancy CW. Association of coronavirus disease 2019 (COVID-19) with myocardial injury and mortality. JAMA Cardiol. 2020;5(7):751-753. doi:10.1001/jamacardio.2020.1105 [ PubMed : 32219362 ] [ CrossRef ] 40. Bernstein L, Cha A. Doctors keep discovering new ways the coronavirus attacks the body. The Washington Post. https://www ​.washingtonpost ​.com/health/2020 ​/05/10/coronavirus-attacks-body-symptoms/?arc404=true 41. Human Diagnosis Project. Coronavirus (COVID-19) self-assessment tool. Accessed May 15, 2020. https://www.humandx.org/covid-19/assessment 42. Wiegel G, Ramaswamy A, Sobel L, Salganicoff A, Cubanski J, Freed M. Opportunities and barriers for telemedicine in the U.S. during the COVID-19 emergency and beyond. KFF. Accessed May 12, 2020. https://www ​.kff.org/womens-health-policy ​/issue-brief/opportunities-and-barriers-for-telemedicine-in-the-u-s-during-the-covid-19-emergency-and-beyond/ 43. Smith AC et al. Telehealth for global emergencies: implications for coronavirus disease 2019 (COVID-19). J Telemed Telecare. 2020;26(5):309-313. doi:10.1177/1357633X20916567 [ PMC free article : PMC7140977 ] [ PubMed : 32196391 ] [ CrossRef ] 44. Moonesinghe SR, Jackson AIR, Boney O, et al. Systematic review and consensus definitions for the Standardised Endpoints in Perioperative Medicine initiative: patient-centred outcomes. Br J Anaesth. 2019;123(5):664-670. doi:10.1016/j.bja.2019.07.020 [ PubMed : 31493848 ] [ CrossRef ] 45. Jerath A, Austin PC, Wijeysundera DN. Days alive and out of hospital: validation of a patient-centered outcome for perioperative medicine. Anesthesiology. 2019;131(1):84-93. doi:10.1097/ALN.0000000000002701 [ PubMed : 31094760 ] [ CrossRef ] 46. Szarek M, Steg PG, DiCenso D, et al. Alirocumab reduces total hospitalizations and increases days alive and out of hospital in the ODYSSEY OUTCOMES Trial. Circ Cardiovasc Qual Outcomes. 2019;12(11):e005858. doi:10.1161/CIRCOUTCOMES.119.005858 [ PubMed : 31707826 ] [ CrossRef ] 47. Fanaroff AC, Cyr D, Neely ML, et al. Days alive and out of hospital: exploring a patient-centered, pragmatic outcome in a clinical trial of patients with acute coronary syndromes. Circ Cardiovasc Qual Outcomes. 2018;11(12):e004755. doi:10.1161/CIRCOUTCOMES.118.004755 [ PMC free article : PMC6347414 ] [ PubMed : 30562068 ] [ CrossRef ] 48. Anesi G. Coronavirus disease 2019 (COVID-19): critical care and airway management issues. UpToDate, Inc. Accessed July 30, 2020. https://www ​.uptodate ​.com/contents/coronavirus-disease-2019-covid-19-critical-care-and-airway-management-issues 49. Shah S, Majmudar, K, Stein A, et al. Novel use of home pulse oximetry monitoring in COVID-19 patients discharged from the emergency department identifies need for hospitalization. Acad Emerg Med. 2020;27(8):681-692. doi:10.1111/acem.14053 [ PMC free article : PMC7323027 ] [ PubMed : 32779828 ] [ CrossRef ] 50. Horby P, Lim, WS, Emberson, J, et al. Effect of dexamethasone in hospitalized patients with COVID-19: preliminary report. Accessed July 30, 2020. medRxiv. https://www ​.medrxiv.org/content/10 ​.1101/2020 ​.06.22.20137273v1 51. Beigel JH, Tomashek KM, Dodd LE, et al. Remdesivir for the treatment of Covid-19—preliminary report. N Engl J Med. 2020;383:1813-1826. doi:10.1056/NEJMoa2007764 [ PubMed : 32649078 ] [ CrossRef ] 52. Caputo ND, Strayer RJ, Levitan R. Early self-proning in awake, non-intubated patients in the emergency department: a single ED's experience during the COVID-19 pandemic. Acad Emerg Med. 2020;27(5):375-378. doi:10.1111/acem.13994 [ PMC free article : PMC7264594 ] [ PubMed : 32320506 ] [ CrossRef ] 53. Elharrar X, Trigui Y, Dols AM, et al. Use of prone positioning in nonintubated patients with COVID-19 and hypoxemic acute respiratory failure. JAMA. 2020;9;323(22):2336-2338. doi:10.1001/jama.2020.8255 [ PMC free article : PMC7229532 ] [ PubMed : 32412581 ] [ CrossRef ] 54. Parker-Pope T. What's a pulse oximeter, and do I really need one at home? New York Times. Updated June 18, 2020. Accessed June 18, 2020. https://www ​.nytimes.com ​/2020/04/24/well/live ​/coronavirus-pulse-oximeter-oxygen ​.html 55. Rezai S. A new war plan for COVID-19 with Richard Levitan. RebelEM podcast. April 24, 2020. Accessed June 26, 2024. https://rebelem ​.com/rebel-cast-ep80-a-new-war-plan-for-covid-19-with-richard-levitan/ 56. Read J. Use of pulse oximeters to monitor novel coronavirus 2019 (COVID-19) among individuals with laboratory-confirmed SARS-CoV-2 infection. Vermont Department of Health. Accessed July 30, 2020. https://www.healthvermont.gov/sites/default/files/documents/pdf/COVID-19-HAN-PulseOximetryProgram.pdf 57. Survivor Corps Facebook page. Accessed July 30, 2020. https://www.facebook.com/groups/COVID19survivorcorps/ 58. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50. doi:10.1186/1748-5908-4-50 [ PMC free article : PMC2736161 ] [ PubMed : 19664226 ] [ CrossRef ] 59. King DK, Shoup JA, Raebel MA, et al. Planning for implementation success using RE-AIM and CFIR frameworks: a qualitative study. Front Public Health. 2020;8:59. doi:10.3389/fpubh.2020.00059 [ PMC free article : PMC7063029 ] [ PubMed : 32195217 ] [ CrossRef ] 60. Safaeinili N, Brown-Johnson C, Shaw JG, Mahoney M, Winget M. CFIR simplified: pragmatic application of and adaptations to the Consolidated Framework for Implementation Research (CFIR) for evaluation of a patient-centered care transformation within a learning health system. Learn Health Syst. 2020;4(1):e10201. doi:10.1002/lrh2.10201 [ PMC free article : PMC6971122 ] [ PubMed : 31989028 ] [ CrossRef ] 61. Sinnott C, Kelly MA, Bradley CP. A scoping review of the potential for chart stimulated recall as a clinical research method. BMC Health Serv Res. 2017;17(1):583. doi:10.1186/s12913-017-2539-y [ PMC free article : PMC5567630 ] [ PubMed : 28830405 ] [ CrossRef ] 62. Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Adm Policy Ment Health. 2009;36(1):24-34. doi:10.1007/s10488-008-0197-4 [ PMC free article : PMC3808121 ] [ PubMed : 19104929 ] [ CrossRef ] 63. Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65-76. doi:10.1007/s10488-010-0319-7 [ PMC free article : PMC3068522 ] [ PubMed : 20957426 ] [ CrossRef ] 64. Centers for Disease Control and Prevention. Trends in number of COVID-19 cases and deaths in the US reported to CDC, by state/territory. Accessed April 23, 2021. https://covid ​.cdc.gov ​/covid-data-tracker ​/#trends_dailytrendscases 65. Wu Z, McGoogan JM. Characteristics of and important lessons from the coronavirus disease 2019 (COVID-19) outbreak in China: summary of a report of 72 314 cases from the Chinese Center for Disease Control and Prevention. JAMA. 2020;323(13):1239-1242. doi:10.1001/jama.2020.2648 [ PubMed : 32091533 ] [ CrossRef ] 66. Gandhi RT, Lynch JB, Del Rio C. Mild or moderate Covid-19. N Engl J Med. 2020;383(18):1757-1766. doi:10.1056/NEJMcp2009249 [ PubMed : 32329974 ] [ CrossRef ] 67. Asch DA, Sheils NE, Islam MN, et al. Variation in US hospital mortality rates for patients admitted with COVID-19 during the first 6 months of the pandemic. JAMA Intern Med. 2021;181(4):471-478. doi:10.1001/jamainternmed.2020.8193 [ PMC free article : PMC7756246 ] [ PubMed : 33351068 ] [ CrossRef ] 68. Garg S, Patel K, Pham H, et al. Clinical trends among US adults hospitalized with COVID-19, March to December 2020: a cross-sectional study. Ann Intern Med. 2021;174(10):1409-1419. doi:10.7326/M21-1991 [ PMC free article : PMC8381761 ] [ PubMed : 34370517 ] [ CrossRef ] 69. Wong LE, Hawkins JE, Murrell KL, et al. Where are all the patients? Addressing Covid-19 fear to encourage sick patients to seek emergency care. NEJM Catalyst. 2020. doi:10.1056/CAT.20.0193 [ CrossRef ] 70. Hartnett KP, Kite-Powell A, DeVies J, et al. Impact of the COVID-19 pandemic on emergency department visits—United States, January 1, 2019–May 30, 2020. MMWR Morb Mortal Wkly Rep. 2020;69(23):699-704. doi:10.15585/mmwr.mm6923e1 [ PMC free article : PMC7315789 ] [ PubMed : 32525856 ] [ CrossRef ] 71. Baum A, Schwartz MD. Admissions to Veterans Affairs hospitals for emergency conditions during the COVID-19 pandemic. JAMA. 2020;324(1):96-99. doi:10.1001/jama.2020.9972 [ PMC free article : PMC7275263 ] [ PubMed : 32501493 ] [ CrossRef ] 72. Huang C, Wang Y, Li X, et al. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. Lancet. 2020;395(10223):497-506. doi:10.1016/S0140-6736(20)30183-5 [ PMC free article : PMC7159299 ] [ PubMed : 31986264 ] [ CrossRef ] 73. Kilaru AS, Lee K, Snider CK, et al. Return hospital admissions among 1419 Covid-19 patients discharged from five US emergency departments. Acad Emerg Med. 2020;27(10):1039-1042. doi: 10.1111/acem.14117 [ PMC free article : PMC7461233 ] [ PubMed : 32853423 ] [ CrossRef ] 74. Wolff D, Nee S, Hickey NS, Marschollek M. Risk factors for Covid-19 severity and fatality: a structured literature review. Infection. 2021;49(1):15-28. doi:10.1007/s15010-020-01509-1 [ PMC free article : PMC7453858 ] [ PubMed : 32860214 ] [ CrossRef ] 75. Haimovich AD, Ravindra NG, Stoytchev S, et al. Development and validation of the quick COVID-19 severity index: a prognostic tool for early clinical decompensation. Ann Emerg Med. 2020;76(4):442-453. doi: 10.1016/j.annemergmed.2020.07.022 [ PMC free article : PMC7373004 ] [ PubMed : 33012378 ] [ CrossRef ] 76. Gupta S, Hayek SS, Wang W, et al. Factors associated with death in critically ill patients with coronavirus disease 2019 in the US. JAMA Intern Med. 2020;180(11):1436-1446. doi:10.1001/jamainternmed.2020.3596 [ PMC free article : PMC7364338 ] [ PubMed : 32667668 ] [ CrossRef ] 77. HealthShare Exchange. Current participants. Accessed March 30, 2021. https://www ​.healthshareexchange ​.org/current-participants 78. Austin PC, Stuart EA. Moving towards best practice when using inverse probability of treatment weighting (IPTW) using the propensity score to estimate causal treatment effects in observational studies. Stat Med. 2015;34(28):3661-3679. doi:10.1002/sim.6607 [ PMC free article : PMC4626409 ] [ PubMed : 26238958 ] [ CrossRef ] 79. Lopes RD, Macedo AVS, De Barros E, Silva PGM, et al. Effect of discontinuing vs continuing angiotensin-converting enzyme inhibitors and angiotensin II receptor blockers on days alive and out of the hospital in patients admitted with COVID-19. JAMA. 2021;325(3):254. doi:10.1001/jama.2020.25864 [ PMC free article : PMC7816106 ] [ PubMed : 33464336 ] [ CrossRef ] 80. Van der Laan M, Polley E, Hubbard A. Super learner. Statistical applications in genetics and molecular biology. Stat Appl Genet Mol Biol. 2007;6(1):25. doi:10.2202/1544-6115.1309 [ PubMed : 17910531 ] [ CrossRef ] 81. Van der Laan MJ, Rose S. Targeted Learning: Causal Inference for Observational and Experimental Data. Springer Science & Business Media; 2011. 82. Kricke G, Roemer P, Banrard C, et al. Rapid implementation of an outpatient Covid-19 monitoring program. NEJM Catalyst. 2020. doi:10.1056/CAT.20.0214 [ CrossRef ] 83. Hutchings OR, Dearing C, Jagers D, et al. Virtual health care for community management of patients with COVID-19 in Australia: observational cohort study. J Med Internet Res. 2021;23(3):e21064. doi:10.2196/21064 [ PMC free article : PMC7945978 ] [ PubMed : 33687341 ] [ CrossRef ] 84. Shaw JG, Sankineni S, Olaleye CA, et al. A novel large scale integrated telemonitoring program for COVID-19. Telemed J E Health. 2021;27(11):1317-1321. doi:10.1089/tmj.2020.0384 [ PubMed : 33544043 ] [ CrossRef ] 85. Moreland A, Herlihy C, Tynan MA, et al. Timing of state and territorial COVID-19 stay-at-home orders and changes in population movement—United States, March 1–May 31, 2020. MMWR Morb Mortal Wkly Rep. 2020;69(35):1198-1203. doi:10.15585/mmwr.mm6935a2 [ PMC free article : PMC7470456 ] [ PubMed : 32881851 ] [ CrossRef ] 86. Sen S, Karaca-Mandic P, Georgiou A. Association of stay-at-home orders with COVID-19 hospitalizations in 4 states. JAMA. 2020;323(24):2522-2524. doi:10.1001/jama.2020.9176 [ PMC free article : PMC7254451 ] [ PubMed : 32459287 ] [ CrossRef ] 87. Friedman AB, Barfield D, David G, et al. Delayed emergencies: the composition and magnitude of non-respiratory emergency department visits during the COVID-19 pandemic. J Am Coll Emerg Physicians Open. 2021;2(1):e12349. doi:10.1002/emp2.12349 [ PMC free article : PMC7812445 ] [ PubMed : 33490998 ] [ CrossRef ] 88. Deerberg-Wittram J, Knothe C. Do not stay at home: we are ready for you. NEJM Catalyst. 2020;1(3). doi:10.1056/CAT.20.0146 [ CrossRef ] 89. Jeffery MM, D'Onofrio G, Paek H, et al. Trends in emergency department visits and hospital admissions in health care systems in 5 states in the first months of the COVID-19 pandemic in the US. JAMA Intern Med. 2020;180(10):1328-1333. doi:10.1001/jamainternmed.2020.3288 [ PMC free article : PMC7400214 ] [ PubMed : 32744612 ] [ CrossRef ] 90. Bhambhvani HP, Rodrigues AJ, Jonathan SY, Carr JB, Gephart MH. Hospital volumes of 5 medical emergencies in the COVID-19 pandemic in 2 US medical centers. JAMA Intern Med. 2021;181(2):272-274. doi:10.1001/jamainternmed.2020.3982 [ PMC free article : PMC7589046 ] [ PubMed : 33104161 ] [ CrossRef ] 91. Garcia S, Albaghdadi MS, Meraj PM, et al. Reduction in ST-segment elevation cardiac catheterization laboratory activations in the United States during COVID-19 pandemic. J Am Coll Cardiol. 2020;75(22):2871-2872. doi:10.1016/j.jacc.2020.04.011 [ PMC free article : PMC7151384 ] [ PubMed : 32283124 ] [ CrossRef ] 92. Holland M, Burke J, Hulac S, et al. Excess cardiac arrest in the community during the COVID-19 pandemic. Cardiovasc Interv. 2020;13(16):1968-1969. doi:10.1016/j.jcin.2020.06.022 [ PMC free article : PMC7293523 ] [ PubMed : 32819492 ] [ CrossRef ] 93. Benzakoun J, Hmeydia G, Delabarde T, et al. Excess out-of-hospital deaths during COVID-19 outbreak: evidence of pulmonary embolism as a main determinant. Eur J Heart Fail. 2020;22(6):1046-1047. doi:10.1002/ejhf.1916 [ PMC free article : PMC7283748 ] [ PubMed : 32463538 ] [ CrossRef ] 94. Marijon E, Karam N, Jost D, et al. Out-of-hospital cardiac arrest during the COVID-19 pandemic in Paris, France: a population-based, observational study. Lancet Public Health. 2020;5(8):e437-e443. doi:10.1016/S2468-2667(20)30117-1 [ PMC free article : PMC7255168 ] [ PubMed : 32473113 ] [ CrossRef ] 95. Mountantonakis SE, Epstein LM, Coleman K, et al. The association of structural inequities and race with out-of-hospital sudden death during the COVID-19 pandemic. Circ Arrhythm Electrophysiol. 2021;14(5):e009646. doi:10.1161/CIRCEP.120.009646 [ PMC free article : PMC8136460 ] [ PubMed : 33835821 ] [ CrossRef ] 96. The RECOVERY Collaborative Group. Dexamethasone in hospitalized patients with Covid-19—preliminary report. N Engl J Med. 2021; 384:693-704. doi:10.1056/NEJMoa2021436 [ PMC free article : PMC7383595 ] [ PubMed : 32678530 ] [ CrossRef ] 97. Boglione L, Olivieri C, Rostagno R, et al. Role of the early short-course corticosteroids treatment in ARDS caused by COVID-19: a single-center, retrospective analysis. Adv Med Sci. 2021;66(2):262-268. doi:10.1016/j.advms.2021.04.002 [ PMC free article : PMC8064826 ] [ PubMed : 34022675 ] [ CrossRef ] 98. Fadel R, Morrison AR, Vahia A, et al. Early short-course corticosteroids in hospitalized patients with COVID-19. Clin Infect Dis. 2020;71(16):2114-2120. doi:10.1093/cid/ciaa601 [ PMC free article : PMC7314133 ] [ PubMed : 32427279 ] [ CrossRef ] 99. Lee HW, Park J, Lee JK, Park TY, Heo EY. The effect of the timing of dexamethasone administration in patients with COVID-19 pneumonia. Tuberc Respir Dis. 2021;84(3):2170225. doi:10.4046/trd.2021.0309 [ PMC free article : PMC8273022 ] [ PubMed : 34078038 ] [ CrossRef ] 100. Monedero P, Gea A, Castro P, et al. Early corticosteroids are associated with lower mortality in critically ill patients with COVID-19: a cohort study. Crit Care. 2021;25(1):1-13. doi:10.1186/s13054-020-03422-3 [ PMC free article : PMC7780210 ] [ PubMed : 33397463 ] [ CrossRef ] 101. Goyal DK, Mansab F, Iqbal A, Bhatti S. Early intervention likely improves mortality in COVID-19 infection. Clin Med. 2020;20(3):248. doi:10.7861/clinmed.2020-0214 [ PMC free article : PMC7354047 ] [ PubMed : 32357975 ] [ CrossRef ] 102. Asch DA, Muller RW, Volpp KG. Automated hovering in health care—watching over the 5000 hours. N Engl J Med. 2012;367(1):1. doi:10.1056/NEJMp1203869 [ PubMed : 22716935 ] [ CrossRef ] 103. Iqbal FM, Joshi M, Davies G, Khan S, Ashrafian H, Darzi A. Design of the pilot, proof of concept REMOTE-COVID trial: remote monitoring use in suspected cases of COVID-19 (SARS-CoV-2). Pilot Feasibility Stud. 2021;7(1):1-7. doi:10.1186/s40814-021-00804-4 [ PMC free article : PMC7933391 ] [ PubMed : 33673868 ] [ CrossRef ] 104. Bell LC, Norris-Grey C, Luintel A, et al. Implementation and evaluation of a COVID-19 rapid follow-up service for patients discharged from the emergency department. Clin Med. 2021;21(1):e57-e62. doi:10.7861/clinmed.2020-0816 [ PMC free article : PMC7850184 ] [ PubMed : 33355255 ] [ CrossRef ] 105. Bokolo AJ. Application of telemedicine and eHealth technology for clinical services in response to COVID-19 pandemic. Health Technol. 2021;11(2):359-366. doi:10.1007/s12553-020-00516-4 [ PMC free article : PMC7808733 ] [ PubMed : 33469474 ] [ CrossRef ] 106. Ogedegbe G, Ravenell J, Adhikari S, et al. Assessment of racial/ethnic disparities in hospitalization and mortality in patients with COVID-19 in New York City. JAMA Netw Open. 2020;3(12):e2026881. doi:10.1001/jamanetworkopen.2020.26881 [ PMC free article : PMC7718605 ] [ PubMed : 33275153 ] [ CrossRef ] 107. Centers for Disease Control and Prevention. Risk for COVID-19 infection, hospitalization, and death by race/ethnicity. Accessed April 19, 2021. https://www ​.cdc.gov/coronavirus ​/2019-ncov ​/covid-data/investigations-discovery ​/hospitalization-death-by-race-ethnicity ​.html 108. Pew Research Center. Mobile fact sheet. April 7, 2021. https://www ​.pewresearch ​.org/internet/fact-sheet ​/mobile/#:∼:text=The ​%20vast ​%20majority%20of%20Americans,a ​%20cellphone ​%20of%20some%20kind 109. Price-Haywood EG, Burton J, Fort D, Seoane L. Hospitalization and mortality among black patients and white patients with Covid-19. N Engl J Med. 2020;382(26):2534-2543. doi:10.1056/NEJMsa2011686 [ PMC free article : PMC7269015 ] [ PubMed : 32459916 ] [ CrossRef ] 110. Anesi GL, Jablonski J, Harhay MO, et al. Characteristics, outcomes, and trends of patients with COVID-19-related critical illness at a learning health system in the United States. Ann Intern Med. 2021;174(5):613-621. doi:10.7326/M20-5327 [ PMC free article : PMC7901669 ] [ PubMed : 33460330 ] [ CrossRef ] 111. Horwitz L, Jones SA, Cerfolio RJ, et al. Trends in COVID-19 risk-adjusted mortality rates in a single health system. J Hosp Med. 2021;16(2):90-92. doi:10.12788/jhm.3552 [ PubMed : 33147129 ] [ CrossRef ] 112. Yong E. Hospitals are in serious trouble. The Atlantic. January 7, 2022. Accessed January 11, 2022. https://www ​.theatlantic ​.com/health/archive ​/2022/01/omicron-mild-hospital-strain-health-care-workers ​/621193/ 113. Delgado MK, Morgan AU, Asch DA, et al. Comparative effectiveness of an automated text messaging service for monitoring COVID-19 at home. Ann Intern Med. 2022;175(2):179-190. doi:10.7326/M21-2019 [ PMC free article : PMC8722738 ] [ PubMed : 34781715 ] [ CrossRef ] 114. Faro JM, Cutrona SL. Extending a lifeline to nonhospitalized patients with COVID-19 through automated text messaging. Ann Intern Med. 2022;175(2):291-292. doi:10.7326/M21-4273 [ PMC free article : PMC8593887 ] [ PubMed : 34781713 ] [ CrossRef ] 115. Simonson TS, Baker TL, Banzett RB, et al. Silent hypoxaemia in COVID-19 patients. J Physiol. 2021;599(4):1057-1065. doi:10.1113/JP280769 [ PMC free article : PMC7902403 ] [ PubMed : 33347610 ] [ CrossRef ] 116. Tobin MJ, Laghi F, Jubran A. Why COVID-19 silent hypoxemia is baffling to physicians. Am J Respir Crit Care Med. 2020;202(3):356-360. doi:10.1164/rccm.202006-2157CP [ PMC free article : PMC7397783 ] [ PubMed : 32539537 ] [ CrossRef ] 117. Jouffroy R, Jost D, Prunet B. Prehospital pulse oximetry: a red flag for early detection of silent hypoxemia in COVID-19 patients. Crit Care. 2020;24(1):313. doi:10.1186/s13054-020-03036-9 [ PMC free article : PMC7278215 ] [ PubMed : 32513249 ] [ CrossRef ] 118. Rahman A, Tabassum T, Araf Y, Al Nahid A, Ullah MA, Hosen MJ. Silent hypoxia in COVID-19: pathomechanism and possible management strategy. Mol Biol Rep. 2021;48(4):3863-3869. doi:10.1007/s11033-021-06358-1 [ PMC free article : PMC8062941 ] [ PubMed : 33891272 ] [ CrossRef ] 119. Nair CV, Sathyapalan DT, Moni M, Suresh A, Roshni PR. Happy hypoxemia: a perplexing clinical entity in coronavirus disease 2019. J Appl Pharm Sci. 2022;12(1):65-69. doi:10.7324/JAPS.2021.120105 [ CrossRef ] 120. Levitan RM. Pulse oximetry as a biomarker for early identification and hospitalization of COVID-19 pneumonia. Acad Emerg Med. 2020;27(8):785-786. doi:10.1111/acem.14052 [ PMC free article : PMC7323007 ] [ PubMed : 32779867 ] [ CrossRef ] 121. WHO. COVID-19 clinical management: living guidance, 25 January 2021. 2021. https://iris ​.who.int/handle/10665/338882 122. Greenhalgh T, Knight M, Inda-Kim M, Fulop NJ, Leach J, Vindrola-Padros C. Remote management of covid-19 using home pulse oximetry and virtual ward support. BMJ. 2021;372:n677. doi:10.1136/bmj.n677 [ PubMed : 33766809 ] [ CrossRef ] 123. UK National Health Service. COVID oximetry @ home. Accessed December 7, 2021. https://www ​.england.nhs ​.uk/nhs-at-home/covid-oximetry-at-home/ 124. Knight SR, Ho A, Pius R, et al. Risk stratification of patients admitted to hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: development and validation of the 4C Mortality Score. BMJ. 2020;370:m3339. doi:10.1136/bmj.m3339 [ PMC free article : PMC7116472 ] [ PubMed : 32907855 ] [ CrossRef ] 125. Yadaw AS, Li YC, Bose S, Iyengar R, Bunyavanich S, Pandey G. Clinical features of COVID-19 mortality: development and validation of a clinical prediction model. Lancet Digit Health. 2020;2(10):e516-e525. doi:10.1016/S2589-7500(20)30217-X [ PMC free article : PMC7508513 ] [ PubMed : 32984797 ] [ CrossRef ] 126. Bertsimas D, Lukin G, Mingardi L, et al. COVID-19 mortality risk assessment: an international multi-center study. PLoS One. 2020;15(12):e0243262. doi:10.1371/journal.pone.0243262 [ PMC free article : PMC7725386 ] [ PubMed : 33296405 ] [ CrossRef ] 127. Ariti CA, Cleland JG, Pocock SJ, et al. Days alive and out of hospital and the patient journey in patients with heart failure: insights from the candesartan in heart failure: assessment of reduction in mortality and morbidity (CHARM) program. Am Heart J. 2011;162(5):900-906. doi:10.1016/j.ahj.2011.08.003 [ PubMed : 22093207 ] [ CrossRef ] 128. Wasson J, Coleman EA. Health confidence: a simple, essential measure for patient engagement and better practice. Fam Pract Manag. 2014;21(5):8-12. [ PubMed : 25251348 ] 129. Hays RD, Spritzer KL, Schalet BD, Cella D. PROMIS®-29 v2.0 profile physical and mental health summary scores. Qual Life Res. 2018;27(7):1885-1891. doi:10.1007/s11136-018-1842-3 [ PMC free article : PMC5999556 ] [ PubMed : 29569016 ] [ CrossRef ] 130. Boissy A. Getting to patient-centered care in a post-Covid-19 digital world: a proposal for novel surveys, methodology, and patient experience maturity assessment. NEJM Catalyst. 2020;1(4). doi:10.1056/CAT.19.1106 [ CrossRef ] 131. Reichheld FF. The one number you need to grow. Harvard Business Review. 2003;81(12):46-55. [ PubMed : 14712543 ] 132. Pronovost PJ, Cole MD, Hughes RM. Remote patient monitoring during COVID-19: an unexpected patient safety benefit. JAMA. 2022;327(12):1125-1126. doi:10.1001/jama.2022.2040 [ PubMed : 35212725 ] [ CrossRef ] 133. Aalam AA, Hood C, Donelan C, Rutenberg A, Kane EM, Sikka N. Remote patient monitoring for ED discharges in the COVID-19 pandemic. Emerg Med J. 2021;38(3):229-231. doi:10.1136/emermed-2020-210022 [ PubMed : 33472870 ] [ CrossRef ] 134. Grutters LA, Majoor KI, Mattern ESK, Hardeman JA, van Swol CFP, Vorselaars ADM. Home telemonitoring makes early hospital discharge of COVID-19 patients possible. J Am Med Inform Assoc. 2020;27(11):1825-1827. doi:10.1093/jamia/ocaa168 [ PMC free article : PMC7454667 ] [ PubMed : 32667985 ] [ CrossRef ] 135. Patel H, Hassell A, Cyriacks B, Fisher B, Tonelli W, Davis C. Building a real-time remote patient monitoring patient safety program for COVID-19 patients. Am J Med Qual. 2022;37(4):342-347. doi:10.1097/JMQ.0000000000000046 [ PMC free article : PMC9241558 ] [ PubMed : 35213860 ] [ CrossRef ] 136. Tabacof L, Kellner C, Breyman E, et al. Remote patient monitoring for home management of coronavirus disease 2019 in New York: a cross-sectional observational study. Telemed J E Health. 2021;27(6):641-648. doi:10.1089/tmj.2020.0339 [ PubMed : 33232204 ] [ CrossRef ] 137. Huynh DN, Millan A, Quijada E, John D, Khan S, Funahashi T. Description and early results of the Kaiser Permanente Southern California COVID-19 Home Monitoring Program. Perm J. 2021;25(3). doi:10.7812/TPP/20.281 [ PMC free article : PMC8784054 ] [ PubMed : 35348067 ] [ CrossRef ] 138. Lee KC, Morgan AU, Chaiyachati KH, et al. Pulse oximetry for monitoring patients with Covid-19 at home—a pragmatic, randomized trial. N Engl J Med. 2022;386(19):1857-1859. doi:10.1056/NEJMc2201541 [ PMC free article : PMC9006781 ] [ PubMed : 35385625 ] [ CrossRef ] 139. Field MJ, Grigsby J. Telemedicine and remote patient monitoring. JAMA. 2002;288(4):423-425. doi:10.1001/jama.288.4.423 [ PubMed : 12132953 ] [ CrossRef ] 140. Daly B, Nicholas K, Flynn J, et al. Analysis of a remote monitoring program for symptoms among adults with cancer receiving antineoplastic therapy. JAMA Netw Open. 2022;5(3):e221078. doi:10.1001/jamanetworkopen.2022.1078 [ PMC free article : PMC8897754 ] [ PubMed : 35244701 ] [ CrossRef ] 141. Fisher NDL, Fera LE, Dunning JR, et al. Development of an entirely remote, non-physician led hypertension management program. Clin Cardiol. 2019;42(2):285-291. doi:10.1002/clc.23141 [ PMC free article : PMC6712321 ] [ PubMed : 30582181 ] [ CrossRef ] 142. Salehi S, Olyaeemanesh A, Mobinizadeh M, Nasli-Esfahani E, Riazi H. Assessment of remote patient monitoring (RPM) systems for patients with type 2 diabetes: a systematic review and meta-analysis. J Diabetes Metab Disord. 2020;19(1):115-127. doi:10.1007/s40200-019-00482-3 [ PMC free article : PMC7270436 ] [ PubMed : 32550161 ] [ CrossRef ] 143. Ong MK, Romano PS, Edgington S, et al. Effectiveness of remote patient monitoring after discharge of hospitalized patients with heart failure: the Better Effectiveness After Transition—Heart Failure (BEAT-HF) randomized clinical trial. JAMA Intern Med. 2016;176(3):310-318. doi:10.1001/jamainternmed.2015.7712 [ PMC free article : PMC4827701 ] [ PubMed : 26857383 ] [ CrossRef ] 144. Guest G, Namey E, Chen M. A simple method to assess and report thematic saturation in qualitative research. PLoS One. 2020;15(5):e0232076. doi:10.1371/journal.pone.0232076 [ PMC free article : PMC7200005 ] [ PubMed : 32369511 ] [ CrossRef ] 145. Hays DG, Singh AA. Qualitative Inquiry in Clinical and Educational Settings. Guilford Press; 2012:xxiv, 504. 146. Flick U. An Introduction to Qualitative Research. 4th ed. Sage Publications; 2009:xx. 147. McHugh ML. Interrater reliability: the kappa statistic. Biochem Med. 2012;22(3):276-282. doi:10.11613/BM.2012.031 [ PMC free article : PMC3900052 ] [ PubMed : 23092060 ] [ CrossRef ] 148. Nguyen MT, Garcia F, Juarez J, et al. Satisfaction can co-exist with hesitation: qualitative analysis of acceptability of telemedicine among multi-lingual patients in a safety-net healthcare system during the COVID-19 pandemic. BMC Health Serv Res. 2022;22(1):195. doi:10.1186/s12913-022-07547-9 [ PMC free article : PMC8842908 ] [ PubMed : 35164746 ] [ CrossRef ] 149. Culyer AJ, Bombard Y. An equity framework for health technology assessments. Med Decis Making. 2012;32(3):428-441. doi:10.1177/0272989X11426484 [ PubMed : 22065143 ] [ CrossRef ] 150. Stiles-Shields C, Cummings C, Montague E, Plevinsky JM, Psihogios AM, Williams KDA. A call to action: using and extending human-centered design methodologies to improve mental and behavioral health equity. Front Digit Health. 2022;4:848052. doi:10.3389/fdgth.2022.848052 [ PMC free article : PMC9081673 ] [ PubMed : 35547091 ] [ CrossRef ] 151. Sieck CJ, Sheon A, Ancker JS, Castek J, Callahan B, Siefer A. Digital inclusion as a social determinant of health. NPJ Digit Med. 2021;4(1):52. doi:10.1038/s41746-021-00413-8 [ PMC free article : PMC7969595 ] [ PubMed : 33731887 ] [ CrossRef ] Related Publications •. Delgado MK, Morgan AU, Asch DA, et al. Comparative effectiveness of an automated text messaging service for monitoring COVID-19 at home. Ann Intern Med. 2022;175(2):179-190. doi:10.7326/M21-2019. [ PMC free article : PMC8722738 ] [ PubMed : 34781715 ] [ CrossRef ] •. Lee KC, Morgan AU, Chaiyachati KH, et al. Pulse oximetry for monitoring patients with covid-19 at home—a pragmatic, randomized trial. N Engl J Med. 2022;386(19):1857-1859. doi:10.1056/NEJMc2201541 [ PMC free article : PMC9006781 ] [ PubMed : 35385625 ] [ CrossRef ] •. Chaiyachati KH, Shea JA, Ward M, et al. Patient and clinician perspectives of a remote monitoring program for COVID-19 and lessons for future programs BMC Health Serv Res. 2023;23:698. doi:10.1186/s12913-023-09684-1 [ PMC free article : PMC10304230 ] [ PubMed : 37370059 ] [ CrossRef ] Acknowledgments The authors thank P. J. Brennan, C. William Hanson III, Susan Day, Michael Y. Kopinsky, Roy Rosin, the COVID Watch nursing and clinical team, Bill Marella and colleagues at HealthShare Exchange (Philadelphia, Pennsylvania), and their patient and stakeholder advisory board: Jill Baren, Brooke Feldman, Janet Williams, Dennis Burroughs, Utibe R. Essien, and Carlos Pascual Sanchez. In addition to the grant from PCORI (COVID-2020C2-10830, to Dr Delgado), this research received support from the NIH (K23HD090272001, to Dr Delgado, and K08AG065444, to Dr Chaiyachati) and the Abramson Family Foundation (to Dr Delgado). Data Sharing Plan The University of Pennsylvania, in agreement with the University of Michigan, will provide a copy of the “Evaluating the Effectiveness and Implementation of an Automated Remote Monitoring Program for COVID-19 Patients” full data package that was funded by PCORI along with any other related documentation to the Inter-university Consortium for Political and Social Research (ICPSR). As a part of the full data package, the University of Pennsylvania will share from aims 1 and 2 (1) electronic restricted-use data including, but not limited to, the deidentified analyzable dataset, full protocol, metadata, data dictionary, full statistical analysis plan, and analytic code; (2) codebook files; and (3) ICPSR modifications of those original research project files. The purpose of this is to facilitate data sharing and allow ICPSR to share these deidentified data for examination, processing, and distribution by qualified researchers. Research reported in this report was funded through a Patient-Centered Outcomes Research Institute® (PCORI®) Award (COVID-2020C2-10830). Further information available at: https://www.pcori.org/research-results/2020/testing-effectiveness-text-message-based-home-monitoring-program-and-oxygen-monitoring-patients-covid-19-covid-watch-study Appendix Aim 3 Interview Guides (PDF, 372K) Original Project Title: Evaluating the effectiveness and implementation of an automated remote monitoring program for COVID-19 patients PCORI ID: COVID-2020C2-10830 ClinicalTrials.gov ID: NCT04581863 Suggested citation: Delgado MK, Morgan AU, Asch DA, et al. (2024). Testing the Effectiveness of a Text-Message-Based Home Monitoring Program and Oxygen Monitoring for Patients with COVID-19—The COVID Watch Study . https://doi.org/10.25302/02.2024.COVID.2020C210830 . Patient-Centered Outcomes Research Institute (PCORI). Disclaimer The [views, statements, opinions] presented in this report are solely the responsibility of the author(s) and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute® (PCORI®), its Board of Governors or Methodology Committee. Copyright © 2024. University of Pennsylvania, Perelman School of Medicine. All Rights Reserved. This book is distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License which permits noncommercial use and distribution provided the original author(s) and source are credited. (See https://creativecommons.org/licenses/by-nc-nd/4.0/ Bookshelf ID: NBK621497 PMID: 41911372 DOI: 10.25302/02.2024.COVID.2020C210830 Share Views PubReader Print View Cite this Page Delgado MK, Morgan AU, Asch DA, et al. Testing the Effectiveness of a Text-Message-Based Home Monitoring Program and Oxygen Monitoring for Patients with COVID-19—The COVID Watch Study [Internet]. Washington (DC): Patient-Centered Outcomes Research Institute (PCORI); 2024 Feb. doi: 10.25302/02.2024.COVID.2020C210830 PDF version of this title (3.7M) In this Page Background Participation of Patients and Other Stakeholders Aim 1 Aim 2 Aim 3 Discussion Conclusions References Related Publications Acknowledgments Appendix Other titles in this collection PCORI Final Research Reports Related information NLM Catalog Related NLM Catalog Entries PMC PubMed Central citations PubMed Links to PubMed Recent Activity Clear Turn Off Turn On Testing the Effectiveness of a Text-Message-Based Home Monitoring Program and Ox... Testing the Effectiveness of a Text-Message-Based Home Monitoring Program and Oxygen Monitoring for Patients with COVID-19—The COVID Watch Study Your browsing activity is empty. Activity recording is turned off. Turn recording back on See more... Follow NCBI Twitter Facebook LinkedIn GitHub NCBI Insights Blog Connect with NLM Twitter Facebook Youtube National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894 Web Policies FOIA HHS Vulnerability Disclosure Help Accessibility Careers NLM NIH HHS USA.gov