A Patient Reported Outcomes Program at Scale at a Cancer Center - PMC Skip to main content An official website of the United States government Here's how you know Here's how you know Official websites use .gov A .gov website belongs to an official government organization in the United States. Secure .gov websites use HTTPS A lock ( Lock Locked padlock icon ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites. Search Log in Dashboard Publications Account settings Log out Search… Search NCBI Primary site navigation Search Logged in as: Dashboard Publications Account settings Log in Search PMC Full-Text Archive Search in PMC Journal List User Guide PERMALINK Copy As a library, NLM provides access to scientific literature. Inclusion in an NLM database does not imply endorsement of, or agreement with, the contents by NLM or the National Institutes of Health. Learn more: PMC Disclaimer | PMC Copyright Notice JCO Clin Cancer Inform . Author manuscript; available in PMC: 2026 Apr 22. Published in final edited form as: JCO Clin Cancer Inform. 2025 Apr 22;9:e2400253. doi: 10.1200/CCI-24-00253 Search in PMC Search in PubMed View in NLM Catalog Add to search A Patient Reported Outcomes Program at Scale at a Cancer Center Fernanda C G Polubriaginof Fernanda C G Polubriaginof , MD, PhD 1. Digital Informatics and Technology Solutions, Memorial Sloan Kettering Cancer Center, New York, NY Find articles by Fernanda C G Polubriaginof 1 , Allison Lipitz-Snyderman Allison Lipitz-Snyderman , PhD 1. Digital Informatics and Technology Solutions, Memorial Sloan Kettering Cancer Center, New York, NY Find articles by Allison Lipitz-Snyderman 1 , Susan Chimonas Susan Chimonas , PhD 1. Digital Informatics and Technology Solutions, Memorial Sloan Kettering Cancer Center, New York, NY Find articles by Susan Chimonas 1 , Peter D Stetson Peter D Stetson , MD, MSc 1. Digital Informatics and Technology Solutions, Memorial Sloan Kettering Cancer Center, New York, NY Find articles by Peter D Stetson 1 , Gilad J Kuperman Gilad J Kuperman , MD, PhD 1. Digital Informatics and Technology Solutions, Memorial Sloan Kettering Cancer Center, New York, NY Find articles by Gilad J Kuperman 1 Author information Article notes Copyright and License information 1. Digital Informatics and Technology Solutions, Memorial Sloan Kettering Cancer Center, New York, NY ✉ Corresponding author: Gilad J. Kuperman, MD, PhD, 633 3 rd Avenue, Memorial Sloan Kettering Cancer Center, 2 nd Floor, Department of Digital Informatics & Technology Solutions, New York, NY, USA, [email protected] Issue date 2025 Apr. PMC Copyright notice PMCID: PMC12017341 NIHMSID: NIHMS2062770 PMID: 40262062 The publisher's version of this article is available at JCO Clin Cancer Inform Abstract Background: Incorporating patient-reported outcomes (PROs) into health care processes can improve engagement with patients; however, adopting PROs at scale is challenging. Objective: To describe the design, development, and adoption at scale of a comprehensive PRO program for standard of care and research at a cancer center. Methods: Requirements for a PRO program were obtained from multiple stakeholders. Components of the program included a governance process to assure a consistent and satisfactory experience for patients completing PRO questionnaires, tools to create and manage questionnaires and related content, methods to send questionnaires to relevant patients at the appropriate time, interactive tools for patients to complete the questionnaires as part of their portal experience, and integration of PRO data into workflows for clinicians. We used descriptive statistics to assess the use of the program from 2016 to 2023. Results: From program launch (on February 1, 2016) until December 31, 2023, 189 unique questionnaires were developed (101 for standard-of-care, 70 for research, 18 for quality improvement). Of the 432,497 unique patients who were assigned at least one questionnaire, 314,685 (73%) completed at least one. Of 5,948,464 questionnaires sent, 3,098,574 (52%) were completed. The median completion time was 2 minutes. Conclusion: Large-scale adoption of PROs at a cancer center is feasible. Key considerations for success include governance processes, attention to patient experience and clinician workflow, and the ability to manage complex inclusion criteria and timing of delivery of questionnaires. These principles should be disseminated so the full potential of PROs in healthcare can be realized. Keywords: Patient Reported Outcome Measures, Surveys and Questionnaires, Quality Improvement, Delivery of Health Care Context Summary Key objective: What are the characteristics of an electronic patient-reported outcomes program adopted at scale at a major cancer center? Knowledge generated: Over an 8-year period, 189 questionnaires for standard of care, quality improvement and research purposes were developed, over 5.9 million instances of the questionnaires were generated, and over 3 million responses were received. The program includes a governance process to oversee content development, technical capabilities to deliver questionnaires to patients, and technology and workflows to make the data available to the care team. Relevance: * This study provides a comprehensive strategic framework for academic medical centers to implement patient-reported outcomes into oncology practice, demonstrating how integrated governance, technical infrastructure, and workflow design can systematically scale patient-centered care delivery across clinical, research, and quality improvement domains. *Relevance section written by JCO CCI Associate Editor Frank Po-Yen Lin, FRACP, MBChB, FAIDH, PhD INTRODUCTION Patient-reported outcomes (PROs) are evaluations of a patient’s health status reported directly by the patient, rather than by a healthcare provider. 1 Examples of PROs include symptom assessments, physical functioning, health-related social needs, health behaviors, and quality of life measures. Capturing PROs and incorporating them into care processes can increase patient engagement, support clinical care, and produce data for research. 2 , 3 In oncology, PROs can be used to assess the patient’s perceptions of their disease, the extent of disease-induced symptoms and the side effects of treatments. The monitoring of PROs by the care team can reduce healthcare visits, improve physical functioning, improve quality of life, and improve survival. 4 – 8 Over the last 15 years, the increasing prevalence of electronic health records and patient portals means that PROs can be collected more routinely electronically. Additionally, due to the advent of consumer technology, patients now often expect to interact digitally with their health care providers, which further increases the opportunity to incorporate PROs into care processes. Despite these important enablers, broadly incorporating PROs into routine workflow at provider organizations for research and standard of care remains uncommon and several challenges must be addressed. 9 , 10 First, for the domains of interest, PRO questionnaires must be obtained or developed. Libraries of validated questionnaires have been developed for several domains, but these may not cover all potential PRO requirements. 11 To minimize patient burden and maximize the likelihood of patient response, PRO questionnaires should be dynamic, with the ability to branch on captured data and other patient characteristics. 12 As PRO questions are being developed, the text should be crafted to be suitable for a lay audience, and should be consistent when an organization is implementing multiple questionnaires. 13 The patient should be able to access PRO questionnaires from their patient portal, and the interactive technology should follow user-centered design principles. 10 , 14 The logic that triggers a questionnaire to be sent to a patient may be complex and needs to be configurable based on clinical events -- for example, following a surgical procedure, just before an office visit, etc. – or on the presence of new clinical data, for example a positive response on a family history screening questionnaire. Finally, to achieve its intended benefit, PRO data must be easily available to clinical and research staff and incorporated into routine workflow. Our organization began studying the use of PROs in oncology in the mid-2000s. 15 , 16 In 2015, we initiated the development of a new PRO program intended to be scalable and usable for research and standard of care scenarios. This report describes the goals, design, development, and adoption of that program from 2016 to 2023. METHODS Setting: This work took place at Memorial Sloan Kettering Cancer Center (MSK), a high-volume specialty cancer center in the New York City metropolitan area. MSK provides 750,000 ambulatory encounters and 23,000 inpatient admissions annually. MSK has an active research program; currently, there are about 1800 research studies underway, approximately half of which are interventional clinical trials. In 2015, MSK initiated a new program to support PROs. The program included new technology and new processes to support the use of PROs at scale. The program leveraged an internally developed patient portal, MyMSK, available to patients on the web and on a mobile platform. The design requirements for the PRO program were derived from a comprehensive process involving multiple stakeholder groups, including clinicians, clinical researchers, administrators, operational staff, the patient education and communication team, legal staff, medical records leadership, patients, caregivers and technical staff. In 2019, an institution-wide workshop was convened with 90 participants to discuss future directions and to gather input on additional requirements for the program. Table 1 summarizes the key design requirements identified by stakeholders to support a comprehensive and scalable PRO program. Table 1. Design requirements to support a comprehensive and scalable PRO program. Requirement Description Technical Environment for Questionnaire Management A technical environment should be established to develop, maintain, and review questionnaires, including features such as alerting thresholds. Dynamic Questionnaires with Branching Logic Questionnaires should be dynamic, displaying only relevant questions based on past responses and patient information to enhance efficiency. Clinical and Research Accessibility PROs must be accessible for both clinical and research purposes. Centralized Governance and Patient-Centered Language A centralized governance process should ensure questionnaires are reviewed and approved, with an emphasis on patient-friendly wording. Technical Methods for Cohort Identification and Timing Technical systems should be in place to (i) identify the patient cohorts to receive each questionnaire, and (ii) create the logic to send a questionnaire to a patient at a particular point in time (for example, at a certain interval following surgery or at a certain time prior to a clinic appointment) or based on the presence of a clinical data element. Notification of Assigned Questionnaires Methods should be implemented to notify patients when a questionnaire is assigned to them. Integration with Patient Portal The questionnaire completion interface should be integrated into the patient portal, available on both web and mobile platforms. Threshold-Based Alerts for Clinical Team PRO results exceeding predefined thresholds should trigger notifications to the clinical team for timely intervention. Clinician Review and Integration into Workflow There should be methods to support clinicians in reviewing and interpreting PRO data within their routine workflow and integrating results into clinical documentation. Open in a new tab A multifaceted socio-technical program was created to address the requirements. The components of the program included ( Figure 1 ): Figure 1. Open in a new tab Components of the MSK Patient Reported Outcomes Program A governance process to oversee the development of questionnaires and related content . Clinical or research sponsors generate requests for PRO questionnaires. The MSK eForms Committee assures alignment with the program requirements, and reviews and considers (i) the overall number of questions in the questionnaire, to mitigate against patient burden, (ii) the wording of questions, to assure patient-friendliness, (iii) the branching logic required, (iv) threshold alerting rules and routing logic, i.e., who will be notified when an alert is detected, (v) cohort identification, i.e., which patients should receive the questionnaire, (vi) scheduling logic, i.e., when the questionnaire should be triggered for a patient, (vii) how long a questionnaire should remain available (“open”) for a patient after it has been sent, (viii) documentation templates in the electronic health record (EHR), if any, that need to be modified to incorporate PRO data, and (ix) any educational materials that need to accompany the questionnaire. Research questionnaires are reviewed and approved by the IRB. Final versions of questionnaires and related content are stored in a searchable Question Library. Once a questionnaire is approved, the multidisciplinary eForms Workgroup oversees development and testing of the questionnaire, maps workflows related to the questionnaire and designs and carries out an implementation plan. Weekly meetings are used to discuss details about the creation of the questionnaire including level of literacy, branching logic, definition of patient cohort, scheduling logic, alert rules and routing, documentation templates, along with current and future state workflow mapping. Launch management activities including communication and training are also addressed. The sponsors can interact with the questionnaire before it is released to patients to assure it is working as intended. More details of the content development process are contained in Supplementary Materials, Appendix 1 . Methods to assign questionnaires to patients . A Questionnaire Assignment Engine, which is part of the MSK data warehouse infrastructure, uses the cohort definition logic and the scheduling logic, along with data in the patient database, to assign questionnaires to patients. There are some instances where a single patient may be assigned the same questionnaire multiple times, for example, in one program, discharged post-operative patients receive a “Post Operative Symptom Questionnaire” each day for 10 days. These count as 10 separate questionnaire assignments. The assignment logic keeps track of the post-operative day. The branching logic can make use of the value of the post-operative day and the patient may receive a different set of questions each day, even though the base questionnaire is the same for all 10 days. The Assignment Engine periodically re-evaluates the scheduled timing of questionnaires to account, for example, for appointments that are rescheduled or canceled or other data that has changed. Tools to enable patients to complete questionnaires . Questionnaires are made available to the patient in the MSK internally developed portal, MyMSK. When a questionnaire is assigned, a feature of MyMSK notifies the patient (via secure message or email, depending on the patient’s preferences). The notification has a link for immediate access to the questionnaire. The patient also receives a reminder to complete outstanding questionnaires before an upcoming clinical visit. If there are outstanding questionnaires at the time of a visit, the patient has the option to complete a questionnaire in the waiting room using an iPad with assistance from staff, if necessary. MSK Engage is the interactive module within MyMSK that allows the patient to complete a questionnaire. MSK Engage was developed using responsive web design, which supports the patient experience on both web and mobile platforms. The MSK Engage user interface and interactive features were developed with input from patients through interviews, focus groups, and usability testing. For pending questionnaires, MSK Engage displays the name of the questionnaire, the date it’s due, whether the assessment is related to a clinic visit, and the estimated time for completion. An example of the screen that informs patients of pending questionnaires is shown in Supplementary Materials, Appendix 2 . Integration of PRO data into workflows . Once the PRO data has been entered by the patient, there are several workflow and technology approaches that make the data available to clinicians and researchers. Some clinical documentation templates in the EHR have been configured to automatically import PRO data. An example is the New Visit Nursing Intake Form. The patient’s responses to these questions may be modified by the nurse as they are being reviewed with the patient. Final data are saved in the Outpatient Nursing Health Assessment clinical note. Other PRO data can be imported into a clinician’s note “on demand.” This allows the PRO data to be reviewed with other clinical data, which facilitates decision making. To support research workflows, research teams can be notified when questionnaires have been completed. Data from these questionnaires are added to data warehouses for analyses. Tableau dashboards can be created to display metrics. As mentioned, based on rules that can be created in the questionnaire authoring environment, alerts are generated when the patient’s PRO entries exceed specified predefined thresholds. Alert notifications often are implemented as secure messages that are sent to the clinical team. Alerts also can generate referral orders, for example, to a smoking cessation program for patients that are smokers. Alerts can be categorized as medium or high severity (as described in Appendix 1 ). Care teams develop escalation pathways (for example, from nurse to physician, etc.) to manage the workflows related to alerts. PRO data also are stored in a database that can be used for analytics and operational reporting purposes, for example, to calculate response rates for questionnaires and for specific questions, and to determine average values for reported outcomes. Historical PRO data has been used to characterize the average responses of a cohort, which an individual’s response then can be compared to. In one program, a patient’s self-reported level of pain post-operatively is compared to the group norm. Then the patient is informed whether the level of pain they report is the same, higher or lower than the group norm. This led to decreased patient anxiety and a reduction in phone calls to the office. 18 An example of this type of display is shown in Supplementary Materials, Appendix 3 . Evaluation of Usage: We conducted a retrospective analysis of data from the launch of the program on February 1, 2016, through December 31, 2023. We report on the number of questionnaires developed, the number of questionnaires sent to and completed by patients, patient burden as measured by questionnaire completion time, and the mode of completion of the questionnaires (via the patient portal, or in the waiting room using an MSK-supplied tablet). We provided additional detail on completion time, drop off rates, median time to completion, and age distribution for questionnaires completed in 2023. This study was reviewed and approved by the MSK Institutional Review Board. RESULTS Summary statistics for the PRO program from initiation in 2016 until the end of 2023 are shown in Table 2 . In that time, 189 unique questionnaires were developed -- 101 (53%) for standard of care, 70 (37%) for research protocols, 18 (10%) for quality improvement ( Table 2 ). Research questionnaires occasionally may be converted into standard of care. As of the end of 2023, there were 8 questionnaires that had been converted in this manner. Examples of programs that leverage PRO questionnaires are shown in Table 3 . In the evaluation period, 432,497 patients were assigned at least one questionnaire, with 314,685 patients (73%) completing at least one. A total of 5,948,464 questionnaires were sent, of which 3,098,574 (52%) were completed. The median time for a patient to complete a questionnaire was 2 minutes. Most questionnaires were completed via the patient portal (80%); the remainder were completed using tablets in the waiting room. Table 2. Summary metrics from MSK PRO Program, February 1, 2016 – December 31, 2023 Measure N Questionnaires created 189 Questionnaire category Standard of care 101 (53%) Research 70 (37%) Quality improvement 18 (10%) Patients receiving at least one questionnaire 432,497 Patients completing at least one questionnaire 314,685 (73%) Total number of questionnaires sent 5,948,464 Total number of questionnaires completed 3,098,574 (52%) Method of completion Questionnaires completed via portal (web or mobile) 80% Questionnaires completed via in-clinic tablets 20% Median time to complete a questionnaire 2 minutes Open in a new tab Table 3. Examples of programs that leverage PROs, by category Standard of care (examples) Purpose Questionnaire Externally-developed questionnaires Measure anxiety in outpatient Psychiatry consults. GAD-7, PHQ-9 Assess pain in supportive care. Brief Pain Inventory (BPI), Memorial Symptom Assessment Scale (MSAS), European Quality of Life Locally-developed questionnaires (reviewed and approved by MSK eForms Committee) Review symptoms and health behaviors in cancer survivors. Survivorship Patient Self Assessment Assess patient readiness for discharge. Goals to Discharge Track surgery-related symptoms. Post-operative symptom tracking Capture Pre-Bone Marrow / Stem Cell transplant health history data collection (regulatory requirement). Cellular Therapy Health History Questionnaire Assess health goals, symptoms, and lifestyle factors relevant to integrative medicine. Integrative Medicine Questionnaire Assess experience for patients who have completed a sleep study. Post Sleep Study Questionnaire Research (examples) Purpose Questionnaire Externally-developed questionnaires Assess health status in multiple myeloma. PROMIS_29 Assess sarcoma, gastric mixed tumor outcomes. PROMIS v1.2 Global Health Evaluate telemedicine. Functional Assessment of Cancer Therapy - General (FACT-G) Assess bladder cancer outcomes. Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) Investigator-developed questionnaires (reviewed and approved by IRB) Assess satisfaction with shared decision making in prostate cancer. Satisfaction Decision Making Questionnaire. Assess value-concordant care in serious illness. Patient Values Assessment Evaluate genetic risk in patients and at-risk relatives. Cancer and Genetic Screening History Quality improvement (examples) Purpose Questionnaire Externally-developed questionnaires Assess satisfaction with surgical care (part of Enhanced Recovery After Surgery program) Modified Consumer Assessment of Healthcare Providers and Systems surgical care survey (S-CAHPS) Locally-developed questionnaires (review and approved by MSK eForms Committee) Assess financial toxicity and connect with Patient Financial Services if they qualify for financial assistance. Financial Burden Questionnaire Gauge patient satisfaction of telehealth visits to continue to improve services at MSK. Telemedicine Visit Satisfaction Survey Assess opioid prescribing practices for post-surgical patients. Opioid Use After Discharge Assess satisfaction with home blood pressure monitoring device. Home Blood Pressure Monitoring Survey Open in a new tab Over the course of the 8-year analysis period, several questionnaires were consistently completed by large numbers of patients. The five most frequently completed questionnaires included: COVID-19 symptom screening, with a total of 346,920 questionnaires completed by 78,964 patients; Pre-CT scan screening, which included questions about dye reactions, relevant medical problems and treatments, amounting to 185,474 questionnaires completed by 68,119 patients; Post-operative symptom monitoring, conducted 10 days after surgery, which garnered 157,044 responses from 21,261 patients; Pre-MRI screening, addressing issues such as past injuries, procedures, and implants, which saw 152,008 completions from 62,521 patients; and the New Visit Nursing Intake questionnaire, with 151,521 questionnaires completed by 148,999 patients. The number of questions per individual instance of a questionnaire varies significantly due to branching logic in the questionnaires. The maximum number of questions for each questionnaire is usually no greater than 30. An exception is the New Visit Intake questionnaire, which could include up to 89 questions, though almost no patient would have to answer that many. To assess the impact of age on questionnaire completion rates, we analyzed completion rates in 2023 across different age groups (in decades). Overall, completion rates remained consistent across most age groups, with a flat trend observed up until the 9th decade of life. However, patients aged 80–89, 90–99, and over 100 years had lower completion rates compared to younger age groups. A graph illustrating these age-based differences in completion rates is shown in Figure 2 . Figure 2. Open in a new tab Completion rates of questionnaires by age group for the year 2023. We compared completion rates, drop-off rates (defined as a questionnaire started but not completed), and time to completion across three categories of questionnaires: research, quality improvement, and standard of care. The data are summarized in Table 4 . Table 4: Completion rates, drop-off rates and time to completion by questionnaire category. Categories Questionnaires sent Completion rate Drop off rate Median time to completion, minutes (IQR) Research 20,796 51% 4% 3.1 (1.5, 7.2) Quality improvement 177,477 47% 3% 1.5 (0.8, 2.5) Standard of care 1,194,202 65% 2% 1.7 (0.7, 5.0) Open in a new tab We observed that the standard of care questionnaires had the highest completion rates (65%) and the lowest drop-off rates (2%). In contrast, research questionnaires had the longest median time to completion (3.1 minutes), and the lowest completion rate (51%). Quality improvement questionnaires showed moderate completion rates (47%) and drop-off rates (3%), with the shortest median time to completion (1.5 minutes). DISCUSSION The main goal of the program was to enable adoption at scale of PROs at our cancer center. The 189 questionnaires we created and 3 million questionnaire responses we collected over 8 years are evidence of success. The program supported innovative clinical programs and numerous research and quality improvement projects. We attribute success to the following factors: Centralized governance to review and approve questionnaires . Questionnaire requests could be initiated by various staff, but a single multidisciplinary governing body assured a consistent look and feel, reuse where relevant and adherence to basic reading levels. Careful attention to patient experience to increase adoption . We included patients in the design of the PRO data entry application and the design of notification methods regarding new questionnaires. The governing body strived to keep questionnaires as short as possible to minimize patient burden. Specialized patient-facing displays of PRO data, for example, post-operative symptom reports showing how the patient compares to the norm also encourages PRO completion by providing value-added content on review. Design of clinical workflows as part of the implementation . For each questionnaire, clinical workflows were mapped to indicate how the PRO results would be used in the clinic. For example, in a pediatric population, a psychosocial assessment tool assesses financial toxicity, care burden and other stressors, with findings assessed by an interdisciplinary group to consider what kinds of social and other services might be available to assist the family in navigating its challenges. As mentioned, plans are created for the management of medium and high severity alerts. Additionally, staff are trained on how to incorporate PRO results into clinical documentation. Giving significant attention to identifying the right patient population to receive a questionnaire, and the right timing for questionnaire delivery . The criteria for sending a questionnaire to a patient must support a variety of scenarios. For example, a Survivorship patient who ordinarily would receive a pre-visit symptom questionnaire might be excluded if he or she is participating in another program that is capturing symptoms data. The presence of a consent may be the trigger for a questionnaire to be sent to a research patient. Procedure codes may serve as inclusion criteria for patients who are participating in post-operative symptom assessment or discharge readiness assessment. During pilots, only patients from certain services or with certain visit types may receive questionnaires. Similarly, the timing of the sending of the questionnaires may be complex. A questionnaire to assess the patient’s understanding of their illness is sent first prior to the 4 th visit and then every 3 months in conjunction with visits. Post-operative symptom assessments are obtained daily for 10 days. A financial burden questionnaire may be obtained at the 2nd treatment visit to MSK and at intervals of 4 months. Inclusion criteria and timing scenarios need to be thoroughly tested. As programs evolve, logic frequently must be modified. Processes are needed to manage the evolution of the logic over time. Several studies have been done that document the benefit of the use of questionnaires for specific use cases. 17 , 18 Because of the diversity of use cases supported by this patient reported outcomes program, an overall statement of benefit would be hard to elucidate, and it could be the focus of future research. We observed higher completion rates, lower dropoff rates and a lower median time to completion for standard of care questionnaires as opposed to the other categories. Several factors may account for this. First, the workflows related to standard of care questionnaires receive a high level of scrutiny. Workflows might be changed to assure reliable, effective, and efficient capture of the patient-reported data. Quality improvement questionnaires may not fit into clinical workflows as well as do standard of care questionnaires, which may affect completion rates. Research questionnaires generally aim to collect a greater amount of data than the other questionnaire types, which leads to a longer median time to completion. Also, the primary approval body for research questionnaires is the IRB and their content is not scrutinized by the governance structure as closely as the other questionnaire types. Many challenges remain to realizing the full potential of PROs. Future work should include understanding how best to display PRO data for patients in ways that can increase engagement and participation in shared decision making, including further exploration of the use of normative data. More understanding is needed for how best to integrate PROs with other patient-generated data, for example, from such monitoring devices as activity trackers, blood pressure cuffs, glucometers, and pulse oximeters. The use of voice technologies may allow broader capture of PROs, for example, from patients who are not comfortable with standard interactive technology or who have limiting symptoms such as tremors. Multi-language support -- though challenging due to issues with translation accuracy and the need to keep dynamic questionnaires up to date – is critical to minimize disparities. The principles we share here are agnostic to any particular technology. For example, our institution is currently transitioning its EHR platform from a best of breed approach to Epic. While the technology and clinician and patient experience for PROs will change, we are moving forward with the overall PRO program and building on the learnings described. Elements such as governance, question reuse, attention to patient experience and workflow integration are being carried forward as we transition to our new EHR vendor. Our study has some limitations including being a single center study. Also, we did not evaluate the impact of the overall program on patient outcomes, although some components have shown an impact on outcome. 17 However, this study was conducted in a busy cancer center which serves a large, diverse population in the NY, NJ and CT tri-state area and may reflect opportunities for other large medical centers. Also, as EHRs advance and patient engagement with technology becomes more widespread, the opportunities described here should be more broadly achievable. CONCLUSION In the 1970s, Warner Slack, a pioneer in health information technology, noted that “patients are the largest and most underutilized resource in healthcare”. 19 A well-constructed PRO program can help to leverage that resource. We demonstrate that large-scale adoption of PROs at a cancer center is feasible. Governance of the questionnaire content and associated knowledge, consideration of patient experience and clinician workflows, and attention to inclusion criteria and timing of questionnaire delivery all require careful thought and planning. Dissemination of these principles and more consideration of how best to incorporate these data into care processes will help to realize the full potential of PROs in healthcare. Supplementary Material PV Data Supplement NIHMS2062770-supplement-PV_Data_Supplement.pdf (334.8KB, pdf) ACKNOWLEDGMENTS The authors acknowledge the significant contributions of the MSK Engage team, current and past, who have improved patient care at MSK by enabling the functionality described in this article. We also acknowledge the MSK Patient and Family Advisory Committee for Quality (PFACQ), who have been active participants in this initiative and have lent invaluable insights over the years about what patients will find most helpful and valuable. We also thank Nirjhar Chakraborty for assistance with formatting and submission. FUNDING This research was funded in part through the NIH/NCI Cancer Center Support Grant P30 CA008748. Footnotes COMPETING INTERESTS None REFERENCES 1. Snyder CF, Aaronson NK. Patient-reported outcomes (PROs) and patient-reported outcome measures (PROMs). Health Serv Insights. 2013;6:61–68. doi: 10.4137/HSI.S11093. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 2. Lavallee DC, Chenok KE, Love RM, Petersen C, Holve E, Segal CD, Franklin PD. Incorporating patient-reported outcomes into health care to engage patients and enhance care. Health Aff (Millwood). 2016;35(4):575–582. doi: 10.1377/hlthaff.2015.1362. [ DOI ] [ PubMed ] [ Google Scholar ] 3. Basch E Patient-reported outcomes—harnessing patients’ voices to improve clinical care. N Engl J Med. 2017;376(2):105–108. doi: 10.1056/NEJMp1611252. [ DOI ] [ PubMed ] [ Google Scholar ] 4. Basch E, Deal AM, Dueck AC, et al. Clinical Utility and User Perceptions of a Digital System for Electronic Patient-Reported Symptom Monitoring During Routine Cancer Care: Findings From the PRO-TECT Trial. JCO Clin Cancer Inform. 2020;(4):947–957. doi: 10.1200/CCI.20.00081. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 5. Sledge GW Jr, Harwin W, Stanton K, et al. Association between remote monitoring and acute care visits in high-risk patients initiating intravenous antineoplastic therapy. JCO Oncol Pract. 2022;18(8). doi: 10.1200/OP.22.00294. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 6. Denis F, Basch E, Septans M, et al. Effect of Electronic Symptom Monitoring on Patient-Reported Outcomes Among Patients With Metastatic Cancer: A Randomized Clinical Trial. JAMA. 2022;328(5):448–457. doi: 10.1001/jama.2022.9265. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 7. Denis F, Lethrosne C, Pourel N, et al. Two-Year Survival Comparing Web-Based Symptom Monitoring vs Routine Surveillance Following Treatment for Lung Cancer. JAMA. 2019;321(3):306–307. doi: 10.1001/jama.2018.18085. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 8. Kotronoulas G, Kearney N, Maguire R, et al. What is the value of the routine use of patient-reported outcome measures toward improvement of patient outcomes, processes of care, and health service outcomes in cancer care? A systematic review of controlled trials. J Clin Oncol. 2014;32(14):1480–1501. doi: 10.1200/JCO.2013.53.5948. [ DOI ] [ PubMed ] [ Google Scholar ] 9. Erickson JM, Blackwood A, Sievers M, et al. Systemwide implementation of patient-reported outcomes in routine clinical care at a children’s hospital. Jt Comm J Qual Patient Saf. 2018;44(6):350–359. doi: 10.1016/j.jcjq.2018.01.002. [ DOI ] [ PubMed ] [ Google Scholar ] 10. Hornik K, Moser P, Klaghofer R, et al. Seamless integration of computer-adaptive patient-reported outcomes into an electronic health record. Appl Clin Inform. 2023;14(4):612–624. doi: 10.1055/a-2235-9557. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 11. HealthMeasures. PROMIS - Patient-Reported Outcomes Measurement Information System. Accessed August 16, 2024. https://www.healthmeasures.net/explore-measurement-systems/promis 12. Haley SM, Coster WJ, Andres PL, et al. Use of computerized adaptive testing to develop more concise patient-reported outcome measures. JBJS Open Access. 2020;5(1). doi: 10.2106/JBJS.OA.19.00052. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 13. Stover AM, Tompkins-Strout B, Hamlin W, et al. Patient-level barriers and facilitators to completion of patient-reported outcomes measures. Qual Life Res. 2022;31(3):749–758. doi: 10.1007/s11136-021-02999-8. [ DOI ] [ PubMed ] [ Google Scholar ] 14. Wu Y, Glaser A, Sharpe L, et al. Patient and healthcare provider views on a patient-reported outcomes portal. J Am Med Inform Assoc. 2019;26(5):416–423. doi: 10.1093/jamia/ocy111. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 15. Basch E, Iasonos A, Barz A, et al. Long-term toxicity monitoring via electronic patient-reported outcomes in patients receiving chemotherapy. J Clin Oncol. 2007;25(34):5374–5380. doi: 10.1200/JCO.2007.11.2243. [ DOI ] [ PubMed ] [ Google Scholar ] 16. Basch E, Jia X, Heller G, et al. Patient-reported outcomes and the evolution of adverse event reporting in oncology. J Clin Oncol. 2009;27(20):3384–3390. doi: 10.1200/JCO.2007.12.4784. [ DOI ] [ PubMed ] [ Google Scholar ] 17. Pusic AL, Temple LK, Carter J, Stabile CM, Assel MJ, Vickers AJ, et al. A Randomized Controlled Trial Evaluating Electronic Outpatient Symptom Monitoring After Ambulatory Cancer Surgery. Ann Surg. 2021;274(3):441–448. doi: 10.1097/SLA.0000000000005005. [ DOI ] [ PubMed ] [ Google Scholar ] 18. Schnipper JL, Gormley EA, Schiff GD, et al. Association between electronic patient symptom reporting with alerts and potentially avoidable urgent care visits after ambulatory cancer surgery. JAMA Surg. 2021;156(8):729–737. doi: 10.1001/jamasurg.2021.1798. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 19. Wu Y, Abernethy AP, Nagelschneider AA, et al. Patients and consumers (and the data they generate): an underutilized resource. J Am Med Inform Assoc. 2021;28(10):2098–2102. doi: 10.1093/jamia/ocab040. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] Associated Data This section collects any data citations, data availability statements, or supplementary materials included in this article. Supplementary Materials PV Data Supplement NIHMS2062770-supplement-PV_Data_Supplement.pdf (334.8KB, pdf) ACTIONS View on publisher site PDF (571.0 KB) Cite Collections Permalink PERMALINK Copy RESOURCES Similar articles Cited by other articles Links to NCBI Databases Cite Copy Download .nbib .nbib Format: AMA APA MLA NLM Add to Collections Create a new collection Add to an existing collection Name your collection * Choose a collection Unable to load your collection due to an error Please try again Add Cancel Follow NCBI NCBI on X (formerly known as Twitter) NCBI on Facebook NCBI on LinkedIn NCBI on GitHub NCBI RSS feed Connect with NLM NLM on X (formerly known as Twitter) NLM on Facebook NLM on YouTube National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894 Web Policies FOIA HHS Vulnerability Disclosure Help Accessibility Careers NLM NIH HHS USA.gov Back to Top