Skip to main content An official website of the United States government Here's how you know Here's how you know Official websites use .gov A .gov website belongs to an official government organization in the United States. Secure .gov websites use HTTPS A lock ( Lock Locked padlock icon ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites. Search Log in Dashboard Publications Account settings Log out Search… Search NCBI Primary site navigation Search Logged in as: Dashboard Publications Account settings Log in Search PMC Full-Text Archive Search in PMC Journal List User Guide PERMALINK Copy As a library, NLM provides access to scientific literature. Inclusion in an NLM database does not imply endorsement of, or agreement with, the contents by NLM or the National Institutes of Health. Learn more: PMC Disclaimer | PMC Copyright Notice J Natl Cancer Inst Monogr . 2024 Aug 7;2024(66):218–223. doi: 10.1093/jncimonographs/lgad029 Search in PMC Search in PubMed View in NLM Catalog Add to search Data quality in a survey of registered medical cannabis users with cancer: nonresponse and measurement error Jeanette Y Ziegenfuss Jeanette Y Ziegenfuss , PhD 1 Center for Evaluation and Survey Research, HealthPartners Institute, Minneapolis, MN, USA Conceptualization, Data curation, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing - original draft, Writing - review & editing Find articles by Jeanette Y Ziegenfuss 1, ✉ , Helen M Parsons Helen M Parsons , PhD 2 University of Minnesota, Minneapolis, MN, USA Conceptualization, Investigation, Methodology, Validation, Writing - review & editing Find articles by Helen M Parsons 2 , Anne H Blaes Anne H Blaes , MD, MS 3 University of Minnesota, Minneapolis, MN, USA Conceptualization, Funding acquisition, Investigation, Methodology, Writing - review & editing Find articles by Anne H Blaes 3 , Bruce Lindgren Bruce Lindgren , MS 4 University of Minnesota, Minneapolis, MN, USA Conceptualization, Formal analysis, Investigation, Methodology, Validation, Visualization, Writing - review & editing Find articles by Bruce Lindgren 4 , Julia Andersen Julia Andersen , MPH 5 Center for Evaluation and Survey Research, HealthPartners Institute, Minneapolis, MN, USA Data curation, Project administration, Resources, Validation, Writing - review & editing Find articles by Julia Andersen 5 , Susan Park Susan Park , PhD 6 Minnesota Department of Health, Saint Paul, MN, USA Data curation, Investigation, Project administration, Resources, Writing - review & editing Find articles by Susan Park 6 , Patricia I Jewett Patricia I Jewett , PhD 7 University of Minnesota, Minneapolis, MN, USA Investigation, Methodology, Writing - review & editing Find articles by Patricia I Jewett 7 , Arjun Gupta Arjun Gupta , MD 8 University of Minnesota, Minneapolis, MN, USA Investigation, Methodology, Writing - review & editing Find articles by Arjun Gupta 8 , Dylan M Zylla Dylan M Zylla , MD, MS 9 HealthPartners Cancer Research Center, Minneapolis, MN, USA Conceptualization, Funding acquisition, Investigation, Methodology, Writing - review & editing Find articles by Dylan M Zylla 9 Author information Article notes Copyright and License information 1 Center for Evaluation and Survey Research, HealthPartners Institute, Minneapolis, MN, USA 2 University of Minnesota, Minneapolis, MN, USA 3 University of Minnesota, Minneapolis, MN, USA 4 University of Minnesota, Minneapolis, MN, USA 5 Center for Evaluation and Survey Research, HealthPartners Institute, Minneapolis, MN, USA 6 Minnesota Department of Health, Saint Paul, MN, USA 7 University of Minnesota, Minneapolis, MN, USA 8 University of Minnesota, Minneapolis, MN, USA 9 HealthPartners Cancer Research Center, Minneapolis, MN, USA ✉ Correspondence to: Jeanette Y. Ziegenfuss, PhD, Center for Evaluation and Survey Research, HealthPartners Institute, 8170 33rd Avenue South 21112R, PO Box 1524, Minneapolis, MN 55440-1524, USA (e-mail: [email protected] ). Roles Jeanette Y Ziegenfuss : PhD , Conceptualization, Data curation, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing - original draft, Writing - review & editing Helen M Parsons : PhD , Conceptualization, Investigation, Methodology, Validation, Writing - review & editing Anne H Blaes : MD, MS , Conceptualization, Funding acquisition, Investigation, Methodology, Writing - review & editing Bruce Lindgren : MS , Conceptualization, Formal analysis, Investigation, Methodology, Validation, Visualization, Writing - review & editing Julia Andersen : MPH , Data curation, Project administration, Resources, Validation, Writing - review & editing Susan Park : PhD , Data curation, Investigation, Project administration, Resources, Writing - review & editing Patricia I Jewett : PhD , Investigation, Methodology, Writing - review & editing Arjun Gupta : MD , Investigation, Methodology, Writing - review & editing Dylan M Zylla : MD, MS , Conceptualization, Funding acquisition, Investigation, Methodology, Writing - review & editing Received 2023 Jun 9; Revised 2023 Aug 11; Accepted 2023 Aug 30; Collection date 2024 Aug. © The Author(s) 2024. Published by Oxford University Press. All rights reserved. For permissions, please email: [email protected] This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model ( https://academic.oup.com/pages/standard-publication-reuse-rights ) PMC Copyright notice PMCID: PMC13070549 PMID: 39108233 Abstract Cannabis use among individuals with cancer is best understood using survey self-report. As cannabis remains federally illegal, surveys could be subject to nonresponse and measurement issues impacting data quality. We surveyed individuals using medical cannabis for a cancer-related condition in the Minnesota Medical Cannabis Program (MCP). Although survey responders are older, there are no differences by race and ethnicity, gender, or receipt of reduced cannabis registry enrollment fee. Responders made a more recent purchase and more recently completed an independent symptom assessment for the registry than nonresponders, suggesting some opportunity for nonresponse error. Among responders, self-report and MCP administrative data with respect to age, race, gender, registry certification, and cannabis purchase history were similar. Responders were less likely to report receipt of Medicaid than would be expected based on registry low-income enrollment eligibility. Although attention should be paid to potential for nonresponse error, surveys are a reliable tool to ascertain cannabis behavior patterns in this population. Clinicians, researchers, and policy makers, among others, often rely on self-reported data from surveys to measure beliefs and behaviors that are unavailable in administrative claims or electronic medical records or are infeasible to observe in other settings. Because medical cannabis is not routinely covered by payers and use is not routinely captured in a standardized manner in oncology clinical encounters, patients’ self-reported patterns of use represent a critical source of information. Based on studies such as those included in this monograph and beyond, we know 1) nearly 25% of patients undergoing cancer care report cannabis use, 2) most patients use cannabis to alleviate symptoms from the cancer or its treatment, and 3) nearly 2 out of 3 patients desire more information and education about use of cannabis ( 1 , 2 ). As is the case with any data, when using self-report, it is important to consider the quality of the data. Evaluation of survey data can be best done through thinking about who is providing the data and their willingness and ability to respond overall and to any specific question. There is ample evidence to support the reliability of self-report of other health behaviors across a myriad of settings such as general health, physical activity, and nutrition. It is generally understood that measurement error emerges when questions are poorly designed or ambiguous and/or if the survey responder is not able to or chooses not to disclose a factual or true underlying value ( 3 ). Thus, self-report is generally thought to be less reliable when survey topics are potentially sensitive, as may be the case with cannabis, because of its historical status as an illicit substance. In fact, there is evidence from others that a history of drug use is associated with a lower probability of response ( 4 , 5 ). In the case of cannabis, its sensitivity may vary by population. Even while cannabis is increasingly decriminalized and legal recreationally and/or medically in many states, Black persons are 3.6 times more likely to be arrested for marijuana possession than their White counterparts despite approximately equal use ( 6 , 7 ). When a topic is perceived as sensitive, individuals may choose not to respond to the survey at all or to report something other than a true or complete underlying value to preserve what the responder perceives will be viewed as a more favorable standing. When respondents skew or falsify self-report to maintain favorability, it is called social desirability bias ( 8 ). Although paper surveys reduce this likelihood because of the absence of an interviewer, it is not completely eliminated ( 3 ). Even though high response rates do not preclude nonresponse error, and there is evidence that nonresponse is not correlated with survey error ( 9 , 10 ), it is important to ascertain the potential impact of nonresponse error by comparing responders with nonresponders on domains that are thought to be correlated with the self-reported outcomes of interest collected in each survey ( 11 ). Leverage-salience theory posits that the willingness to complete a survey is correlated with one’s interest (positive or negative) in a topic and/or perceived personal relevance of the same ( 12 ). In the case of medical cannabis for use in the treatment and management of cancer symptoms, the topic may be polarizing. Thus, it may have higher salience among some groups than others, underscoring the need to understand potential nonresponse bias as a source of error. In 2015, Minnesota residents with certain diagnoses became eligible to purchase cannabis to treat their condition when certified by a registered clinician. Individuals with cancer and a diagnosis of cancer-related cachexia, nausea, or pain are eligible. A survey of medical cannabis use among a subset of these individuals known to have purchased cannabis through the Minnesota Medical Cannabis Program (MCP) provides a unique opportunity to quantify the quality of self-reported data in a population for which the topic should have high salience. Specifically, the following questions are addressed: To what extent is there the potential for nonresponse bias? Do survey responders differ from nonresponders with respect to demographic characteristics and interaction with the MCP, inclusive of how long they have been certified, who certified them, whether they completed a self-assessment collected by the registry, their symptoms reported, and their purchase history? Is there evidence of measurement error? To what extent is self-report in a survey of cannabis use correlated with administrative records available in the MCP? Methods Individuals aged 18 years or older certified for a cancer-related indication within the MCP and who had made a purchase within the 3 months prior to the study were identified and included in the sample population. In December 2021, nearly 1400 individuals with cancer were registered with the MCP. Of these, 797 (57%) had filled a prescription in the 3 months prior and were thus eligible. Postal mail was used for all survey contact because it is the most secure form of self-administered survey administration and the full coverage of postal mail addresses in the sample frame. All individuals were mailed a survey packet from the Minnesota Department of Public Health. Included in the survey packet was an invitation printed on Minnesota Department of Public Health letterhead signed by the study principal investigator who is also an oncologist, the survey, and a postage-paid self-addressed envelope in which to return the survey. The cover letter included the elements of informed consent, and the study was waivered from documentation of consent. There was no incentive, and there was only 1 survey mailing. Mailings that were not delivered because of bad addresses were removed from the study denominator for response rate calculations. Existing administrative data from the MCP were linked to the survey responses using a unique identifier printed on each survey. Administrative data available for responders and nonresponders included demographic characteristics (age, gender, race, and low income as represented as eligible for reduced registry enrollment fee), MCP registration information (months since certification and type of provider that certified the individual), and purchasing data (months since first purchase, months since last purchase, and total number of purchases). Individuals were eligible for a reduced registry enrollment fee if they were considered low income as indicated by having one of the following forms of medical assistance: Supplemental Security Income, Social Security Disability including those transitioned to retirement benefits, medical assistance, MinnesotaCare, Indian Health Service (IHS), railroad disability, US Department of Veterans Affairs (VA) dependency and indemnity compensation, or Veteran’s disability benefits. Also available from the MCP for the entire sample were patient or proxy report of symptoms through the registry’s online patient self-evaluation for which completion is required before each medical cannabis purchase. Measures included in analysis were months between last patient self-evaluation completion and average date of survey completion, to eliminate confounding with response latency, date, total number of patient self-evaluations completed, and actual symptom rating (first ratings and most recent ratings for those with more than 1 assessment are assessed). The one-time, cross-sectional survey was designed using Teleform Elite scannable software. The survey was 11 pages; had 41 questions, some with multiple subquestions; and was designed to take approximately 20 minutes to complete. Questions asked about current and past use of cannabis; frequency, duration, and mode of use; therapeutic reasons for use; perception of the benefits or risks and/or harms; discussion of use with clinical providers; and estimates of financial costs. The survey contained items from existing surveys available in the public domain with known psychometric properties where available, including the National Cancer Institute’s Core Measures Questionnaire and Health Information National Trend Survey, as well as the Centers for Disease Control and Prevention’s Behavioral Risk Factor Surveillance System. Additional questions were developed by the study team (including oncologists, survey methodologists, health services researchers, and public health professionals) using known best practices for survey item development ( 13 ). The survey was reviewed for face validity by independent experts and patient advocates and modified iteratively. To ascertain potential nonresponse error, that is, the extent that responders were similar or different than nonresponders, each group was characterized with respect to the variables available from MCP administrative data. Responders were compared with nonresponders using χ 2 or Fisher exact test for categorical variables and the 2-sample t test or Wilcoxon rank sum test for continuous or ordinal variables. Where similar or identical data were available for responders from the survey and administrative data, potential measurement error was quantified comparing the 2 sources. Means are reported with standard deviations and medians with range, and Kappa or Spearman correlation coefficients are reported. A Bonferroni adjustment is made to account for multiple comparisons, reducing the alpha considered statistically significant to 0.002 in the nonresponse analysis and 0.008 in the measurement error analysis. All analyses was conducted using SAS 9.4 software (SAS Institute Inc, Cary, NC, USA). This study was reviewed and approved by the University of Minnesota institutional review board and the Minnesota Department of Health Office of Medical Cannabis. The Strengthening of the Reporting of Observational Studies in Epidemiology guidelines were used to aid in the organization and comprehensiveness of this manuscript ( 14 ). Results Of 797 invited individuals, 220 returned their completed surveys resulting in a 28% response rate ( 15 ). Five additional surveys were returned but not eligible because they were either returned blank or the survey responder self-reported not eligible in the first question asking if the individual had ever had cancer. The most common primary cancer sites included breast (24%), lung (14%), and colorectal (10%). Most (46%) had stage IV cancer (results not shown). Comparing responders with nonresponders ( Table 1 ), responders were older (average age 61.1 years compared with 57.3 years; P < .001). There were no statistically significant differences between responders and nonresponders with respect to race and ethnicity (94.6% non-Hispanic White compared with 89.3% BIPOC; P = .03), gender (51.8% female compared with 49.1% male; P = .61) or receipt of reduced cannabis registry enrollment fee (34.1% compared with 39.5% not receiving reduced registry enrollment; P = .17), although there was a slight difference in the conceptual definition between the 2 sources with the survey reporting receipt of Medicaid or other state program compared with another type of insurance coverage or being uninsured. Of note, race and ethnicity could not be reported with more granularity because of small sample sizes. Months since certification was not statistically different between responders and nonresponders ( P = .12) reflecting similar duration of time in the program. There were no differences in certifying provider between responders and nonresponders ( P = .04) ( Table 1 ). Table 1. Characteristics of the study population from MCP administrative data by survey response status to assess impact of nonresponse error Demographic variables from Minnesota Medical Cannabis Program registry Nonresponders Responders P a No. (%) No. (%) (n = 580) (n = 220) Age, mean (SD), y 57.3 (14.1) 61.1 (12.1) <.001 Age groups, y <.001 18-29 23 (4.0) 5 (2.3) 30-39 56 (9.7) 8 (3.6) 40-49 79 (13.6) 27 (12.3) 50-59 156 (26.9) 45 (20.5) 60-69 160 (27.6) 87 (39.6) 70-79 84 (14.5) 42 (19.1) 80 and older 22 (3.8) 6 (2.7) Gender .61 Female 285 (49.1) 114 (51.8) Male 289 (49.8) 105 (47.7) Refused 6 (1.0) 1 (0.5) Non-Hispanic White 518 (89.3) 208 (94.6) .03 Low-income eligible for reduced registry enrollment fee 229 (39.5) 75 (34.1) .17 Registry process information Certifying health-care practitioner .04 Physician 417 (71.9) 145 (65.9) Physician assistant 46 (7.9) 13 (5.9) Advanced practice registered nurse 117 (20.2) 62 (28.2) Months from certification to survey administration, February 2022 .12 2.5-6.0 151 (26.0) 58 (26.4) 6.0-12.0 154 (26.6) 39 (17.7) 12.0-24.0 108 (18.6) 45 (20.5) 24.0-48.0 124 (21.4) 57 (25.9) 48.0-78.0 43 (7.4) 21 (9.6) Months from most recent self-assessment to average survey completion date, March 2022 <.001 0-2.0 177 (30.5) 86 (39.1) 2.0-4.0 163 (28.1) 78 (35.5) 4.0-6.0 191 (32.9) 43 (19.6) 6.0-8.0 49 (8.5) 13 (5.9) Total number of assessments 15.5 (25.3) 15.1 (19.7) .04 Median (min/max) 6.0 (1/233) 8.0 (1/177) Patient self-assessment data Symptom ratings, mean (SD), first measurement, for where total assessment ≥ 1 Anxiety 6.0 (3.0) 5.7 (3.1) .13 Lack of appetite 5.3 (3.4) 4.9 (3.3) .08 Depression 4.9 (3.1) 4.5 (3.1) .06 Disturbed sleep 6.7 (2.9) 6.9 (2.8) .38 Fatigue 7.1 (2.2) 6.9 (2.3) .31 Nausea 4.4 (3.4) 4.0 (3.4) .14 Pain 6.6 (2.7) 6.6 (2.8) .97 Vomiting 2.2 (3.2) 1.6 (2.8) .01 Symptom ratings, mean (SD), most recent measurement, where total assessments > 1 n = 508 n = 202 Anxiety 4.7 (3.2) 4.3 (3.1) .15 Lack of appetite 3.9 (3.3) 3.4 (3.1) .04 Depression 3.7 (3.2) 3.2 (2.9) .04 Disturbed sleep 5.1 (3.2) 4.7 (3.1) .07 Fatigue 6.0 (2.7) 5.6 (2.7) .13 Nausea 3.3 (3.3) 2.9 (3.2) .15 Pain 5.9 (2.7) 5.5 (2.9) .04 Vomiting 1.8 (3.0) 1.3 (2.5) .04 Symptom ratings, mean (SD), change from first to last measurement for those with total number of assessments ≥ 2 n = 508 n = 202 Anxiety 1.4 (3.2) 1.4 (2.9) .99 Lack of appetite 1.3 (3.6) 1.4 (3.8) .76 Depression 1.2 (3.1) 1.3 (3.1) .73 Disturbed sleep 1.6 (3.5) 2.3 (3.1) .01 Fatigue 1.0 (2.7) 1.3 (3.0) .23 Nausea 1.0 (3.5) 1.0 (3.6) .98 Pain 0.7 (2.7) 1.3 (2.7) .01 Vomiting 0.3 (3.5) 0.2 (2.6) .42 Medical cannabis purchasing data, median (range) Months from first purchase to average survey completion date, March 2022 11.1 (3.3-71.9) 14.0 (3.3-78.7) .03 Months from last purchase to average survey completion date, March 2022 2.8 (0.7-6.3) 2.0 (0.8-6.2) .001 Total number of purchase dates 12 (1-254) 16 (1-206) .045 Open in a new tab a To account for multiple comparisons using a Bonferroni adjustment, P < .002 is considered statistically significant. Statistically significant P -values are bolded. Statistical methods: Responders and nonresponders were compared using the χ 2 or Fisher exact test for categorical variables and the 2-sample t test or Wilcoxon rank sum test for continuous or ordinal variables. MCP = Minnesota Medical Cannabis Program. Responders were more likely to have a recent completion of an online MCP registry symptom assessment via the patient self-evaluation in the 2 months before the cannabis survey was mailed than were nonresponders ( P < .001), however, there was no difference in number of completed patient self-evaluations over the course of time on the registry. There were no statistically significant differences with respect to symptom assessment in the patient self-evaluation considering first report, most recent report, or change over time ( Table 1 ). Responders were more likely to have a more recent purchase than nonresponders (median 2.0 months since first purchase compared with median of 2.8 months; P = .001). There were no differences with respect to duration of purchasing history (median 14.0 months since first purchase compared with median of 11.1 months; P = .027) or total instances of making a purchase (16 for responders compared with 12; P = .045) ( Table 1 ). Among responders, self-report of demographic data to the survey was overall consistent with administrative data available from MCP, which included similar measures of age, gender, and race (correlation = 0.97, kappa = 0.98, and kappa = 0.55, respectively; all at P < .001). Responders were less likely to self-report Medicaid or other state programs on the survey compared with MCP registry data showing qualifying for reduced enrollment fee (8.0% compared with 34.1%; kappa = 0.27; P < .001). By definition, everyone in the sample, and thus all responders, were certified and had purchased medical cannabis within the prior 3 months according to registry information. Nearly everyone reported certification and purchase within the prior three months in the survey (99.5% reporting each) ( Table 2 ). Table 2. Difference between survey self-reported and Minnesota Medical Cannabis Program (MCP) administrative data among survey responders to assess measurement error Demographic MCP administrative data Survey self-report Kappa or Spearman No. (%) No. (%) (n = 220) (n = 220) P a Age, mean (SD), y 61.1 (12.1) 60.4 (12.4) Corr = 0.97 < .001 Age categories, y 18-29 5 (2.3) 6 (2.8) 30-39 8 (3.6) 8 (3.7) 40-49 27 (12.3) 26 (12.1) 50-59 45 (20.5) 44 (20.5) 60-69 87 (39.6) 85 (39.5) 70-79 42 (19.1) 40 (18.6) 80 and older 6 (2.7) 6 (2.8) Missing 5 Gender Female 114 (51.8) 113 (52.3) k = 0.98 b < .001 Male 105 (47.7) 103 (47.7) No answer, missing 1 (0.5) 4 Non-Hispanic White Yes 208 (94.6) 211 (97.2) k = 0.55 < .001 Not Applicable, other 12 (5.5) 6 (2.8) Missing 3 Cannabis registry data Medical assistance Yes 75 (34.1) 14 (8.0) k = 0.27 b < .001 No, other coverage 145 (65.9) 161 (92.0) Missing 45 Percent certified 100% 99.5% Medical cannabis purchasing data Any purchase? 100% 99.5% Open in a new tab a To account for multiple comparisons using a Bonferroni adjustment, P < .008 is considered statistically significant. Corr = Spearman correlation coefficient. b Includes those who answered both questions. Discussion Our study suggests nonresponse bias is a greater potential threat to medical cannabis research among populations with cancer and experience with cannabis than is measurement error. In this assessment of data quality in a survey of individuals with cancer who had made a recent cannabis purchase through the MCP, we cannot rule out nonresponse error because of differences between responders and nonresponders theoretically correlated with outcomes of interest, albeit relatively small differences with much equivalence observed between responders and nonresponders. Specifically, we found that responders were more likely to be older than nonresponders, but they were similar with regard to race and ethnicity, gender, and our low-income status measure. Responders and nonresponders had similar lengths of time since certification and similar number of purchases; however, responders had a more recent purchase and a more recent completed patient self-evaluation, 2 items that are functionally linked. Symptom reporting did not differ between responders and nonresponders, somewhat inconsistent with data suggesting that feeling better may result in a higher likelihood of survey response in other contexts ( 16 , 17 ). Across settings, survey responders are generally older and of higher socioeconomic status than nonresponders, likely because of smaller opportunity costs for time ( 18 , 19 ). It is of interest that we saw an impact for age but not for other indicators of socioeconomic status, which could be because of the relationship of the population with the topic. However, we cannot conclude if our observed pattern with respect to age was because of the topic and particulars of survey administration or this general trend. In light of leverage salience theory that posits those with more interest in a topic will be more likely to respond to a survey about that topic ( 12 ), it is not surprising that those with more recent cannabis purchases within the MCP would be more likely to participate in the survey. Although the magnitude of the differences was not large in absolute terms, it should be considered when reviewing data about self-reported cannabis use. Individuals responding to surveys may have more recent cannabis-related experience than that which exists in an overall population, even among known cannabis users. Moreover, given observed differential outcomes in cannabis possession between White and Black populations, where we may have expected response differences ( 6 , 7 ), small sample sizes may have limited our ability to make inferences about Black survey recipients or other race or ethnic populations, which is a limitation of this study. Among responders, there was little indication of measurement error for available data. Consistent reports of gender and race and ethnicity across administrative and survey data confirm what has been observed in other contexts ( 20 ). The only observed discrepancy was the underreporting of medical assistance compared with administrative data indicating receipt of reduced cannabis enrollment fee, albeit largely based on receipt of medical assistance. As medical assistance is highly correlated with income and may thus have stigma associated with it, this could be expected. There is a large body of literature on underreporting of medical assistance that is consistent with this finding ( 21 , 22 ). However, administrative data are not necessarily the gold standard, and it is possible that medical assistance is overly documented in the context of MCP because of timing, that individuals may have lost eligibility since last MCP certification, or other factors unobservable in our data. That individuals accurately reported both certification in MCP and history of purchase is promising for measurement of similar constructs in general population surveys. The study of cannabis use patterns relies heavily on survey. That responders may be older and tend to have more recent history of use is an important lens through which to view survey data. Moreover, these differences underscore the essential practice of conducting nonresponse bias whenever survey data are disseminated ( 11 ). Survey postprocessing, such as weighting, can mitigate impact of nonresponse bias where differences are observed. Although nonmedical use of cannabis was illegal in Minnesota at the time of this survey, responders were willing to report MCP certification status and purchase history, suggesting there was little social desirability bias. This is an important finding for other contexts with the same or less restrictive legal environments. A strength of this study was its rich data available for responders and nonresponders. However, this study was conducted in the context of 1 state at one point in time with individuals who had known experience with cannabis for 1 specific diagnosis. Moreover, the survey was only implemented in English and through only 1 mode, which may have negatively affected response rates. Although these implementation decisions are justified given the available frame data; the more secure nature of mail compared with email, important for a potentially sensitive subject; and the desire to avoid an interviewer-administered survey, results should be considered in that context. Moreover, because of the quickly changing legal landscape for cannabis, it will be important to revisit these findings in other contexts. Our response rate was relatively low and our request for completion singular, which may influence our nonresponse findings. The continuum of resistance theory posits that later responders are more similar to nonresponders than are responders ( 23-25 ), suggesting that more aggressive contact protocols may have brought in responders more similar to nonresponders in this study. In other survey studies that use incentives, mixed modes, or other strategies resulting in higher response rates may not result in as many differences between responders and nonresponders ( 26 , 27 ) Nonetheless, the use of surveys to understand cannabis use is not only necessary but also appropriate given the findings herein. Overall, this study highlights that, although attention should be paid to potential for nonresponse error, surveys are a reliable tool to ascertain cannabis behavior patterns in this population. Contributor Information Jeanette Y Ziegenfuss, Center for Evaluation and Survey Research, HealthPartners Institute, Minneapolis, MN, USA. Helen M Parsons, University of Minnesota, Minneapolis, MN, USA. Anne H Blaes, University of Minnesota, Minneapolis, MN, USA. Bruce Lindgren, University of Minnesota, Minneapolis, MN, USA. Julia Andersen, Center for Evaluation and Survey Research, HealthPartners Institute, Minneapolis, MN, USA. Susan Park, Minnesota Department of Health, Saint Paul, MN, USA. Patricia I Jewett, University of Minnesota, Minneapolis, MN, USA. Arjun Gupta, University of Minnesota, Minneapolis, MN, USA. Dylan M Zylla, HealthPartners Cancer Research Center, Minneapolis, MN, USA. Data availability Individual de-identified survey data is kept at the University of Minnesota and will be shared. For access, contact the corresponding author Jeanette Ziegenfuss ([email protected]). Minnesota Medical Cannabis Registry data reported in this manuscript were provided by the Minnesota Department of Health and will be shared on request to the corresponding author Jeanette Ziegenfuss ([email protected]) with permission of the Minnesota Department of Health. Contact for the Minnesota Department of Health: Susan Park ([email protected]). Author contributions Jeanette Y Ziegenfuss, PhD (Conceptualization; Data curation; Investigation; Methodology; Project administration; Resources; Software; Supervision; Validation; Visualization; Writing—original draft; Writing—review & editing), Helen M Parsons, PhD, MPH (Conceptualization; Investigation; Methodology; Validation; Writing—review & editing), Anne H Blaes, MD, MS (Conceptualization; Funding acquisition; Investigation; Methodology; Writing—review & editing), Bruce Lindgren, MS (Conceptualization; Formal analysis; Investigation; Methodology; Validation; Visualization; Writing—review & editing), Julia Andersen, MPH (Data curation; Project administration; Resources; Validation; Writing—review & editing), Susan Park, PhD (Data curation; Investigation; Project administration; Resources; Writing—review & editing), Patricia I Jewett, PhD (Investigation; Methodology; Writing—review & editing), Arjun Gupta, MD (Investigation; Methodology; Writing—review & editing), and Dylan Zylla, MD, MS (Conceptualization; Funding acquisition; Investigation; Methodology; Writing—review & editing). Funding This study was supported by the National Cancer Institute P30 Supplement—Grant #3P30CA077598-22S2. Monograph sponsorship This article appears as part of the monograph “Cannabis Use Among Cancer Survivors,” sponsored by the National Cancer Institute. Conflicts of interest The authors have no conflicts of interest to disclose. No competing financial interests exist. References 1. Hawley P, Gobbo M, Afghari N. The impact of legalization of access to recreational cannabis on Canadian medical users with cancer. BMC Health Serv Res. 2020;20(1):977. doi: 10.1186/s12913-020-05756-8. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 2. Pergam SA, Woodfield MC, Lee CM, et al. Cannabis use among patients at a comprehensive cancer center in a state with legalized medicinal and recreational use. Cancer. 2017;123(22):4488-4497. doi: 10.1002/cncr.30879. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 3. Tourangeau R, Rips LJ, Rasinski K. The Psychology of Survey Response. Cambridge, United Kingdom: Cambridge University Press; 2000. [ Google Scholar ] 4. Johnson TP. Sources of error in substance use prevalence surveys. Int Sch Res Notices. 2014;2014:923290. doi: 10.1155/2014/923290. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 5. McCabe SE, West BT. Selective nonresponse bias in population-based survey estimates of drug use behaviors in the United States. Soc Psychiatry Psychiatr Epidemiol. 2016;51(1):141-153. doi: 10.1007/s00127-015-1122-2. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 6. Edwards E, Greytak E, Madubuonwu B, et al. American Civil Liberties Union (ACLU): Research Report: A Tale of Two Countries: Racially Targeted Arrests in the Era of Marijuana Reform . 2020. https://www.aclu.org/report/tale-two-countries-racially-targeted-arrests-era-marijuana-reform . 7. Sheehan BE, Grucza RA, Plunk AD. Association of racial disparity of cannabis possession arrests among adults and youths with statewide cannabis decriminalization and legalization. JAMA Health Forum. 2021;2(10):e213435. doi: 10.1001/jamahealthforum.2021.3435. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 8. Paulhus DL. Two-component models of socially desirable responding. J Pers Soc Psychol. 1984;46(3):598-609. [ Google Scholar ] 9. Davern M. Nonresponse rates are a problematic indicator of nonresponse bias in survey research. Health Serv Res. 2013;48(3):905-912. doi: 10.1111/1475-6773.12070. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 10. Groves RM. Nonresponse rates and nonresponse bias in household surveys. Public Opin Q. 2006;70(5):646-675. doi: 10.1093/poq/nfl033. [ DOI ] [ Google Scholar ] 11. Johnson TP, Wislar JS. Response rates and nonresponse errors in surveys. JAMA. 2012;307(17):1805-1806. doi: 10.1001/jama.2012.3532. [ DOI ] [ PubMed ] [ Google Scholar ] 12. Groves RM, Singer E, Corning A. Leverage-saliency theory of survey participation: Description and an illustration. Public Opin Q. 2000;64(3):299-308. doi: 10.1086/317990. [ DOI ] [ PubMed ] [ Google Scholar ] 13. Fowler FJ Jr. Improving Survey Questions: Design and Evaluation. Vol. 38. Thousand Oaks, CA: SAGE Publishing; 1995. [ Google Scholar ] 14. Cuschieri S. The STROBE guidelines. Saudi J Anaesth. 2019;13(suppl 1):S31-S34. doi: 10.4103/sja.SJA_543_18. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 15. The American Association for Public Opinion Research (AAPOR). Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys Revised 2023. https://aapor.org/standards-and-ethics/standard-definitions/ . 16. Keyes KM, Rutherford C, Popham F, Martins SS, Gray L. How healthy are survey respondents compared with the general population?: using survey-linked death records to compare mortality outcomes. Epidemiology. 2018;29(2):299-307. doi: 10.1097/EDE.0000000000000775. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 17. Lallukka T, Pietilainen O, Jappinen S, Laaksonen M, Lahti J, Rahkonen O. Factors associated with health survey response among young employees: a register-based study using online, mailed and telephone interview data collection methods. BMC Public Health. 2020;20(1):184. doi: 10.1186/s12889-020-8241-8. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 18. Groves RM, Cialdini RB, Couper MP. Understanding the decision to participate in a survey. Public Opin Q. 1992;56(4):475-495. [ Google Scholar ] 19. Groves RM, Couper MP. Nonresponse in Household Interview Surveys. Ann Arbor, MI: John Wiley & Sons, Inc.; 1998. [ Google Scholar ] 20. Ziegenfuss JY, Easterday CA, Dinh JM, JaKa MM, Kottke TE, Canterbury M. Impact of demographic survey questions on response rate and measurement: a randomized experiment. Surv Pract 2021;14(1):1. doi: 10.29115/SP-2021-0010. [ DOI ] [ Google Scholar ] 21. Call KT, Davern ME, Klerman JA, Lynch V. Comparing errors in Medicaid reporting across surveys: evidence to date. Health Serv Res. 2013;48(2 Pt 1):652-664. doi: 10.1111/j.1475-6773.2012.01446.x. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 22. Pascale J, Fertig AR, Call KT. Assessing the accuracy of survey reports of health insurance coverage using enrollment data. Health Serv Res. 2019;54(5):1099-1109. doi: 10.1111/1475-6773.13191. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 23. Clarsen B, Skogen JC, Nilsen TS, Aaro LE. Revisiting the continuum of resistance model in the digital age: a comparison of early and delayed respondents to the Norwegian counties public health survey. BMC Public Health. 2021;21(1):730. doi: 10.1186/s12889-021-10764-2. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 24. Davern M, McAlpine D, Beebe TJ, Ziegenfuss J, Rockwood T, Call KT. Are lower response rates hazardous to your health survey? An analysis of three state telephone health surveys. Health Serv Res. 2010;45(5 Pt 1):1324-1344. doi: 10.1111/j.1475-6773.2010.01128.x. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 25. Lin IF, Schaeffer NC. Using survey participants to estimate the impact of nonparticipation. Public Opin Q. 1995;59(2):236-258. [ Google Scholar ] 26. Olson K, Smyth JD, Wood HM. Does giving people their preferred survey mode actually increase survey participation rates? An experimental examination. Public Opin Q. 2012;76(4):611-635. doi: 10.1093/poq/nfs024. [ DOI ] [ Google Scholar ] 27. Sakshaug JW, Cernat A, Raghunathan TE. Do sequential mixed-mode surveys decrease nonresponse bias, measurement error bias, and total bias? An experimental study. J Surv Stat Methodol. 2019;7(4):545-571. doi: 10.1093/jssam/smy024. [ DOI ] [ Google Scholar ] Associated Data This section collects any data citations, data availability statements, or supplementary materials included in this article. Data Availability Statement Individual de-identified survey data is kept at the University of Minnesota and will be shared. For access, contact the corresponding author Jeanette Ziegenfuss ([email protected]). Minnesota Medical Cannabis Registry data reported in this manuscript were provided by the Minnesota Department of Health and will be shared on request to the corresponding author Jeanette Ziegenfuss ([email protected]) with permission of the Minnesota Department of Health. Contact for the Minnesota Department of Health: Susan Park ([email protected]). Articles from Journal of the National Cancer Institute. Monographs are provided here courtesy of Oxford University Press ACTIONS View on publisher site PDF (601.5 KB) Cite Collections Permalink PERMALINK Copy RESOURCES Similar articles Cited by other articles Links to NCBI Databases Cite Copy Download .nbib .nbib Format: AMA APA MLA NLM Add to Collections Create a new collection Add to an existing collection Name your collection * Choose a collection Unable to load your collection due to an error Please try again Add Cancel Follow NCBI NCBI on X (formerly known as Twitter) NCBI on Facebook NCBI on LinkedIn NCBI on GitHub NCBI RSS feed Connect with NLM NLM on X (formerly known as Twitter) NLM on Facebook NLM on YouTube National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894 Web Policies FOIA HHS Vulnerability Disclosure Help Accessibility Careers NLM NIH HHS USA.gov Back to Top