Independence and interdependence of non-medical practitioners in the emergency care skill-mix (SKILLMix-ED): a multi-method study to develop a measurement tool

Independence and interdependence of non-medical practitioners in the emergency care skill-mix (SKILLMix-ED): a multi-method study to develop a measurement tool - NCBI Bookshelf An official website of the United States government Here's how you know The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site. The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely. Log in Show account info Close Account Logged in as: username Dashboard Publications Account settings Log out Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation Bookshelf Search database Books All Databases Assembly Biocollections BioProject BioSample Books ClinVar Conserved Domains dbVar Gene Genome GEO DataSets GEO Profiles GTR Identical Protein Groups MedGen MeSH NLM Catalog Nucleotide OMIM PMC Protein Protein Clusters Protein Family Models PubChem BioAssay PubChem Compound PubChem Substance PubMed SNP SRA Structure Taxonomy ToolKit ToolKitAll ToolKitBookgh Search term Search Browse Titles Advanced Help Disclaimer NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health. Independence and interdependence of non-medical practitioners in the emergency care skill-mix (SKILLMix-ED): a multi-method study to develop a measurement tool Health and Social Care Delivery Research Francesca Taylor , Mary Halter , Vari M Drennan , Jonathan Gabe , Heather Gage , Heather Jarman , Celayne Heaton-Shrestha , and Catriona Brice . Author Information and Affiliations Authors Francesca Taylor , 1 ,* Mary Halter , 1 Vari M Drennan , 1 Jonathan Gabe , 2 Heather Gage , 3 Heather Jarman , 4 Celayne Heaton-Shrestha , 1 and Catriona Brice 1 . Affiliations 1 Faculty of Health, Science, Social Care and Education, Kingston University, Kingston Upon Thames, UK 2 School of Law and Social Sciences, Royal Holloway, University of London, Egham, UK 3 School of Biosciences, Faculty of Health and Medical Sciences, University of Surrey, Guildford, UK 4 St George’s University Hospitals NHS Foundation Trust, London, UK * Corresponding author ; Email: [email protected] Southampton (UK): National Institute for Health and Care Research ; 2026 Apr 1 . Copyright and Permissions Copyright © 2026 Taylor et al. This work was produced by Taylor et al . under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This is an Open Access publication distributed under the terms of the Creative Commons Attribution CC BY 4.0 licence, which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. See: https://creativecommons.org/licenses/by/4.0/ . For attribution the title, original author(s), the publication source – NIHR Journals Library, and the DOI of the publication must be cited. Abstract Background: In addressing the many challenges that face them, some emergency departments are changing their staffing to include new roles, such as nurse practitioners and physician associates. Known collectively as non-medical practitioners, they work with varying levels of independence and supervision, which, evidence suggests, may influence outcomes. As part of a larger study investigating the impact of these staff skill-mix changes, we sought to measure quantitatively how such staff work independently or with supervision. Objectives: To develop a prototype-structured observational tool for quantitative measurement of levels of independence and supervision of non-medical practitioners in emergency departments and urgent treatment centres in England. Design: A multimethod study using an incremental and iterative process underpinned by theories suggesting autonomy can be measured by practice independence behaviour. Method: We undertook three interconnected research activities in March 2021–April 2023: (1) literature review to clarify concepts, and identify classifications and tools associated with independence and supervision; (2) 12 ethnographic observations of non-medical practitioners and resident doctors to describe the enactment of independence and supervision, with data coded and thematically analysed; (3) collaborative prototype tool development incorporating three collaborative sessions with 28 clinicians, non-medical practitioners and patients, and three further stages of stakeholder feedback. Results: Twenty-six articles were included in the literature review. Nine articles included concepts associated with independence and supervision, the main concepts being clinical decision-making, competence, responsibility and autonomy. These multifaceted concepts were found to be intertwined in complex ways with those of collaboration, teamwork and interdependence. Seventeen articles included classifications or tools to measure levels of independence or supervision. Shared decision-making was the most frequently measured concept. There were no tools or classifications identified for measuring levels of independence or supervision within a skill-mix team. Our clinical observations found that participant clinicians practised with varying levels of independence within an interdependent team. The extent of guidance, education and direction received in discussion with the clinician-in-charge varied widely. In addition to the clinician-in-charge structure, clinical discussion and advice seeking were often with colleagues known and trusted to have specific knowledge. Spatial, temporal and resource constraints also encouraged collaborative working. Patients played an important team role. During collaborative development of the prototype tool, key issues raised by stakeholders and addressed in the prototype included: concerns that an inappropriate binary might be perceived between independence as ‘good’ and supervision as ‘bad’; clinicians practise interdependently with most episodes of care involving some interaction, consultation or advice seeking; and the importance of collecting contextual data. Limitations: The literature review was limited by excluding non-English-language papers. Our clinician observations included two sites with self-selecting participants which may have introduced biases. Objectivity and validity of the tool were not tested. Conclusions: Our data led us to develop a prototype observational tool reflecting non-medical practitioners working within an interdependent skill-mix team. The tool discerns tasks and actions undertaken with varying levels of independence or supervision. Further testing of the tool is required. Future research: The prototype tool was designed for use in later stages of the larger study. Funding: This article presents independent research funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme as award number NIHR131356. Plain language summary Emergency departments face many pressures, which has led some to employ non-medical practitioners. These are qualified health staff doing some of the work done by doctors as agreed at senior level. This study aimed to find a way to measure how independently or with how much supervision non-medical practitioners work in emergency departments. It is part of a larger study looking at different balances of non-medical practitioners, doctors and nurses, and which work best. The study had three phases: (1) looking at what is known about the independence and supervision of non-medical practitioners, and what ways there are to measure this; (2) 12 observations in two emergency departments in England looking at what non-medical practitioners and doctors do and how independently they work; (3) designing a measure of independence and supervision with our patient and public involvement, non-medical practitioner and clinician stakeholders. We identified 26 articles on how independence and supervision were described or measured. Descriptions were varied and complex but connected closely with team working and relying on each other. We found no measures of levels of independence and supervision within a health team. Our observations showed that non-medical practitioners and doctors worked with others as a team rather than independently. They received varied amounts of guidance and direction from senior doctors. Patients were important team members. Our stakeholders raised several key points included in the designed measure: independence should not be seen as ‘good’ and supervision ‘bad’; clinicians generally consult or ask advice of others; information should be collected on the emergency department environment. Our findings led us to develop a way of measuring how non-medical practitioners work with others in an emergency department team, doing different tasks and actions with different levels of independence and supervision. It will be used in later stages of the larger study. Background Emergency departments (EDs) and urgent treatment centres (UTCs) in England, as in other countries, have experienced significant pressures over recent years as a consequence of increasing patient numbers and acuity; higher rates of specialty consultations; overcrowding and staffing shortages, particularly of doctors. 1 – 4 Some reshaping of the workforce has resulted, including employment within the ED/ UTC skill-mix of advanced clinical practitioners (ACPs) from professional backgrounds, such as nursing and physiotherapy, and physician associates/assistants (PAs), collectively termed in this paper non-medical practitioners (NMPs). 5 However, there is currently no evidence to guide decisions on the appropriate skill-mix, including NMPs in EDs/UTCs and the level of supervision or independence from senior medical staff, nor evidence of the influence of different skill-mix on care outcomes. 6 The study reported here on the development of a quantitative measure of independence and interdependence formed part of a larger mixed-methods ‘SKILLMix-ED’ study designed to address these evidence gaps about the implementation of NMPs in the ED/ UTC skill-mix. 6 The NMPs in England and in many countries globally work with varying levels of independence, supervision and oversight reflective of the employing organisations’ policies and individual NMP ’s levels of training and experience. 7 – 11 Evidence indicates that these variations in practice may influence costs, processes and outcomes. For example, a Canadian study modelling ED resource requirements and costs suggested these should allow for variation in doctors’ supervision responsibilities given differing independence levels among PAs and nurse practitioners (NPs). 12 An evaluation of an Australian ED workforce found that while doctors perceived NPs’ ability to treat independently less-complex patient conditions as contributing positively to ED efficiency, they were reluctant to supervise NPs in treating more complex patients due to concerns about negative efficiency impacts. 9 It has also been suggested that better understanding of supervisory and collegial support and authority in decision-making within ED skill-mix teams may offer insights into how different roles and teams interact with the potential to improve patient outcomes. 13 In this context, an analytical tool to measure quantitatively levels of independence and supervision of NMPs was selected as a method to help deliver the overall study aim to explore the impact of different skill-mix, including NMPs in EDs and UTCs in acute NHS trusts in England. The tools in use for emergency doctors in training, such as the direct observation of procedural skills, the extended supervised learning event, case-based discussions and mini clinical evaluation exercises, are designed to measure specific clinical skills and/or clinical decision-making, not the complexities of different levels of independent and supervised working within a skill-mixed team. Additionally, a preliminary literature search identified no quantitative tool available to measure and characterise the levels of independence and supervision of NMPs working in the skill-mix in EDs/UTCs, and capture the inter-relationship between the two dimensions. 6 Consequently, it was necessary to develop a tool to measure these factors objectively and consistently. The study was underpinned by theories that suggest the degree of autonomy can be measured as a function of practice independence behaviour, 14 , 15 focusing on: concepts of readiness (taking responsibility and being accountable for actions); empowerment (providing quality services through one’s actions); actualisation (having a sense of professionalism) and valuation (accepting the consequences of choices made). 15 We were also cognisant of research papers that addressed measurement of these concepts in other healthcare specialties. 16 – 18 Aim and objectives The study aim was to develop a prototype-structured observational tool for quantitative measurement of levels of independence and supervision of NMPs working in EDs/UTCs. The objectives were to: Identify and clarify concepts associated with the independence and supervision of NMPs in EDs/UTCs. Identify classifications and tools from any medical specialty used to measure clinician levels of independence and supervision. Describe how independently or with what level of supervision clinicians work in EDs/UTCs. Work collaboratively with stakeholders to design the prototype-structured observational tool. Methods Patient and public involvement Patient and public involvement (PPI) in healthcare improvement studies has been shown to inform and enhance service-related tool development, 19 and positively influence quality outcome data. 20 , 21 However, this requires that the involvement and engagement be ‘active’, with collaborative decision-making in partnership with other stakeholders throughout the process. 22 An established university patient and public research expert group, supported by the study PPI lead, advised on the development of the study prior to funding. This group raised a number of issues the study should focus on pertinent to development of the tool, subsequently given consideration in the study design: interstaff teamwork, patient communication, consideration of the case-mix of patients seen by clinicians, responsibility for patient assessment within the team, and strengthening how issues of diversity and inclusion would be addressed given variation in patient experience of ED/ UTC care. A study-specific PPI panel ( n = 10) was formed with member recruitment through PPI leads in two NHS trust study sites. PPI representatives in the two trusts ( n = 28) were sent an e-mail invitation to participate throughout the study by the study researchers, with information on the study aims, skills and experience required, and what participation would involve. It was made clear in this material that the aim was for diversity in demographic characteristics of the PPI panel. Equality, diversity and inclusion The PPI panel was initially oversubscribed; therefore, selection was made by the study PPI lead as pre-alerted to applicants, on the basis of creating as diverse a membership as possible. Study stakeholders involved in the collaborative prototype tool development lived and worked in locations that differed widely in terms of their socioeconomic characteristics and health outcomes. Collaborative sessions were offered online or face-to-face to maximise involvement and participation. Design We undertook a multimethod study, 23 using an incremental and iterative process. 24 The strength of this process was that it allowed development of the tool to evolve through loops of learning with constant revisions and testing until a satisfactory prototype tool was in place. 24 The study involved three interconnected strands of work: (1) clarifying concepts and identifying classifications and tools associated with independence and supervision, (2) describing the enactment of independence and supervision in EDs/UTCs and (3) collaborative prototype tool development ( Figure 1 ). FIGURE 1 Interconnected strands of the prototype tool development. We used guidance from the reporting of PPI in research (Guidance for Reporting Involvement of Patients and the Public 2 short form) 25 and the standards for reporting qualitative research. 26 The study was undertaken between March 2021 and April 2023. Strand 1: clarifying concepts and identifying classifications and tools associated with independence and supervision A systematised literature review was undertaken in order to address the two research questions: What is the available evidence on concepts associated with the independence and supervision of NMPs in EDs/UTCs? What classifications and tools from any medical specialty are used to measure clinician levels of independence and supervision? To answer these questions, evidence was assessed from three data sources. A sub-review was undertaken within a larger scoping and systematic review to understand the empirical evidence on skill-mix involving NMPs in the ED/ UTC ; the full methodology is outlined in the protocol published and registered on International Prospective Register of Systematic Reviews. 27 Papers identified as belonging to an independence and supervision category were not included in the main review but were double-blind read against specific eligibility criteria for this study (see Appendix 1 ). Full-text papers were retrieved, where relevant, and read double-blind. Additionally, all other papers in the main skill-mix review were scrutinised during data extraction for any implicit reference to the concepts of independence and supervision. Citation pearl searching 28 of the three papers referring to the concept of measuring autonomy, independence or supervision 16 – 18 identified in a literature search undertaken for the ‘SKILLMix-ED’ study bid submission 6 (see Appendix 2 for search terms). The searching involved three activities: use of the Scopus and Google Scholar citation functions; selection of the top five related articles using the PubMed function or Google Scholar where the included paper was not indexed on PubMed; and screening of the reference lists. Searches were performed in January 2022. Papers citing the three ‘pearls’ were double screened, and those identified as belonging to either the concept or classification/tool category were double-blind read against the specific eligibility criteria for the independence and supervision review category (see Appendix 1 ). Full-text papers were retrieved where appropriate and read double-blind. Searches of Google Scholar and PubMed of the first 120 unique search results (from the first 12 search engine results pages), using additional search terms (see Appendix 3 ) identified by stakeholders attending the collaborative sessions (strand 2). Searches were undertaken in June 2022. The same screening process was undertaken as detailed above (2), but for the classification/tool category alone, and from any healthcare specialties. Data were extracted and tabulated for concepts associated with independence or supervision, or for classifications and tools for independence or supervision. Data analysis by the study researchers identified potential conceptual ingredients, and content from the classifications and tools for building the prototype-structured observational tool. Stakeholders involved in strand 3 (collaborative prototype tool development) considered whether and how these relevant constructs might be made tangible in the prototype tool. Strand 2: describing the enactment of independence and supervision in emergency departments/urgent treatment centres A small-scale qualitative descriptive study was undertaken using non-participant observation in an ethnographic approach, 29 to address the research question: How is the independence and supervision of NMPs and resident doctors in EDs/UTCs demonstrated in practice? Data collection The observations were conducted in the EDs/UTCs of two NHS acute trusts in England. In one trust, the NMPs were long-established with a trust strategy for this workforce’s development. In the other trust, the NMPs were a newer addition to the ED/ UTC workforce. In both sites, a clinician-in-charge was always present and available for supervision. The observations were intended to be undertaken in October 2021–January 2022, prior to the collaborative sessions of strand 3 (collaborative prototype tool development); however the COVID-19 pandemic delayed fieldwork until May–October 2022. The initial invitation for volunteer participants was via posters placed in the ED/ UTC staff bases, and an e-mail from the clinical lead to all eligible staff with the study outline and researcher contact details. Study researchers sent any staff who expressed interest a participant information sheet outlining the study purpose and what participation would involve, and a consent form, separate to contact with the clinical leads. Sampling was purposive to include up to six NMPs in various roles, and up to six resident doctors (qualified doctors in clinical training), included because they were on the same part of the staffing roster as NMPs, not for comparative purposes. One observation was undertaken of the supervisory activities of a clinician-in-charge, throughout the consultation a consultant doctor, to gain further insight into the supervision and independence interface. Informed consent was given by clinician participants in advance of the planned 2-hour observation, and assent was sought in advance of the clinicians-in-charge (medical and nursing) rostered for that time period. Patient assent (or that of an accompanying relative/carer) was gained by the clinician requesting permission of the patient for the researcher’s presence. Patients who lacked capacity to assent to observation, and were without an accompanying relative/carer, were excluded. If the participant clinician or their patient considered procedures to be intrusive or issues to be sensitive, they could ask the researcher to withdraw from observing at that point, without impact on the remaining observation period. At the end of the observed participant interaction with each assenting patient, the shadowing researcher provided the patient with a leaflet giving additional study information and contact details. Information notices about the study and researcher presence, with the observation time periods, were offered for display in the ED/ UTC for patients and staff. Each participating NMP and resident doctor was shadow observed 30 in their interactions with patients and colleagues, by one study researcher over a 2-hour time period. It was anticipated this would encompass any of the steps of a patient’s ED journey: assessment, diagnosis, treatment interventions and discharge. The observations were thought likely to include the treatment of at least two patients in the ‘majors’ or several in the ‘minors’ stream of the ED in that time period. Each observation was undertaken by one of two study researchers Mary Halter (MH) and Francesca Taylor (FT) with prior experience of observational ED research. The researchers independently kept hand-written field notes 31 of what they saw and heard in relation to the participant’s tasks, activities and interactions at the time of observation. Contextual elements in the ED were also noted, for example, where patients were attended, proximity of clinician-in-charge and where equipment was sourced. No patient identifiable data were recorded. To prevent any intrusion or interruption of the functioning of the ED , the researchers did not seek clarification from participants of what was observed. Notes were unstructured but guided by Spradley’s 32 dimensions of social situations – space, object, act, activity, event, time, actor, goal, feeling – as applied to independence and supervision. To ensure consistency in the scope and granularity of their note-taking, after initial observations the researchers compared and discussed the level of detail recorded. Each researcher recorded reflective analytical notes post-observation. 33 Data analysis All field notes were transcribed following data collection. Transcripts from three of the observations were scrutinised and discussed by the two researchers involved in the observations (MH, FT) and two other researchers [Jonathan Gabe and Catriona Brice (CB)] to identify preliminary codes and categories. The researchers who had undertaken the observations and another researcher with no ED research experience, Celayne Heaton-Shrestha, then read and discussed all the transcripts prior to one researcher developing a coding index, subsequently refined through discussion. All data were then coded and thematically analysed by the three researchers using deductive coding framed by ‘theoretical knowingness’ arising from the study literature review, and inductive (data-driven) coding approaches. 34 The process was collaborative and reflexive rather than seeking consensus on meaning, the aim being to develop themes that were ‘analytic outputs’ interpretive of the data. 35 Researchers’ visual memories from the observations, 36 stimulated by the analytic process, provided additional insight to data interpretation. As analysis progressed, significant data were compressed to adhere around key analytic themes. Where data did not fit into existing themes, new ones were developed or existing ones modified until all data were coded by theme. Dedoose v9.05 software, [Sociocultural Research Consultants, Limited Liability Company (Manhattan Beach, California, USA)] was used for data coding and analysis. Strand 3: collaborative prototype tool development Members of the overall study Steering Committee, PPI and NMP panels, and Management Group – encompassing patients, carers, clinicians, ED/ UTC healthcare leaders, national healthcare policy leaders and researchers – were invited to one of three collaborative sessions held in February and March 2022. A modified Nominal Group Technique 37 was used to ensure all stakeholder participants’ views of tool content options were heard and considered. Two study researchers (MH, FT) facilitated each 3-hour session using a structured agenda. After the identified literature on independence and supervision concepts, classifications and tools had been summarised, an outline tool mock-up was shown, followed by feedback and open discussion anchored by three specific questions: What would need to be observed to capture adequately independent clinical activity by a NMP or resident doctor? What does ‘independence’ within a skill-mixed team look like for observation purposes? What does ‘being supervised’ within a skill-mixed team look like for observation purposes? Key findings from the discussions were recorded by the two study researchers and later analysed deductively. Iterative development of the prototype tool Development of an initial prototype tool was based on relevant constructs related to independence and supervision derived from the literature review undertaken in strand 1, stakeholder feedback data from the collaborative sessions, and preliminary learning from the strand 2 clinician observations. Potential items for measuring these constructs were taken from classifications and tools identified in the literature review and adapted to the ED/ UTC context. Where no items to measure relevant constructs could be found in existing classifications and tools, novel items were designed. The initial prototype tool was e-mailed to all members of the study advisory groups in August 2022 for comment. This process resulted in additional feedback that was incorporated in the next iteration of the tool sent to study advisory group members, together with initial findings from the strand 2 clinician observations, prior to discussion at separate meetings of the overall study Steering Committee, PPI and NMP panels, and Management Group (September–October 2022). Responses from these meetings were used to make additional revisions to the tool. A first ‘working prototype’ was then tested by three study researchers (FT, MH, CB) using publicly available documentary television recordings of staff and patients in NHS EDs. Based on feedback and discussion among the researchers, the tool was modified further (December 2022–January 2023). The proposed prototype tool was then circulated (March 2023) to all study advisory group members for face validation and final feedback. It was then piloted (June–July 2023), during clinician observations of four NMPs in the two trusts that had participated in strand 2, using the same recruitment methodology. Results We report by the different strands of research activity. Strand 1: clarifying concepts and identifying classifications and tools associated with independence and supervision A total of 177 potential articles were identified from across the 3 data sources of the literature review and screened by title and abstract. Sixty-six full-text articles were assessed for eligibility, and 26 articles were included in the review. Figure 2 displays a flow chart of article selection. FIGURE 2 Flow chart of article selection. Key concepts of independence and supervision of non-medical practitioners Nine articles were identified as including references to concepts associated with the independence or supervision of NMPs in EDs/UTCs. Countries of study included the USA ( n = 3), 38 – 40 Australia ( n = 2), 9 , 41 Ireland ( n = 2) 42 , 43 and Canada ( n = 1), 44 with one study undertaken in both Finland and Norway. 45 All studies included NMP populations; six studies focused on NMP populations alone, 38 – 40 , 42 – 44 and three studies included additional ED staff populations. 9 , 41 , 45 NPs were the most frequently included NMP population ( n = 5), 9 , 38 – 41 followed by advanced nurse practitioners (ANPs) ( n = 2), 42 , 43 PAs ( n = 2) 39 , 44 and nurse specialists ( n = 1). 45 While all nine studies were directly related to ED settings, one study investigated three additional healthcare settings. 44 The studies were all published between 2014 and 2020. Key concept descriptions and definitions were extracted from each of the included articles (see Appendix 4 , Table 1 ). Three studies included descriptions of clinical decision-making in relation to independence and supervision. 39 , 42 , 43 One study used case studies to describe how the clinical decision-making process was impacted by the number and type of a patient’s problems, while the complexity of patient data obtained and reviewed by the NP/ PA and consultation with other team members contributed to the complexity level of the decision-making. 39 Another study considered clinical decision-making in relation to knee fractures, emphasising the importance of anatomy and physiology knowledge in order to manage patients and determine the requirement for immediate orthopaedic consultation, and recommended ‘a high index of suspicion to hasten diagnosis and reduce the risk of complications’. 42 A third study used a case example to contrast an ANP ’s intuitive clinical decision-making based on experience, and more deliberative and analytical decision-making, and advised the sharing of research, knowledge and clinical experience within an ED multidisciplinary team to ensure the decision-making was safe and effective. 43 Two studies described competence. 38 , 45 Boman et al . 45 associated competence with a range of personal and interpersonal skills and attributes, while Hoyt and Proehl 38 linked attaining a sufficient level of competence with being entrusted to undertake unsupervised activities. The concept of collaboration was considered in two studies. 40 , 44 One study referred to NMP ‘collaborative practice and authority to treat patients with a doctor’ in the context of practice independence, 40 while another study discussed how NMPs’ capacity and preparedness to undertake roles within existing frameworks helped to establish collaborative care models. 44 Supervision was described in two studies. 9 , 44 Wise 9 referred to NMPs receiving and providing supervision, defined as instruction by a colleague in ‘the appropriate course of action for a patient’s care and/or educating them in aspects of a condition, procedure or treatments’. 9 Burrows et al . 44 reported that supervision needed both supervisor trust in the NMP to seek advice, and NMP confidence in the supervisor to be available when required. We identified two studies that referred to teamwork, both with Wise as first author. 9 , 41 One study defined the concept as ‘the skills, behaviours and attitudes that allow team members to interact dynamically, interdependently, and adaptively towards a common goal’. 9 The other study described these characteristics as necessary because of the urgency and unpredictability of ED work, and being facilitated by permanent staff knowing their team members’ roles. 41 Both these studies associated closely the concepts of teamwork and interdependence; team members described as understanding their roles as interdependent through needing to support each other to achieve quality patient care while maintaining patient flow. 9 , 41 One study referred to NMP independence, defining practice independence as working collaboratively and having authority to treat patients together with a doctor using ‘advanced practice skills and knowledge, and self-directed medical judgements’. 40 The concept of autonomy was defined in one study as being ‘empowered to initiate tasks, exercise discretion and participate in decisions regarding (their) work’, while enhanced autonomy ‘aims to improve skill utilisation and encourage employees to display adaptive and proactive behaviour’. 9 The same study described the concept of responsibility in relation to two types: ‘core duties’, the responsibilities of a particular role, and ‘shared tasks’ undertaken by a particular role or delegated. 9 The identified concepts were often described as inter-related and unified. For example, one study described NMPs as having the competence to work more autonomously and take on a supervisory role in their specific area of expertise, with a particular skill set suitable for specific ED scenarios. 45 Another study reported that in settings with multiple supervising physicians there could be varying levels of autonomy that necessitated NMPs adjusting their approach to satisfy different supervisors. 44 Additionally, Wise 9 described task responsibility in the ED as being an aspect of teamwork; rather than team members taking an individual or role-based view of task responsibility, responsibility for each role completing its task was perceived as shared within the team. We found no shared conceptual definitions of the independence and supervision of NMPs. The concepts of clinical decision-making, competence, responsibility and autonomy associated with independence and supervision not only incorporated a variety of multifaceted components but also were intertwined in complex ways with the concepts of collaboration, teamwork and interdependence. Therefore, to measure quantitatively levels of independence and supervision of NMPs, we needed to take account of observable behaviours related to these interconnected conceptual ingredients. Classifications and tools to measure clinician levels of independence and supervision Seventeen articles were identified as including classifications or tools to measure the levels of independence or supervision of clinicians from any medical specialty (see Appendix 5 , Table 2 ). Of these, eight articles 18 , 46 – 52 were identified from the sub-review within the larger scoping and systematic review, 14 and the citation paper searching. The remaining nine articles 53 – 61 were identified from additional searches instigated following feedback from the strand 3 collaborative sessions. Measurement tools were described in 10 of the articles, 46 , 47 , 53 , 55 – 61 mapping and delineation tools ( n = 4), 18 , 48 – 50 and classifications ( n = 3). 51 , 52 , 54 Four studies were based in Australia, 48 , 49 , 54 , 61 USA ( n = 4), 46 , 47 , 50 , 55 Sweden ( n = 2), 56 , 57 with one each in Belgium, 59 Canada, 60 Denmark, 53 England, 52 Norway, 58 Switzerland 18 and Uganda. 51 All the studies were undertaken between 1993 and 2022. The target population of the classifications and tools included NMPs ( n = 4), 48 – 50 , 52 NMPs working in EDs ( n = 1), 52 doctors working in EDs ( n = 3), 18 , 51 , 52 supervisors of NMPs ( n = 1), 51 hospital patients ( n = 4), 53 , 56 , 57 , 60 primary care patients ( n = 3), 56 , 57 , 60 hospital doctors ( n = 5), 18 , 51 , 46 , 52 , 58 hospital nurses ( n = 3), 46 , 47 , 58 hospital healthcare workers in general ( n = 1) 59 and medical trainees and students ( n = 3). 54 , 55 , 61 Shared decision-making was the most frequently measured concept ( n = 9 studies), 47 , 53 , 55 – 61 while clinical decision-making was a measured concept in four studies, 18 , 46 , 47 , 52 including within a skill-mix team ( n = 2). 46 , 47 However, apart from two studies, 47 , 61 patient-shared decision-making was not considered within clinician decision-making classifications and tools, but separately. Further concepts measured were consultation ( n = 5), 47 – 50 , 52 collaboration ( n = 5), 46 , 48 – 50 , 61 responsibility ( n = 3), 47 , 54 , 55 complete care/see a patient completely ( n = 3), 48 – 50 autonomy ( n = 2), 47 , 55 supervision ( n = 2), 51 , 54 accountability ( n = 1), 55 co-ordination ( n = 1), 46 competence ( n = 1), 54 entrustment ( n = 1), 54 independence ( n = 1), 47 interdependence ( n = 1) 54 and leadership ( n = 1). 61 There were no tools or classifications identified for measuring levels of independence or supervision within a skill-mix healthcare team. Strand 2: describing the enactment of independence and supervision in emergency departments/urgent treatment centres A total of 12 clinician observations were conducted. Six observations were undertaken with NMPs [one each of ACP , trainee ACP , trainee emergency practitioner (EP) and PA , and two EPs]. Five observations were undertaken with resident doctors (two foundation year 2, two core trainee 3, one specialist trainee 6), and one with a consultant-in-charge. When observed, the clinicians were working in the following sections: majors ( n = 5), minors ( n = 3), resuscitation ( n = 1), paediatrics ( n = 1), UTC ( n = 1), and one clinician worked in both majors and resuscitation. The data analysis process resulted in identification of four inter-related themes around the independence and supervision of NMPs and resident doctors: (1) Interdependent clinical discussion and advice seeking; (2) Spatial, temporal and resource constraints encouraged collaborative working; (3) Collaborative team working practices; and (4) Patients’ involvement and role in teamwork. The themes are discussed below and see Appendix 6 , Table 3 for observation excerpts related to the themes. Themes Interdependent clinical discussion and advice seeking Most observed episodes of patient care by all types of clinicians were undertaken interdependently: the clinician at some point had a clinical discussion or sought advice with a team colleague, clinician-in-charge or specialist. While some tasks or activities with patients were completed by participants without a clinical discussion, this was always observed to be within interdependent teamwork practice. For example, sharing with a colleague the checking of a patient’s vital signs. Participants were observed following patterns of consulting a clinician-in-charge. In situations of choice rather than requirement whether to discuss a task or action with the clinician-in-charge, the participant’s experiential knowledge, confidence, and attitudes to manging risk appeared to be influential in their decision-making. For example, a NMP with several years’ experience of working in the minors section was observed undertaking a treatment procedure without prior discussion with the clinician-in-charge, although a team colleague in the same role had expressed caution about themselves undertaking the procedure. In addition to the clinician-in-charge structure, clinical discussion and advice seeking by participants seemed often to be with those colleagues known and trusted to have specific knowledge and experience, and was not always hierarchical. For example, one NMP participant was observed using their experiential knowledge to provide supported decision-making for a clinician-in-charge, providing advice as to whether or not a hand injury required plastic surgery. It was observed that when consulted by participants, the clinician-in-charge did not always make decisions independently. There were several examples recorded in the field notes where before making a decision, a clinician-in-charge directed or guided participants to discuss the issue with a specialist, or simply discussed the issues with the participant and reached what appeared to be a mutual decision. Other clinical discussion was observed at patient handovers from or to a participant, intershift or at shift changes, where specifics such as test results, diagnoses, treatment procedures and disposition plans were shared. Additionally, incidental clinical discussions were often observed between participants and team colleagues who found themselves sitting side by side inputting patient details into electronic health records. Advice seeking was sometimes observed via online information sources for a clinical guideline, protocol or pathway, to find out what to do next. In a few situations, this approach appeared to be taken as an alternative strategy to move the patient forward when the clinician-in-charge was not immediately available to the observed participant. Wide variation was observed in the level of guidance, education, and direction received by participants during their discussions with a clinician-in-charge. These observed decision-making interfaces were generally brief and to the point, using a shared economical and unembellished lexicon. Observed participants might discuss a task or action for just one of their patients or for several patients sequentially. During some discussions of a task or action the observed participant outlined their action plan of care; in others they appeared to be more reliant on the clinician-in-charge to lay this out. The content and outcome of these observed discussions followed several different orientations. In some cases, after discussing a task or action the observed participant was not asked to make any change to their plan of care for the patient or the patient’s trajectory. For example: Participant to clinician-in-charge: ‘I think he might have COVID. Nothing else.’ Clinician-in-charge: ‘Chest X-ray?’ Participant: ‘Plan is I order X-ray. If fine, he can go home after that’. Clinician-in-charge: ‘OK’. NMP08 During other discussions, the observed participants were persuaded or steered by the advice of the clinician-in-charge towards an action plan of care, or to change the action plan they had mentioned. For example: Participant to clinician-in-charge. ‘ 61-year gentleman. Here end May, jaundice, pale stools, obstructive pancreatic mass. Awaiting … Came in two weeks ago, RUQ pain, radiating, no vomiting. He’s alright but white count’s … (both look at laboratory results). Not any more jaundiced. Just wondering if it’s an obstruction type picture’ . Clinician-in-charge: ‘ Has he had surgery?’ Participant: ‘ No, a stent’ . Clinician-in-charge: ‘ So what shall we do?’ Participant: ‘ I was thinking refer to medics or surgeons’ . Clinician-in-charge: ‘ Who is he under?’ Participant: ‘ Surgeons’ . Clinician-in-charge: ‘We should probably re CT and refer to medics’. Participant: ‘So medics for gastro, analgesia, should I?’ Clinician-in-charge: ‘ I suppose because before 5pm you could ring gastro, see if they will see, but need the CT anyway’ . D09 Discussions were also observed during which a participant was told or taught by the clinician-in-charge the reasons for and how to change their plan of care, or the patient’s trajectory. For example: Participant to clinician-in-charge: ‘Can I ask a question? Woman with abdominal pain, severe, bowels not opened three days. She had a FIT test under the two-week rule, then missed procedures as couldn’t put cannula in – sounds a bit odd but that’s what happened’. Clinician-in-charge: ‘How old?’ Participant: ‘59. Not hard; tender. CRP raised and trending upwards’. Clinician-in-charge: ‘ So have a CT scan. Has she had painkillers? Observations fine?’ Participant: ‘ Tachy’ . Clinician-in-charge: ‘ Tachy? Pain diffuse? All over? Query mass as well’. Participant: ‘ I was thinking that’ . Clinician-in-charge: ‘ So that’s what you say to radiologist’ . NMP03 During another discussion, an observed participant was taught by the clinician-in-charge both the reasons why and how they needed to change their action plan for the patient: Participant to clinician-in-charge: ‘ I’ve seen another patient’. Describes symptoms, assessment and results. ‘ Hope can then go home’. Clinician-in-charge: ‘ PUQE?’ Gives some education about this to NMP . ‘ Check PUQE score. More than 30 equals admission. Depends on hydration status … For admission, check hydration, how long felt’. Clinician-in-charge and participant check admission criteria linked to PUQE score. Clinician-in-charge: ‘ Any itching?’ Participant: ‘I’ve not asked that question’. Consultant explains and shows participant details on screen. Participant: ‘ Why have I never heard of that before ?’ Takes a mobile phone picture of the screen. ‘Do gynae ask this?’ Clinician-in-charge: ‘Trust me’. Participant: ‘Always learning’. Clinician-in-charge: ‘You can always learn something new’. NMP08 After some observed discussions, the clinician-in-charge decided to review the participant’s patient themselves. In a few situations, subsequent to reviewing the patient the clinician-in-charge assumed management of the patient. Spatial, temporal and resource constraints encouraged collaborative working Participants were observed negotiating a multitude of spatial, temporal, and resource constraints. Shortage of rooms and beds to undertake patient assessments and treatment were common obstacles observed, with collaborative working with team colleagues seemingly required to steward patient flow by juggling room and bed use. Some participants were observed struggling to find a functioning computer or diagnostic test equipment, requiring them to negotiate usage with team colleagues. Accessing the clinician-in-charge to discuss a patient’s diagnosis or treatment often appeared to involve queuing. Many participants were observed making a judgement call on whether or not to join a queue dependent on its length. Collaborative team working practices Several collaborative shared team working practices were observed. Patient flow was a constant issue for discussion in the context in which the observations took place. Participants repeatedly gave attention to computer and ED/ UTC screens that continuously showed patient flow spreadsheets, while clinicians and nurses-in-charge frequently focused discussion on bed availability and requested staff information on patient discharge progress. Observed participants, particularly NMPs with ED situational knowledge, demonstrated awareness and understanding of the skills and experience of other permanent team members. Similarly, some observed participants were recognised by clinicians-in-charge to have the experiential knowledge and capacity to fill gaps in rosters. For example, a NMP participant was entrusted by the clinician-in-charge to step out of their normal role working in majors to assist in resuscitation. Team working was also fostered by the professional regulations of role practice. In particular, PAs were observed discussing patients with resident doctors as part of a request to prescribe medications or order X-rays or computed tomography scans. Professional hierarchy did not seem to be a pronounced feature of team working. Some observed participants self-completed tasks or activities they wanted to delegate when the staff member concerned seemed too busy. Professional boundaries were more apparent in relation to specialists where clinical discussions with observed participants occurred predominantly by phone and were relatively formal. The discussion outcomes were mainly directed by the specialist. Patients’ involvement and role in teamwork Most observed participants were perceived to make efforts to quickly establish a relationship with their patients (or their relatives/carers) through communicating empathy and care. In this context, patients were observed to share information on their symptoms, experience and understanding of their health issues, personal and work context, medications and allergies. Participants also tended to communicate and explain test results, diagnoses and treatment plans to their patients. A few participants were observed talking with patients about what a diagnosis might or might not be as a way of helping them determine a diagnosis. Several participants also made concerted efforts to keep patients informed and updated on the progress of their trajectory through the ED/ UTC , for example, on expected length of time until a diagnostic procedure could be performed, and delays in getting test results. The need to discuss their patients’ case with a more senior clinician was frequently communicated to patients by participants using phrases such as, ‘ I’ll have a chat with the boss ’ and ‘ I’ll have to talk to my senior ’. The results of these discussions were mostly reported back to their patients by participants. Nevertheless, observed participants seemed rarely to involve their patients in the decision-making process, although a few ‘expert’ patients interceded in decision-making around their diagnosis and treatment. For example, raising questions based on their experience of a long-term condition. There were several instances, however, where patients were empowered to take responsibility for their health care following discharge from the ED/ UTC . Some patients were encouraged to see their general practitioner (GP) to request further diagnostic tests and procedures for conditions not deemed an emergency, or to make an appointment with a hospital specialist. Others were provided with education about their condition or information about their health behaviours, which, it was conveyed, would help keep them out of the ED in future. ‘Red flag’ symptoms were also highlighted to patients, with an explanation of how they should respond. Strand 3: collaborative prototype tool development Collaborative sessions A total of 28 stakeholders participated across three collaborative sessions ( PPI representatives, n = 8; clinicians, n = 11; ED/ UTC healthcare or national policy leads, n = 5; researchers, n = 4). Six key findings were extracted from analysis of data from the discussions arising from presentation of the literature on independence and supervision concepts and tools, and an outline tool mock-up: An inappropriate and restrictive binary could be perceived between independence as ‘good’ and supervision as ‘bad’. Clinicians were assumed to practise with diverse levels of independence and supervision within an interdependent skill-mixed team, with most episodes of care thought to involve some interaction, consultation or advice seeking from team colleagues or the clinician-in-charge. A spectrum of levels of independence alongside dissimilar categories of supervision. Independence was understood to range from a clinician being able to undertake a few tasks but not the complete patient journey independently, to having the experience, authority and accountability to make all clinical decisions and complete a patient’s care without senior supervisory input. Both ‘hard’ and ‘soft’ aspects to independence . Independence was thought to have a ‘hard’ side, for example, being able to order diagnostic tests, make decisions on likely prognosis, and prescribe, as well as a ‘soft’ side involving, for example, tacit skills, relationships, confidence and attitudes to risk. Importance of collecting contextual data . The ED/ UTC context was perceived as a complex social system with many different variables potentially impacting levels of independence and supervision: types of patient presentation, patient acuity, ED/ UTC area where patient seen, staff workload, social relationships, organisational culture and protocols, rules on ‘check-in’ with clinicians-in-charge, clinician attitudes to risk, clinician lived experience, and trust between clinician and supervisor. Patients as well as clinicians should be involved in decision-making . Our patient representatives provided valuable insight and feedback based on their own experience regarding participation in decision-making around diagnosis and treatment in EDs/UTCs or hospital care in general. They stressed that patients should be an integral part of the skill-mix team together with ED/ UTC staff. Need for further literature review searches for tools measuring additional concepts : authority/authorised; accountable/accountability; complete care/see a patient completely; patient participation in clinical decision-making. Iterative development of the prototype tool The initial prototype tool was based on relevant constructs related to independence and supervision derived from the literature review undertaken in strand 1, preliminary learning from the strand 2 clinician observations, and stakeholder feedback data from the collaborative sessions. It was in three sections designed to be completed by researchers during a 2-hour clinician observation. The first section contained a vertical scale that distinguished between 36 observable tasks and actions for patient assessment and investigation, online information gathering, diagnosis and treatment, information sharing and communication with patient, patient participation in decision-making, disposition of the patient, and patient handover. The horizontal scale included four independent and interdependent criteria: no discussion, discussion with team colleague, discussion with consultant-in-charge, and discussion with specialist, alongside five discussion outcomes. Pre-coded context data for collection during the observation made up the second section with items on location, time, proximity of clinician-in-charge and resource availability, together with context data for collection from the trust on patient wait times and staff numbers. The closing section included the perceptions of the clinician observed on time pressures, workplace experience and knowledge of team members’ skills and experience. The a priori assumption being that diverse levels of independence and interdependence would be observed in different clinicians inter-related with contextual issues. Two further stages of stakeholder discussion and feedback on the circulated initial prototype tool showed the overall design and constructs incorporated to be acceptable and in concordance with stakeholder proposals from the collaborative sessions. Nevertheless, stakeholders considered it important to make several changes to improve the clarity and coherence of the tool in relation to: NMP involvement in advice seeking and clinical decision-making; differential diagnoses; clinician attitudes and behaviour to risk; definition of ‘patient’; NMP grades and roles; context data collection; interobserver reliability; and guidance for researcher use and completion of the tool. For a summary of these changes, see Appendix 7 . The subsequent testing and piloting by researchers of the revised prototype tool resulted in further modifications. The key issue was the amount of time required to complete the tool given its length and complexity. Our initial intention had been to include all tasks and activities that might be observed. However, with a pragmatic lens, the tasks and activities were reduced in number to include only those perceived to measure clinician independence or interdependence, being ones that could potentially be discussed by a clinician with team colleague, clinician-in-charge or specialist. The other main change was inclusion of whether the observed clinician has an action plan or not in a discussion with a team colleague, clinician-in-charge or specialist. The subsequent prototype tool with user guide was circulated to all stakeholders and accepted with only minor modifications suggested to some wording. The final prototype tool contained the same three sections as the initial prototype. The first section included a vertical scale of 28 observable clinician tasks and activities under five headings: patient assessment and investigation (5 items); information sharing and decision-making with patient (5 items); online information seeking (2 items); advice/clinical decision asked for by team colleagues (2 items); diagnosis and treatment (8 items) and disposition of patient (6 items). The horizontal scale included different independence and interdependence criteria for each observed task and action: undertaken without discussion; outcomes of discussion with ‘a team colleague’, ‘clinician-in-charge’ or ‘specialist’; and whether a discussion is ‘with’ or ‘without action plan proposed by clinician observed’. Figure 3 shows the horizontal scale and exemplar tasks and actions as an illustration of the vertical scale. FIGURE 3 Independence and interdependence criteria used in the tool and exemplar tasks and actions. The second section of the tool was for context data collected during the observation with pre-coded items on location, time, proximity of clinician-in-charge, resource availability and patients not assenting to observation. The final section was in two parts: the first for contextual data collected by the researcher from the trust, including patient wait times and staff numbers on duty; the second for self-completion by the clinician observed, including their perceptions of time pressures, workplace experience, knowledge of team members’ skills and experience, and managing risk and uncertainty. A user guide provided practical instructions and advice for researchers completing the tool. Discussion This multimethod study 23 used an incremental and iterative process 24 incorporating a literature review, observation of clinical practice, and stakeholder collaboration to develop what is, to the best of our knowledge, the first structured observational tool to measure quantitatively the levels of independence NMPs have in an interdependent skill-mixed team in EDs/UTCs. Each iteration of the prototype tool was based on improvements identified in three rounds of stakeholder collaboration until all stakeholders could accept the tool design. Many issues were considered, discussed and decided upon. The different emergent concerns prompted a need to balance rigour to include all identified relevant items for measurement against practical matters, such as complexity and length of time taken to complete the tool. Compromise was sometimes required, especially where there was no apparent right solution. Lessons learnt We discuss in turn the key learnings from this study that have guided development of the prototype tool: NMPs work interdependently; task urgency and complexity influences levels of interdependence; different categories of independent and interdependent behaviours; patient involvement; different types of clinical decision-making discussion; incorporation of observable and measurable clinician behaviours. Rather than oppositional tension between independence and supervision, our findings suggest that individual NMPs and resident doctors undertake different tasks and activities with varying levels of independence within an interdependent team. Sebok-Syer et al . 62 identified two types of interdependence among clinical healthcare teams: (1) supportive interdependence prompted by a team member’s deficient expertise to perform within their full scope of practice and (2) collaborative interdependence prompted by recognition that patient care requires team member contribution. We found evidence of both types of interdependence, which underpinned the rationale to avoid dichotomous language in the prototype tool we developed. From a sociological perspective, ED/ UTC team members demonstrated communitarian elements 63 in their work. Relationships and interconnections ‘crisscrossed and reinforced one another’, 63 based on a shared responsibility, commitment and identity around efficient patient care and patient flow. Reeves et al . 64 argued that such components of shared team identity and responsibility, and interdependence, are core elements of interprofessional teamwork suited to unpredictable, urgent and complex team tasks; characteristic of tasks in EDs/UTCs. 41 Consequently, our prototype tool enables both the measurement of the level of interdependence of observed tasks and also contextual factors influencing the task urgency and complexity. Clinicians-in-charge have the professional autonomy to make their own clinical decisions with that judgement accepted as authoritative by their team, 65 but not necessarily so when interacting with other specialties. In this study, authority was identified as an important concept. The authority to make clinical decisions without senior input being observed as one end of a spectrum of levels of independence. The significance of authority was also demonstrated by NMPs frequently communicating to patients the need for their case to be discussed with a senior doctor. The prototype tool therefore allows for measurement of different categories of observable independent and interdependent behaviour including whether clinical decision-making discussions were with ‘a team colleague’, ‘clinician-in-charge’ or ‘specialist’ (see Figure 3 ). Our study PPI representatives argued that patients should be considered integral members of ED/ UTC interdependent teams, and our clinician observation findings showed that patients played an important team role although seldom brought into the diagnostic decision-making process. Evidence shows that when patients can establish relationships with staff, they feel able to share more information and contribute to decision-making. 66 The prototype tool was therefore designed to give recognition that patients are also participants in decision-making. Existing evidence reports increasing numbers of high-acuity ED patients, 67 often meriting more complex thinking and decision-making. 68 In this study, we observed that participants accessed available knowledge from team colleagues, as well as protocols and clinical guidance using personal smartphones and ED/ UTC computers, or by drawing on experiential knowledge. They also chose or were required by NHS trust regulations to discuss decision-making with a clinician-in-charge. Accordingly, the prototype tool allows for assessment of different types of clinical decision-making discussion with different team members and different outcomes. Our prototype tool distinguishes between distinct types of independence and supervision within an interdependent team which differentiates it from existing classifications and tools that measure these concepts. The tool is also distinctive in that it incorporates observable clinician behaviours related to these concepts in order that they can be measured; our literature review identified only one tool that included observation, its development referring to aspects of teamwork ‘that could be reworded as observable behaviours’. 61 Our collaborative stakeholder tool development process proved a strength not only in helping identify which concepts to measure, but also how to translate them into observable behaviours. The use of documentary television recordings showed promise as a methodology to support this process. Limitations This was a multistage and multimethod study with strengths and limitations in each component. A limitation of the literature review is that it excluded non-Englishlanguage papers, meaning some potentially relevant studies may not have been included. Despite our comprehensive search strategy, the evidence rarely focused on the independence and supervision of NMPs in EDs/UTCs. Our clinician observations involved a small number of participants recruited from two hospital ED sites. This may have restricted our ability to understand some of the mediating factors associated with local context. However, in-depth investigation at two sites illuminated the nature of independent and interdependent skill-mixed team working. The volunteer participants were self-selecting rather than purposively recruited, which could have introduced some partiality in the sample. Nevertheless, a mixed sample of NMPs and resident doctors was recruited with differences in background and training and ED areas they worked. Our observations included only day shifts, albeit we aimed to reduce bias by observing on a range of weekdays. Although we tried to minimise observer influence, there was potential for the Hawthorne effect, 69 in that clinicians and patients may have acted and responded differently in the presence of external observers. While it was not possible to assess whether there were any differences between international medical graduate doctors and local resident doctors, we were able to collect contextual data on mandatory supervision requirements. All resident doctors and NMPs observed were required to discuss their patient cases with the clinician-in-charge. While the collaborative tool development process was a strength of this study, and it sought to incorporate the views of all stakeholders, including PPI and NMP representatives, and to work with them as equals, they were not all involved in every aspect of decision-making. Separate meetings of the research team took place alongside the collaborative sessions and stakeholder meetings. The need for pragmatic compromise in its final iteration meant the prototype tool may not encompass all preferences. Establishing the validity of any measurement tool is a formal ongoing process. This study was able to show face validity, but the objectivity and validity of the prototype tool were not tested. Future research The prototype tool was designed for use in later stages of the parent ‘SkillMix-ED’ study. The findings from the parent study will provide insights into the application of the tool. Future studies are required to determine the objectivity, reliability and validation of the tool. Conclusion This study developed a prototype-structured observational tool for quantitative measurement of levels of independence and supervision of NMPs working in EDs/UTCs. Our data led us intentionally to develop a prototype observational tool that reflected NMPs working within an interdependent skill-mix team, rather than a tool that reflected a decontextualised spectrum from independence to supervision as initially planned. The prototype tool has been designed to discern various tasks and actions being undertaken, with more or less independence or supervision, in different areas of the ED/ UTC by more or less skilled and experienced NMPs and other clinicians, within a supportive and collaborative interdependent team. Our iterative methodology engaging clinicians, practitioners, patients and carers added robustness to the process. Further testing of the tool is required. Additional information CRediT contribution statement Francesca Taylor ( https://orcid.org/0000-0003-3751-6003 ): Conceptualisation, Methodology, Project administration, Formal analysis, Investigation, Funding acquisition, Writing – original draft, Writing – reviewing and editing. Mary Halter ( https://orcid.org/0000-0001-6636-0621 ): Conceptualisation, Methodology, Formal analysis, Investigation, Visualisation, Supervision, Funding acquisition, Writing – reviewing and editing. Vari M Drennan ( https://orcid.org/0000-0002-8915-5185 ): Conceptualisation, Methodology, Investigation, Supervision, Funding acquisition, Writing – reviewing and editing. Jonathan Gabe ( https://orcid.org/0000-0002-3765-4935 ): Conceptualisation, Methodology, Funding acquisition, Writing – reviewing and editing. Heather Gage ( https://orcid.org/0000-0002-2049-9406 ): Conceptualisation, Methodology, Funding acquisition, Writing – reviewing and editing. Heather Jarman ( https://orcid.org/0000-0002-4820-3291 ): Conceptualisation, Methodology, Funding acquisition, Writing – reviewing and editing. Celayne Heaton-Shrestha ( https://orcid.org/0000-0002-6021-5877 ): Writing – reviewing and editing. Catriona Brice ( https://orcid.org/0009-0002-8885-0941 ): Writing – reviewing and editing. Acknowledgements The authors would like to thank all staff participants for their time in taking part in the study, and all members of our study advisory groups for contributing time, experience and expertise to the study. Data-sharing statement The data sets used and/or analysed during the study are available from the corresponding author on reasonable request. Ethics statement Ethical approval for the study was received from NHS North East-Tyne and Wear South Research Ethics Committee (REC number 21/NE/0071) on 1 April 2021. Information governance statement Kingston University is committed to handling all personal information in line with the UK Data Protection Act (2018) and UK General Data Protection Regulation. This is specified in the university’s data protection policy and in the research protocol. Kingston University, as sponsor of this research, is the Data Controller. You can find out more about how we handle personal data, including how to exercise your individual rights, and can access the contact details for the Kingston University Data Protection Officer here: [email protected] . Disclosure of interests Full disclosure of interests: Completed ICMJE forms for all authors, including all related interests, are available in the toolkit on the NIHR Journals Library report publication page at https://doi.org/10.3310/GJMH1818 . Primary conflicts of interest: No competing interests were disclosed. Department of Health and Social Care disclaimer This publication presents independent research commissioned by the National Institute for Health and Care Research (NIHR). The views and opinions expressed by the interviewees in this publication are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, MRC, NIHR Coordinating Centre, the Health and Social Care Delivery Research programme or the Department of Health and Social Care. This article was published based on current knowledge at the time and date of publication. NIHR is committed to being inclusive and will continually monitor best practice and guidance in relation to terminology and language to ensure that we remain relevant to our stakeholders. Study registration The study is registered as ClinicalTrials.gov NCT04770766 . Funding This article presents independent research funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme as award number NIHR131356. Box This article reports research award Implementation of the non-medical practitioner workforce into the urgent and emergency care system skill-mix in England: a mixed methods study of configurations and impact . For more information about this research, (more...) About this article The contractual start date for this research was in March 2021. This article began editorial review in April 2025 and was accepted for publication in August 2025. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The Health and Social Care Delivery Research editors and publisher have tried to ensure the accuracy of the authors’ article and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this article. Copyright Copyright © 2026 Taylor et al . This work was produced by Taylor et al . under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This is an Open Access publication distributed under the terms of the Creative Commons Attribution CC BY 4.0 licence, which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. See: https://creativecommons.org/licenses/by/4.0/ . For attribution the title, original author(s), the publication source – NIHR Journals Library, and the DOI of the publication must be cited. Copyright and credit statement Every effort has been made to obtain the necessary permissions for reproduction, to credit original sources appropriately and to respect copyright requirements. However, despite our diligence, we acknowledge the possibility of unintentional omissions or errors and we welcome notifications of any concerns regarding copyright or permissions. Glossary Advanced clinical practitioner Advanced clinical practitioners are registered healthcare professionals, educated to master’s level or equivalent, with the skills and knowledge to allow them to expand their scope of practice to better meet the needs of the people they care for. Advanced clinical practitioners are deployed across all healthcare settings and work at a level of advanced clinical practice that pulls together the four advanced clinical practitioner pillars of clinical practice, leadership and management, education and research. 70 Advanced nurse practitioner A term used for nurses who meet the criteria for advanced clinical practitioners as described by National Health Service England. 71 Emergency department The emergency department is for serious injuries and life-threatening emergencies. It is also known as accident and emergency and casualty. The emergency department is open 24 hours a day, every day. 72 Emergency practitioner The emergency practitioner role is an extended role rather than an advanced role. Most emergency practitioners work at an enhanced level of practice with specific knowledge and skills in a field of expertise. Practitioners working at an enhanced level of practice must make complex decisions using specific knowledge and skills in a field of expertise. They manage all aspects of a patient’s care in their current level of practice, which will be particular to a specific context. 73 General practitioner A general practitioner is a doctor who is a consultant in general practice. General practitioners have distinct expertise and experience in providing whole-person medical care while managing the complexity, uncertainty and risk associated with the continuous care they provide. General practitioners work at the heart of their communities, striving to provide comprehensive and equitable care for everyone, taking into account their healthcare needs, stage of life and background. 74 National Health Service The National Health Service in Great Britain, a comprehensive public health service under government administration, was established by the National Health Service Act of 1946 and subsequent legislation. Virtually the entire population is covered, and health services are free except for certain minor charges. 75 Non-medical practitioner A generic term used to incorporate a wide range of professions, including advanced practitioners and physician associates, working in roles that were traditionally performed by doctors. Some of these are autonomous, whereas others provide support to medical staff. 76 Nurse practitioner Another term used for advanced nurse practitioner. 71 Physician associate Physician associates are healthcare professionals with a generalist healthcare education who work alongside doctors and surgeons, providing medical care as an integral part of the multidisciplinary team. Physician associates work under the supervision of a named senior doctor (such as a named General Medical Council registered consultant or general practitioner) but can work autonomously with appropriate support. 77 Patient and public involvement Public involvement in research means research that is done ‘with’ or ‘by’ the public, not ‘to’, ‘for’ or ‘about’ them. It means that patients or other people with relevant experience contribute to how research is designed, conducted and disseminated. 78 Resident doctor Resident doctors are fully qualified doctors who are either currently in postgraduate training or gaining experience as locally employed doctors. It is a generic term for doctors in foundation programmes, specialty and general practitioner training or locally employed. It does not include consultants who are the most senior doctor with overall responsibility for the care of patients in hospital. 79 Urgent treatment centre Urgent treatment centres provide urgent medical help when it is not a life-threatening emergency. Open at least 12 hours a day, every day, urgent treatment centres are equipped to investigate, diagnose and deal with many of the most common injuries and illnesses people attend emergency departments (accident and emergency) for. 80 List of abbreviations ACP advanced clinical practitioner ANP advanced nurse practitioner ED emergency department EP emergency practitioner GP general practitioner NMP non-medical practitioner NP nurse practitioner PA physician associate PPI patient and public involvement UTC urgent treatment centre References 1. Gettel CJ, Courtney DM, Janke AT, Rothenberg C, Mills AM, Sun W, Venkatesh AK. The 2013 to 2019 emergency medicine workforce: clinician entry and attrition across the US geography. Ann Emerg Med 2022;80:260–71. 10.1016/j.annemergmed.2022.04.031 [ PMC free article : PMC9398978 ] [ PubMed : 35717274 ] [ CrossRef ] 2. Kenny JF, Chang BC, Hemmert KC. Factors affecting emergency department crowding. Emerg Med Clin North Am 2020;38:573–87. 10.1016/j.emc.2020.04.001 [ PubMed : 32616280 ] [ CrossRef ] 3. Di Somma S, Paladino L, Vaughan L, Lalle I, Magrini L, Magnanti M. Overcrowding in emergency department: an international issue. Intern Emerg Med 2015;10:171–5. 10.1007/s11739-014-1154-8 [ PubMed : 25446540 ] [ CrossRef ] 4. Stock L, Turcotte J, Johnson A, Holbert SE, Siska M, Pipkin K, Patton C. Evaluating the success of an inpatient PA and NP program through trends in ED consults. JAAPA 2024;37:41–6. 10.1097/01.JAA.0000995648.20577.85 [ PubMed : 38051811 ] [ CrossRef ] 5. Drennan VM, Halter M, Taylor F, Gabe J, Jarman H. Non-medical practitioners in the staffing of emergency departments and urgent treatment centres in England: a mixed qualitative methods study of policy implementation. BMC Health Serv Res 2023;23:1221. 10.1186/s12913-023-10220-4 [ PMC free article : PMC10631061 ] [ PubMed : 37936220 ] [ CrossRef ] 6. Halter M, Drennan VMD, Wang C, Webb J, Gabe J, Taylor F, Gage H, Jarman J. Implementation of the Non-medical Practitioner Workforce into the Urgent and Emergency Care System Skill-Mix in England: A Mixed Methods Study of Configuration and Impact. Study Protocol V 2.0. NIHR131356. 2023. URL: https: ​//fundingawards ​.nihr.ac.uk/award/NIHR131356 (accessed 20 August 2025). 7. Clark A, Amanti C, Sheng AY. Supervision of advanced practice providers. Emerg Med Clin North Am 2020;38:353–61. 10.1016/j.emc.2020.02.007 [ PubMed : 32336330 ] [ CrossRef ] 8. Pines JM, Courtney DM. How to effectively integrate advanced practice providers into the ED workforce. Acad Emerg Med 2022;30:154–6. 10.1111/acem.14633 [ PubMed : 36445097 ] [ CrossRef ] 9. Wise S. The Flexibility of Healthcare Teams. A Mixed Methods Study of Doctors and Nurses in an Emergency Department. PhD thesis. Sydney, NSW, Australia: University of Technology Sydney; 2018. 10. Phillips AW, Klauer KM, Kessler CS. Emergency physician evaluation of PA and NP practice patterns. JAAPA 2018;31:38–43. 10.1097/01.JAA.0000532118.98379.f1 [ PubMed : 29698370 ] [ CrossRef ] 11. Horvath S, Visekruna S, Kilpatrick K, McCallum M, Carter N. Models of care with advanced practice nurses in the emergency department: a scoping review. Int J Nurs Stud 2023;148:104608. 10.1016/j.ijnurstu.2023.104608 [ PubMed : 37801938 ] [ CrossRef ] 12. Lim ME, Worster A, Goeree R, Tarride JE. Simulating an emergency department: the importance of modeling the interactions between physicians and delegates in a discrete event simulation. BMC Med Inform Decis Mak 2013;13:59. 10.1186/1472-6947-13-59 [ PMC free article : PMC3664080 ] [ PubMed : 23692710 ] [ CrossRef ] 13. Merrick E, Fry M, Busby Grant J. Team structure, skill mix and clinical outcomes. Emerg Med Austral 2017;29:584–6. 10.1111/1742-6723.12813 [ PubMed : 28568241 ] [ CrossRef ] 14. Bahadori A, Fitzpatrick JJ. Level of autonomy of primary care nurse practitioners. J Am Acad Nurse Pract 2009;21:513–9. 10.1111/j.1745-7599.2009.00437.x [ PubMed : 19845809 ] [ CrossRef ] 15. Dempster JS. Autonomy in Practice: Conceptualization, Construction, and Psychometric Evaluation of an Empirical Instrument. PhD thesis. San Diego, CA: University of San Diego; 1990. 16. Varjus SL, Suominen T, Leino-Kilpitt H. Autonomy among intensive care nurses in Finland. Intensive Crit Care Nurs 2003;19:31–40. 10.1016/S0964-3397(03)00007-7 [ PubMed : 12590892 ] [ CrossRef ] 17. Weston MJ. Validity of instruments for measuring autonomy and control over nursing practice. J Nurs Scholarsh 2009;41:87–94. 10.1111/j.1547-5069.2009.01255.x [ PubMed : 19335682 ] [ CrossRef ] 18. Hausmann D, Zulian C, Battegay E, Zimmerli L. Tracing the decision-making process of physicians with a Decision Process Matrix. BMC Med Inform Decis Mak 2016;16:133. 10.1186/s12911-016-0369-1 [ PMC free article : PMC5070075 ] [ PubMed : 27756369 ] [ CrossRef ] 19. Bamber Y, Baker GR, Orlando E, Fancott C, Bhati P, Casalino S, et al. Engaging patients to improve quality of care: a systematic review. Implement Sci 2018;13:98. 10.1186/s13012-018-0784-z [ PMC free article : PMC6060529 ] [ PubMed : 30045735 ] [ CrossRef ] 20. Bergerum C, Thor J, Josefsson K, Wolmesjö M. How might patient involvement in healthcare quality improvement efforts work – a realist literature review. Health Expect 2019;22:952–64. 10.1111/hex.12900 [ PMC free article : PMC6803394 ] [ PubMed : 31044517 ] [ CrossRef ] 21. LeMaster JW. PPIE in studies: randomized trials and clinical quality improvement. Health Expect 2020;23:1–2. 10.1111/hex.13009 [ PMC free article : PMC6978877 ] [ PubMed : 31976613 ] [ CrossRef ] 22. Ocloo J, Matthews R. From tokenism to empowerment: progressing patient and public involvement in healthcare improvement. BMJ Qual Saf 2016;25:626–32. 10.1136/bmjqs-2015-004839 [ PMC free article : PMC4975844 ] [ PubMed : 26993640 ] [ CrossRef ] 23. Morse JM. Principles of Mixed Methods and Multimethod Research Design. In Tashakkori A, Teddlie C, editors. Handbook of Mixed Methods in Social and Behavioral Research. Thousand Oaks, CA: SAGE Publications Ltd; 2003. pp. 189–208. 24. Egholm CL, Packness A, Stokholm J, Rasmussen K, Ellervik C, Simonsen E, et al. Questionnaire development for the Lolland-Falster health study, Denmark: an iterative and incremental process. BMC Med Res Methodol 2020;20:52. 10.1186/s12874-020-00931-1 [ PMC free article : PMC7055066 ] [ PubMed : 32126988 ] [ CrossRef ] 25. Staniszewska S, Brett J, Simera I, Seers K, Mockford C, Goodlad S, et al. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. BMJ 2017;358:j3453. 10.1136/bmj.j3453 [ PMC free article : PMC5539518 ] [ PubMed : 28768629 ] [ CrossRef ] 26. O’Brien B, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med 2014;89:1245–51. 10.1097/ACM.0000000000000388 [ PubMed : 24979285 ] [ CrossRef ] 27. Halter M, Taylor F, Drennan VM. A Review of the Configurations and Impact of Different Skill-Mix Involving Non-medical Practitioners in Emergency and Urgent Care Hospital-Based Settings: Protocol for a Systematic Review. PROSPERO 2022:CRD42022378511. 2022. URL: www ​.crd.york.ac.uk/PROSPERO ​/display_record ​.php?ID=CRD42022378511 (accessed 20 August 2025). 28. De Brún C, Pearse-Smith N. Citation Pearl Searching. In De Brúún C, Pearse-Smith N, editors. Searching Skills Toolkit: Finding the Evidence. London: John Wiley & Sons; 2014. pp. 98–101. 29. Hammersley M, Atkinson P. Research Design: Problems, Cases and Samples. In Hammersley M, Atkinson P, editors. Ethnography: Principles in Practice. Abingdon: Routledge; 2019. pp. 21–43. 30. Czarniawska, B. Fieldwork Techniques for Our Times: Shadowing. In Ciesielska M, Jemielniak D, editors. Qualitative Methodologies in Organization Studies. Cham: Springer Nature; 2018. pp. 53–74. 31. Holloway I, Galvin K. Ethnography. In Holloway I, Galvin K, editors. Qualitative Research in Nursing and Health Care. Chichester: Wiley-Blackwell; 2016. pp. 159–78. 32. Spradley JP. Making Descriptive Observations. In Spradley JP, editor. Participant Observation. New York: Holt, Rinehart and Winston; 1980. pp. 73–84. 33. Mays N, Pope C. Quality in Qualitative Research. In Pope C, Mays N, editors. Qualitative Research in Health Care. Chichester: Wiley-Blackwell; 2020. pp. 211–34. 34. Pope C, Mays N. The Role of Theory in Qualitative Research. In Pope C, Mays N, editors. Qualitative Research in Health Care. Chichester: Wiley-Blackwell; 2020. pp. 15–26. 35. Braun V, Clarke V. Reflecting on reflexive thematic analysis. Qual Res Sport Exer Health 2019;11:589–97. 10.1080/2159676X.2019.1628806 [ CrossRef ] 36. Coleman S. On Remembering and Forgetting in Writing and Fieldwork. In Collins P, Gallinat A, editors. The Ethnographic Self as Resource. Writing Memory and Experience into Ethnography. New York: Berghahn Books; 2010. pp. 215–27. 37. Islam R. Group decision making through nominal group technique: an empirical study. J Int Bus Entrep Dev 2010;5:134–53. 10.1504/JIBED.2010.036998 [ CrossRef ] 38. Hoyt KS, Proehl JA. Making a case for entrustable professional activities for nurse practitioners in emergency care. Adv Emerg Nurs J 2017;39:77–80. 10.1097/TME.0000000000000146 [ PubMed : 28463862 ] [ CrossRef ] 39. Shea SS, Hoyt KS. Medical decision making in emergency care. Adv Emerg Nurs J 2014;36:360–96. 10.1097/TME.0000000000000038 [ PubMed : 25356896 ] [ CrossRef ] 40. Trautman J, Epstein E, Rovnyak V, Snyder A. Relationships among moral distress, level of practice independence, and intent to leave of nurse practitioners in emergency departments: results from a national survey. Adv Emerg Nurs J 2015;37:134–45. 10.1097/TME.0000000000000060 [ PubMed : 25929224 ] [ CrossRef ] 41. Wise S, Duffield C, Fry M, Roche M. Clarifying workforce flexibility from a division of labor perspective: a mixed methods study of an emergency department team. Human Resour Health 2020;18:17. 10.1186/s12960-020-0460-7 [ PMC free article : PMC7060538 ] [ PubMed : 32143632 ] [ CrossRef ] 42. McBrien B. Assessment and management of patients with tibial plateau fractures in emergency departments. Emerg Nurse 2019;27:26–30. 10.7748/en.2019.e1981 [ PubMed : 31571436 ] [ CrossRef ] 43. Smyth O, McCabe C. Think and think again! Clinical decision making by advanced nurse practitioners in the emergency department. Int Emerg Nurs 2017;31:72–4. 10.1016/J.IENJ.2016.08.001 [ CrossRef ] 44. Burrows KE, Abelson J, Miller PA, Levine M, Vanstone M. Understanding health professional role integration in complex adaptive systems: a multiple-case study of physician assistants in Ontario, Canada. BMC Health Serv Res 2020;20:365. 10.1186/s12913-020-05087-8 [ PMC free article : PMC7189743 ] [ PubMed : 32349738 ] [ CrossRef ] 45. Boman E, Levy-Malmberg R, Fagerstrom L. Differences and similarities in scope of practice between registered nurses and nurse specialists in emergency care: an interview study. Scan J Caring Sci 2020;24:492–500. 10.1111/scs.12753 [ PubMed : 31638277 ] [ CrossRef ] 46. Baggs JGB. Development of an instrument to measure collaboration and satisfaction about care decisions. J Adv Nurs 1994;20:176–82. 10.1046/j.1365-2648.1994.20010176.x [ PubMed : 7930118 ] [ CrossRef ] 47. Blegen MA, Goode C, Johnson M, Maas M, Chen L, Moorhead S. Preferences for decision-making autonomy. Image J Nurs Sch 1993;25:339–44. 10.1111/j.1547-5069.1993.tb00269.x [ PubMed : 8288303 ] [ CrossRef ] 48. Chang AM, Gardner GE, Duffield C, Ramis MA. A Delphi study to validate an advanced practice nursing tool. J Adv Nurs 2010;66:2320–30. 10.1111/j.1365-2648.2010.05367.x [ PubMed : 20626481 ] [ CrossRef ] 49. Chang AM, Gardner GE, Duffield C, Ramis MA. Advanced practice nursing role development: factor analysis of a modified role development tool. J Adv Nurs 2012;68:1369–79. 10.1111/j.1365-2648.2011.05850 [ PubMed : 22032475 ] [ CrossRef ] 50. Mick DJ, Ackerman MH. Advanced practice nursing role delineation in acute and critical care: application of the Strong Model of Advanced Practice. Heart Lung 2000;29:210–21. 10.1067/mhl.2000.106936 [ PubMed : 10819802 ] [ CrossRef ] 51. Rice B, Pickering A, Lawrence C, Kizito PM, Leff R, Kisingin SJ, et al. Emergency medicine physician supervision and mortality among patients receiving care from non-physician clinicians in a task-sharing model of emergency care in rural Uganda: a retrospective analysis of a single-centre training programme. BMJ Open 2022;12:e059859. 10.1136/bmjopen-2021-059859 [ PMC free article : PMC9244677 ] [ PubMed : 35768107 ] [ CrossRef ] 52. Thompson S, Moorley C, Barratt J. A comparative study on the critical decision-making processes of nurse practitioners vs. medical doctors using scenarios in a secondary care environment. J Adv Nurs 2017;73:1097–110. 10.1111/jan.13206 [ PubMed : 27859497 ] [ CrossRef ] 53. Berg SK, Faerch J, Cromhout PF, Tewes M, Pedersen PU, Rasmussen TB, et al. Questionnaire measuring patient participation in health care: scale development and psychometric evaluation. Eur J Cardiovasc Nurs 2020;19:600–8. 10.1177/1474515120913809 [ PubMed : 32324044 ] [ CrossRef ] 54. Brown J, Reid H, Dornan T, Nestel D. Becoming a clinician: trainee identity formation within the general practice supervisory relationship. Med Educ 2020;54:993–1005. 10.1111/medu.14203 [ PubMed : 32350873 ] [ CrossRef ] 55. Djulbegovic M, Beckstead JW, Fraenkel L. The Patient Care Ownership Scale: development of an instrument to measure patient care ownership among internal medicine trainees. J Gen Intern Med 2019;34:1530–7. 10.1007/s11606-019-05066-8 [ PMC free article : PMC6667594 ] [ PubMed : 31161566 ] [ CrossRef ] 56. Eldh AC, Luhr K, Ehnfors M. The development and initial validation of a clinical tool for patients’ preferences on patient participation: the 4Ps. Health Expect 2015;18:2522–35. 10.1111/hex.12221 [ PMC free article : PMC5810670 ] [ PubMed : 24938672 ] [ CrossRef ] 57. Eldh AC. Facilitating patient participation by embracing patients’ preferences – a discussion. J Eval Clin Pract 2019;25:1070–3. 10.1111/jep.13126 [ PubMed : 30916440 ] [ CrossRef ] 58. Heggland LH, Mikkelsen A, Øgaard T, Hausken K. Measuring patient participation in surgical treatment decision-making from healthcare v professional’s perspective. J Clin Nurs 2014;23:482–91. 10.1111/jocn.12184 [ PubMed : 23551641 ] [ CrossRef ] 59. Malfait S, Eeckloo K, Van Dael J, Van Hecke A. The Patient Participation Culture Tool for healthcare workers (PaCT-HCW) on general hospital wards: a development and psychometric validation study. Int J Nurs Stud 2016;61:187–97. 10.1016/j.ijnurstu.2016.05.015 [ PubMed : 27372433 ] [ CrossRef ] 60. Pomey MP, Clavel N, Normandin L, Del Grande C, Ghadiri DP, Fernandez-McAuley I, et al. Assessing and promoting partnership between patients and health-care professionals: co-construction of the CADICEE tool for patients and their relatives. Health Expect 2021;24:1230–41. 10.1111/hex.13253 [ PMC free article : PMC8369086 ] [ PubMed : 33949739 ] [ CrossRef ] 61. Thistlewaite J, Dallest K, Moran M, Dunston R, Roberts C, Eley D, et al. Introducing the individual Teamwork Observation and Feedback Tool (iTOFT): development and description of a new interprofessional teamwork measure. J Interprof Care 2016;30:526–8. 10.3109/13561820.2016.1169262 [ PubMed : 27269996 ] [ CrossRef ] 62. Sebok-Syer SS, Chahine S, Watling CJ, Goldszmidt M, Cristancho S, Lingard L. Considering the interdependence of clinical performance: implications for assessment and entrustment. Med Educ 2018;52:970–80. 10.1111/medu.13588 [ PMC free article : PMC6120474 ] [ PubMed : 29676054 ] [ CrossRef ] 63. Etzioni A. The Responsive Communitarian Platform: Rights and Responsibilities. In Etzioni A, editor. Rights and the Common Good: The Communitarian Perspective. New York: St Martin’s Press; 1995. pp. 11–24. 64. Reeves S, Xyrichis A, Zwarenstein M. Teamwork, collaboration, coordination and networking: why we need to distinguish between different types of interprofessional practice. J Interprof Care 2018;32:1–3. 10.1080/13561820.2017.1400150 [ PubMed : 29131697 ] [ CrossRef ] 65. Gabe J, Bury M, Elston MA. Health Work and the Division of Labour. In Gabe J, Bury M, Elston MA, editors. Key Concepts in Medical Sociology. Thousand Oaks, CA: SAGE Publications Ltd; 2004. pp. 163–212. 66. Walsh A, Bodaghkhani E, Etchegary H, Alcock L, Patey C, Senior D, et al. Patient-centered care in the emergency department: systematic review and meta-ethnographic synthesis. Int J Emerg Med 2022;15:36. 10.1186/s12245-022-00438-0 [ PMC free article : PMC9367087 ] [ PubMed : 35953783 ] [ CrossRef ] 67. Authority of the House of Lords. Emergency Healthcare: A National Emergency. 2023. URL: www ​.publications.parliament ​.uk/pa/ld5803 ​/ldselect/pubserv/130/130.pdf (accessed 20 August 2025). 68. Schreyer KE, Kuhn D, Norton V; American Academy of Emergency Medicine Board of Directors. Physician productivity and supervision. West J Emerg Med 2023;24:372–6. 10.5811/westjem.60876 [ PMC free article : PMC10284533 ] [ PubMed : 37278799 ] [ CrossRef ] 69. McCambridge J, Witton J, Elbourne DR. Systematic review of the Hawthorne effect: new concepts are needed to study research participation effects. J Clin Epidemiol 2014;67:267–77. 10.1016/j.jclinepi.2013.08.015 [ PMC free article : PMC3969247 ] [ PubMed : 24275499 ] [ CrossRef ] 70. National Health Service (NHS) England. Multi-Professional Framework for Advanced Practice. 2025. URL: www ​.advanced-practice ​.hee.nhs.uk/multi-professional-framework-for-advanced-practice/ (accessed 20 August 2025). 71. Royal College of Nursing. RCN Professional Development Framework: Advanced Level Nursing. 2024. URL. www ​.rcn.org.uk/Professional-Development ​/publications ​/rcn-professional-development-framework-advanced-level-nursing-uk-pub-011-127 (accessed 20 August 2025). 72. National Health Service (NHS). Urgent and Emergency Care Services. 2025. URL: www ​.nhs.uk/nhs-services ​/urgent-and-emergency-care-services (accessed 20 August 2025). 73. National Health Service (NHS) England. Urgent and Emergency Care: Emergency Practitioner Framework. 2023. URL: https://www ​.hee.nhs.uk ​/sites/default/files ​/documents/Urgent%20and ​%20Emergency%20Care ​%20%E2%80%93%20Emergency ​%20Practitioner%20Framework.docx (accessed 20 August 2025). 74. Royal College of General Practitioners (RCGP). The RCGP Curriculum: Being a General Practitioner. Definition of a GP. 2025. URL: www ​.rcgp.org.uk/mrcgp-exams ​/gp-curriculum ​/rcgp-curriculum-being-gp (accessed 20 August 2025). 75. Britannica. National Health Service (NHS). URL: www ​.britannica.com/topic ​/National-Health-Service (accessed 20 August 2025). 76. The College of Emergency Medicine. Non-medical Practitioners in the Emergency Department. 2015. URL: www ​.rcem.ac.uk/wp-content ​/uploads/2021/11 ​/Non_Medical_Practitioners_in_the_ED ​.pdf (accessed 20 August 2025). 77. General Medical Council (GMC). Who Are Physician Associates and Anaesthesia Associates? 2024. URL: www ​.gmc-uk.org/about ​/what-we-do-and-why/regulating-physician-associates-and-anaesthesia-associates ​/who-are-pas-and-aas (accessed 20 August 2025). 78. National Health Service (NHS) Health Research Authority (HRA). Public Involvement. 2024. URL: www ​.hra.nhs.uk/planning-and-improving-research ​/best-practice/public-involvement/ (accessed 20 August 2025). 79. British Broadcasting Corporation (BBC) News. NHS Junior Doctors to be Known as resident Doctors after Job Title Change. 2024. URL: www ​.bbc.co.uk/news/articles/c4geg72vjx9o (accessed 20 August 2025). 80. National Health Service (NHS) United Kingdom (UK). Urgent and Emergency Care Services. When to Visit an Urgent Treatment Centre. 2025. URL: www ​.nhs.uk/nhs-services ​/urgent-and-emergency-care-services ​/when-to-visit-an-urgent-treatment-centre/ (accessed 20 August 2025). Appendix 1. Eligibility criteria for independence and supervision review category Reference to the concepts of, measurement of, and/or outcomes pertaining to one or more of the following groups: Refers to at least one of the following concepts in the ED/ UTC setting: mentor/mentoring/mentorship/mentee supervision/supervisor/supervisee autonomy/autonomous independent/independence/interdependence consultation decision-making/clinical decision-making clinical judgement competence/competency empowerment responsibility Describes a classification system for independence or supervision in any clinical setting, with a classification system being defined as one or more of: model theory framework audit criterion explicit measure categories arising from empirical data Develops or utilises a specific tool that measures: any of above terms role delineation advanced practice scope of practice. Appendix 2. Preparatory literature review search strategy Search strategy for Cumulative Index to Nursing and Allied Health Literature Plus and Medical Literature Analysis and Retrieval System Online (2000–20) ‘non-medical practitioner’ OR ‘paramedic’ OR ‘nurse practitioner’ OR ‘pharmacist’ OR ‘physiotherapy practitioner’ OR ‘physician assistant/associate’ AND ‘independence’ OR ‘supervision’ OR ‘autonomy’. Appendix 3. Search strategy for additional terms after collaborative sessions Search strategy for PubMed (at June 2022) (accountable) AND (tool) AND (clinician) (authority) AND (tool) AND (clinician) (complete care) AND (tool) (patient participation) AND (tool) Search strategy for Google Scholar (at June 2022) ‘accountable’ AND ‘tool’ AND ‘clinician’ ‘authority’ AND ‘tool’ AND ‘clinician’ ‘complete care’ AND ‘tool’ ‘patient participation’ AND ‘tool' Appendix 4 TABLE 1 Concepts associated with independence or supervision View in own window Concept/s Author/year Country Population Setting Study design Key descriptions Competence Boman et al . (2020) 45 Finland and Norway Nurse specialists, registered nurses, nurse leaders Two EDs (corresponding to level 3 trauma centres) Semistructured interviews Competence relates to knowledge, understanding, and judgement combined with a range of cognitive, technical or psychomotor and interpersonal skills, as well as a range of personal attributes and attitudes. SupervisionCollaboration Burrows et al . (2020) 44 Canada PAs ED case study setting – urban (4 EDs) and rural (2 EDs) – within four case study settings Case studies of semistructured interviews, and document analysis of site-specific and stakeholder policy documents on PA role The physician must trust the PA to seek help, and the PA needs to feel confident that the supervising physician is readily available for consultation when required. Willingness to work and ability to define their roles within existing frameworks allows for the establishment of interprofessional collaborative person-centred care. Competence/entrustment Hoyt and Proehl (2017) 38 USA NPs EDs Editorial article Entrustment decisions involve clinical skills and abilities as well as more general facets of competence such as understanding one’s own limitations and knowing when to ask for help. An individual could be entrusted to perform unsupervised activities once he or she has attained a sufficient level of competence. Clinical decision-making McBrien (2019) 42 Ireland Registered ANP Single ED Clinical case study [U]nderstanding of the anatomy and physiology of the knee is essential to manage patients … and determine who requires immediate orthopaedic consultation. [R]egistered ANPs should retain a high index of suspicion to hasten diagnosis and reduce the risk of complications. Clinical decision-making Shea and Hoyt (2014) 39 USA NPs, PAs EDs/UTCs Clinical case studies The medical decision-making (MDM) process is impacted by the number and types of problems encountered during the visit. The complexity of the data obtained and reviewed by the provider contributes to the MDM level. Consultation with other members of the health care team and primary care providers also impacts the MDM process. Clinical decision-making Smyth and McCabe (2017) 43 Ireland ANPs EDs Clinical case study Intuitive decision-making uses heuristics (rules of thumb) and shortcuts that represent previous exposure to similar case scenarios … analytical decision-making is slower, more deliberate methodical thinking that involves reviewing all data collected. Consistent and regular sharing of research and knowledge and clinical experiences between medical and nursing staff, educators and clinicians in a multidisciplinary setting such as the ED in the interest of patient safety is essential to safe and effective clinical decision-making. IndependenceCollaboration Trautmann et al . (2015) 40 USA NPs, currently working or had worked in an ED EDs Cross-correlation design using survey methods For NPs, level of practice independence includes collaborative practice and authority to treat patients with a physician, using advanced practice skills and knowledge, and self-directed medical judgements. Autonomy Interdependence Teamwork Responsibility Supervision Wise (2018) 9 Australia Registered nurses, NPs, junior doctors, doctors Single site ED – Fast track (minors only) Explanatory sequential mixed methods design – work observations and qualitative field notes, and qualitative interviews [e]mpowered to initiate tasks, exercise discretion and participate in decisions regarding their work. Enhanced autonomy aims to improve skill utilisation and to encourage employees to display adaptive and proactive behaviour, improving organisational responsiveness. Working interdependently to prioritise and reprioritise each role’s core duties to keep patients flowing through the ED. The concept of ‘teamwork’ incorporates the skills, behaviours and attitudes that allow team members to interact dynamically, interdependently, and adaptively towards a common goal. Doctors, NPs and RNs conceptualised the responsibilities of each role as comprising ‘core duties’ and ‘shared tasks’. Core duties were tasks perceived to be primarily the responsibility of a particular role, part of their occupational specialisation. A shared task was one that a doctor or NP could perform themselves, or delegate to an RN, a consequence of their overlapping roles. [W]here the participant was instructing (Supervisor) or being instructed (Supervisee) by a colleague in the appropriate course of action for a patient’s care and/or educating them in aspects of a condition, procedure or treatment. Interdependence/Teamwork Wise et al . (2020) 41 Australia Registered nurses, NPs, junior doctors, doctors Single site ED – Fast track (minors only) Explanatory sequential mixed methods design – work observations and qualitative field notes, and qualitative interviews The team understood their roles were interdependent and worked towards a shared goal of timely, quality patient care. This teamwork behaviour was necessitated by the urgency and unpredictability of ED work and facilitated by the permanent ED clinical staff who worked side-by-side to develop a deep knowledge of each other’s roles. Appendix 5 TABLE 2 Tools and classification systems for independence or supervision View in own window Tool/classification Author/year Country Target population Concept/s measured Key features Identified from scoping and systematic review, and citation pearl searching Collaboration and Satisfaction about Care Decisions (CSACD) Baggs (1994) 46 USA Nurses and physicians working in intensive care units Clinical decision-making within skill-mix team, collaboration, co-ordination Tool for measuring critical attributes and amount of nurse-physician collaboration in making patient care decisions. Seven-point scale from strongly agree to strongly disagree. Content and construct validation. Autonomy Scale of Blegen Blegen et al . (1993) 47 USA Registered hospital staff nurses and head nurses Autonomy, clinical decision-making within skill-mix team, consultation, independence, responsibility, shared decision-making (incl. with patients) Tool for measuring decision-making autonomy. Contains 41 decisions (21 patient care and 21 unit operation activities) across three categories (decisions agreed on at independent level, decisions agreed on at shared level, decisions without agreement) with a 5-point scale from ‘have no authority and accountability’ to ‘have full independent authority and accountability’. Strong Model of Advanced Practice Role Delineation Tool Chang et al . (2010 and 2012) 48 , 49 Mick and Ackerman (2000) 50 Australia USA Advanced practice nurses Collaboration, complete care/see a patient completely, consultation Modified tool for delineating advanced nursing practice. Contains 41 activities in 5 domains: direct comprehensive care, support of systems, research, education, publication and professional leadership. Five-point Likert scale from ‘a very great extent' to ‘not at all’. Construct validation of the five domains. Decision process matrix (DPM) Hausmann et al . (2016) 18 Switzerland Physicians involved in medical decision-making, especially in EDs Clinical decision-making Tool for mapping individual diagnostic and decision-making processes. A three-phase pattern: option generation in terms of suspected diagnoses; option verification; and determination of final diagnosis. Nine confidence ratings from ‘practically impossible’ to ‘practically certain’. Internal and external validation. Levels of emergency medicine physician supervision Rice et al . (2022) 51 Uganda Emergency medicine physicians supervising NMPs in EDs Supervision Qualitative classification of three different levels of supervision: ‘Direct supervision’, ‘Indirect supervision’ and ‘Independent care’. Information processing theory and think aloud approach Thompson et al . (2017) 52 England Secondary care NPs and doctors in specialist areas of NP-led hospital care (including EDs) Clinical decision-making, consultation Qualitative classification of nine discrete phases of cognitive processes in clinical consultation decision-making: four stages of clinical reasoning and five stages involving clinical inference and therapeutic decision-making. Identified from searches instigated following collaborative sessions Patient Participation Questionnaire (PPQ) Berg et al . (2020) 53 Denmark Hospital patients admitted for cardiac disease, pulmonary disease or cancer diagnoses Shared decision-making Tool for measuring patient assessment of their participation in hospital care. Questionnaire of 17 items categorised into 4 subscales (participation, information, communication and relationship between patient and healthcare staff). Items scored on a 4-point Likert-type scale from ‘not at all’ to ‘a great extent’. Convergent construct validation. Trajectory of trainee and supervisor identities Brown et al . (2020) 54 Australia Supervisors and medical trainees within general practice training Entrustment, competence, interdependence, responsibility, supervision Classification model based on the trajectory of change in four trainee and supervisor reciprocal identities: junior learner and expert clinician, apprentice assistant and master coach, co-clinicians, and lead clinician and advisor; three associated social discourses: attribution of clinical authority, ownership of medical knowledge, and measures of clinical competence; and associated actions and facilitators Patient Care Ownership Scale Djulbegovic et al . (2019) 55 USA Hospital internal medicine trainees Accountability, autonomy, shared decision-making, responsibility Tool for measuring patient care ownership. Fifteen-items categorised into eight dimensions (advocacy; responsibility, accountability and follow-through, knowledge, communication, initiative, continuity of care, autonomy, perceived ownership). Items scored on a 7-point Likert-type scale from ‘strongly disagree’ to ‘strongly agree’. Preliminary face validity. Patient Preferences for Patient Participation Eldh et al . (2015 and Eldh 2019) 56 , 57 Sweden Acute, long-term and primary care adult patients Shared decision-making Tool for measuring patient preferences for patient participation through 12 attributes (as items). Items scored against four responses from ‘unimportant’ to ‘crucial’. Tool can also be used to measure extent to which patient participation has occurred, the 12 items being scored against four responses from ‘not at all’ to ‘entirely’. Initial content validation. Patient participation in decision-making in surgical treatment Heggland et al . (2014) 58 Norway Nurses and physicians working in surgical wards Shared decision-making Self-report measurement tool of shared decision-making from the nurse and physician perspective. Sixteen-item tool categorised into four factors (information dissemination, formulation of options, integration of information, control). Content and construct validation. Patient Participation Culture Tool for Healthcare Workers (PaCT-HCW) Malfait et al . (2016) 59 Belgium Healthcare workers on general and university hospital wards Shared decision-making Tool for measuring patient participation and information sharing and dialogue from the healthcare worker perspective. Contains 52 item across 8 components. Items scored on a 4-point Likert scale. Content and construct validation of seven components (excluding information and dialogue component). CADICEE tool Pomey et al . (2021) 60 Canada Patients in different clinical contexts (hospitalisation, consultation, chronic/acute disease) Shared decision-making Tool for measuring the degree of partnership in care desired by patients in their relationship with healthcare professionals. Consists of 24 items within 7 dimensions (confidence/trust, autonomy, participation in decision-making, sharing information, personal context, empathy, expertise). Face and construct validation. Advanced Individual Teamwork Observation and Feedback Tool (Advanced iTOFT) Thistlewaite et al . (2016) 61 Australia Senior students and junior health professionals undertaking interprofessional tasks Collaboration, leadership, shared decision-making (incl. with patients) Tool to measure and document observation of professional teamwork behaviour and give formative feedback. Ten observable behaviours under four headings: ‘shared decision-making’, ‘working in a team’, ‘leadership’, and ‘patient safety’, with four assessment options: ‘not applicable to this activity’, ‘inappropriate’, ‘appropriate’, and ‘responsive’. Appendix 6 TABLE 3 Observation excerpts related to themes View in own window Theme Field note extracts Interdependent clinical discussion and advice seeking Knowledge of when to consult clinician-in-charge. D04: Participant to researcher, ‘ As a F2 I need to see the consultant and queue up. More senior you are you just do on your own ’. D01: Participant to researcher: ‘ As a CT3 could discuss if wanted or discharge on own. Here (this ED) is consultant consultation-heavy. In other places would get laughed at if consulted on every case’. Advice provided to colleague. NMP02: C olleague asks participant to give a second opinion on their patient who has had a fall. Goes into a room with this colleague and goes up to their patient and looks at their eyes. Colleague to participant, ‘ I might need your X-ray capability’. NMP10 : EP asks participant to look at the x-ray of patient’s foot. Together they look at the X-ray. Participant: ‘ That looks like – how old – growth’ . Both discuss the X-ray. Advice given by colleague. D01: Nurse to participant: ‘Probably need follow-up. I wouldn’t do bloods’. Ad hoc clinical discussions. CiC12: F ailed chest drain being discussed by three staff in pink, blue and teal uniforms. a Another staff member in blue uniform leans over and asks whether patient had chest pains. So, four staff talking about this patient. Clinical discussion with specialists. NMP03: Radiologist to participant: ‘ What’s the problem?’ Participant: (patient age and gender) Complaining of severe abdominal pain, insidious today, 8–9/10. Bowels not open 3 days. 2/52 rule post FIT test. PR empty. We’re thinking mass with query obstruction or plain obstruction. Oh, and renal function is fine ’. Radiologist says that is fine. D09: Participant on phone to gastroenterologist: (describes patient symptoms and diagnostic test results) ' biopsy … stent … yes, his bilirubin is 27 … yes abdominal pain … would you give intravenous antibiotics … that’s what I though t’. Patient handover discussion D04: Participant grabbed by colleague in teal uniform for a patient handover. Asked to look at a male patient who had a washout procedure as in some pain last week. Colleague tells participant the medication the man is on and about his liver. D05: Participant to doctor colleague: ‘ Tell me what you’ve got and I’ll take over … What is it? More than a retention of urine I suspect? Any diagnosis yet ?’ Participant online information seeking. D01: Participant looks up clinical guidelines named ‘children’s ED afebrile seizure’. Guidance, education, support provided by clinician-in-charge. CiC12: Clinician-in-charge to resident doctor: ‘ In ED have to decide cardiac or not. My suggestion is, learn from other side one feature that suggests not cardiac. Start, that all CP kills ’. Resident doctor: ‘ Too short, pain is lack of O 2 ’. Clinician-in-charge: ‘ Other one negative. TROP. Pain has been there too long. If pain there 24 hours constant, should get TOP release so no TROP ’. D01: Clinician-in-charge to participant: ‘ Important to explain this is viral to parents. Have you enough to have a review? What’s the appropriate referral?’ Participant: ‘ So first fit. I haven’t referred before ’. Clinician-in-charge: ‘ OK so I will do that. So how would you do that ?’ Participant: ‘ I would refer to guidelines’. Clinician-in-charge reviews participant’s patient. D10: Clinician-in-charge to participant after reviewing their patient: ' What I did was to name it, call it out. Don’t be afraid to name things and say it’s not them. And I didn’t have to start, you did the ground work with your assessment and I went in with another assessment, it’s not the same. I could go in focused because you had done the first assessment. It’s like when you try to get a cannula in and can’t then (name of F2 walking past) comes and gets it in and you think you’ve failed but it’s no. That person has the band width because you’ve done the work already'. Referral to specialists. NMP03: Clinician-in-charge to participant: ‘ Speak with thoracics, as had surgical intervention and air leak there. Do they want to follow up in clinic ’. NMP10: Participant to patient: ‘ Spoken to orthopaedics. Better, still impacted. We’ll refer to a fracture clinic. Orthopaedics will discuss with consultant in the morning ’. Spatial, temporal and resource constraints encouraged collaborative working Space limitations. NMP08: Participant goes to look for a room and to get patient. No room available so speaks to patient in the resus waiting room. D11: Participant goes to get another patient from the waiting room. On the way looks for an empty room. No spare room so has to wait in corridor for one. Resource constraints. NMP10: Participant walks around a few places to find ultrasound machine and explains to someone else why needs to take machine. First machine doesn’t work. Has to go to find another machine. D04: Participant says to researcher, ‘ first task is finding a computer’. Finds one in the staff base. Cannot log in so goes to find another one that works. Wonders around looking at several computers to find one not in use that can log into. Watch participant try at least six computers. Eventually at 16.15 after 16.00 start, is able to log into a computer. But is still not able to access patient data. Finally, at 16.25 is able to look at patient data on a computer. Inputting patient notes online. D09: 16.40 Participant starts typing up notes looking up history and bloods and copying in. Writing very detailed notes still at 16.50. NMP02: Participant spends more time on computer doing notes and says to researcher: ‘ Big part of our day is notes, notes, notes’. Waiting to access clinician-in-charge. D11: Participant at front of queue at the moment. Another teal uniform now seems to have jumped ahead in the queue. A further teal uniform joins the queue to see the clinician-in-charge so there are now five people waiting. NMP08: Participant starts to give information on their patient to the clinician-in-charge. Clinician-in-charge ignores the participant and asks a grey uniform about another patient. Then clinician-in-charge takes a phone call. So, participant gives up and goes back to their computer. Collaborative team working practices Patient flow. CiC12: Clinician-in-charge asks two staff nearby, one in blue uniform and one in teal uniform, whether there are any patients that can be discharged or who are waiting to be discharged. NMP02: Participant finishes off notes. Says to researcher: ‘ Just doing bullet notes as conscious patients waiting’. Sharing of knowledge and experiences. D04: Participant listens in and looks at X-ray colleague in teal uniform and clinician-in-charge are discussing. NMP06 : Participant talking emergency NP through trauma guidelines. Emotional support. D05: Clinician-in-charge to participant: ‘ What do you have left to do that’s stopping you having lunch? Do you want to have a break and document late r?’ NMP08: Participant gives friendly pat to the shoulder of colleague in teal scrubs. Practical support. NMP03: Clinician-in-charge to participant: ‘ When you refer to SWOT clinic, which i-clip (local computer system) do you use, there’s two? ’ Participant: ‘I didn’t know there were two’. Clinician-in-charge, nurse and participant gather and decide which is correct one to use. D09: Participant to colleague, ‘ I can’t recall now who this refers to, I just want to give gastro a call, do you have a bleep ?’ Teamwork and team membership. NMP10: Participant to patient: ‘ Just waiting for a colleague who will do a cast for us ’. NMP03 : Participant goes to nurse to ask if can take a D-dimer for patient. Says to nurse, ‘ if a problem I will come and do it’. D09: Nurse comes back with auroscope and together with participant try to get it working ‘Soft’ professional hierarchies. D04: Colleague in teal scrubs asks to be ahead of participant. NMP07: Participant tries to see clinician-in-charge again. Next in line, as CiC has teal uniform with them. But another teal uniform asks to be ahead of participant. Patients’ involvement and role in teamwork Communication around diagnostic tests. NMP03: Participant to patient: ‘ So blood tests not back yet and we’ve just done the D-dimer but you have got a pneumothorax ’. D11: Participant to patient: ‘ You shouldn’t be getting this chest pain. I’ll go through your chest X-ray, chat with the consultant, and then make a plan ’. Communication around diagnoses and treatment plans. D01: Participant to patient’s parent: ‘ Good news, protein and blood. I think probably had urine infection. Complaining of abdominal pain, soft, keep an eye. If rigid, come back. Observations normal, no signs dehydration. Keep going as you’ve been doing. No reason for antibiotics. Sent urine and if it grows we’ll contact you, not expecting it though ’. NMP02: Participant to patient: ‘ From the electro-cardiac I’ll discuss with the doctor-in-charge. But it looks alright to me’. Different diagnosis considered. D09: Participant to patient: ‘ I think given pain is very likely to be muscular; don’t think chest or heart but waiting for blood tests. Your ECG is OK. If everything OK with blood tests, physio via GP . Can be trapped nerve, physio can help with that as well’. Empowering patient. D04: Participant to patient: ‘ Need to discuss with GP . The (medication) will even out your hormones. After a week see GP to arrange gynaecology and obstetrics appointment. Usually takes 2–3 week to even out’. NMP07 : Participant to patient: ‘ Come back to (hospital name) for TWOK (trial without catheter) with open mind on Friday. But on Friday they may not do it. Say to them I failed TWOK last time, I’d like to see the urologist. They’re elusive. I’ve bleeped them twice to talk to them. I’ve made sure the urine has been sent to lab. I’d like them to as you’re in to see them up there. I’d be forceful about that’. Disposition plan communicated. NMP08: Participant to patient: ‘ We don’t need to do anything more with you. We will discharge you. Have you someone who can collect you?. Go home. Get rest and lots of water. If anything changes go to your local GP or local hospital’. Alerted to ‘red flag’ symptoms. NMP02: Participant to patient: ‘ Come to see us if any of these things’. Shows section in leaflet to patient. Writes on the ointment box is giving to patient how much to use. ‘Apply for five days. If anything abnormal come and see us here’. a Staff uniform colours referenced in field notes: teal (resident doctor), blue (staff nurse), pink (registrar). Uniform colours vary between NHS trusts and trust departments. Appendix 7 TABLE 4 Summary of key changes in response to issues raised during iterative tool development View in own window Category Issue raised Changes made Seeking of advice or clinical decision from NMPs Inappropriate to assume clinical decision-making always involves NMPs seeking to discuss tasks and actions with others. A team colleague may ask for advice or a clinical decision from a NMP , particularly from those with specific expertise, for example, minor injuries, pharmacy. Inclusion of additional ‘Task and Actions’ category: ‘ Advice/clinical decision asked for by team colleague’. Diagnosis Difficult to differentiate between the tasks ‘ Differential diagnosis made’ , and ‘ Different diagnoses considered on the basis of information gathered’. ‘Different diagnoses’ considered hard to identify in observation as it relates to clinician thought processes, but important to include since junior clinicians thought to sometimes jump to an obvious diagnosis without discounting other diagnoses. ‘ Different diagnoses considered based on information gathered’ included as a task and ‘ differential diagnosis made’ removed. Context In the ‘Context data collected from Trust’ section, not clear whose criteria will be used to establish whether the patient and staff numbers collected are determined to be ‘ low ’, ‘medium’ or ‘ high ’. Divergent views on completion of the ‘Perspective of clinician observed’ section as to whether the questions be asked of the observed clinician by a researcher or self-completed by the clinician. Three-option choice for each question thought preferable to a scale. ‘Context data collected from Trust’ section to be requested by the researcher from the nurse-in-charge and clinician-in-charge. The ratings of ‘ low ’, ‘ medium ’ or ‘ high ’ to be based on Trust averages. The observed clinician to self-complete the ‘Perspective of clinician observed’ section: eight choice-based questions with three options. Clinician attitude and behaviour with regard to risk Important to capture how comfortable and confident the observed clinician perceives themselves to be in managing risk. NMPs working in minors may have a different risk profile than NMPs working in resus or paediatrics. Comparison could be made with observed behaviour; whether clinical risks are discussed with a team colleague or clinician-in-charge. The clinician to self-complete ‘Perspective of clinician observed’ section to include their perceptions on ‘ managing risk’ and ‘ managing uncertainty’. Definition of ‘patient’ Patient representatives highlighted that children and people without capacity may be accompanied in the ED/ UTC by a carer, relative, or guardian. Clinicians may involve them in discussions when the patient does not have capacity. Explained in the ‘User Guide’ that ‘ patient ’ includes carer/guardian/advocate and that the clinician maybe observed discussing and sharing decision-making with them as well as the patient. NMP grades and roles Wide variation in NMPs’ roles between NMPs at different grades and within and between EDs/UTCs. For example, a band 3 emergency NP may see just minor injuries while a band 8 ACP may have a leadership role. The title, grade, and role of observed NMPs to be recorded. Trust rules and protocols for supervision Entrustable tasks and activities are important for preparing NMPs for practice and are used to asses levels of supervision required. There is variation in NMPs' scope of practice between NMPs at different levels, and between different EDs/UTCs. Some types of patient are ‘ must discuss’ because of trust rules. The trust rules and protocols for supervision to be requested by the researchers for inclusion in the context data collected from the trust. Researcher guidance on use and completion of tool Tool considered potentially difficult to complete without guidance. A protocol thought helpful in order to clarify: how and where a clinician is to be observed type and level of NMPs and doctors-in-training to be observed how each section of the tool is to be completed and by whom how to record the same tasks and actions undertaken by observed clinicians for multiple patients meaning of ambiguous terms such as ‘ prompted change’ , ‘ directed change’ , and ‘ no change’ Detailed user guide developed setting out how the tool is to be used and completed, including: instructions for how each section of the tool should be filled out and by whom definitions of levels of interdependence explanation of the team members that different types of discussion might be undertaken with guidance on what is involved and included under different tasks and activities explanation that a number (1, 2, 3, etc.) be used to denote each patient alongside a tally/hash mark for each task and action Inter-observer reliability For inter-observer reliability would need two researchers doing the same thing at the same time but this was considered too obtrusive in the confined spaces of EDs/UTCs. Researcher adherence to the user guide plus structured debriefs to promote as similar as possible observation approaches among researchers together with calibration through practice. Nonetheless, inter-observer reliability was acknowledged as a limitation. About the Series Health and Social Care Delivery Research ISSN (Electronic): 2755-0079 Article history The contractual start date for this research was in March 2021. This article began editorial review in April 2025 and was accepted for publication in August 2025. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The Health and Social Care Delivery Research editors and publisher have tried to ensure the accuracy of the authors’ article and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this article. Last reviewed: April 2025; Accepted: August 2025. Copyright © 2026 Taylor et al . This work was produced by Taylor et al . under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This is an Open Access publication distributed under the terms of the Creative Commons Attribution CC BY 4.0 licence, which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. See: https://creativecommons.org/licenses/by/4.0/ . For attribution the title, original author(s), the publication source – NIHR Journals Library, and the DOI of the publication must be cited. Bookshelf ID: NBK621567 DOI: 10.3310/GJMH1818 Share Views PubReader Print View Cite this Page Taylor F, Halter M, M Drennan V, et al. Independence and interdependence of non-medical practitioners in the emergency care skill-mix (SKILLMix-ED): a multi-method study to develop a measurement tool [Internet]. Southampton (UK): National Institute for Health and Care Research; 2026 Apr 1. doi: 10.3310/GJMH1818 PDF version of this title (1.0M) In this Page Background Aim and objectives Methods Results Discussion Conclusion Additional information Glossary List of abbreviations References Eligibility criteria for independence and supervision review category Preparatory literature review search strategy Search strategy for additional terms after collaborative sessions Appendix 4 Appendix 5 Appendix 6 Appendix 7 Other titles in this collection Health and Social Care Delivery Research Related information NLM Catalog Related NLM Catalog Entries PMC PubMed Central citations PubMed Links to PubMed Recent Activity Clear Turn Off Turn On Independence and interdependence of non-medical practitioners in the emergency c... Independence and interdependence of non-medical practitioners in the emergency care skill-mix (SKILLMix-ED): a multi-method study to develop a measurement tool Your browsing activity is empty. Activity recording is turned off. Turn recording back on See more... Follow NCBI Twitter Facebook LinkedIn GitHub NCBI Insights Blog Connect with NLM Twitter Facebook Youtube National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894 Web Policies FOIA HHS Vulnerability Disclosure Help Accessibility Careers NLM NIH HHS USA.gov