2026-5-4
Linking Behaviour and Perception to Evaluate Meaningful Human Control over Partially Automated Driving Ashwin George1,2,*,+ , Lucas Elbert Suryana1,3,* , Lorenzo Flipse1,2 , Bart van Arem1,3 , David A. Abbink1,2 , Simeon Craig Calvert1,3 , Luciano Cavalcante Siebert1,4 and Arkady Zgonnikov1,2 1 Centre for Meaningful Human Control, 2 Department of Cognitive Robotics, 3 Department of Transport & Planning, 4 Department of
arXiv:2605.00556v1 [cs.HC] 1 May 2026
Intelligent Systems; Delft University of Technology, * Authors contributed equally and listed in alphabetical order, + Corresponding author: [email protected], Mekelweg 2, 2628 CD, Delft, Zuid Holland, The Netherlands.
Partial driving automation creates a tension: drivers remain legally responsible for vehicle behaviour, yet their active control is significantly reduced. This reduction undermines the engagement and sense of agency needed to intervene safely. Meaningful human control (MHC) has been proposed as a normative framework to address this tension. However, empirical methods for evaluating whether existing systems actually provide MHC remain underdeveloped. In this study, we investigated the extent to which drivers experience MHC when interacting with partially automated driving systems. Twenty-four drivers completed a simulator study involving silent automation failures under two modes haptic shared control (HSC) and traded control (TC). We derived behavioural metrics from telemetry data, subjective perception scores from post-trial surveys and used them to test hypothesised relations between them derived from the properties of systems under MHC. The confirmatory analysis showed a significant negative correlation between the perception of the automated vehicle (AV) understanding the driver and conflict in steering torques. An exploratory analysis also revealed a surprising positive correlation between reaction times and the perception of sufficient control. Qualitative feedback from open-ended post-experiment questionnaires revealed that mismatches in intentions between the driver and automation, lack of safety, and resistance to driver inputs contribute to the reduction of perceived MHC, while subtle haptic guidance aligned with driver intent had a positive effect. These findings suggest that future designs should prioritise effortless driver interventions, transparent communication of automation intent, and context-sensitive authority allocation to strengthen meaningful human control in partially automated driving.
1. Introduction Responsibility goes hand-in-hand with the level of control [1]. For example, the driver of a car has a much higher degree of responsibility and control over the vehicle as compared to a passenger in a train. This balance of responsibility and control can be disrupted when automation systems take up part of the control while the drivers of such vehicles are still held responsible for accidents [2]. Therefore, understanding driver’s perception of responsibility and control in partially automated vehicles is essential for designing vehicle automation. Vehicle automation is widely seen as a promising way to improve road safety and traffic efficiency [3]. Early deployments of fully automated systems have demonstrated benefits, such as reductions in specific crash types [4]. In practice, however, most vehicles today operate at SAE levels 2–3 [5], where automation handles some driving tasks while human drivers must supervise the system and intervene when necessary. These levels create a paradox: drivers are expected to remain constantly attentive even though their control role is significantly reduced [6]. One manifestation of this paradox is driver complacency, an over-reliance on automation in which drivers’ reactions slow when manual intervention is required due to reduced vigilance. This decline occurs when automation shifts drivers’ role from active control to passive supervision, reducing engagement [7]. In human factors terms, this reflects a vigilance decrement caused by cognitive underload during passive monitoring
Linking Behaviour and Perception to Evaluate Meaningful Human Control over Partially Automated Driving
Subjective Perception
Perceive
Act
Human reacts Human intention AV intention
Human - AV Interaction
Control
Act
Behavioural Metrics
Perceive
Haptic Shared Control
Control Modes
Both human and automation in control
Traded Control
Either human or automation in control
Figure 1 | Understanding the relationship between the subjective perception of human drivers and behavioural metrics observable to machines is vital for designing automated systems under meaningful human control. This study explores how different control modes — haptic share control (HSC) and traded control (TC) affect perceptions of control and responsibility when the human and automated system have conflicting reasons. [6, 8]. Such disengagement is critical in scenarios where the driver must retake control unexpectedly: if unprepared, the driver may respond too slowly, increasing accident risk. At the same time, some commercial systems are marketed as being capable of “autonomous driving”, even though manufacturers specify that drivers must remain ready to intervene when needed. These conflicting expectations reinforce the above paradox of automation: while the systems are presented as relieving the driver of control, safe operation still depends on sustained human vigilance raising questions about whether drivers can reasonably be assigned full moral and legal responsibility under these conditions. Beyond the paradox of automation, partial automation can also erode drivers’ felt responsibility for vehicle behaviour, even though they remain legally accountable. This reduced sense of responsibility is often explained through the sense of agency (SoA), the subjective experience of initiating and controlling an action. Prior work shows that SoA tends to decline as automation increases, with higher autonomy reducing awareness and engagement [9, 10, 11]. When individuals feel they did not cause an action, they are also less likely to accept moral responsibility for its outcomes [12]. Taken together, complacency and declining SoA create a layered challenge: systems expect drivers to remain attentive and accountable, yet their design undermines the very capacities required to do so. This mismatch produces misattribution of responsibility [13, 14], where it is unclear who should be responsible (and held accountable) for the actions of an automated system. Recent evaluations of commercial driving systems illustrate these gaps: drivers often lack understanding of how well the system can react (or how well they themselves can react) while manufacturers frequently distance themselves from responsibility through disclaimers [15]. To mitigate such responsibility gaps, meaningful human control (MHC) has been proposed as a normative stance that humans should maintain some form of control and responsibility over the behaviour of artificial systems, even if these systems perform tasks autonomously [16]. Automated systems (including driving automation) must meet two conditions to be under MHC — 1) the tracking condition which asserts that the behaviour of automated systems must track the reasons of relevant humans and 2) the tracing condition which asserts that it should be possible to trace the responsibility for the behaviours of the automated system to at least one human [16]. Based on these two fundamental conditions, subsequent work proposed a range of practice-oriented frameworks to operationalize tracking and tracing, such as actionable properties of Cavalcante Siebert et al. [17] or MHC operationalisation framework of Calvert [18].
2
Linking Behaviour and Perception to Evaluate Meaningful Human Control over Partially Automated Driving
While MHC has often been discussed as a guiding principle for system design [16, 19, 17, 18], a necessary first step is to evaluate whether an existing system satisfies its conditions in practice. Without such evaluation, MHC risks being practically under-validated. Previous studies have proposed several ways to evaluate MHC of specific systems [15, 20] or general classes of such systems [21], but assess it only indirectly. For example, MHC has been assessed indirectly through post-crash analyses [15] or user interviews [20]. However, evaluations based on post-crash analyses do not capture the nuances of real-time interaction between the driver and the automated system, while those based solely on user interviews rely on questions not specifically designed to measure MHC, even though partial alignment with MHC principles can be inferred. More generally, the current literature lacks methods that directly evaluate whether the interaction between human drivers and automated driving systems provides for meaningful human control. In this work, we address this gap by combining subjective and behavioural measures to evaluate meaningful human control in driver interactions with automated driving systems. Specifically, in our evaluation methodology (Fig. 1), we integrate detailed driving interaction data with a questionnaire explicitly designed to assess MHC based on the established properties [17]. This approach is inspired by Verhagen et al. [22], who combined subjective reports and interaction metrics to evaluate MHC in human-robot interaction for firefighting systems. To demonstrate our evaluation methodology, we focus on interaction strategies currently implemented in automated vehicles. We use two common interaction strategies, haptic shared control (HSC) and traded control (TC), as representative approaches to explore how such systems align with MHC in practice. In HSC, both the driver and the automation act simultaneously through force feedback on the steering wheel, supporting smoother collaboration and reducing conflict [23, 24, 25]. In contrast, TC relies on explicit handovers, which can be simpler to implement but risk disorientation if poorly timed [26]. With these two interaction strategies in mind, we address the following research questions (RQ): RQ1 To what extent do stereotypical strategies of human-automation interaction enable drivers to experience meaningful human control? RQ2 How are objective metrics associated with driver’s subjective experience of meaningful human control? We investigate these questions in a controlled driving simulator study where participants interact with an automated vehicle under both HSC and TC conditions in safety-critical situations. The main contributions of this paper are: • A methodology that integrates behavioural metrics, subjective questionnaires, and qualitative insights to evaluate how meaningful the control of human drivers is when interacting with driving automation • An experiment-based comparison of two established human-automation interaction strategies in terms of the driver’s perception of control and responsibility and the relation of those perceptions to observable behaviour.
2. Methods We conducted a driving simulator experiment (Section 2.1) and used surveys to gauge drivers’ subjective perception of interacting with driving automation through different control modes (Section 2.2). These subjective measurements were complemented with objective behavioural metrics derived from the telemetry data (Section 2.3), which were further analysed to study the influence of control modes, and relations between subjective perceptions and behavioural metrics (Section 2.4). The data and analysis scripts are publicly available along with supplementary materials1 . Twenty-four participants (13 male, 11 female) 23 to 36 years old (with average and SD 29.2 ± 3.75 years) were recruited from the student and research community at Delft University of Technology between December 2023 and April 2024 via flyers distributed through personal contacts and snowball sampling, where enrolled participants referred others. Eligible candidates must (i) have held a valid driving licence for at least one year, (ii) have normal or corrected-to-normal vision without spectacles (contact lenses were permitted), and (iii) have no history of epilepsy or other conditions that could be aggravated by virtual reality. The study protocol was approved by the Human Research Ethics Committee (HREC) of Delft University of Technology (ID: 111053). Written informed consent was obtained from all participants prior to the experiment. 1 Link to supplementary materials: https://osf.io/.. (will be updated after acceptance).
3
Linking Behaviour and Perception to Evaluate Meaningful Human Control over Partially Automated Driving
“Undetected” rider (collision target)
Ego vehicle Reference Trajectory Other vehicles
Figure 2 | Driving simulator experiment: Experimental setup: the fixed-base driving simulator with a virtual reality headset and haptic steering wheel, and the repeated overtaking scenario simulating a silent automation failure on a two-lane, two-way road. The ego vehicle (white car) overtakes other vehicles and motorcyclists in the presence oncoming traffic (yellow). To simulate silent automation failure (i.e., the system failing to detect a motorcyclist), the reference trajectory was used that crossed the path of a motorcyclist, leading to a collision in case the participant did not intervene. Only one motorcyclist served as the collision target, while the other travelled without conflict. The figure illustrates one representative configuration; the position of the vehicles and motorcycles, including the target motorcycle, were counterbalanced across trials using a Latin square design across trials. At the beginning of the experiment, participants were reminded of their right to withdraw at any time without penalty. A €10 voucher was provided to each participant upon completion of the experiment. All data were anonymised and stored using unique participant identification codes. 2.1. Driving simulator experiment The participants were asked to drive through a section of a rural road supported by driving automation; this was done repeatedly over a sequence of trials. In each trial, participants had to complete a sequence of overtaking manoeuvres in a bidirectional road, requiring them to repeatedly swerve into the opposite lane while managing oncoming traffic, while interacting with an automated driving system through a haptic steering wheel. Participants were presented a first-person view of the driving scene using Varjo VR-3 virtual reality headset while seated in front of a SensoWheel SD-LC steering wheel capable of providing high-fidelity haptic feedback (Fig. 2). The experiment was configured in JOAN [27] – the framework for running experiments in the CARLA environment [28]. Sony WH-1000XM3 noise-cancelling headphones were used to reduce auditory distractions. As the participants drove through a bidirectional rural road with straight and winding sections, the speed of the ego vehicle was fixed at 50 km/h under cruise control. Participants had no control over the accelerator or brake pedals, controlling only the steering wheel. This was done to isolate the lateral control task and ensure that participants’ behaviour primarily reflected their interaction with the steering-based haptic guidance, rather than confounding influences from longitudinal control. While driving, participants encountered right-hand traffic travelling in both directions at a constant speed of 40 km/h: nine vehicles travelling in the same direction, and five in the opposite direction (Fig. 2). Due to the speed difference, participants had to overtake the vehicles travelling in the same direction by swerving into the opposite lane. The vehicles were spaced in such a way that, with appropriate steering, it was possible for the participants to overtake all vehicles while maintaining
4
Linking Behaviour and Perception to Evaluate Meaningful Human Control over Partially Automated Driving
sufficient safety margins to oncoming vehicles. The experiment included three conditions: fully manual control (baseline; participants had complete control over steering), haptic shared control (HSC), and traded control (TC). In the two latter conditions, the steering was controlled by a simple automated driving system which used a pure pursuit controller to follow a pre-recorded reference trajectory. In HSC, both the automation and the driver could apply torques on the steering wheel at the same time and thus steer the vehicle together. In TC, if the driver applied a torque above a threshold (0.2 N m), the automation’s authority decreased to zero, giving the driver complete authority, and was only restored after the driver’s torque dropped below the threshold for more than one second. If the torque applied by the human remained below the threshold for more than one second, automation torque gradually increased until it regained full authority or was intervened upon by the driver. Both HSC and TC were implemented based on the four design choices architecture [29]. To familiarise themselves with the control modes and driving task, each participant completed four familiarisation trials: (i) manual driving without traffic, (ii) manual driving with traffic, (iii) driving with traffic under HSC, and (iv) driving with traffic under TC. The main experiment then consisted of nine trials: Trial 1 involved manual driving, while Trials 2–9 featured automated driving in either TC or HSC, with four trials per mode. The order of automated trials was randomized independently for each participant. To investigate participants’ interaction with automation in safety-critical scenarios, we simulated a silent automation failure in every automated driving trial. Specifically, the ego vehicle was programmed to follow a trajectory leading toward a potential side collision with a motorcycle (Fig. 2). Only one motorcycle served as the collision target per trial; its path intersected with the automation’s reference trajectory, while the other motorcycle travelled without conflict. The target motorcycle’s was counterbalanced with those of other vehicles and motorcycles across trials using a Latin square design, with position distributions balanced across participants and control modes to avoid learning effects. Participants were instructed to remain in their lane (unless overtaking is needed), avoid collisions, and stay on the road. They were also told: “Please note that the automation system is still in development and may falsely detect objects, potentially leading to collisions. You need to intervene when it behaves unsafely ”. Participant’s subjective perceptions were measured after each trial using a post-trial survey (Section 2.2.1). After completing all trials, a descriptive post-experiment questionnaire was used to qualitatively assess the subjective perceptions about the interaction (Section 2.2.2). 2.2. Subjective perception questionnaires Post-trial surveys with a rating scale were used to quantify the perception of the participants (Section 2.2.1) and open ended post-experiment surveys were used to get a deeper understanding of the cues that influenced to their perception (Section 2.2.2). 2.2.1. Post-trial subjective scores Seven Likert-scale (1 to 10) questions (Table 1) were asked after each HSC or TC trial to gauge how participants experienced the interaction with the driving automation (referred to as the automated vehicle (AV) in the questionnaires) and to ultimately assess the extent to which the driving automation operated under MHC. The questions were derived from the previously proposed actionable properties of systems under MHC [17]. Based on tracking and tracing conditions [16], these properties require that: P1 a moral operational design domain (moral ODD; extending the traditional notion of operational design domain towards relational and moral aspects) is clearly defined and mechanisms are in place for keeping the system in that domain; P2 representations which humans and automation have of each other, the environment, and the moral ODD, are shared, i.e., mutually compatible; P3 humans have sufficient ability and authority to control the outcome of the automation; P4 automation actions can be explicitly linked to at least one human who is aware of their responsibility. Since here we focused on the interaction between the human and the automation, rather than on the design of the automation itself, we designed the control modes HSC and TC to have identical moral operational design domains. Hence, in our comparison of HSC and TC we did not include any questions for Property 1. Property 5
Linking Behaviour and Perception to Evaluate Meaningful Human Control over Partially Automated Driving
Post-trial questions – automated driving trials (HSC and TC) A1: I felt that I had sufficient control over the automated vehicle A2: I felt that the automated vehicle understood my intentions during the driving task A3: I felt that I had sufficient understanding about the behaviours of automated vehicle A4: I felt that the automated vehicle and I were working together towards the same goal A5: I felt responsible for the driving task when I was using the automated vehicle A6: I felt safe in the automated vehicle during the driving task A7: I trusted the automated vehicle during the driving task
Property Sufficient control (P3) AV understood me (P2) I understood AV (P2) Working together (P2) Responsible (P4) Safe Trust
Post-trial questions – manual driving trials M1. I felt that I had sufficient control over the vehicle. M2. I felt responsible for the driving task when I was using the vehicle. M3. I felt safe in the vehicle during the driving task.
Sufficient control Responsible Safe
Table 1 | Post-trial questions that participants had to answer on a Likert scale (1 to 10) after interacting with the automated driving system through haptic shared control or traded control, and their relation to the properties of systems under meaningful human control [17]. Questions A6 and A7 were designed to assess perceived safety and trust, concepts that are related to MHC but do not map directly onto the four MHC properties. The questions for manual driving were redacted since there was no automation involved. 2 (shared representations) was divided into three components: AV understood me, I understood AV, and working together. This division reflects Cavalcante Siebert et al.’s [17] description that shared representations between human and AI systems depend on how both agents understand each other and are able to update their representations in response to changing reasons. The remaining properties were each represented by one questionnaire item: sufficient control for Property 3 (ability and authority to control) and responsible for Property 4 (actions are linked to a responsible human). In addition to items directly related to the MHC properties [17], we included questions on perceived safety and trust; as previous research has demonstrated, they can strongly influence evaluations of MHC based on how drivers experience interactions with driving automation [30]. As the manual driving trials did not include driving automation, only three of the above questions were applicable to these trials; different formulations were used to account for the absence of automation. Thus, answers to questions M1, M2 and M3 were used to evaluate participants’ baseline perception of sufficient control, responsibility and safety, respectively (Table 1). 2.2.2. Post-experiment open-ended questions After all the driving trials, an open-ended post-experiment questionnaire was administered to get a deeper understanding of the perceptions of the participant and the factors that might have influenced these perceptions. The questions related to the same concepts mentioned in post-trial questions (sufficient control, AV understood me, I understood AV, working together responsibility, safety, and trust). Two questions per concept asked the participant to recall positive and negative experience related to that concept. For example, the questions related to sufficient control were: D1: “Were there any situations where you felt you had sufficient control over the automated vehicle operation? Please describe.” and, D2: “Were there any situations where you felt you did not have sufficient control over the automated vehicle operation? Please describe.” An exhaustive list of the questions is included in the supplementary material. These items were designed to record in greater detail the overall impressions the participants had of HSC and TC, to elicit scenarios that positively or negatively affected their perception of the qualities being measured in the post-trial questionnaire, and to compare their experiences across conditions. In particular, they addressed aspects that could not be evaluated meaningfully on a trial-by-trial basis, such as general system preference, system characteristics that affect workload, and qualitative reflections on system behaviour. This approach
6
Linking Behaviour and Perception to Evaluate Meaningful Human Control over Partially Automated Driving
follows [22] in employing post-experiment reflections to evaluate MHC. In our study, such qualitative insights provide valuable context for understanding the Likert-scale scores recorded after each trial. 2.3. Behavioural metrics and hypotheses To study how subjective perceptions are related to behavioural aspects of driver-automation interaction, we formulated trial-level behavioural metrics that quantify interaction dynamics between the driver and the automation, driver’s performance, and the characteristics of vehicle trajectories. We then formulated hypotheses regarding a) correlation of these metrics to certain subjective perception scores, and b) differences in these metrics between HSC and TC (Fig. 3). Brief descriptions of metrics and the hypotheses linked to them are mentioned below; all metrics except reaction time and time to collision were calculated by aggregating the data over the whole duration of each trial. Detailed definitions of each metric are provided in supplementary materials. Reaction time: In experiment trials where a simulated silent automation failure triggered an incursion manoeuvre towards a motorcyclist, the reaction time was quantified as the time between the initiation of the automation manoeuvre and the first detectable human response. The automation initiation was identified as the first moment when the gradient of the automation torque exceeded a threshold (0.18 Nm/s), while the human response was determined based on conflict onset in HSC and authority in TC. We hypothesised that reaction times would be negatively correlated with subjective scores for item A1 (‘sufficient control’). Because of the continuous nature of the interaction in HSC, we hypothesised that reaction times would be shorter for HSC than for TC. Average conflict in steering torques: In the experiment, the human driver interacted with the automation through the steering wheel. We used the conflict in steering torques — defined as the negative signed product of human and automation applied torques that exceed a threshold (0.2 Nm) — to measure the disagreement between the human and the automation. In line with prior research [31], we hypothesised that conflict is negatively correlated with subjective scores A2, A3, and A4 (‘AV understood me’, ‘I understood AV’, and ‘working together’, respectively). Based on the assumption that interactions with HSC will be smoother, we hypothesised that conflict will be lower for HSC than for TC. Maximum steering torque: When interacting with the automation, the steering torque exerted by the participant reflects their control effort. Thus, we used maximum value of the human-applied torque during the trial as a metric of effort. Earlier studies have shown that drivers exert maximum torque when resisting lane-keeping or lane-departure assistance [32]. Thus, we hypothesised that maximum steering torques are negatively correlated with scores of ‘sufficient control’ (A1) and ‘working together’ (A4). Due to the smoothness of the interaction in HSC, we hypothesised that maximum steering torques will be lower for HSC than in TC. Steering reversal rate: Another established method for quantifying the control effort of a driver considers steering reversals: the greater the rate of steering reversals, the greater the control effort of the driver [33]. Steering reversal rate was computed as the number of steering reversals per second across a full trial, where a reversal is counted when the rate of change of steering angle is zero and the difference between two extremes exceeds 1° [34]. We hypothesised that the steering reversal rate is negatively correlated with the scores A1, A3, and A4 (‘sufficient control’, ‘I understood AV’ and ‘working together’, respectively). Based on the continuous nature of HSC, we hypothesised that the steering reversal rate would be lower for HSC than for TC. Jerk of vehicle trajectories: The smoothness of the trajectories of the ego vehicle was quantified as the rootmean-squared jerk (longitudinal and lateral jerks combined) computed over the whole trial. We hypothesised that trajectory jerk would be negatively correlated with the score A1 (‘sufficient control’). Also, assuming that HSC would result in smoother trajectories, we hypothesised that jerk would be smaller for HSC than TC. Time to collision (TTC): In each trial where a simulated silent automation failure could lead to a collision with a motorcyclist, the minimum TTC between the ego vehicle and the motorcyclist can be used to measure the criticality of the interaction: the lower the TTC, the more critical the interaction is. The TTC was calculated using a two-dimensional TTC algorithm [35], with the minimum taken over the duration of the failure event. We hypothesised that low min-TTC values (higher interaction criticality) would be associated with reduced perception of ‘sufficient control’; that is, we expected a positive correlation between TTC and A1. Furthermore, as the continuous engagement of the driver can reduce the criticality of the interaction in HSC, we hypothesised that TTC would be larger for HSC than for TC . 7
Linking Behaviour and Perception to Evaluate Meaningful Human Control over Partially Automated Driving
Property 1
Property 2
Moral operational design domain
MHC Properties
Shared representations
Property 3
Property 4
Ability and authority to control
Actions are linked to a responsible human
[17] Questions based on properties
Subjective perception
Sufficient control
AV understood me
I understood AV
Working together
Responsible
Hypothesised relations between subjective perception and objective metrics
+
-
Behavioural Metrics
Reaction time
Conflict
Maximum steering torque
Steering reversal rate
Jerk
Time to Collision
Take overs
Trajectory Deviation
HSC < TC
HSC < TC
HSC < TC
HSC < TC
HSC < TC
HSC > TC
HSC > TC
HSC > TC
Hypothesised relations between objective metrics and control modes
Figure 3 | Deriving hypotheses about survey questions and behavioural metrics from the properties of systems under meaningful human control (MHC): Survey questions related to MHC properties were used to evaluate subjective perception of MHC. The figure also shows the hypothesised dependence of subjective perception of MHC on behavioural metrics and control modes. Number of takeovers: When a driver is interacting with the automation through TC, a takeover happens if the torque applied by the driver exceeds the pre-defined threshold which would disengage the automation. For HSC, a takeover was counted each time the conflict signal transitioned from zero to positive. The total number of such events was summed per trial. We assumed that takeovers would be triggered when the behaviour of the automation was not aligned with the intentions of the driver. Hence, we hypothesised that the number of takeovers would be negatively correlated with scores A2 (‘AV understood me’) and A4 (‘working together’). At the same time, we hypothesised that the number of takeovers would be positively correlated with the score A5 (‘responsible’), as the driver would have more influence on the trajectory of the ego vehicle with more takeovers. We hypothesised that participants would be more engaged in HSC and that the number of takeovers would be greater in the case of HSC than for TC. Trajectory deviation: The deviation of the trajectory of the ego vehicle from the reference trajectory can be used to capture the extent to which the driver contributed to the behaviour of the vehicle. As an example, if the trajectory doesn’t deviate at all from the reference, that would mean that the driver fully relied on automation. This was quantified as the root-mean-squared error between the actual and reference trajectories, computed over the whole duration of a trial. The reference point for each time step was taken as the nearest point on the automation’s reference trajectory. We hypothesised that root-mean-squared trajectory deviation is positively correlated with the score A5 (‘responsible’). Also, assuming that drivers are more engaged in HSC, we hypothesised that trajectory deviation is higher for HSC than TC. Overtaking time: To quantify how driver’s preferences might deviate from the reference trajectory, we also included the overtaking time defined as the total time spent by the ego vehicle in the opposite lane during a trial. Since the preferences of individual drivers might lead to longer or shorter overtaking times, we did not associate any hypotheses with the this metric; it was only used for exploratory analyses. By providing objective insight into the behaviour of drivers, these metrics complement the data collected about the subjective perception of various concepts related to MHC. Detailed definitions of these metrics can be found in supplementary materials.
8
Linking Behaviour and Perception to Evaluate Meaningful Human Control over Partially Automated Driving
2.4. Analysis Behavioural data and subjective scores collected during the experiment were analysed quantitatively to test hypotheses (Section 2.4.1) and the answers to post-experiment questionnaires were analysed qualitatively to garner deeper insights into the factors affecting them (Section 2.4.2). 2.4.1. Quantitative analysis To analyse the effect of the control modes (HSC and TC) on the behaviour of participants, we fit linear mixed-effect models (LMMs) for each behavioural metric (Section 2.3): behavioural-metric ∼ control-mode + (1|participant) . Here, behavioural metrics (z-scored) were the dependent variables, control mode was the independent variable with a fixed effect, and participant ID was a random intercept. To examine how participant’s perceptions during the experiment were related to their behaviour and control modes, we analysed the post-trial subjective scores (Section 2.2.1) as a function of control mode and behavioural metrics. For each subjective perception item, one LMM was fit with the subjective score as the dependent variable, control modes and all behavioural metrics as fixed-effect independent variables, and participant ID as a random intercept: subjective-score ∼ all-behavioural-metrics + control-mode + (1|participant) . Control mode is included as a fixed effect to account for the effect of interaction type on subjective scores. The results of these LMMs were used in the confirmatory analyses of our hypotheses about the relationships between behavioural metrics and subjective scores (Fig. 3). For the confirmatory analyses, we used the statistical significance level 𝛼 = 0.05. In addition to testing our hypotheses, we conducted an exploratory analysis (based on the above LMMs) to identify potential relations between subjective scores and behavioural metrics that might have been overlooked in the hypotheses. The purpose of the exploratory analysis was not to make further claims, but to highlight possible relationships to investigate in future experiments. In our initial hypotheses, we had assumed that the relationships between subjective scores and behavioural metrics would be the same across control modes. However, some responses to the post-experiment questionnaire hinted that the nature of the interaction, as determined by the control mode, might also influence the relationships between subjective scores and behavioural metrics. To study whether control modes influence these fixed effects, we split the data and fit separate LMMs for each control mode. For the exploratory analysis, statistical significance of slopes was assessed at 𝑝 < 0.05. No correction for multiple comparisons was applied as we prioritised minimizing Type-II errors (false negatives) over Type-I errors (false positives). We did not formulate hypotheses about the effect of control mode on subjective scores; these relationships were therefore explored as part of the exploratory analysis only. 2.4.2. Qualitative analysis To analyse participants’ responses to the post-experiment questionnaire, we categorised their free-text answers into thematic topics using a two-coder approach. Both coders reviewed participants’ responses in their entirety to ensure familiarity with the dataset. An initial coding pass was conducted by one coder to identify all possible topics mentioned by participants, without restricting the scope to pre-defined categories. This exploratory, data-driven approach allowed for the inclusion of topics beyond those anticipated by the MHC framework. Following the initial coding, similar topics were consolidated into broader thematic categories. Within each category, related ideas were organised into subtopics that together captured the range of participant experiences. This hierarchical structure enabled the preservation of nuanced perspectives while reducing redundancy and complexity in the dataset. Once the preliminary categorisation was complete, a second coder independently reviewed the grouped topics. Both coders then developed labels and classifications for each factor, determining the most appropriate and precise naming. This independent labelling stage ensured that classification decisions were made without bias from prior discussion. 9
Linking Behaviour and Perception to Evaluate Meaningful Human Control over Partially Automated Driving
Reaction time b = 1.33, p < 0.001
ns
3 Z-scored metrics
Maximum Steering Torque b = 0.57, p < 0.001
Conflict b = -0.10, p = 0.42
Steering reversal rate b = -0.00, p = 0.99
Take overs b = -1.37, p < 0.001
TTC b = -0.35, p = 0.032
ns
***
***
Jerk b = -0.10, p = 0.36
Trajectory deviation b = -0.10, p = 0.48
Overtaking Time b = 0.46, p < 0.001
ns
***
*** ns
2
*
1 0 1 2 3
HSC
TC
HSC
TC
HSC
TC
HSC
TC
HSC
TC
HSC
TC
HSC
TC
HSC
TC
HSC
TC
(a) Control Modes Control Modes Control Modes Control Modes Control Modes Control Modes Control Modes Control Modes Control Modes Sufficient control b = -1.03, p < 0.001
Subjective Score
10
ns
I understood AV b = -0.85, p < 0.001
***
Working together b = -0.79, p < 0.01
**
Responsible b = -0.28, p = 0.21
Safe b = -1.17, p < 0.001
Manual HSC TC
Manual HSC TC
ns
***
Trust b = -0.68, p = 0.015
*
8 6 4 2
(b)
***
AV understood me b = -0.38, p = 0.15
Manual HSC TC
Control Modes
HSC
TC
Control Modes
HSC
TC
Control Modes
HSC
TC
Control Modes
Control Modes
Control Modes
HSC
TC
Control Modes
Figure 4 | The effects of control mode on behavioral metrics (a) and subjective perception scores (b). Statistical comparisons in panel (a) are based on confirmatory analyses using linear mixed-effect models (Section 2.4.1; Table 2). Comparisons in panel (b) are exploratory and indicated for reference only. Statistical tests are based on LMMs used for testing the relationships between behavioural metrics and subjective scores (Section 2.4.1) which also included control mode as the fixed effect; significance codes: ∗∗∗ 0.001 ∗∗ 0.01 ∗ 0.05. For visualization purposes, outliers (z-scores > 3) were excluded in panel (a). To assess coding reliability, both coders independently evaluated the presence or absence of each identified factor within participant responses. Each factor–response pair constituted one coded item for the reliability analysis. Inter-coder agreement was quantified using Cohen’s kappa [36] and Gwet’s AC1 [37]. See supplementary materials on osf.io for more details about the calculation of these metrics. Discrepancies between coders were resolved through discussion until full consensus was achieved. This consensus-based, investigator-triangulation approach follows established best practices for directed content analysis [38, 39, 40].
3. Results 3.1. Quantitative findings The quantitative results are reported in Tables 2 and 3 and Figs. 4 and 5. The confirmatory analysis tested two sets of hypotheses: the effect of control mode on each behavioural metric (Fig. 4 and Table 2), and the relationships between behavioural metrics and subjective scores (Table 3). The exploratory analysis examined additional relationships between behavioural metrics and subjective scores that were not part of the original hypotheses (Fig. 5). For each behavioural metric below, we first report the confirmatory relationship with control mode, then with subjective score, and finally any additional findings.
10
Linking Behaviour and Perception to Evaluate Meaningful Human Control over Partially Automated Driving
Behavioural Metric Reaction time Conflict Maximum steering torque Steering reversal rate Jerk TTC Takeovers Trajectory deviation
Hypothesised relation
Observed relation
HSC < TC HSC < TC HSC < TC HSC < TC HSC < TC HSC > TC HSC > TC HSC > TC
< <
>
Slope 𝛽
p-value
1.33 -0.10 0.57 -0.00 -0.10 -0.35 -1.37 -0.10
< 0.001∗∗∗ 0.42 < 0.001∗∗∗ 0.99 0.36 0.03 ∗ < 0.001∗∗∗ 0.48
Hypothesis accepted ✓ ✓
✓ ✓
Table 2 | Confirmatory analysis – behavioural metrics vs control modes: Hypothesised relations and results of statistical tests based on LMMs: behavioural-metric ∼ control-mode + (1|participant) (Section 2.4.1); significance codes: ∗∗∗ 0.001 ∗∗ 0.01 ∗ 0.05. Reaction time: In accordance with our hypothesis, reaction times in simulated automation failures were significantly lower for HSC than for TC ( 𝛽 = 1.33, 𝑝 < 0.001). However, contrary to our hypothesis, reaction times were positively correlated with the score for ‘sufficient control’ ( 𝛽 = 0.24, 𝑝 = 0.01). The exploratory analysis also revealed a positive correlation between reaction time and the score of ‘responsible’ for HSC ( 𝛽 = 0.18, 𝑝 = 0.04). Conflict in steering torques: There was no evidence for the hypothesized difference in conflict in steering torques between HSC and TC ( 𝛽 = −0.1, 𝑝 = 0.42). At the same time, the hypothesis that conflict in steering is negatively correlated with the score for ‘AV understood me’ was confirmed ( 𝛽 = −0.34, 𝑝 = 0.001). The exploratory analysis suggested that conflict might be positively correlated with the score of ‘sufficient control’ for HSC ( 𝛽 = 0.39, 𝑝 = 0.01) and negatively correlated with the score of ‘responsible’ for TC ( 𝛽 = −0.27, 𝑝 = 0.03). Maximum steering torque: Consistent with our hypothesis, maximum steering torque was lower for HSC than for TC ( 𝛽 = 0.57, 𝑝 < 0.001). The exploratory analysis indicated a possible positive correlation between maximum steering torque and the score of ‘responsible’ for TC ( 𝛽 = 0.235, 𝑝 = 0.04). Steering reversal rate: There was no evidence for relationships between steering reversal rate and control mode or subjective scores. Jerk of vehicle trajectories: The data showed no evidence for differences in the jerk of vehicle trajectories between HSC and TC. None of the hypothesised correlations of jerk with subjective scores were substantiated in the confirmatory analysis. The exploratory analysis showed that trajectory jerk could be negatively correlated with the score of ‘safe’ ( 𝛽 = −0.22, 𝑝 = 0.01), especially for TC ( 𝛽 = −0.314, 𝑝 = 0.01). Time to collision: The data did not show a significant difference in minimum TTC during silent automation failure between the two control modes. There was no evidence for the hypothesised correlation between TTC and the score of ‘sufficient control’ either. However, the exploratory analysis suggested a positive correlation between minimum TTC and the score for ‘AV understood me’ for TC ( 𝛽 = −0.253, 𝑝 = 0.03). Number of takeovers: As per our hypothesis, the number of takeovers was larger for HSC than for TC ( 𝛽 = −1.37, 𝑝 < 0.001). No evidence was however found for correlations between the number of takeovers and any of the subjective scores. Trajectory deviation: No evidence was found for difference in deviation of the vehicle trajectory from prerecorded reference between HSC and TC. The confirmatory analysis did not yield evidence for any of the hypothesised correlations of trajectory deviation with subjective scores. The exploratory analysis indicated that trajectory deviation can be negatively correlated with the subjective score for ‘sufficient control’ ( 𝛽 = −0.19, 𝑝 = 0.02) and positively correlated with the score for ‘trust’ ( 𝛽 = 0.24, 𝑝 = 0.01), especially for TC ( 𝛽 = 0.29, 𝑝 = 0.05). Overtaking time: Overtaking time was not part of our hypotheses; exploratory analyses suggested that overtaking time was larger for TC than for HSC ( 𝛽 = 0.46, 𝑝 < 0.001). Overtaking time might be positively
11
Linking Behaviour and Perception to Evaluate Meaningful Human Control over Partially Automated Driving
Hypothesised Slope 𝛽 relation
p-value
Subjective score
Behavioural metric
Sufficient control
Reaction time Maximum steering torque Steering reversal rate Jerk TTC
+
0.238 0.038 -0.178 -0.148 0.033
0.014 ∗ 0.666 0.086 0.083 0.621
AV understood me
Conflict Takeovers
-
-0.344 -0.019
0.001∗∗∗ 0.880
I understood AV
Conflict Steering reversal rate
-
-0.130 -0.038
0.203 0.720
Working together
Conflict Maximum steering torque Steering reversal rate Takeovers
-
-0.110 -0.080 0.061 -0.176
0.307 0.402 0.565 0.167
Responsible
Takeovers Trajectory deviation
+ +
0.182 -0.022
0.900 0.769
Hypothesis accepted
✓
Table 3 | Confirmatory analysis — subjective scores vs behavioural metrics: Hypothesised relations and results of statistical tests based on LMMs subjective-score ∼ behavioural-metric+control_mode+ (1|participant) (Section 2.4.1); significance codes: ∗∗∗ 0.001 ∗∗ 0.01 ∗ 0.05. ‘+’ indicates that the perception score was hypothesized to be positively correlated with the corresponding behavioural metric; ‘-’ indicates hypothesised negative correlation. correlated with subjective score for ’safety’ ( 𝛽 = 0.22, 𝑝 = 0.04) and ’trust’ ( 𝛽 = 0.24, 𝑝 = 0.05). Additionally, for HSC, we found potential positive correlation between overtaking time and the score for ‘sufficient control’ ( 𝛽 = 0.35, 𝑝 = 0.01). 3.2. Qualitative findings We performed a topic analysis of participants’ responses to the post-experiment questionnaire to gain deeper insight into the factors that influenced their subjective experience by categorising those experiences into thematic topics. As part of the qualitative analysis, inter-coder agreement was assessed on a set of 57 coded factors. The observed agreement between coders was 66.67%, with Cohen’s kappa [36] of –0.1993 and Gwet’s AC1 [37] of 0.5385. The negative kappa value was attributable to the absence of cases where both coders labeled a factor as absent (see contingency table in supplementary materials for details), which can produce prevalence bias in kappa calculations [41]. The agreement between both coders was on the presence of a factor in 38 cases, with disagreement in 19 cases (nine coded as present by Coder 1 only, ten coded as present by Coder 2 only), and no instances where both coded the factor as absent. Given these conditions, Gwet’s AC1 was taken as the more robust measure of agreement. Full consensus was reached after discussions between coders consolidated overlapping factors and resolved disagreements by jointly evaluating the arguments presented by each coder. This process resulted in 47 final factors that either positively or negatively affected perceptions related to MHC. These factors along with remarks for control modes are summarised in Table 4. We classified the factors into three categories: (1) attitudinal factors ○ relating to mental states of participants, (2) interaction factors ■ relating to human-AV interaction through the steering wheel, and (3) trajectory factors ▶ relating to the motion of vehicles. We follow this categorisation when describing the findings for each type of perception. Participant quotes are referenced by anonymised IDs (e.g., ID61, ID90); these are randomised identifiers and do not reflect the sequential numbering of the 24 participants.
12
Linking Behaviour and Perception to Evaluate Meaningful Human Control over Partially Automated Driving
- H U N
7 7 &
7 D N H R Y H U V
7 U D M H F W R U \ G H Y L D W L R Q
2 Y H U W D N L Q J 7 L P H
6 X E M H F W L Y H V F R U H
7 U X V
6 D
7 U X V
6 D
W
I H
7 U X V
6 D
6 X I $ 9 F R Q I L F L H Q X Q W U R W G H O U V W , X P R R G Q G H H U V W R $ R G : R 9 W R J U N H W L Q 5 H K H U J V S R Q V L E O H
6 X E M H F W L Y H V F R U H
W
I H
& R Q I O L F W 0 D [ L P X P 6 W H H U L Q J 7 R U T X H 6 W H H U L Q J U H Y H U V D O U D W H
6 X I $ 9 F R Q I L F L H Q X Q W U R W G H O U V W , X P R R G Q G H H U V W R $ R G : R 9 W R J U N H W L Q 5 H K H U J V S R Q V L E O H
W
I H
7 U D G H G & R Q W U R O
6 X I $ 9 F R Q I L F L H Q X Q W U R W G H O U V W , X P R R G Q G H H U V W R $ R G : R 9 W R J U N H W L Q 5 H K H U J V S R Q V L E O H
+ 6 & 7 &
% H K D Y L R X U D O P H W U L F V
+ D S W L F 6 K D U H G & R Q W U R O
5 H D F W L R Q W L P H
6 X E M H F W L Y H V F R U H
Figure 5 | Exploratory analysis — Subjective scores vs behavioural metrics: The slope coefficients for each metric obtained from the linear mixed-model regressions with the formula: subjective-score ∼ all-behavioural-metrics + control-mode + (1|participant); p<0.05 is indicated by ∗ . Sufficient Control: Within the epistemic dimension, participants reported that HSC gave them a greater sense of control, describing that in most cases they felt they had sufficient control over the AV’s operation (ID61, ID90). Regarding trajectory-related factors, one participant noted inability to influence vehicle speed as a factor that reduced their perception of having sufficient control (ID58). In terms of interaction, the ease of overriding the AV’s actions and continuous access to the steering wheel were identified as factors that enhanced the perception of control (ID17, ID58, ID172). Conversely, steering forces against driver intention in both TC and HSC (ID106, ID203) created a sense that control was being taken away rather than shared. Furthermore, readaptation effort after abrupt takeovers (ID61, ID90, ID172, ID242), need for continuous vigilance (ID19), and invasive forces in TC (ID203) were cited as reducing the sense of control. AV understood me: Perception of whether the AV understood the driver often depended on its ability to anticipate intentions accurately and act safely. In terms of attitudinal factors, TC was sometimes linked to unsafe lane changes near adjacent motorcycles (ID106, ID172). In terms of trajectory-related factors, participants reported that accurate anticipation of manoeuvre intentions, such as overtaking or aligning within the lane, they felt the AV understood them (ID167, ID172). Conversely, mismatched anticipation, such as returning to the lane too early and creating a potential crash situation, made them feel misunderstood (ID229). Finally, from the perspective of human–automation interaction, low steering resistance (ID17) and immediate handover upon intervention (ID61) were taken as behaviours that aligned with the perception that the system understood the driver. Conversely, strong steering resistance against driver preferences (ID264) reduced understanding. I understood AV: Understanding of the AV’s behaviour was often shaped by the predictability of its actions. In terms of attitudinal factors, participants noted that familiarity with how HSC and TC operation, gained during the experiment, made them understand the AV’s intentions (ID17, ID58). As for trajectory-related factors, several negative experiences were reported, including erratic deviations from the reference trajectory, which could occur when driver inputs conflicted with the automation’s path-following controller (ID302),unsafe manoeuvres that could lead to a crash (ID454), and mismatched manoeuvre timing that were either too early or too late (ID137, ID203). These actions influenced participants’ confidence in anticipating the AV’s next move. Regarding human–automation interaction, subtle torque feedback was perceived as a helpful indicator for understanding the AV’s intentions (ID58, ID61), while overly strong torque feedback (ID172) or unnecessary steering jerks (ID17) were seen as intrusive factors.
13
Linking Behaviour and Perception to Evaluate Meaningful Human Control over Partially Automated Driving
Positive
Negative
Control mode
Sufficient Control ○ Need for continuous vigilance
■ Continuous access to control (HSC) ■ Abruptness of takeovers (TC) ■ Override capability ■ Force against driver intention ■ Control overshoots ■ Disruptions in fluency
○ Greater sense of
control for HSC ■ Forces in TC more invasive
AV understood me ○ Anticipation of intention
■ Subtle torque feedback ■ Low steering resistance ■ Override capability (HSC) ▶ Timing ▶ Smooth lane alignment
○ Mismatched anticipation of intention ○ Lack of safety (TC) ■ High steering resistance
▶ Mismatched timing
I understood AV ○ Transparency of AV intention
○ Lack of transparency of AV intention
■ Clear takeovers ■ Subtle torque feedback
○ Lack of safety ■ Stronger torque feedback ■ Unnecessary steering jerks ▶ Erratic trajectories ▶ Mismatched timing
Working together ○ Effortless coordination (HSC)
○ Drivers preferring
○ Shared direction (HSC)
autonomy, preferred HSC
○ Safety
■ Subtle AV corrections matching driver intent
○ Lack of safety ■ Driver corrections due to mismatched intent
Responsible ○ Need for supervision (TC),
& Expectation to correct (HSC) ○ Different driving style ○ Safety-critical situations ○ Novelty and untrustworthiness ○ Accountability (moral, legal) ■ Low steering resistance
○ Redundancy of human input
○ Control mode
defined responsibility ○ Shared-intent moments (TC) ○ Monotonous driving
■ Physically disengaged (TC) ■ Force against driver intention (HSC)
Table 4 | Factors affecting the subjective perception: The qualitative analysis of post-experiment answers revealed factors that participants believed to positively and negatively affect their subjective perception. Remarks pertaining to the nature of the control modes were also summarised. These factors were classified into three categories as 1) those pertaining to attitudinal ○ factors, 2) those pertaining to the interaction ■ between the human and the automation, and 3) those pertaining to the trajectory ▶ of the vehicle. Similar themes repeated across columns are mentioned in the same row whenever possible. Control modes in parentheses, (HSC) and (TC), indicate factors that were associated with a specific control mode. 14
Linking Behaviour and Perception to Evaluate Meaningful Human Control over Partially Automated Driving
Working together: Among attitudinal factors, the perceived sense of cooperation between the driver and the AV was associated with HSC (ID172). In trajectory-related factors, safe manoeuvres such as avoiding a collision with the vehicle in front (ID227) were viewed as signs of effective teamwork, whereas unsafe manoeuvres that could potentially lead to a crash with other vehicles undermined this perception (ID224). For human–automation interaction, subtle steering corrections provided by the AV (ID224), minimal override pressure in HSC (ID17), and sense of shared execution (ID40) reinforced the feeling that participants and the AV were working together. However, driver corrections due to mismatched intent were seen as undermining the feeling of working together with the automation systems (ID58). Responsibility: Participants’ awareness of responsibility was shaped by how they understood their role, the driving situation, and the nature of their interaction with the AV. In terms of attitudinal factors, recognition of operational role influences awareness of responsibility to supervise the automation as supervisor in TC and as driving as manual mode in HSC (ID61), both of which positively influenced their awareness of responsibility. Others acknowledged knowing they were the responsible party even when they were not actively controlling the vehicle (ID172). Counterintuitively, low level of trust in the system positively influenced participants’ perception of responsibility, with some attributing this to their first time driving such a system (ID203). However, perception of partial responsibility, in which the participant could intervene with the AV but also recognised that the AV could complete the manoeuvre without their input, led to ambiguity that reduced their feeling of responsibility (ID17). As for trajectory-related factors, safety-critical situations such as overtaking (ID430), led to a stronger perception of responsibility, while monotonous driving on a straight road (ID278) and low-traffic situations (ID203) were associated with feeling less responsible. Finally, for human-automation interaction, both HSC and TC were reported as causing negative perceptions of responsibility. In TC, passive disengagement from the steering wheel reduced the sense of responsibility (ID90). In HSC, feeling forced into certain moves made them feel less responsible (ID17).
4. Discussion This study set out to examine how meaningful human control can be evaluated in the context of partially automated driving, with a particular focus on linking objective behavioural measures to drivers’ subjective experience. Rather than treating control as a purely technical property of the system, we approach meaningful human control as something that emerges through human–automation interaction and is reflected in both observable behaviour and perceived experience. In doing so, we introduced a mixed-methods approach that enables these different perspectives to be analysed in relation to one another. The following discussion synthesises our quantitative and qualitative findings and reflects on how the MHC approach can inform the understanding, design, and evaluation of human–automation interaction. 4.1. Factors related to subjective perception of control and responsibility 4.1.1. Sufficient Control (MHC Property 3) None of our hypotheses regarding correlation of behavioural metrics with the perception of ‘sufficient control’ were accepted in the confirmatory analysis. Interestingly, we found a positive association between the reaction time in silent automation failures and the subjective score for ’sufficient control’; this contradicted our hypothesis, as we originally assumed that faster reaction times would be associated with stronger perception of control. However, stronger perception of control might have made participants more complacent causing them to react late as found in previous studies [42, 43]. This would mean that participants would have greater reaction times for the control mode for which they had greater sense of control. However, when comparing the control modes, HSC has a higher score for ‘sufficient control’ and a lower reaction time than TC, and thereby contradicts the complacency explanation. Another plausible explanation might lie in the dynamics of the interaction: when participants react early, they have to make smaller corrections than when they react late, which might lead to greater sense of control in cases where they react late. This presents an interesting contradiction where humans might be getting greater sense of agency for late reactions of greater magnitude than smaller reactions of small magnitude. Future research can explore this dynamic with regard to other factors associated with the ‘sense of agency’ such as sensory attenuation (where agents perceive their own actions to be of a smaller magnitude) and intentional binding (where agents perceive that the time interval between them doing some action and its outcome is smaller than the actual time 15
Linking Behaviour and Perception to Evaluate Meaningful Human Control over Partially Automated Driving
elapsed) [44, 45]. Sensory attenuation might cause humans to feel that they are not doing much when they are performing small actions when reacting fast [46]. Intentional binding might be caused by the additional cognitive load triggered when humans are being active [47], which might also explain why reaction times are higher when they are active. 4.1.2. AV understood me (MHC Property 2) In accordance with our hypothesis, the score for ‘AV understood me’ was negatively correlated with conflict in steering torques. This is consistent with participants’ qualitative reports that force against their intention or invasive steering torque undermined the feeling that the AV understood them. In other words, when drivers experienced strong resistance from the automation, they interpreted it as misalignment of intent. Furthermore, the exploratory analysis for TC alone revealed a possible positive correlation between time to collision (TTC) and the score for ‘AV understood me’. This correlation with TTC, along with the qualitative responses mentioning the negative impact of ‘mismatched intentions’ and ‘mismatched timing’, suggests that the perception of ‘AV understood me’ is sensitive to event-level responsiveness of the automation system and that it might vary within trials. Thus, instead of aggregating perception scores and behavioural metrics per trial, future work could explore how perception varies in real time with respect to behavioural signals of finer temporal resolution. Also, it must be noted that in our experiment, the AV never truly ‘understands’ the driver. The controller operates with fixed parameters that do not adapt to individual drivers during interaction. This means that the perception of ‘AV understood me’ reflects how well the design of the automation aligned with driver expectations, rather than responsiveness to driver intent. Consequently, our findings should be interpreted as evidence that certain controller characteristics, such as low conflict in steering torques, can give rise to the feeling of being understood, even in the absence of real adaptive behaviour. This has implications for future AV design: fostering the perception of mutual understanding may not require intelligence or adaptivity per se, but rather careful tuning of interaction parameters. 4.1.3. I understood AV (MHC Property 2) The lack of evidence for relationships between the subjective score for ‘I understood AV’ and the behavioural metrics might suggest that the metrics employed did not capture the interaction properties most relevant to this perception. The possible difference in scores between HSC and TC suggested by the exploratory analysis could in turn indicate that this perception is sensitive to control mode characteristics. This motivates future work on identifying behavioural metrics that are more sensitive to the aspects of human–automation interaction that underlie this perception. Qualitative reports indicated that mismatched manoeuvre timing, intrusive torque feedback, and erratic or unsafe manoeuvres undermined perceived understanding (see Section 3.2), consistent with the weak negative trend observed for takeovers. Taken together, the results indicate that perceived understanding is less about about aggregate measures of steering input computed over an entire trial, and more about specific moments of alignment or misalignment between the AV’s actions and the driver’s expectations. Future analyses should therefore prioritise event-based indicators (e.g., timing of manoeuvres, mismatched torque events) over global metrics (e.g., root-mean-squared trajectory deviation), which are too coarse to capture these dynamics. 4.1.4. Working together (MHC Property 2) Exploratory analyses suggested that the scores for ‘working together’ differed significantly between HSC and TC; if confirmed by future studies, this would align with qualitative reports where participants described HSC as fostering a greater sense of cooperation. However, similar to ‘I understood AV’, no significant correlations were found between the scores for ‘working together’ and any of the behavioural metrics. Thus, our behavioural metrics do not capture perceptions of ‘working together’ with the automation; future research should explore the use of other metrics for this purpose. These results suggest that the perception of working together might not be explained simply by metrics aggregated over trials. Instead, qualitative reports suggest that the perception of ’working together’ was sensitive to the interactional qualities for specific moments or portions of the interaction. For example, whether corrections feel subtle or forceful, whether overrides are smooth or resisted, and whether driver and AV actions are aligned in timing and intent. Relatedly, [33] emphasized the complementary perspective of conflict in 16
Linking Behaviour and Perception to Evaluate Meaningful Human Control over Partially Automated Driving
cooperation, noting that to capture such moments of opposition (the “inverse” of working together), it is more informative to compute transient torque variations around specific events rather than relying solely on global measures aggregated across entire trials. 4.1.5. Responsibility (MHC Property 4) The hypothesised relationships between the score for ‘responsible’ and behavioural metrics (number of takeovers and trajectory deviation from the reference) were not confirmed by the data. That said, the exploratory analysis for HSC indicated a potential positive correlation between the score for ‘responsible’ and reaction time. Further, it revealed that for TC, the score for ‘responsible’ might be negatively correlated with conflict and positively correlated with maximum steering torque. These exploratory findings may suggest that larger or later interventions are more salient to drivers, though this interpretation remains tentative. Qualitative findings similarly linked responsibility to situations requiring supervision or correction. However, these effects must be considered alongside safety, as increasing the need for intervention may conflict with safe system behaviour. Overall, these results hint that the perception of being ‘responsible’ could be diminished in cases of only subtle actions by the driver. Together with earlier findings that outside observers might wrongly blame the drivers of partially automated vehicles for accidents [2], this warrants the need for objective metrics for ascribing responsibility in traffic interactions [48, 49, 50]. 4.1.6. Safety and trust Perceived safety [51, 52, 53] and trust [42, 43, 54, 55] have been extensively studied in the context of humanAV interactions. Even though perceived safety and trust were not central to our investigation of driver’s MHC perceptions, we opted to include them to explore potential relationships between these concepts and the behavioural metrics that are potential MHC correlates. Our exploratory analysis suggested that the scores for ‘safe’ might be negatively correlated with the jerk of vehicle trajectories and positively correlated with overtaking time. This would be aligned with the earlier work which hypothesized that smoother vehicle trajectories would improve the sense of safety [51]. The exploratory analysis also suggested positive associations between scores for ‘trust’ and trajectory deviation and overtaking time. However, these findings should be interpreted with caution, as they may reflect context-specific factors rather than general behavioural patterns. In particular, in our experimental scenario overtaking time may be linked to perceived safety, which could in turn influence trust. As such, these results may not generalise beyond our specific setting. 4.2. Which interaction mode facilitates more meaningful control? In this study, we illustrated the proposed MHC assessment methodology by evaluating drivers’ interaction with automation through two control modes well-established in the literature — haptic shared control (HSC) and traded control (TC). It must be noted that the following discussion only pertains to control modes HSC and TC as we have implemented them, and that these might not necessarily generalise to all forms of HSC and TC, for instance, implementations that issue takeover requests as auditory cues. Exploratory analyses of subjective scores collected from post-trial surveys indicate that HSC might lead to better perceptions of MHC compared to TC (Section 3.1). In our implementation, the driving automation (i.e., its control algorithm) did not adapt to individual participants; this might have contributed to the observed lack of difference in the scores of ‘AV understood me’ for the two control modes. The exploratory analysis pointed to participants reporting higher scores for ‘sufficient control’, ‘I understood the AV’, and ‘working together’ for HSC. In their qualitative answers, participants mentioned how the forces in TC were more invasive and that HSC offered greater sense of control, especially for drivers who preferred being autonomous. Together, these map onto MHC properties 2 and 3, indicating better-aligned representations and improved control ability for HSC. Furthermore, ‘perceived safety’ and ‘trust’ were also rated higher for HSC compared to TC. In terms of behavioural outcomes, participants showed faster reaction times under HSC, more takeovers, and lower maximum steering torque compared to TC, all aligning with our hypotheses (Table 2).These patterns suggest that HSC keeps drivers more engaged and reduces the need for forceful corrective input. Overall, the quantitative results indicate that HSC yields a more favourable MHC profile. However, the factors underlying 17
Linking Behaviour and Perception to Evaluate Meaningful Human Control over Partially Automated Driving
these perceptions, particularly the unexpected positive correlation between reaction time and the perception of sufficient control,remain to be clarified. Notably, both HSC and TC have relatively high scores for ‘responsible‘, indicating that participants felt morally responsible for the interaction irrespective of the control mode. The qualitative results also mirror this, with participants saying that the control mode defined their exact responsibilities — with participants expected to supervise in case of TC and correct in case of HSC. The finding that being informed of their responsibility prior to operating the system was sufficient for drivers to internalise it, even when the system had limitations that may have compromised their ability to fulfil it, echoes the concern that drivers of partially automated vehicles were problematically held responsible for accidents even when they were incapable of preventing them [2]. These notions of responsibility attribution might change as people get more exposure to capable automation systems that they feel comfortable trusting. To make sense of this ever changing landscape, we distil our findings into a framework for further exploring the perception of MHC in partially automated systems. 4.3. Evaluating meaningful human control Our methodology adds to the existing literature on evaluating MHC by providing strong empirical grounding to the established, yet so far mostly abstract, notion of MHC. This is achieved by combining behavioural metrics (specifically chosen to represent the established MHC properties [17]) with detailed questionnaires (designed to capture subjective perception of related MHC aspects). Further, although we developed this methodology for evaluating MHC specifically in partially automated driving, it could have broader implications for evaluating MHC from the perspective of human operators interacting with a more general class of automated systems. 4.3.1. Relation to existing MHC evaluation studies This work contributes to the empirical assessment of meaningful human control by translating the four MHC properties [17] into observable indicators, linking these indicators to behavioural evidence, and offering a reproducible structure to investigate perceptions of MHC as a function of behavioural metrics and control mode. Beyond the specific comparison of HSC and TC, our findings relate to ongoing debates on complacency, sense of agency, and responsibility in human–automation interaction. First, we translated the four properties of systems under MHC proposed by [17] into Likert-scale, selfreported quantitative questions. Prior work evaluated MHC mainly through narrative descriptions, with little empirical investigations [20, 15], without questions explicitly designed to assess MHC. Our approach complements this by employing items that are intentionally constructed to measure the MHC properties. Another approach quantified MHC for human–robot teams in a computer-based firefighting game based on the tracing condition [22]. They argue that while subjective measures offer valuable insights into human experiences, objective metrics are essential to verify whether the system genuinely supports MHC. Such objective data helps assess the system’s effectiveness in real-time decision-making scenarios and ensures that humans can be held accountable for robot actions, thereby preventing responsibility gaps. While their method did not regard the tracking condition, our study is grounded in the four MHC properties [17] which capture both tracking and tracing conditions. Further, in contrast to the simple firefighting game which provides only a simplified representation of complex interactions with discrete actions, our driving simulator study presents a more ecologically valid scenario with continuous actions and thereby enables a more realistic evaluation of MHC. Taken together, this positions our contribution as an early empirical operationalisation of MHC in scenarios similar to real-world interactions. Beyond the operationalisation of MHC introduced above, driver complacency, which can lead to delayed reactions, is a recurring concern in Level 2–3 automation, where drivers might slip into passive monitoring roles [56, 54]. Our results suggest that HSC may counteract this tendency: participants under HSC showed shorter reaction times and more frequent takeovers than under TC, indicating greater readiness to act. In contrast, TC, with its explicit but infrequent handovers, risks creating phases of over-reliance during which drivers disengage until prompted. Our findings resonate with prior discussions on the two modes of control, where one of the identified pitfalls of TC is that drivers may fall into complacency [57, 56, 26]. These patterns suggest that certain interaction designs may undermine meaningful human control, particularly its tracing dimension, as discussed by [58], where the capability to control the system is eroded not due to the absence of human ability, but because the system’s design discourages its exercise. This highlights the importance of designing shared control mechanisms that actively sustain driver engagement, rather than leaving humans in 18
Linking Behaviour and Perception to Evaluate Meaningful Human Control over Partially Automated Driving
purely supervisory roles. Beyond engagement, our findings also speaks to driver’s sense of agency (SoA). Evaluating the SoA of drivers is important as it is directly related to their engagement and how responsible they feel [59]. Prior research shows that higher levels of automation often reduce drivers’ SoA, undermining engagement and willingness to accept responsibility [60, 45, 61, 59]. In our study, HSC elicited significantly higher ratings of sufficient control, understanding the AV, and working together. These perceptions map directly onto SoA, suggesting that continuous, transparent, and negotiated interaction helps drivers maintain a subjective feeling of authorship over vehicle behaviour. By contrast, TC can undermine SoA by producing disorientation during abrupt handovers or by diffusing the sense of joint action. The qualitative finding that mismatches in intentions can undermine the sense of MHC supports earlier studies that showed how SoA was maintained when automation behaviour matched the intentions of the driver [62]. The responsibility ascribed to the driver should be commensurate with their agency and, as the level of automation increases, their responsibility should decrease [1]. Our results complement prior studies on theoretical and perceived responsibility of humans interacting with automation systems [63, 64] which highlight the importance of considering whether the human can meaningfully contribute to the outcome of the system. Quantitatively, responsibility ratings did not differ significantly between HSC and TC. Qualitative reports suggest that responsibility was shaped less by control mode itself than by situational demands and drivers’ understanding of their operational role. For example, under HSC, some drivers felt forced into responsibility by counter-torque or abrupt interventions, whereas under TC, others felt disengaged or only partially responsible during cooperative execution. These findings highlight that responsibility is not fixed but shaped by how authority is balanced between human and automation: it can be reduced both when the system exerts too much pressure on the driver (dominance) and when it operates too independently, leaving the driver disengaged (independence). For designers and policymakers, bridging the gap between assigned and experienced responsibility is essential to prevent unfair attribution of blame when control is shared between humans and automation [1, 65, 64, 2]. Finally, although our study focused on automotive shared control, the broader implications extend to other domains where humans and automation jointly act, such as aviation, robotics, and healthcare. Across these domains, complacency [56, 54], reduced agency [61, 45], and blurred responsibility [2, 1] recur as central challenges. Meaningful human control offers a principled framework for addressing these challenges by guiding the design of systems that sustain engagement, preserve a sense of authorship, and maintain appropriate accountability. In this way, our work connects empirical findings on HSC and TC to broader societal debates on the responsible integration of automation. 4.3.2. Implications First, our evaluation results point to concrete design implications for driving automation systems that aim to comply with the four properties of MHC. Property 1 (moral operational design domain): Drivers require system transparency and familiarisation to interpret automated behaviour. Training sessions, such as those we provided in our experiment, can help users understand system logic across routine and safety-critical scenarios without exposing them to real-world risks. Importantly, participants were explicitly informed that they remained fully responsible for the AV’s actions and were required to stay aware at all times. Such reminders of accountability are essential for aligning drivers’ understanding of their role with the moral ODD of the system. Property 2 (sufficient control): Because higher trust can delay takeovers, systems should ensure that interventions remain easy and smooth, even after abrupt transitions. Continuous steering access, minimal override friction, and intuitive re-engagement mechanisms are critical design priorities. Property 3 (shared representations): Our findings show that drivers evaluate automation at the event level rather than across entire trips. Thus, design should prioritise the quality of individual interactions—for example, ensuring subtle corrections, smooth overrides, and trajectory adaptations aligned with driver preferences. Aggregate smoothness metrics may miss these dynamics. Property 4 (responsibility): Responsibility perceptions vary with system design and driving context. Designers should avoid extremes where drivers feel either forced by high conflict or disengaged through passivity. Interfaces and adaptive control strategies that sustain an appropriate sense of accountability, 19
Linking Behaviour and Perception to Evaluate Meaningful Human Control over Partially Automated Driving
even in low-demand driving, represent a key area for future research. Together, these implications illustrate how our operationalisation of MHC can not only evaluate but also guide the design of shared-control systems. Second, in this work, starting from high-level MHC properties, we followed a hypothesis-driven approach to design an experiment for evaluating objective metrics and subjective perceptions of a human interacting with an automation system in relation to design choices (the control modes). To aid future research, we summarize the process of designing such experiments (Table 5). Table 5 | Framework for designing meaningful human control evaluation experiments: When studying how humans interact with an automation system, the following steps can be used to design experiments for evaluating their subjective perceptions pertaining to meaningful human control and how they relate to behavioural metrics. Step 1: Identify a scenario where there is potential conflict between the reasons of the human operator and the automation system which could be caused by mismatches in beliefs, desires or intentions. Step 2: Identify the properties of systems under MHC that are relevant to this scenario. Step 3: Based on identified properties, identify subjective perceptions to be evaluated in the post-trial surveys and post-experiment questionnaires. Step 4: Identify task-relevant objective metrics that pertain to the experiment scenario based on hypothesised relations to subjective perceptions. Step 5: Conduct the experiment, collect data, and analyse the results. Step 6: (Optional) Revise objective metrics and (questions for) subjective perceptions based on the participant responses to open-ended post-experiment questionnaire. Following the steps in the framework, similar experiments could be designed to build a knowledge base of how the subjective perceptions pertaining to MHC are related to different objective metrics. Potential use cases for different target groups are mentioned below. For automation manufacturers: MHC evaluation experiments could be used to direct the design process. Further, results of MHC assessments of final products could be used as evidence in safety cases presented for type approval of AVs by regulatory authorities. For regulating automated driving systems: The framework can be applied to design experiments for evaluating novel designs of driving automation systems from the perspective of the human operators interacting with them. In case of deficits in MHC, regulatory authorities could request design updates from automation manufacturers or mandate more training for operators. For assessment of human drivers and remote operators: For organisations engaged in the training and licensing of drivers and remote operators, the proposed framework can be used to test the skills of humans in interacting with automation systems. Importantly, the automation systems used in such assessments should be carefully selected to ensure they support MHC, otherwise, the evaluation risks measuring how humans cope with poorly designed automation rather than assessing human drivers’ and remote operators’ competences. 4.4. Limitations and future research 4.4.1. Limitations First, the automation system used in this study is relatively artificial in that drivers were only responsible for lateral control and had no input over longitudinal control. This choice was deliberate: by fixing longitudinal control, we isolated the lateral interaction between driver and automation, providing controlled and repeatable conditions for testing meaningful human control. However, this simplification limits the generalisability of our findings to real-world driving systems where drivers must manage both lateral and longitudinal control simultaneously, which may alter their perceptions of meaningful human control. Second, our sample was limited to 24 participants, predominantly students and individuals working in academia, many of whom had limited real-world exposure to automated vehicle technology. This sample size 20
Linking Behaviour and Perception to Evaluate Meaningful Human Control over Partially Automated Driving
was determined by feasibility constraints and may not fully reflect the perspectives of early adopters or more experienced users of automated systems, thereby limiting the representativeness of our findings. However, since our primary contribution is the evaluation methodology rather than definitive conclusions about specific perceptions of MHC, the limited sample size and representativeness does not fundamentally undermine the validity of our approach. Third, our evaluation focused exclusively on the driver as the human agent, disregarding other relevant human agents such as other road users, traffic regulators, or system designers. While this does not undermine our conclusions, which are scoped to the driver interacting directly with the automation, it does restrict the broader applicability of the findings. Fourth, subjective scores were collected from post-trial surveys, which cannot capture how driver perceptions may have varied within a trial, limiting the temporal resolution of our findings. 4.4.2. Future research First, in this study, the human outperforms the automation (the AV deliberately fails to steer clear of some road users); an important direction for future research concerns situations in which the automation outperforms the human (e.g., emergency braking or being aware of vehicles in the blind spot). Such experiments are necessary to identify theoretical bounds for human and automation performance with regard to different levels of driving complexity [66], and for designing appropriate transfers of authority. Second, the relatively short duration of the experiment cannot capture how driving behaviour might adapt over days of exposure to partial automation. These adaptations in human behaviour along with updates in automation behaviour can lead to unforeseen emergent behaviours that could be detrimental to meaningful human control [67, 68]. Thus, to manage emergence, longer studies that account for adaptation are needed to properly validate automated systems. Third, future studies should include a broader and more diverse participant pool, such as incorporating drivers with varied ages, professional backgrounds, driving experience, and geographical locations, to provide additional insights into how perceptions of MHC vary across different user groups. Fourth, more continuous information about driver perception could be obtained by measuring physiological signals such as heart rate, blood pressure, muscle activation, or brain activity, for example, relating the sense of agency to electroencephalography (EEG) signals [60]. Fifth, extending the evaluation framework to include other human agents with varying degrees of authority and influence, such as other road users, traffic regulators, or system designers, would provide a more complete picture of MHC in partially automated driving.
5. Conclusion Starting from the four properties of systems under meaningful human control (MHC), we developed and demonstrated an MHC evaluation methodology, instantiated here through a driving simulator experiment, to study the perception of responsibility and control in drivers of partially automated vehicles operating under two control modes — based on haptic shared control (HSC) and traded control (TC). We integrated behavioural metrics from telemetry data, subjective perception scores from post-trial surveys and qualitative feedback from open-ended post-trial questionnaires. Linear mixed-effect model (LMMs) fits confirmed our hypothesis that conflict in steering torques is negatively correlated with the perception of the AV understanding the driver. However, contrary to our hypothesis, the analysis revealed a surprising positive correlation between reaction time and the perception of sufficient control which contradicted our hypothesis. Such contradictions highlight the need for conducting experiments for evaluating design choices regarding the interaction between driver and automated driving systems. From the qualitative feedback, we identified three categories of factors — attitudinal factors, interaction factors and trajectory factors — that positively and negatively influence the perception of MHC. Mismatches in intentions, lack of safety and resistance to driver inputs were frequently associated with reduction in perceived MHC. The perception of being responsible on the other hand were more dependent on the expectations associated with being accountable for the novel and untrustworthy automation, especially in safety-critical 21
Linking Behaviour and Perception to Evaluate Meaningful Human Control over Partially Automated Driving
situations and less dependent on the control mode. Finally the exploratory analysis suggested that HSC supports meaningful human control better than TC; specifically the results for the perceptions of ‘sufficient control’, ‘understanding the AV’, and ‘working together’. Based on our findings, guidelines for designing the interaction of drivers with the automation are summarised as: (1) ensure that driver interventions remain effortless and smooth, (2) improve mutual communication of intentions between the driver and the automation, and (3) conduct experiments to validate design choices. Ultimately, we presented a framework for designing experiments starting from the properties of systems under MHC, to guide future research on the perception of MHC for humans interacting with automated systems,
CRediT author statement Conceptualization: AG, LES, LF, DA, SCC, LCS, AZ; Methodology: AG, LES, LF, SCC, LCS, AZ; Software: AG, LES, LF; Validation: AG, LES; Formal analysis: AG, LES. LF; Investigation: AG, LES, LF; Resources: DA, AZ; Data curation: AG, LES; Writing - original draft preparation: AG, LES; Writing - review and editing: AG, LES, BvA, DA, SCC, LCS, AZ; Visualization: AG, LES; Supervision: BvA, DA, SCC, LCS, AZ;
Acknowledgements This research was funded by the TU Delft AI Initiative and Indonesia Endowment Fund for Education Agency (LPDP). The authors would like to thank members of the Centre for Meaningful Human Control and AiTech for all their valuable suggestions, feedback and discussions during our meetings.
References [1] F. Flemisch, M. Heesen, T. Hesse, J. Kelsch, A. Schieben, and J. Beller, “Towards a dynamic balance between humans and automation: Authority, ability, responsibility and control in shared and cooperative control situations,” Cognition, Technology & Work, vol. 14, pp. 3–18, Mar. 2012. [2] N. Beckers, L. C. Siebert, M. Bruijnes, C. Jonker, and D. Abbink, “Drivers of partially automated vehicles are blamed for crashes that they cannot reasonably avoid,” Scientific Reports, vol. 12, p. 16193, Sept. 2022. [3] D. J. Fagnant and K. Kockelman, “Preparing a nation for autonomous vehicles: opportunities, barriers and policy recommendations,” Transportation Research Part A: Policy and Practice, vol. 77, pp. 167–181, 2015. [4] K. D. Kusano, J. M. Scanlon, Y. H. Chen, T. L. McMurry, T. Gode, and T. Victor, “Comparison of waymo rider-only crash rates by crash type to human benchmarks at 56.7 million miles,” Traffic Injury Prevention, pp. 1–13, 2025. [5] SAE International, “Taxonomy and definitions for terms related to on-road motor vehicle automated driving systems,” Standard J3016_202104, 2021. [6] M. R. Endsley, “Autonomous driving systems: A preliminary naturalistic study of the tesla model s,” Journal of Cognitive Engineering and Decision Making, vol. 11, no. 3, pp. 225–238, 2017. [7] Y. Chu and P. Liu, “Automation complacency on the road,” Ergonomics, vol. 66, no. 11, pp. 1730–1749, 2023. [8] N. Merat, H. A. Jamson, F. Lai, and O. Carsten, “Human factors of highly automated driving: results from the easy and citymobil projects,” in Road vehicle automation, pp. 113–125, Springer, 2014. [9] J. W. Moore, “What is the sense of agency and why does it matter?,” Frontiers in psychology, vol. 7, p. 1272, 2016. [10] P. Cornelio, P. Haggard, K. Hornbaek, O. Georgiou, J. Bergström, S. Subramanian, and M. Obrist, “The sense of agency in emerging technologies for human–computer integration: A review,” Frontiers in Neuroscience, vol. 16, p. 949138, 2022. [11] B. Berberian, J.-C. Sarrazin, P. Le Blaye, and P. Haggard, “Automation technology and sense of control: a window on human agency,” PloS one, vol. 7, no. 3, p. e34075, 2012. [12] G. Moretto, E. Walsh, and P. Haggard, “Experience of agency and sense of responsibility,” Consciousness and cognition, vol. 20, no. 4, pp. 1847–1854, 2011. [13] A. Matthias, “The responsibility gap: Ascribing responsibility for the actions of learning automata,” Ethics and information technology, vol. 6, pp. 175–183, 2004. [14] F. Santoni de Sio and G. Mecacci, “Four responsibility gaps with artificial intelligence: Why they matter and how to address them,” Philosophy & Technology, vol. 34, no. 4, pp. 1057–1084, 2021. 22
Linking Behaviour and Perception to Evaluate Meaningful Human Control over Partially Automated Driving
[15] S. C. Calvert, B. van Arem, D. D. Heikoop, M. Hagenzieker, G. Mecacci, and F. S. de Sio, “Gaps in the control of automated vehicles on roads,” IEEE intelligent transportation systems magazine, vol. 13, no. 4, pp. 146–153, 2020. [16] F. Santoni de Sio and J. Van den Hoven, “Meaningful human control over autonomous systems: A philosophical account,” Frontiers in Robotics and AI, vol. 5, p. 15, 2018. [17] L. Cavalcante Siebert, M. L. Lupetti, E. Aizenberg, N. Beckers, A. Zgonnikov, H. Veluwenkamp, D. Abbink, E. Giaccardi, G.-J. Houben, C. M. Jonker, J. van den Hoven, D. Forster, and R. L. Lagendijk, “Meaningful human control: Actionable properties for AI system development,” AI and Ethics, May 2022. [18] S. C. Calvert, “Principles and framework for the operationalisation of meaningful human control over autonomous systems,” Science and Engineering Ethics, vol. 31, no. 5, p. 27, 2025. [19] G. Mecacci and F. Santoni de Sio, “Meaningful human control as reason-responsiveness: the case of dual-mode vehicles,” Ethics and Information Technology, vol. 22, no. 2, pp. 103–115, 2020. [20] L. E. Suryana, S. Nordhoff, S. Calvert, A. Zgonnikov, and B. van Arem, “Meaningful human control of partially automated driving systems: Insights from interviews with tesla users,” Transportation Research Part F: Traffic Psychology and Behaviour, vol. 113, pp. 213–236, 2025. [21] S. C. Calvert and A. Zgonnikov, “A lack of meaningful human control for automated vehicles: Pressing issues for deployment and regulation,” Frontiers in Future Transportation, vol. 6, p. 1534157, 2025. [22] R. S. Verhagen, M. A. Neerincx, and M. L. Tielman, “Meaningful human control and variable autonomy in human-robot teams for firefighting,” Frontiers in Robotics and AI, vol. 11, p. 1323980, 2024. [23] D. A. Abbink, T. Carlson, M. Mulder, J. C. De Winter, F. Aminravan, T. L. Gibo, and E. R. Boer, “A topology of shared control systems—finding common ground in diversity,” IEEE Transactions on Human-Machine Systems, vol. 48, no. 5, pp. 509–525, 2018. [24] Z. Wang, R. Zheng, T. Kaizuka, K. Shimono, and K. Nakano, “The effect of a haptic guidance steering system on fatigue-related driver behavior,” IEEE Transactions on Human-Machine Systems, vol. 47, no. 5, pp. 741–748, 2017. [25] M. Li, H. Cao, X. Song, Y. Huang, J. Wang, and Z. Huang, “Shared control driver assistance system based on driving intention and situation assessment,” IEEE Transactions on Industrial Informatics, vol. 14, no. 11, pp. 4982–4994, 2018. [26] J. C. de Winter, S. M. Petermeijer, and D. A. Abbink, “Shared control versus traded control in driving: a debate around automation pitfalls,” Ergonomics, vol. 66, no. 10, pp. 1494–1520, 2023. [27] N. Beckers, O. Siebinga, J. Giltay, and A. van der Kraan, “Joan: A framework for human-automated vehicle interaction experiments in a virtual reality driving simulator,” Journal of Open Source Software, vol. 8, no. 82, p. 4250, 2023. [28] A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun, “Carla: An open urban driving simulator,” in Conference on robot learning, pp. 1–16, PMLR, 2017. [29] M. R. van Paassen, R. P. Boink, D. A. Abbink, M. Mulder, and M. Mulder, “Four design choices for haptic shared control,” in Advances in Aviation Psychology, Volume 2, pp. 237–254, Routledge, 2017. [30] L. E. Suryana, S. Nordhoff, S. C. Calvert, A. Zgonnikov, and B. Van Arem, “A meaningful human control perspective on user perception of partially automated driving systems: a case study of tesla users,” in 2024 IEEE intelligent vehicles symposium (IV), pp. 409–416, IEEE, 2024. [31] R. Boink, M. M. Van Paassen, M. Mulder, and D. A. Abbink, “Understanding and reducing conflicts between driver and haptic shared control,” in 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 1510–1515, IEEE, 2014. [32] Z. Ercan, A. Carvalho, H. E. Tseng, M. Gökaşan, and F. Borrelli, “A predictive control framework for torque-based steering assistance to improve safety in highway driving,” Vehicle system dynamics, vol. 56, no. 5, pp. 810–831, 2018. [33] F. Mars, M. Deroo, and J.-M. Hoc, “Analysis of human-machine cooperation when driving with different degrees of haptic shared control,” IEEE transactions on haptics, vol. 7, no. 3, pp. 324–333, 2014. [34] G. Markkula and J. Engström, “A steering wheel reversal rate metric for assessing effects of visual and cognitive secondary task load,” in Proceedings of the 13th ITS World Congress, Leeds, 2006. [35] Y. Jiao, “A fast calculation of two-dimensional Time-to-Collision,” Mar. 2023. [36] J. Cohen, “A coefficient of agreement for nominal scales,” Educational and psychological measurement, vol. 20, no. 1, pp. 37–46, 1960. [37] K. L. Gwet, “Computing inter-rater reliability and its variance in the presence of high agreement,” British Journal of Mathematical and Statistical Psychology, vol. 61, no. 1, pp. 29–48, 2008. [38] V. Braun and V. Clarke, “Using thematic analysis in psychology,” Qualitative research in psychology, vol. 3, no. 2, pp. 77–101, 2006.
23
Linking Behaviour and Perception to Evaluate Meaningful Human Control over Partially Automated Driving
[39] C. E. Hill, B. J. Thompson, and E. N. Williams, “Consensual qualitative research: An update,” Journal of Counseling Psychology, vol. 52, no. 2, pp. 196–205, 2005. [40] J. L. Campbell, C. Quincy, J. Osserman, and O. K. Pedersen, “Coding in-depth semistructured interviews: Problems of unitization and intercoder reliability and agreement,” Sociological methods & research, vol. 42, no. 3, pp. 294–320, 2013. [41] A. R. Feinstein and D. V. Cicchetti, “High agreement but low kappa: I. the problems of two paradoxes,” Journal of clinical epidemiology, vol. 43, no. 6, pp. 543–549, 1990. [42] W. Payre, J. Cestac, and P. Delhomme, “Fully automated driving: Impact of trust and practice on manual control recovery,” Human factors, vol. 58, no. 2, pp. 229–241, 2016. [43] V. V. Dixit, S. Chand, and D. J. Nair, “Autonomous vehicles: disengagements, accidents and reaction times,” PLoS one, vol. 11, no. 12, p. e0168054, 2016. [44] W. Wen and H. Imamizu, “The sense of agency in perception, behaviour and human–machine interactions,” Nature Reviews Psychology, vol. 1, pp. 211–222, Feb. 2022. [45] B. Berberian, J.-C. Sarrazin, P. Le Blaye, and P. Haggard, “Automation Technology and Sense of Control: A Window on Human Agency,” PLoS ONE, vol. 7, p. e34075, Mar. 2012. [46] P. M. Bays, J. R. Flanagan, and D. M. Wolpert, “Attenuation of Self-Generated Tactile Sensations Is Predictive, not Postdictive,” PLoS Biology, vol. 4, p. e28, Jan. 2006. [47] D. Wenke and P. Haggard, “How voluntary actions modulate time perception,” Experimental Brain Research, vol. 196, pp. 311–318, July 2009. [48] A. George, L. C. Siebert, D. A. Abbink, and A. Zgonnikov, “Feasible Action Space Reduction for Quantifying Causal Responsibility in Continuous Spatial Interactions,” May 2025. [49] S. Shalev-Shwartz, S. Shammah, and A. Shashua, “Vision Zero: On a Provable Method for Eliminating Roadway Accidents without Compromising Traffic Throughput,” Jan. 2019. [50] I. Remy, D. Fridovich-Keil, and K. Leung, “Learning responsibility allocations for multi-agent interactions: A differentiable optimization approach with control barrier functions,” Oct. 2024. [51] C. Peng, S. Horn, R. Madigan, C. Marberger, J. D. Lee, J. Krems, M. Beggiato, R. Romano, C. Wei, E. Wooldridge, R. Happee, M. Hagenzieker, and N. Merat, “Conceptualising user comfort in automated driving: Findings from an expert group workshop,” Transportation Research Interdisciplinary Perspectives, vol. 24, p. 101070, Mar. 2024. [52] K. Chen, Z. Li, P. Liu, V. L. Knoop, Y. Han, and Y. Jiao, “Evaluating the safety and efficiency impacts of forced lane change with negative gaps based on empirical vehicle trajectories,” Accident Analysis & Prevention, vol. 203, p. 107622, Aug. 2024. [53] E. Papadimitriou, H. Farah, G. van de Kaa, F. Santoni de Sio, M. Hagenzieker, and P. van Gelder, “Towards common ethical and safe ‘behaviour’ standards for automated vehicles,” Accident Analysis & Prevention, vol. 174, p. 106724, Sept. 2022. [54] S. Nordhoff, J. D. Lee, S. C. Calvert, S. Berge, M. Hagenzieker, and R. Happee, “(mis-) use of standard autopilot and full self-driving (fsd) beta: Results from interviews with users of tesla’s fsd beta,” Frontiers in psychology, vol. 14, p. 1101520, 2023. [55] L. J. Molnar, L. H. Ryan, A. K. Pradhan, D. W. Eby, R. M. S. Louis, and J. S. Zakrajsek, “Understanding trust and acceptance of automated vehicles: An exploratory simulator study of transfer of control between automated and manual driving,” Transportation research part F: traffic psychology and behaviour, vol. 58, pp. 319–328, 2018. [56] F. Flemisch, D. A. Abbink, M. Itoh, M.-P. Pacaux-Lemoine, and G. Weßel, “Joining the blunt and the pointy end of the spear: Towards a common framework of joint action, human–machine cooperation, cooperative guidance and control, shared, traded and supervisory control,” Cognition, Technology & Work, vol. 21, pp. 555–568, Nov. 2019. [57] D. A. Abbink, M. Mulder, and E. R. Boer, “Haptic shared control: smoothly shifting control authority?,” Cognition, Technology & Work, vol. 14, pp. 19–28, 2012. [58] S. C. Calvert and G. Mecacci, “A conceptual control system description of cooperative and automated driving in mixed urban traffic with meaningful human control for design and evaluation,” IEEE Open Journal of Intelligent Transportation Systems, vol. 1, pp. 147–158, 2020. [59] J. W. Moore, “What Is the Sense of Agency and Why Does it Matter?,” Frontiers in Psychology, vol. 7, Aug. 2016. [60] S. Yun, W. Wen, Q. An, S. Hamasaki, H. Yamakawa, Y. Tamura, A. Yamashita, and H. Asama, “Investigating the relationship between assisted driver’s SoA and EEG,” in Converging Clinical and Engineering Research on Neurorehabilitation III (L. Masia, S. Micera, M. Akay, and J. L. Pons, eds.), (Cham), pp. 1039–1043, Springer International Publishing, 2019.
24
Linking Behaviour and Perception to Evaluate Meaningful Human Control over Partially Automated Driving
[61] P. Cornelio, P. Haggard, K. Hornbaek, O. Georgiou, J. Bergström, S. Subramanian, and M. Obrist, “The sense of agency in emerging technologies for human–computer integration: A review,” Frontiers in Neuroscience, vol. 16, Sept. 2022. [62] W. Wen, S. Yun, A. Yamashita, B. D. Northcutt, and H. Asama, “Deceleration Assistance Mitigated the Trade-off Between Sense of Agency and Driving Performance,” Frontiers in Psychology, vol. 12, June 2021. [63] N. Douer and J. Meyer, “Theoretical, Measured, and Subjective Responsibility in Aided Decision Making,” ACM Transactions on Interactive Intelligent Systems, vol. 11, pp. 1–37, Mar. 2021. [64] N. Douer and J. Meyer, “Judging One’s Own or Another Person’s Responsibility in Interactions With Automation,” Human Factors: The Journal of the Human Factors and Ergonomics Society, vol. 64, pp. 359–371, Mar. 2022. [65] L. Methnani, A. Aler Tubella, V. Dignum, and A. Theodorou, “Let Me Take Over: Variable Autonomy for Meaningful Human Control,” Frontiers in Artificial Intelligence, vol. 4, p. 737072, Sept. 2021. [66] J. C. F. De Winter, S. M. Petermeijer, and D. A. Abbink, “Shared control versus traded control in driving: A debate around automation pitfalls,” Ergonomics, vol. 66, pp. 1494–1520, Oct. 2023. [67] T. Melman, N. Beckers, and D. Abbink, “Mitigating undesirable emergent behavior arising between driver and semi-automated vehicle,” arXiv preprint arXiv:2006.16572, 2020. [68] S. C. Calvert, S. O. Johnsen, and A. George, “Designing automated vehicle and traffic systems towards meaningful human control,” in Research Handbook on Meaningful Human Control of Artificial Intelligence Systems, United Kingdom: Edward Elgar Publishing, 2023 (Accepted).
25