Partner Capability Estimation for Task-Agnostic Adaptation in Ad-Hoc Teamwork

Unknown

Conceptio › Archive › arXiv CS

arXiv CSopen access

Partner Capability Estimation for Task-Agnostic Adaptation in Ad-Hoc Teamwork

Unknown · 2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

Open Source ↗Direct PDF ↓

artificialintelligence knowledgerepresentation reasoning

artificial intelligence, reasoning, knowledge representation

Partner Capability Estimation for Task-Agnostic Adaptation in Ad-Hoc Teamwork Peter Tisnikara,b,1,∗, Maja Swieczkowskaa , Benteng Maa , Gerard Canala , Matteo Leonettia,∗ a

Department of Informatics, King’s College London, 30 Aldwych, London, WC2B 4BG, England, United Kingdom b Department of Engineering Science, University of Oxford, Parks Road, Oxford, OX1 3PJ, England, United Kingdom

arXiv:2607.27177v1 [cs.AI] 29 Jul 2026

Abstract Effective collaboration with novel and diverse partners is a crucial skill for autonomous agents. Most current ad-hoc teamwork (AHT) approaches assume that agents will collaborate on a single, fixed task and that the partner’s capabilities, their ability to successfully execute the desired action, are already known. In reality, a partner’s true capabilities are often hidden, and human collaborators may act sub-optimally on tasks with multiple valid strategies. To address these limitations, we extend ad-hoc teamwork into a multi-task setting by re-framing it as a problem of joint planning with decentralised execution under hidden partner capabilities. We introduce CE-CM (Capability Estimation via Contextual Models), an approximate Bayesian method that infers task-invariant capability vectors. By using simulation-based sampling, the agent estimates capabilities and induces a contextual Multi-agent Markov Decision Processes for planning. This approach requires no population pre-training and refines its beliefs online from just a few tasks. To account for human unpredictability, we propose CE-CM-Div, an extension that evaluates capability hypotheses against diverse planner rollouts rather than a single optimal trajectory. Simulated experiments demonstrate that CE-CM rapidly recovers hidden capabilities, reduces infeasible action assignments, and adapts to changes over time. Furthermore, in an offline human study of 225 trajectories from 15 participants, CE-CM-Div substantially improved capability estimates over the baseline CE-CM ∗

Corresponding author. Work done whilst at King’s College London. This work was supported by UK Research and Innovation (grant number EP/S023356/1), in the UKRI Centre for Doctoral Training in Safe and Trusted Artificial Intelligence (www.safeandtrustedai.org). 1

method. Our results suggest capability-based modelling is a promising interpretable, task-agnostic representation in the studied settings, demonstrating that accounting for behavioural diversity is essential for robust human-AI teaming. Keywords: ad-hoc teamwork, human–agent interaction, behaviour modelling 1. Introduction Autonomous agents will increasingly be required to collaborate with diverse partners. In open-world settings, an agent cannot assume that its partners will be homogeneous; they may differ in their preferences, skill level, physical or cognitive capabilities, and other traits. Furthermore, these traits may only be revealed to the agent through repeated interaction. Adapting to such collaborators, especially human ones, has therefore been recognised as a crucial requirement for intelligent embodied agents [1, 2]. Consider, for example, a household robot collaborating with a new human partner on a series of chores. If the human leaves the robot to carry a bulky object or repeatedly avoids part of the environment, such behaviour is difficult to interpret: it may reflect a deliberate strategy, but it may also indicate that the partner is simply unable to perform that part of the task. This distinction is important for coordination, because it changes which actions should be assigned to which agent and which joint plans are likely to succeed. We argue that such adaptation requires learning an explicit, reusable model of the current partner, rather than only a policy that is robust to partner variation within a single task. Ad-hoc teamwork (AHT) [3, 4] provides a natural framework for this problem, as it studies agents that must collaborate with previously unknown partners without pre-coordination. However, much recent AHT work learns robust ego policies against populations of partners in a fixed task, often using model-free Multi-agent reinforcement learning (see Section 2 for a more detailed discussion). Such policies can generalise within the training task, but they usually do not provide an explicit model of the current partner, nor do they transfer directly across tasks. We therefore focus on repeated collaboration, where the agent must learn something about the particular partner and reuse that knowledge in future tasks. This leads us to the central question of the paper: what kind of partner representation can be learned from interaction, reused across tasks, and used to adapt to the current individual? We argue that this requires shifting the focus of AHT toward transferable partner models. In particular, we propose to represent partners through their capabilities: latent, task-invariant constraints on their ability to realise transitions in a joint task. Whereas preferences and goals describe what a partner may want to do, capabilities describe what they are fundamentally able to do. Policies 2

Infer Capabilities

Observation 1) Sample

2) Simulate

3) Compare

4) Update beliefs

Plan on New Task

New Task

Execute Plan in New Task

Figure 1: Overview of CE-CM. After completing a task with a partner, the agent uses the observed joint trajectory and the task’s known goal to infer what the partner is capable of doing. It samples candidate capability vectors ĉ1 , . . . , ĉN , simulates the joint trajectory that each would induce in the observed task, and compares the simulated trajectories with the observation. Capability hypotheses that explain the observation are used to update the agent’s belief c∗ , producing an explicit partner model Mc∗ (·). The model is then used by the agent to compute a joint plan ρ∗ for the new task. Whilst the agent computes a joint plan, it only executes its own actions. After the new task is completed, its joint trajectory becomes new evidence, and the inference-planning loop repeats.

are tied to particular tasks and observed actions are local to particular states, while capabilities can explain a broader family of feasible and infeasible interactions across different tasks. By modelling what the partner can and cannot do, rather than only what they happened to do before, the agent can accumulate knowledge about an individual partner and use it to adapt its own behaviour in future tasks. If the partner is modelled in terms of hidden capabilities, adaptation becomes a problem of reasoning about how those capabilities constrain feasible joint behaviour. In cooperative settings with shared goals, the agent must infer which joint plans are possible under different assumptions about what the partner can and cannot do, and allocate work accordingly. This leads naturally to viewing the interaction as joint planning with decentralised execution under hidden partner capabilities (Section 3): the team’s behaviour is reasoned about jointly, but each agent ultimately executes only its own actions. Under this view, the same capability-conditioned model supports both explanation and adaptation: observed trajectories reveal which capability assignments best explain prior interactions, and the resulting estimate is used to plan the next task with the same partner. Inferring such hidden capabilities from interaction is challenging because a hypothesis can only be evaluated through the joint behaviour it induces. In Section 4, we therefore propose Capability Estimation via Contextual Models (CECM, overview in Figure 1). This approximate Bayesian method infers task-invariant capability vectors through a simulate-and-compare approach: it samples candidate 3

hypotheses, generates joint plans under them, and matches the resulting trajectories against observations. The inferred estimate then induces a capability-conditioned model for planning future tasks with the same partner. CE-CM requires no population pre-training and can refine its estimate online from a small number of interactions. However, in realistic human–AI teaming, human partners rarely act like optimal planners. Because human behaviour often involves diverse or suboptimal strategies, comparison to a single planned trajectory becomes brittle even when the capability representation is appropriate. We therefore introduce CE-CM-Div, an extension that evaluates each capability hypothesis against diverse planner rollouts, improving robustness to suboptimality and behavioural variability. To evaluate our capability-based framework, we deploy it across two distinct collaborative domains: a symbolic household setting modelled in PDDL (TidyUP) and a simulator with under-specified rewards (Overcooked). In TidyUP, we show that CE-CM rapidly recovers representative capability estimates from only a few tasks, improves feasibility-aware coordination, and adapts when partner capabilities change over time. In Overcooked, we further test CE-CM in a dynamic coordination domain, where the results reveal both the promise of capability-based modelling and the limitations of assuming optimal partner behaviour. To bridge this gap, we collected 225 human Overcooked trajectories from 15 participants, showing that human behaviour is often not well captured by a single planned trajectory. These results motivate CE-CM-Div and show that accounting for diverse plausible behaviours substantially improves capability inference. Together, they position capability estimation as an interpretable, task-agnostic approach to adaptation in ad-hoc teamwork, while clarifying when diversity-aware inference becomes necessary. Overall, we make the following contributions in this paper: • A capability-based formulation of multi-task ad-hoc teamwork: We formalise repeated collaboration with unknown partners as joint planning with decentralised execution under hidden, task-invariant capability constraints. This yields an interpretable, reusable representation of what a partner can and cannot do. • CE-CM for online task-agnostic adaptation: We introduce CE-CM, an approximate Bayesian method that infers discrete capability vectors from observed trajectories and uses them to induce capability-conditioned models for future planning. The method learns explicit partner models from only a small number of interactions and requires no population pre-training. • Empirical evidence for feasibility-aware coordination: Across TidyUP 4

and Overcooked, we show that CE-CM recovers hidden partner capabilities, reduces infeasible action assignments, and transfers estimates across tasks. • CE-CM-Div for behaviourally diverse partners: To address the brittleness of single-trajectory inference, we propose CE-CM-Div, which evaluates capability hypotheses against sets of diverse planner rollouts rather than one optimal prediction. This improves capability estimation when multiple behaviours are plausible under the same capability constraints. • Human-subject evidence for diversity-aware inference: Through a study of 15 participants in Overcooked, we show that human trajectories often differ substantially from planner-generated trajectories. This demonstrates that accounting for behavioural diversity is essential for robust capability inference in realistic human–AI teaming. CE-CM uses the same approximate Bayesian simulate-and-compare loop as our preliminary work on CApability Modelling from Observations (CAMO) [5]: sampling candidate capability hypotheses, simulating them, and scoring them against observed behaviour. The setting and contribution are otherwise different. CAMO inferred continuous physical parameters from passive observations in a collaborative manipulation setting; here, an ego agent participates in repeated ad-hoc teamwork tasks while inferring discrete, task-invariant partner capabilities for future planning. Thus, this work reuses CAMO’s inference principle, but extends it for online adaptation, decentralised execution, and behavioural ambiguity in multi-task human–AI teaming. 2. Related Work 2.1. Collaborating with Diverse Partners Most approaches that aim to create robust agents capable of cooperating with diverse partners have been developed by the AHT community, although some other fields such as zero-shot coordination (ZSC) [6] (also known as convention learning [7]) share a similar problem formulation. ZSC approaches explicitly assume a known reward function that is identical for both agents, while this might not generally be the case in AHT settings [3]. Nevertheless, both fields have converged to similar solution approaches and we therefore review them together. The key aim of the methods is to create an agent’s ego policy (i.e., a mapping between joint states and the agent’s individual actions) that is robust to different partners. To do so, most methods utilise population-based training, which builds 5

on the popular self-play framework [8], where the agent’s robustness to variation is achieved by continuously playing the game against either itself, or its older policy checkpoints [9]. Most research has therefore focussed on the challenge of developing populations that are diverse with respect to some metrics, such as partner trajectory diversity [10, 11], or partner policy diversity [12, 13]. Other methods induce population diversity by modelling additional latent rewards [14], or introduce learned combinations of policies to further enhance diversity without increasing the population size [15]. All above methods utilise model-free, decentralised reinforcement learning algorithms to train the ego policy as a best response to all partners in the population. Another approach is to build a set of policies, or a policy with latent conditioning, rather than a single ego policy. The potential partner distribution is either learned from demonstration by clustering of strategies [16], handcrafted partner models [17, 18], or by varying partners’ reward functions [19, 20]. The correct policy at runtime is then selected by a Bayesian inference algorithm that selects the policy from the set of policies that is considered to be the best response to the partner’s behaviour. Similarly, other methods utilise a monolithic policy, but use latent conditioning to identify the partner’s latent parameters at runtime [20, 21]. While above methods produce agents that are capable of adapting to diverse partners, they focus on training the policy of the ego agent rather than learning an explicit, reusable model of the current partner’s capabilities. Moreover, these methods are typically developed for settings in which the cooperative task is fixed between training and deployment, and the learned policies are not intended to transfer across tasks. They also require either a generative model of the partner population, trajectory demonstrations to induce a latent space over partners, or that the partner policies are available a priori for the ego agent to train with. In contrast, CE-CM does not require demonstrations from all potential partners, or any pre-computation of policies or populations to be able to adapt to the current partner. Furthermore, CECM is able to learn from any task, and learns partner-specific, but task-independent parameters, allowing it to collaborate with the same partner on any future task, not just the ones it observed so far. 2.2. Partner Behaviour Modelling Partner behaviour modelling has been identified as a crucial component for successful teamwork in settings where artificial (embodied) agents team up with other agents [22, 23], or humans [24]. The partner behaviour modelling literature has produced a very diverse set of approaches, and spans work in ad-hoc teamwork, human–agent teaming, human–robot collaboration, and others. 6

One of the simplest techniques to model the partner’s behaviour is to predict their action based on the intelligent agent’s ego policy [25]. However, this method makes an explicit assumption that both agents have the same capabilities and follow the same policy (and conventions) in the task. Instead, methods within AHT literature perform model selection amongst a pool of partner models that are (partially) parametrised by their type [22, 26]. The type space specifies a set of possible partners, and by knowing a partner’s type, the intelligent agent can instantiate the partner’s policy. The methods use these policies to query the partner’s future actions while planning the intelligent agent’s course of action [27, 28, 29]. These methods require that the type space is known and explicit in the sense that each type in the type space induces a policy, and the policies are assumed to be known and given to the methods by the designers. Other methods forgo the specification of a type space and instead learn the partner’s representation directly from observation through repeated interaction. For example, the partner’s policy is iteratively approximated by using behaviour cloning [30], or by biasing the planner’s exploration towards previously observed joint actions [31]. However, these methods require repeated interaction in a single task in order to learn these representations. While they achieve individual adaptation, they do not learn partner models that can be reused in different tasks. Partner behaviour modelling has also been utilised in the context of goal recognition [32, 33, 34, 35, 36], where goal-conditioned partner models can explain the partner’s behaviour and allow the intelligent agent to be able to predict their future actions and adapt its own behaviour in return. The inference over possible partner goals is done using pre-computed goal-conditioned partner policies [33, 34], or with directly querying models learned from data [35, 36]. These methods therefore adapt by understanding the task at hand with their partner, and then pursuing complementary actions based on this information. They do not explicitly learn partner-specific properties that can be carried across tasks. To this end, many methods aim to learn the partner’s preferences in a collaborative task. Preferences are encoded as rewards [37, 38, 39, 40, 41, 42], action costs [43, 44], or are simply accounted for by generating diverse plans [45].They specify the partner’s preferred course of action in a given task, and can be used to predict their behaviour by performing joint planning for the team. They are learned either from passive observations [40], active feedback queries [39, 43, 38], or by swapping the agents’ roles in the task [41, 42]. However, while preferences could in theory capture the partner’s potential diversity and different capability, most learned preferences are specific to the task in which they are learned, and thus have limited transferability. Preferences therefore help explain which feasible behaviour a partner may choose 7

within a task, whereas capabilities determine which behaviours are feasible in the first place. Estimation of partner behaviour models is also formulated as a parameter estimation problem in centralised task allocation and scheduling approaches. The parameters that denote some behavioural descriptor of the partner such as their task proficiency [46, 47, 48], levels of attentiveness and fatigue [49], or capabilities [50, 51]. The parameter estimates are learned through direct feedback on successful execution of the allocated task or team strategy. However, in these settings, the team’s actions are determined by a centralised planner or scheduler, and therefore they violate the assumption made that the partners are uncontrollable. The parameters that govern behaviour can also be latent. Previous methods have considered learning latent decision parameters from observations [52] or from active collaboration [53]. However, these models again rely on previously collected data that they can utilise in (semi-)supervised learning settings to explicitly learn a link function between the latent parameters and the individual actions of the partner. While the modelling landscape is very diverse, the majority of the works focus on learning models that are specific to the context of the task at hand. Little attention, if any, has been given to learning partner models that are transferable across many collaborative tasks with the same individual. This is exactly the focus of CE-CM, as by learning a portable partner model based on their capabilities, the model is specific to the individual partner, but independent of the tasks. Because of this, it can be easily learned, refined, and reused across different tasks, and allows for effective behaviour modelling, prediction, and adaptation. 3. Problem Formulation We now describe the problem of capability inference and adaptation from observations in joint tasks. Consider two agents: Ag1 , who is a controllable autonomous agent, and Ag2 , who is an uncontrollable agent (or human), that is collaborating with Ag1 . The two agents collaborate in an environment to achieve a known shared goal by executing a sequence of actions, maximising some notion of reward. Both agents are characterised by capabilities that remain unchanged over the sequence of tasks. We assume that Ag2 knows the true capabilities of both agents, but Ag2 ’s true capabilities are hidden from Ag1 . In order to successfully adapt, Ag1 must therefore estimate Ag2 ’s capabilities and use the estimate to plan its own contributions.

3.1. Modelling Framework To model this setting, we extend the contextual MDP framework of Hallak et al. [54] by defining a finite, goal-conditioned2 , contextual Multi-agent Markov Decision Process (CMMDP): Mc (g) = ⟨S, A, T (s′ , a, s|c), R(s, a|g), G, C, γ⟩, where: • S is a finite set of discrete joint states (we assume a global shared state between both agents); • A = AAg1 × AAg2 is a finite set of joint actions, composed of individual actions a = {aAg1 , aAg2 }; • T : S × A × S × C → {0, 1} is the context-dependent transition function, which we assume to be deterministic in this paper; • R(s, a|g) is the goal-conditioned reward function, which encodes the goal and other order-inducing information such as the agents’ preferences over joint actions. The reward function is assumed to be known to both agents; • g ∈ G represents the goal that is shared and known to both agents in advance, and is an element of all possible goals in the domain G ⊆ S; and • γ → (0, 1] is the discount factor. We model Ag2 ’s capabilities as D-dimensional vectors of binary variables c → {0, 1}D . Each such vector induces a specific instance of Mc (g), and thus restricts the space of feasible joint plans. The set of context vectors C is therefore the space of Ag2 ’s possible capability vectors over which Ag1 reasons. 3.2. Capabilities as Transitions The capabilities modulate Mc (g) by restricting feasibility of Ag2 ’s actions through the transition function T . Each capability, which can have semantic meaning, can influence one or more transitions, and a single transition can be influenced by more than one capability. For a transition s → s′ to occur given a joint action a = {aAg1 , aAg2 }, there is a set of enabling capabilities that must all be present for the transition to 2

We use goal and task interchangeably throughout this paper, and assume that any task in a goal-conditioned MMDP can be described by its goal state.

change the state. More formally, let c′ ⊆ c be the (possibly empty) subset of capabilities affecting the transition ⟨s, a, s′ ⟩. The effect of the enabling capabilities is as follows: ( 1 if s′ = s, ′ ′ ∃ci ∈ c | ci = 0 =⇒ T (s , a, s | c) = 0 otherwise.

We model missing capabilities as self-loops rather than removing actions from the action set, because capability requirements depend on the state context (e.g., object properties or locations), not only on the action itself. An action may therefore be executable in one state but infeasible in another. 3.3. Inference and Planning Objectives Given that capabilities determine the induced CMMDP, Ag1 must infer c from observations in order to explain past behaviour and align its representation of Ag2 ’s capabilities to the partner. Estimating c allows Ag1 to instantiate the correct transition model T (·|c) and thus reason about feasible joint behaviour in future tasks. To do so, Ag1 seeks to align its model Mc (g) by finding the capabilities vector c such that the predicted joint trajectories, produced by Ag1 ’s planning process, match the team’s executed trajectories. Ag1 has access to observed joint state trajectories in the environment τobs = ⟨si ⟩H i=1 , as well as the known reward function. We assume that the executed trajectories are aligned to Ag2 ’s true capabilityconstrained model. In particular, when Ag1 ’s intended action conflicts with the joint plan induced by Ag2 ’s true capabilities, Ag2 is assumed to have the ability to correct Ag1 and enforce the aligned action instead. As a result, the trajectories observed by Ag1 are assumed to be feasible under the true CMMDP and they reflect coordination under Ag2 ’s joint plan. For simplicity, we do not model these alignment actions explicitly in the CMMDP, unlike communicative teamwork formulations such as that of Pynadath and Tambe [55]. More formally, Ag1 is looking to learn c∗ , the best estimate of c, that minimizes the following expression: X c∗ = argminĉ∈C L τĉ,g , τobs,g , (1) g∈Gseen

across all previous seen goals Gseen ⊂ G. Here, L is a similarity metric, measuring the difference between the joint trajectory planned by Ag1 under the current capability 10

vector estimate ĉ, τĉ,g , and observed joint trajectory τobs . We discuss the choice of similarity metrics in Section 4 and Appendix B. Once Ag1 estimates c∗ , it can induce the corresponding task model Mc∗ and compute a joint plan. Joint planning is therefore used to reason about coordination, not to control Ag2 , as Ag1 computes a team-level plan under c∗ , but executes only its own actions. More formally, the objective of Ag1 is to find a joint plan ρ = ⟨a1 , a2 , ..., aH ⟩ that maximises the reward whilst achieving g, by appropriately allocating the actions to each agent. Ag1 is optimising the following objective in an MMDP induced by the context c∗ , Mc∗ = ⟨S, A, T, R, γ⟩: ∗

ρ (g) = arg max ρ

∞ X

γ t R(st , at |g).

t=1

Ag1 therefore needs to find a joint plan ρ∗ (g) that solves the task while anticipating Ag2 ’s contributions under the estimated capability model. The resulting plan is joint in the sense that it allocates actions to both agents, but execution remains decentralised: Ag1 only carries out its own actions while using the plan to anticipate Ag2 ’s role. In summary, we formulate adaptation as the problem of inferring a hidden partner capability vector from observed joint trajectories and using its estimate to induce a partner model for future coordination. This creates a continual estimation and planning loop, as each newly observed task provides additional evidence about the partner’s capabilities, and the updated estimate is then used to plan subsequent interactions. 4. Methods In this section, we present Capability Estimation via Contextual Models (CECM), our method for inferring partner capabilities from observations of joint trajectories and using those estimates to support coordination in future tasks. CE-CM implements the estimation-planning loop described in Section 3. Given observed trajectories, the agent infers a capability vector that explains past observations, and uses this estimate to induce a model for planning subsequent interactions. At a high level, CE-CM operates iteratively over tasks (Figure 1, Algorithm 1). In the inference step, candidate capability vectors are evaluated by simulating the joint trajectory they induce and comparing them to observations. Hypotheses that are consistent with the observed trajectory are retained, forming an empirical approximation of the posterior. In the planning step, the agent uses the estimate to compute a joint plan for the next task, which is executed in a decentralised manner. 11

This formulation makes use of the same underlying capability model both to explain observed behaviour and to plan future actions. However, when multiple behaviours are compatible with the same capability vector, comparing observations to a single simulated trajectory can be brittle. To address this, we introduce CECM-Div, an extension that evaluates each capability hypothesis against a diverse set of simulated trajectories rather than a single rollout, improving robustness to behavioural variability. Algorithm 1: Capability Estimation via Contextual Models (CE-CM) Input: Sequence of tasks g1 , g2 , . . . , similarity threshold ε, sample budget N Output: Capability estimates ĉ∗ and plans ρ∗ (g) 1 Initialise sample set Q0 ← ∅; 2 for each task gn do 3 Observe trajectory τobs,gn ; 4 Qn ← Qn−1 ; 5 for i = 1 to N do 6 c ∼ Prior(c); 7 τc,gn ← JointPlanner(c, gn ); 8 L ← Similarity(τobs,gn , τc,gn ); 9 if L < ε then 10 Qn ← Qn ∪ c; 11 12 13

ĉ∗ ← Estimate(Qn ); if next task gn+1 exists then ρ∗ (gn+1 ) ← JointPlanner(ĉ∗ , gn+1 );

4.1. Updating the Capability Belief with Approximate Inference To update the intelligent agent’s beliefs over partner’s capabilities, the agent seeks to infer a posterior distribution P (c|τobs ) over the partner’s capability vector given the observed joint trajectory. In principle, the agent could update the posterior using Bayesian updating: P (c|τ ) ∝ L(τ |c) · P (c), where the posterior is proportional to the likelihood of the observed trajectory under a capability hypothesis and a prior over capabilities. However, evaluating the likelihood L(τ |c) is intractable in our setting, as it requires solving the forward planning problem under each candidate capability vector. The behaviour induced by a given c depends on both the reward structure and the capability-conditioned transition function, making direct likelihood computation impractical. We therefore approximate this likelihood using simulation-based inference, specifically Approximate Bayesian Computation (ABC) [56]. The key idea is to evaluate 12

capability hypotheses by generating trajectories under each candidate vector c and comparing them to the observed trajectory, accepting those that are sufficiently similar. Formally, this corresponds to sampling from an approximate posterior (lines 4–9 in Algorithm 1): Pϵ (c | τobs ) = {c : L(τc , τobs ) < ϵ}, where ϵ > 0 is a similarity threshold, and L is a distance metric between trajectories. The choice of L depends on the state representation: we use Jaccard distance for propositional (PDDL) states, and cosine similarity for real-valued state vectors. Let Qn = {c : L(τc,gn , τobs,gn ) < ϵ} be the multi-set of capability vectors that produce trajectories sufficiently similar to the observation in task gn . Intuitively, Qn contains capability hypotheses that are consistent with Sn the observed behaviour in task gn . The union of accepted samples across tasks, j=1 Qj , aggregates evidence over time and acts as an empirical approximation of samples from the posterior. CE-CM uses this accumulated sample set to compute a point estimate ĉ∗ of the partner’s capabilities (line 11 in Algorithm 1). To obtain a parametric estimate, we approximate the posterior as a multivariate Bernoulli distribution. Each capability ci is treated as an independent Bernoulli random variable with parameter θi ≡ P (ci = 1), so that the belief at iteration n is represented as a vector β n (c) = [θ1 , . . . , θD ]⊤ . The parameters in β are updated by computing the empirical mean over accepted samples: Sn β

n+1

+ j=1 Qj (ci )

(ci ) =

j=1 Qj

where Q+ j (ci ) = {ĉ ∈ Qj | ĉi = 1} are all accepted samples in which the capability ĉi was present. Finally, the MAP estimate is obtained by thresholding: ( 1 if β n+1 (ci ) > ψ, ĉ∗i = 0 otherwise. The resulting estimate ĉ∗ can be interpreted as an approximate solution to Equation 1, as it selects the capability vector that best explains the observed trajectories under the ABC approximation, and is used to induce a capability-conditioned model for planning future interactions. 13

4.2. Planning with Capability Estimates Once CE-CM obtains an estimate ĉ∗ of the partner’s capabilities, it uses this estimate to induce a capability-conditioned model Mĉ∗ (g) and compute a joint plan for the next task (line 13 in Algorithm 1). Planning is performed using a joint planner over the induced CMMDP, assigning actions to both agents while respecting the capability constraints. As discussed in Section 3, execution remains decentralised: Ag1 carries out only its own actions, using the joint plan to anticipate the partner’s behaviour. The specific planning implementation depends on the domain representation and is described in Appendix B. 4.3. CE-CM-Div: Extending CE-CM to Suboptimal and Diverse Trajectories CE-CM implicitly assumes that each capability vector induces a single joint trajectory that the partner will follow. For this assumption to hold, the agents’ plans must be optimal with respect to their capabilities, and their preferences must be fully specified to induce strong orderings on behaviours. In practice, agents may act suboptimally, follow unmodelled preferences, or choose among multiple behaviours with equal returns, giving rise to multiple plausible trajectories under the same capabilities. In such cases, comparing observations to a single simulated trajectory can cause CE-CM to incorrectly reject valid capability hypotheses. To address this, we introduce CE-CM-Div, an extension that modifies the likelihood approximation in CE-CM. Instead of generating a single trajectory for each capability hypothesis, CE-CM-Div evaluates each hypothesis against a set of diverse trajectories ∆ that capture different feasible behaviours under that capability vector. For each candidate capability vector ĉ, the planner produces a set of trajectories ∆ĉ,g for the observed goal g, and the hypothesis is accepted when at least one trajectory in this set falls within the ABC tolerance under the trajectory-distance metric L, i.e., when minτ ∈∆ĉ,g L(τ, τobs,g ) ≤ ϵ. This prevents rejecting a capability vector due to the single rollout produced by the planner differs from the human’s chosen behaviour. We consider such rejections as false negatives, because the capability hypothesis is feasible and compatible with the observation, but is incorrectly discarded due to unmodelled behavioural diversity. This modification replaces single-trajectory comparison in CE-CM with a setbased comparison, while leaving the rest of the inference and planning loop unchanged. In practice, the diverse trajectory set ∆ can be generated using any planner capable of producing multiple distinct solutions. In our implementation, we use a diversity-aware variant of Monte Carlo Tree Search (MCTS) to generate a set of trajectories that differ in their visited states, following prior work on diverse planning [57, 58]. Full implementation details are provided in Appendix B.4. 14

5. Experiments We evaluate CE-CM along three core hypotheses: • H1 (Capability inference): CE-CM can recover a partner’s latent capability vector from interaction across tasks. • H2 (Coordination impact): Improved capability estimates lead to better coordination, reflected in reduced misallocation of actions and increased agreement between agents. • H3 (Robustness to behavioural variability): When observed behaviour is not well explained by a single trajectory, CE-CM-Div improves capability inference over CE-CM by accounting for multiple plausible behaviours. We test these hypotheses in two collaborative domains with complementary properties3 . TidyUP is a PDDL 2.1 [59] domain with fully specified rewards and strong ordering over joint plans, allowing us to evaluate capability inference and its downstream impact in a well-defined setting. Overcooked is a simulator-based coordination domain in which multiple joint behaviours can achieve similar outcomes, making it suitable for evaluating the limits of capability-based reasoning and the effects of behavioural variability. In addition to simulated partners, we collect human gameplay data in Overcooked to evaluate performance under realistic behaviour. To evaluate these hypotheses, we use the following metrics: • Capability inference accuracy: Hamming distance between the inferred and ground-truth capability vectors, measuring how well the agent recovers the partner’s capabilities. • Coordination quality: (i) the ratio of corrections, capturing how often the agent proposes actions that conflict with the partner’s behaviour, and (ii) plan overlap, measuring agreement in action allocation between the agents. • Assigned action feasibility: the proportion of unproductive (infeasible) actions assigned to the partner, and the proportion of feasible actions in open-loop planning on unseen tasks. 3

All of our code is available at https://github.com/Ptisni/CE-CM

The remainder of this section is organised as follows. In Section 5.1, we evaluate CE-CM in TidyUP, focusing on capability inference and its impact on coordination in a well-specified setting. In Section 5.2, we analyse CE-CM in Overcooked with simulated partners, highlighting both its strengths and its limitations when multiple behaviours are compatible with the same capabilities. Finally, in Section 5.3, we evaluate CE-CM-Div on human data and show that accounting for behavioural diversity substantially improves capability inference in realistic settings. 5.1. TidyUP

Kitchen

Living Room

Study

Figure 2: TidyUP domain. Left: one of the possible starting states, where the blanket and a dirty bowl are on the couch, the dirty dish is in the study, and the toothbrush is in the kitchen. Right: possible goal state where the blanket is on the bed, the bowl and the dish are clean and in the kitchen, and the toothbrush is in the bathroom.

We first evaluate CE-CM in TidyUP, a PDDL-based household domain in which a human and their robot assistant jointly tidy the apartment by cleaning and rearranging objects across several rooms (see Figure 2 for an example scenario, and Appendix A.1 for a more detailed description). The joint state encodes object locations, object cleanliness, and agent positions. Transitions are deterministic and governed by action preconditions, which correspond directly to capabilities (e.g., an agent can only pick up a plate if can_pick(plate) is true). The reward function specifies action costs, inducing a clear ordering over joint plans. The domain contains 20 capabilities, yielding a 20-dimensional capability vector. The robot is fully capable, while the simulated human exhibits restricted capabilities. We consider two partner types:

1. Low-capability (LC) human, who can move freely and manipulate objects only in the bathroom and kitchen; and 2. High-capability (HC) human, who can perform all actions except placing objects in the bedroom and living room. The simulated human plans using the ground-truth CMMDP Mc and has full knowledge of both agents’ capabilities, while the robot must infer the human’s capabilities from interaction. Both agents plan independently but execute only their own actions. To ensure that the executed trajectory reflects the true capabilityconstrained model, we allow the human to correct the robot’s actions when they conflict with the human’s plan, effectively enforcing alignment during execution. This mirrors the alignment assumption introduced in Section 3. 5.1.1. Experimental Setup and Evaluation Each experimental run consists of a sequence of 5 tasks sampled from the domain’s goal set. After each observed task, CE-CM updates its capability estimate and uses it to plan the next task. We repeat this process over 20 independent runs. This setup directly evaluates H1 (capability inference) and H2 (coordination impact) as the agent accumulates experience over tasks. We compare CE-CM against two baselines. The optimistic baseline assumes the partner is fully capable and does not adapt over time. The pessimistic baseline enables capabilities only after they are directly observed through successful transitions, effectively following a “what you see is what you believe” strategy. While this baseline can perform well in domains with explicit action preconditions, such as PDDL, it relies on knowing the mapping between transitions and capabilities, which may not generalise to other settings. All implementation details for CE-CM and the baselines are provided in Appendix B.1. 5.1.2. Results We first evaluate whether CE-CM recovers the partner’s capabilities from interaction. Figure 3 shows that CE-CM rapidly reduces Hamming distance for both partner types, converging to near-zero error within five observed tasks. In contrast, the optimistic baseline fails to correct its initial assumptions, while the pessimistic baseline converges more slowly due to relying solely on direct observation. These results demonstrate that CE-CM can efficiently infer the partner’s capability vector from a small number of collaborative tasks. We next evaluate whether improved capability estimates lead to better coordination. Figure 4 shows that the proportion of corrected actions decreases substantially as more tasks are observed, dropping from approximately 75% to below 30% for HC 17

1.0 CE-CM Optimistic Baseline Pessimistic Baseline

0.8 0.6 0.4 0.2 0.0

Hamming Distance

1.0

0.8 0.6 0.4 0.2 0.0

CE-CM Optimistic Baseline Pessimistic Baseline

Tasks

(a) High-capability type

(b) Low-capability type

1.00

0.75

Corrections

Figure 3: Hamming distance between ground truth and inferred capability vectors. Solid line indicates the mean, and shaded area denotes the 95% Confidence Interval.

0.50 0.25 0.00

2 Tasks

0.50 0.25 0.00

(a) High-capability type

2 Tasks

(b) Low-capability type

Figure 4: Ratio of corrected actions across tasks. Solid line indicates the mean, and shaded area denotes the 95% Confidence Interval.

and below 5% for LC. This indicates that the agent increasingly assigns actions in a manner consistent with the partner’s capabilities. However, correction rates do not fully vanish. This is partly due to temporal mismatches: even when both agents assign actions to the same agent, differences in execution order may lead to corrections. To better capture agreement in task allocation, we measure plan overlap using IoU (Figure 5). CE-CM achieves substantial improvements, reaching over 75% overlap for HC and above 95% for LC, outperforming both baselines. This indicates that CE-CM correctly identifies which agent should perform which actions, even when execution order varies. In summary, we have shown that in a well-specified domain such as TidyUP, CECM accurately infers the partner’s capabilities and uses them to produce joint plans that align closely with the partner’s behaviour, supporting both H1 and H2.

1.00

0.75

0.75 IoU

IoU

1.00

0.50 0.25 0.00

0.50 0.25

0 CE-CM

2 Tasks

Optimistic Baseline

0.00

Pessimistic Baseline

0 CE-CM

(a) High-capability type

2 Tasks

Optimistic Baseline

Pessimistic Baseline

(b) Low-capability type

Figure 5: Plan overlap (IoU) between joint plans. Solid line indicates the mean, and shaded area denotes the 95% Confidence Interval.

5.2. Overcooked We next evaluate CE-CM in Overcooked, a common collaborative benchmark [23, 60] in which two agents must coordinate to prepare and deliver dishes by gathering ingredients, chopping, plating, and serving (Figure 6, see Appendix A.2 for details). We use a macro-action variant of the environment [61], where low-level controls are abstracted into high-level actions such as get tomato, chop, and deliver. The capability vector in this domain is 10-dimensional. The intelligent agent (blue) is fully capable, while the partner agent (green) may lack capabilities. We consider three partner types: 1. Type 1: cannot handle any vegetables (cannot pick up lettuce, onion, or tomato); 2. Type 2: cannot cut or deliver dishes; 3. Type 3: cannot handle plates or deliver dishes. All partner types are controlled by the same joint planner, operating in the groundtruth CMMDP Mc (g). Compared to TidyUP, Overcooked exhibits significant underspecification: many distinct joint plans can achieve similar returns. As a result, small differences in capabilities can lead to qualitatively different behaviours, and coordination becomes sensitive to conventions rather than feasibility alone. For example, multiple valid strategies exist for assembling a dish, even though the temporal structure of subtasks (e.g., chopping before plating) remains constrained. This makes Overcooked a suitable testbed for evaluating the limits of capability-based reasoning. 5.2.1. Experimental Setup and Evaluation We sample 5 tasks for learning from the set of 9 layout–recipe combinations (Figure 6). After each observed task, CE-CM updates its capability estimate and uses 19

Recipes

Level 3

Level 2

Level 1

Layouts

lettuce-onion-tomato

lettuce-tomato

onion-tomato

lettuce-onion-tomato

lettuce-onion

onion-tomato

lettuce-onion-tomato

lettuce-onion

lettuce-tomato

Figure 6: Overview of Overcooked. There are three distinct layouts, and three possible recipes, creating nine distinct tasks in total. The agents must collaborate to deliver the dish as quickly as possible, ensuring that the dish consists of the correct ingredients.

it to plan the next task. To evaluate open-loop generalisation, we assess performance on the remaining 4 held-out tasks. We conduct two main evaluations. First, we measure H1 capability inference accuracy over 20 runs per partner type using Hamming distance. Second, we evaluate H2 downstream coordination by measuring the proportion of infeasible (unproductive) actions assigned to the partner, and the proportion of feasible actions in open-loop planning on held-out tasks. We compare CE-CM against an optimistic baseline that assumes the partner is fully capable and does not adapt over time. We do not include the pessimistic baseline from TidyUP, as the mapping between individual transitions and capabilities is not explicitly defined in this domain (i.e., the simulator admits the whole capability vector at initialisation). 5.2.2. Results We first evaluate whether CE-CM recovers partner capabilities from repeated interaction. Figure 7 shows that CE-CM consistently reduces Hamming distance across all three partner types, reaching values between 0.18 and 0.2 after five tasks. Importantly, CE-CM is able to reduce error even when the exact capability vector is not sampled from the prior (e.g., Type 1, Figure 7a), indicating that the method can recover a useful approximation of the underlying capability structure. In all cases, CE-CM significantly outperforms the optimistic baseline. We next examine whether improved capability estimates translate into better 20

0.6 0.4 0.2 0.0

3 Tasks

0.8

0.6

0.4 0.2 0.0

Hamming Distance

CE-CM Baseline

Hamming Distance

0.8

(a) Type 1

3 Tasks

0.4 0.2 0.0

(b) Type 2

3 Tasks

Figure 7: Hamming distance between ground truth capabilities and the estimated capability vector. Solid line indicates the mean, and shaded area denotes the 95% Confidence Interval.

0.4 0.2 0.0

3 Tasks

(a) Type 1

0.6 Corrections

Corrections

0.6

0.4 0.2 0.0

3 Tasks

(b) Type 2

0.4 0.2 0.0

3 Tasks

Figure 8: Ratio of corrected actions across tasks. Solid line indicates the mean, and shaded area denotes the 95% Confidence Interval.

coordination. Unlike in TidyUP, Figure 8 shows little change in the proportion of corrected actions for Types 1 and 2, with only a modest reduction for Type 3. This indicates that, despite improving capability estimates, the agent does not converge to the same joint behaviour as its partner. This discrepancy arises from the underspecified nature of the domain. Multiple joint plans can achieve similar returns, meaning that knowing the correct capability constraints does not uniquely determine the partner’s behaviour. We confirm this by examining the proportion of infeasible actions assigned to the partner. Figure 9 shows that CE-CM substantially reduces unproductive actions across all partner types, indicating that it correctly captures feasibility constraints even when it fails to match the partner’s exact behaviour. We further evaluate CE-CM in open-loop planning on held-out tasks. Figure 10 shows that CE-CM consistently produces plans with a high proportion of feasible actions, outperforming the optimistic baseline across all partner types. This demonstrates that CE-CM improves safety and robustness by avoiding infeasible action assignments. In the Overcooked domain, CE-CM successfully recovers capability constraints (H1) and improves feasibility and safety of action assignment, but does not fully 21

0.3 0.2 0.1 0.0

3 Tasks

0.5

Ratio of Unproductive Actions

CE-CM Baseline

0.4

Ratio of Unproductive Actions

0.5

0.4 0.3 0.2 0.1 0.0

(a) Type 1

3 Tasks

0.5 0.4 0.3 0.2 0.1 0.0

(b) Type 2

3 Tasks

1.00

0.95

0.90

0.85 0.80 0.75 0.70 0.65 0.60

Ratio of Safe Actions

1.00

Ratio of Safe Actions

Figure 9: Ratio of unproductive (infeasible) actions across tasks. Solid line indicates the mean, and shaded area denotes the 95% Confidence Interval.

0.85 0.80 0.75 0.70 0.65

CE-CM

(a) Type 1

Baseline

0.60

0.85 0.80 0.75 0.70 0.65

CE-CM

Baseline

(b) Type 2

0.60

CE-CM

Baseline

Figure 10: Proportion of feasible actions in open-loop planning on held-out tasks. Solid line indicates the mean, and shaded area denotes the 95% Confidence Interval.

Robbie's intention

Commands Ingredients Capabilities

Robbie Player

Figure 11: Overview of the Overcooked user interface. In the top left corner, the intelligent assistant agent indicates its next action. The available commands, the current recipe, and the restricted capabilities are displayed on the right hand side.

resolve coordination (H2). This highlights a key limitation: capabilities alone are insufficient to predict behaviour when multiple strategies are equally valid. 5.3. Capability Inference from Human Players The results in Overcooked highlight a key limitation of CE-CM: when multiple behaviours are consistent with the same capability vector, matching observations to a single simulated trajectory becomes unreliable. To evaluate this failure mode in a realistic setting, and to test whether CE-CM-Div addresses it, we conduct a human-subject study in the Overcooked domain. Human behaviour in this setting exhibits substantial variability even within the same capability profile and task. Different players may adopt distinct strategies, conventions, or preferences, all of which can produce valid but diverse trajectories under identical capabilities. This form of behavioural ambiguity directly violates the single-trajectory assumption underlying CE-CM and provides a natural testbed for evaluating CE-CM-Div. Additional details and analysis of the dataset are provided in Appendix E.

5.3.1. Experimental Setup and Evaluation To evaluate CE-CM and CE-CM-Div under realistic behavioural variability, we collect a dataset of human–agent interactions in the Overcooked domain4 . We recruited 15 participants, each of whom completed 15 episodes in collaboration with a fixed-policy assistant agent. Tasks were sampled from the same nine layout–recipe combinations used in previous experiments, with each participant interacting across all three capability types. At each decision point, participants selected high-level actions (e.g., get ingredient, chop, deliver ) and were shown the assistant’s intended action, which they could correct if needed (the interface and the assistant can be seen in Figure 11). This interaction protocol mirrors the correction-based execution model used in our simulations, ensuring consistency between experimental settings. We record sequences of joint states along with the ground-truth capability vector associated with each trajectory, yielding a dataset of 225 trajectories (75 per capability type). Full details of data collection and participant demographics are provided in Appendix D. We evaluate both methods in an offline setting, where capability vectors are inferred directly from recorded trajectories. Our primary objective is to assess whether CE-CM-Div improves capability inference over CE-CM when learning from human behaviour. 5.3.2. Results We first compare CE-CM and CE-CM-Div in terms of their ability to generate consistent posterior updates. Figure 12 shows a stark difference in sample acceptance rates: CE-CM accepts very few capability samples per task (typically fewer than 10), whereas CE-CM-Div admits substantially more (between 120 and 175 after five tasks). This indicates that CE-CM frequently fails to generate a trajectory sufficiently similar to human behaviour, preventing it from updating its belief. In contrast, CE-CM-Div matches observed behaviour more reliably by evaluating each hypothesis against a set of trajectories. We next evaluate capability estimation accuracy directly. Figure 13 shows that CE-CM-Div substantially outperforms CE-CM across all capability types, reducing Hamming distance to approximately 0.29, 0.39, and 0.27 for Types 1, 2, and 3, respectively. In contrast, CE-CM achieves significantly higher error (0.49, 0.75, and 0.46) and frequently fails to update its estimate. When CE-CM does update, it can 4

We obtained permission to conduct this study from King’s College London Research Ethics Office (Study registration number: MRSP-24/25-50047).

100 50 0

3 Tasks

150 100 50 0

150 Posterior Size

Posterior Size

200

CE-CM CE-CM-Div

150

(a) Type 1

3 Tasks

100 50 0

(b) Type 2

3 Tasks

Figure 12: Number of accepted samples in the posterior across observed tasks. Solid line indicates the mean, and shaded area denotes the 95% Confidence Interval.

0.8 0.6 0.4 0.2

3 Tasks

(a) Type 1

1.0

1.0 Hamming Distance

CE-CM CE-CM-Div

Hamming Distance

1.0

0.8 0.6 0.4 1

3 Tasks

(b) Type 2

0.8 0.6 0.4 0.2

3 Tasks

Figure 13: Hamming distance between ground truth and inferred capability vectors. Solid line indicates the mean, and shaded area denotes the 95% Confidence Interval.

produce accurate estimates, but this depends on the human following a trajectory similar to the planner’s prediction. These results demonstrate that CE-CM-Div provides a robust solution to capability inference under behavioural variability, producing richer posterior estimates and significantly improving accuracy over CE-CM. Additional analysis of the influence of diversity parameter δ on CE-CM-Div’s performance is provided in Appendix C.3. 6. Discussion Our experimental results show that CE-CM can reliably recover a partner’s capabilities in well-specified settings, and that these estimates can improve coordination when behaviour is largely determined by feasibility constraints. In TidyUP, CECM rapidly converges to the correct capability vector and produces joint plans that closely align with the partner. In contrast, in Overcooked, although CE-CM still learns meaningful approximations of the capability vector, these estimates do not always translate into improved coordination. CE-CM-Div addresses this limitation, enabling robust capability inference from human demonstrations where behaviour is highly variable. 25

At the same time, our experiments reveal a key insight: correctly estimating a partner’s capabilities is not sufficient to predict their behaviour. Capabilities constrain which plans are feasible, but they do not determine which plan will be executed. In domains such as Overcooked, where multiple joint behaviours can achieve similar outcomes, the absence of strong preferences leads to ambiguity that cannot be resolved through capability inference alone. This explains why improved capability estimates do not necessarily reduce coordination errors in such settings. CE-CM-Div mitigates this limitation by reasoning over a set of plausible trajectories for each capability hypothesis. By matching observations against multiple rollouts rather than a single predicted trajectory, it becomes significantly more robust to behavioural variability and suboptimality, as demonstrated in the human-subject study. However, CE-CM-Div still does not explicitly model preferences, and therefore cannot systematically resolve ambiguity between equally feasible behaviours. In addition, it incurs higher computational cost due to the need for diverse planning. A further limitation of both CE-CM and CE-CM-Div is scalability beyond dyadic interactions. Joint planning scales exponentially with the number of agents, making direct extensions to larger teams impractical. Addressing this limitation will likely require structured approximations, such as decomposed planning, role-based reasoning, or factorised representations of multi-agent interactions. Finally, a promising direction for future work is to jointly model capabilities and preferences. Such an approach would allow agents to distinguish between feasibility constraints and behavioural tendencies, enabling more accurate prediction of partner behaviour. Incorporating preference inference, for example through hierarchical or layered partner models [28], could significantly improve coordination in settings where multiple valid strategies exist. 7. Conclusion In this paper, we introduced CE-CM, a framework for task-agnostic adaptation in ad-hoc teamwork based on inferring latent partner capabilities. By modelling capabilities as task-invariant constraints on feasible behaviour, CE-CM provides an explicit and interpretable representation of what a partner can and cannot do, enabling agents to refine their beliefs across tasks and coordinate through capabilityconditioned planning. Our experiments show that CE-CM can reliably recover partner capabilities in well-specified settings and use these estimates to improve coordination by avoiding infeasible or unsafe action assignments. However, we also demonstrate a key limitation: capabilities alone are not sufficient to predict behaviour when multiple valid 26

strategies exist. In such settings, coordination depends not only on feasibility but also on preferences and conventions. To address this, we introduced CE-CM-Div, which extends capability inference by reasoning over sets of plausible behaviours rather than single trajectories. This significantly improves robustness to behavioural variability, particularly in human–AI teaming scenarios where observed behaviour is diverse and often suboptimal. Overall, our results highlight capability inference as a principled mechanism for generalising across tasks and partners, while also emphasising the importance of modelling behavioural diversity in realistic settings. Future work includes integrating preference inference alongside capabilities, improving computational efficiency for real-time deployment, and extending the approach to larger multi-agent teams.

Appendix A. Domain Descriptions In this section, we provide detailed descriptions of the two experimental domains that we used to evaluate CE-CM and CE-CM-Div. Appendix A.1. TidyUP TidyUP is a PDDL-based human–robot collaboration domain representing a household environment. The domain consists of: • Rooms: {kitchen, living_room, bedroom, study, bathroom} • Objects: {plate, bowl, toothbrush, blanket} • Object states: {dirty, clean} • Actions: {move, pick, place, wash, no_op} States are represented as binary predicate vectors encoding object locations, object states, and agent positions. Joint actions are defined as A = A × A \ {(no_op, no_op)}. The objective is to reach a goal state in which all objects are clean and placed in designated rooms. Action costs induce preferences over joint plans: • The human prefers handling plates and bowls (cost 1 vs. 4 for the robot), • Slight preference for toothbrush (1 vs. 2), • No preference for blanket (cost 1 for both). Transitions are deterministic and governed by action preconditions. A transition occurs only if all required conditions and capabilities are satisfied; otherwise, the action results in a self-loop. For example, picking up an object requires being in the same room, having free hands, and possessing the corresponding capability. The domain contains 20 binary capabilities, summarised in Table A.1. Some capabilities (e.g., washing outside the kitchen) are inactive due to environmental constraints and do not affect behaviour. The robot agent is fully capable. The human agent’s capability vector varies across experiments.

Capability

Description

place_living_room place_kitchen place_bedroom place_bathroom place_study pick_living_room pick_kitchen pick_bedroom pick_bathroom pick_study move_living_room move_kitchen move_bedroom move_bathroom move_study wash_living_room wash_kitchen wash_bedroom wash_bathroom wash_study

Ability to put down objects in the living room Ability to put down objects in the kitchen Ability to put down objects in the bedroom Ability to put down objects in the bathroom Ability to put down objects in the kitchen Ability to pick up objects in the living room Ability to pick up objects in the kitchen Ability to pick up objects in the bedroom Ability to pick up objects in the bathroom Ability to pick up objects in the kitchen Ability to go to the living room Ability to go to the kitchen Ability to go to the bedroom Ability to go to the bathroom Ability to go to the kitchen Ability to clean objects in the living room Ability to clean objects in the kitchen Ability to clean objects in the bedroom Ability to clean objects in the bathroom Ability to clean objects in the kitchen

Table A.1: List of capabilities in the TidyUP environment.

Appendix A.2. Overcooked We use a macro-action version of Overcooked based on [61]. Macro actions abstract low-level movements into high-level commands. The individual movement and interaction commands are produced by an A∗ planner that, given a macro action as a goal, produces a sequence of primitive actions that execute the macro action. The domain consists of: • Agents: {blue_agent, green_agent} • Objects: {lettuce, onion, tomato, plate1 , plate2 , knif e1 , knif e2 , delivery} • Vegetable states: {raw, chopped} • Actions: {stay, get_lettuce, get_tomato, get_onion, get_plate_1, get_plate_2, go_to_knif e_1, go_to_knif e_2, deliver, chop, out_of _the_way}. The state representation combines binary and continuous features encoding object positions, agent positions, and task information. Joint actions are defined as A = A × A \ {(stay, stay)}. Transitions are deterministic and conditioned on capabilities: actions that require unavailable capabilities result in self-loops. The domain contains 29

10 binary capabilities (Table A.2), defining whether the agent can interact with specific objects or perform specific operations. The reward structure is sparse: • −0.1 per step, • +0.5 for correct chopping, • +1 for delivering the correct dish, • −1 for incorrect delivery. We generate 9 tasks using 3 layouts and 3 recipes. Transitions are deterministic and conditioned on capabilities: actions that require unavailable capabilities result in self-loops. The domain contains 10 binary capabilities (Table A.2), defining whether the agent can interact with specific objects or perform specific operations. The blue agent (intelligent agent) is fully capable, while the green agent’s capability vector varies across experiments. Capability

Description

handle_lettuce handle_onion handle_tomato handle_plate_1 handle_plate_2 handle_knif e_1 handle_knif e_2 deliver chop out_of _the_way

Ability to handle raw lettuce Ability to handle raw onion Ability to handle raw tomato Ability to handle plate 1 Ability to handle plate 2 Ability to use knife 1 Ability to use knife 2 Ability to deliver a dish Ability to chop vegetables Ability to move to the middle of the kitchen

Table A.2: List of capabilities in the Overcooked environment.

Appendix B. Methods and Baselines Implementation Details This appendix provides implementation details for CE-CM, CE-CM-Div, and all baselines. The main differences between implementations arise from the choice of planner, similarity metric, and sampling parameters.

Appendix B.1. CE-CM with PDDL We use the POPF planner [62] to generate joint plans in TidyUP. At each iteration, we sample N = 500 capability vectors from a Bernoulli prior with P (ci = 1) = 0.8. Trajectory similarity is measured using Jaccard distance over binary state representations. We set the acceptance threshold to ε = 0.24. The capability estimate is obtained by thresholding the Bernoulli parameters with ψ = 0.5. The optimistic baseline assumes full capabilities and performs no learning. The pessimistic baseline updates capabilities only when observed, using a cumulative OR over inferred capabilities ĉn−1 to update the belief ĉn = ĉn−1 ⊕ cobs . Appendix B.2. CE-CM with MCTS In Overcooked, we implement CE-CM using PUCT MCTS [63]. The planner is ∗ , yielding the selection rule: biased by a pre-trained joint-action policy πpt # " p n(s) ∗ , a∗ = arg max Q̂(s, a) + cp πpt (a|s) · 1 + N (s, a) a∈A with cp = 1. We sample N = 500 capability vectors from the full space (210 ). For efficiency, rollout trajectories are precomputed and reused across tasks. We also assume access to a function f : T × A → {0, 1}, which prunes infeasible actions from the planner’s consideration. We do so to reduce the branching factor of the joint planner and stop it from considering unproductive actions and help it find feasible joint plans quicker. Trajectory similarity is measured using cosine similarity over flattened state vectors, with threshold ε = 0.03. Table B.3 summarises MCTS hyperparameters. Parameter

Value

Number of simulations Number of leaf node rollouts Maximum rollout depth Discount factor γ

200 12 20 0.99

Table B.3: Hyperparameters for the PUCT MCTS planner.

Appendix B.3. Q-learning Baseline We implement the optimistic baseline using a tabular Q-learning policy assuming a fully capable partner. The policy is trained for 70M steps using softmax exploration and replay. Other hyperparameters can be found in Table B.4 below. 31

At inference time, actions are selected using a softmax policy with temperature τ = 0.1. We note here that the behaviour of the Q-policy-based agent approximates to one that uses MCTS to plan its actions assuming a fully-capable partner when the Q-policy has converged. We make this assumption in our evaluation, and thus do not compare to an MCTS agent in our experiments. Parameter

Value

No. of training steps Initial Q values Exploration mechanism Softmax policy temperature Replay buffer size Replay buffer update frequency Learning rate α Discount factor γ

70M 0.01 Softmax / Random (0.8 / 0.2) 1.0 100k 100k 0.1 0.99

Table B.4: Hyperparameters used for the baseline policy.

Appendix B.4. CE-CM-Div CE-CM-Div extends CE-CM by evaluating each capability hypothesis against a set of trajectories ∆ rather than a single rollout. In our implementation, we generate the set of trajectories ∆ using a diversity-aware MCTS procedure [57]. Candidate trajectories are extracted from the MCTS search tree and filtered to ensure diversity. Diversity is measured using trajectory set difference [58]: |Sτ \ Sτ ′ | 1 min . Diversity(τ, Γ) = |Γ| τ ′ ∈Γ |Sτ | We retain up to k = 50 trajectories with diversity above threshold δ = 0.35. Table B.5 summarises the hyperparameters. Algorithm 2 describes the extraction procedure. We emphasise that CE-CM-Div is not tied to this implementation; any method capable of generating diverse trajectories can be used. Parameter

Value

Maximum no. of trajectories k Diversity parameter Minimum trajectory length Maximum trajectory length

50 0.35 Tree depth

Table B.5: Hyperparameters used for the breadth-first search algorithm.

We emphasise that CE-CM-Div is not tied to this specific implementation. Any planning method capable of generating multiple diverse trajectories under a given capability hypothesis can be used in place of DiverseMCTS. The choice of planner and diversity metric may affect performance, but does not change the underlying inference procedure. Algorithm 2: DiverseMCTS Input: Task g, capability estimate c, minimum length Lmin , diversity threshold δ, maximum number of trajectories k Output: Diverse trajectory set ∆ = ⟨τ1 , . . . , τk ⟩ 1 root ← MCTS(g, c); 2 Γ ← ∅; 3 open_list ← [[root]]; 4 while open_list do 5 p ← open_list.pop(); 6 n ← p.last(); 7 L ← |p| − 1; 8 if L ≥ dmax or n.children = ∅ then 9 if L ≥ Lmin then 10 Γ ← Γ ∪ p; 11 12 13

else foreach child c ∈ n.children do open_list ← open_list ∪ [p.extend(c)];

Sort Γ by descending length; ∆ ← ∅; 16 foreach p ∈ Γ do 17 if |∆| ≥ k then 18 break; 14 15

19 20 21 22

d ← Diversity(p, Γ); if d ≥ δ then ∆ ← ∆ ∪ p; return ∆

Appendix C. Additional Results This section provides supplementary results that support the findings presented in Section 5. These experiments offer additional insight into (i) how capability estimates evolve over time, (ii) how CE-CM adapts to changing capabilities, and (iii) how the diversity parameter δ affects the performance of CE-CM-Div. While these results are not central to the main claims of the paper, they provide further evidence of the behaviour and limitations of our methods.

wash living room

0 0

2 3 Tasks

move kitchen

0 0

2 3 Tasks

wash kitchen

0 0

2 3 4 Tasks CE-CM

0 0

2 3 Tasks

move bedroom

0 0

2 3 Tasks

wash bedroom

0 0

2 3 4 Tasks Pessimistic Baseline

2 3 Tasks

pick bathroom

0 0

2 3 Tasks

move bathroom

0 0

2 3 Tasks

wash bathroom

0 0

2 3 Tasks True Capability

Capability

pick bedroom

Capability

2 3 Tasks

Capability

2 3 Tasks

Capability

2 3 Tasks

Capability

move living room

Capability

pick kitchen

place study 1

Capability

2 3 Tasks

Capability

2 3 Tasks

Capability

place bathroom 1

Capability

pick living room

Capability

place bedroom 1

Capability

2 3 Tasks

Capability

place kitchen 1

Capability

Capability Capability

Capability

place living room 1

0 0

2 3 Tasks

pick study

0 0

2 3 Tasks

move study

0 0

2 3 Tasks

wash study

0 0

2 3 Tasks

(a) HC Type

move living room

0 0

2 3 Tasks

wash living room

0 0

2 3 Tasks

pick kitchen

0 0

2 3 Tasks

move kitchen

0 0

2 3 Tasks

wash kitchen

0 0

2 3 4 Tasks CE-CM

0 0

2 3 Tasks

pick bedroom

0 0

2 3 Tasks

move bedroom

0 0

2 3 Tasks

wash bedroom

0 0

2 3 4 Tasks Pessimistic Baseline

Capability

2 3 Tasks

Capability

2 3 Tasks

Capability

place bathroom 1

Capability

pick living room

Capability

place bedroom 1

Capability

2 3 Tasks

Capability

place kitchen

Capability

place living room

0 0

2 3 Tasks

pick bathroom

0 0

2 3 Tasks

move bathroom

0 0

2 3 Tasks

wash bathroom

0 0

2 3 Tasks True Capability

place study

0 0

2 3 Tasks

pick study

0 0

2 3 Tasks

move study

0 0

2 3 Tasks

wash study

0 0

2 3 Tasks

(b) LC Type

Figure C.14: Evolution of individual capability estimates across tasks. Solid lines show the mean over 20 runs, and shaded regions indicate 95% confidence intervals.

Appendix C.1. Evolution of Capability Estimates We visualise the evolution of individual capability estimates across tasks in TidyUP. CE-CM converges to the correct capability values more rapidly than the pessimistic baseline, particularly when multiple capabilities must be inferred jointly. This highlights the advantage of reasoning over the full capability vector rather than updating each capability independently. 34

Appendix C.2. Adaptation to Changing Capabilities We evaluate CE-CM in a setting where the partner’s capabilities change during interaction. The method successfully abandons outdated beliefs and converges to the new capability profile, reducing Hamming distance to approximately 0.1. This demonstrates that maintaining a distribution over capability hypotheses enables rapid adaptation when the underlying partner model changes.

Hamming Distance

0.8

CE-CM Baseline

0.6 0.4 0.2 0.0

1 2 3 4 5 6 7 8 9 10 11 12 13 14 Tasks

Figure C.15: Adaptation to changing capabilities. The partner switches from Type 2 (green) to Type 3 (red) after 5 tasks. Solid line indicates the mean, and shaded area denotes the 95% Confidence Interval.

0.5

Hamming Distance

Average Nτ

Appendix C.3. Effect of the Diversity Parameter in CE-CM-Div

20 0

0.4 0.3 0.2

0.15 0.25 0.35 0.45 0.55 0.65 0.75 Diversity Parameter

(a) Number of trajectories per capability vector

Type 1 Type 2 Type 3 All 0.15 0.25 0.35 0.45 0.55 0.65 0.75 Diversity Parameter (b) Capability estimation accuracy

Figure C.16: Impact of the diversity parameter δ on trajectory diversity and estimation performance.

We analyse the effect of the diversity parameter δ in CE-CM-Div. Increasing δ reduces the number of trajectories retained per capability vector, as stricter diversity constraints filter out more behaviours (Figure C.16a). 35

Estimation performance exhibits a non-monotonic relationship with δ (Figure C.16b). Moderate values yield the best results, while too little diversity fails to capture behavioural variation and too much diversity removes informative trajectories. This highlights a trade-off between coverage and selectivity in diverse inference. Appendix D. Human Study Dataset Collection We collected a dataset of human–agent interactions in the Overcooked domain from 15 participants (8 male, 7 female; mean age 28.27±3.15). Participants reported an average video game experience of 3.00 ± 1.07 on a 5-point Likert scale (1 = no experience, 5 = daily play). All participants provided informed consent prior to the study. Each participant completed 15 episodes, organised into three sets of five tasks corresponding to the three capability types used in the main experiments. Tasks were randomly sampled from the 9 layout–recipe combinations described in Section 5. In each episode, participants collaborated with a fixed-policy assistant agent Robbie, controlled by a Q-learning-based joint policy trained with fully capable agents (see Appendix B.2). Participants selected high-level actions (e.g., get ingredient, chop, deliver ) via keyboard input. At each decision point, Robbie’s intended action was displayed, and participants could override it if desired. This interaction protocol mirrors the correction-based execution model used in the simulated experiments. We recorded sequences of joint states along with the ground-truth capability vector associated with each participant’s assigned type. In total, the dataset contains 225 trajectories (75 per capability type), which are used for offline capability inference in Section 5.3. Appendix E. Human Study Dataset Analysis We first examine the diversity of the collected trajectories to evaluate the hypothesis that human behaviour exhibits substantial variability even within the same capability profile. To quantify this, we compute the average pairwise Euclidean distance between trajectory feature vectors, normalised by the maximum distance observed within each task-type group. We refer to this metric as the normalised distance metric (NDM): N

N DM =

1 X X ||τi − τj ||2 , N i=1 j̸=i Dmax 36

0.8

NDM

0.6 0.4

(level_1, lettuce-onion-tomato) (level_1, lettuce-tomato) (level_1, onion-tomato)

Agent Type (level_2, lettuce-onion) (level_2, lettuce-onion-tomato) (level_2, onion-tomato)

0.0

0.2

(level_3, lettuce-onion) (level_3, lettuce-onion-tomato) (level_3, lettuce-tomato)

Figure E.17: Diversity of the collected trajectories by level and agent type. Each coloured bar represents one task, grouped by agent type.

where N is the number of trajectories in the group, and Dmax = maxτ1 ,τ2 ||τ1 − τ2 ||2 . As shown in Figure E.17, trajectory diversity varies significantly across tasks and capability types. Some task–recipe combinations exhibit near-deterministic behaviour (e.g., level_3 with lettuce-onion), where all participants converge to the same strategy. In contrast, more complex tasks (e.g., level_2 and level_3 with lettuceonion-tomato) exhibit high diversity, indicating multiple viable strategies even under identical capabilities. Next, we compare human trajectories to those generated by the MCTS planner under the same task and capability conditions. We compute cosine distance between flattened trajectory feature vectors, consistent with the similarity metric used in CE-CM. As shown in Figure E.18, planner-generated trajectories differ substantially from human behaviour across all capability types, with considerable variability within each group. This result highlights a key limitation of single-trajectory inference: even when the capability vector is correct, the planner’s trajectory is unlikely to match the observed human trajectory. Consequently, using a single “optimal” rollout provides a weak learning signal for capability inference. This motivates the use of CE-CM-Div, which accounts for behavioural variability by evaluating each capability hypothesis against a set of plausible trajectories rather than a single prediction.

0.5

Cosine Distance

0.4 0.3 0.2

(level_1, lettuce-onion-tomato) (level_1, lettuce-tomato) (level_1, onion-tomato)

Agent Type (level_2, lettuce-onion) (level_2, lettuce-onion-tomato) (level_2, onion-tomato)

0.0

0.1

(level_3, lettuce-onion) (level_3, lettuce-onion-tomato) (level_3, lettuce-tomato)

Figure E.18: Cosine distance between human trajectories and planner-generated trajectories. Bars show the mean distance, with standard deviation indicated in black.

References [1] M. D. Zhao, R. G. Simmons, H. Admoni, The role of adaptation in collective human–ai teaming, Topics in Cognitive Science 17 (2022) 291 – 323. [2] M. Natarajan, E. Seraj, B. Altundas, R. Paleja, S. Ye, L. Chen, R. Jensen, K. C. Chang, M. Gombolay, Human-Robot Teaming: Grand Challenges, Current Robotics Reports 4 (3) (2023) 81–100. doi:10.1007/s43154-023-00103-1. [3] R. Mirsky, I. Carlucho, A. Rahman, E. Fosong, W. Macke, M. Sridharan, P. Stone, S. V. Albrecht, A Survey of Ad Hoc Teamwork Research, arXiv:2202.10450 [cs.MA] (2022). arXiv:2202.10450. URL http://arxiv.org/abs/2202.10450 [4] P. Stone, G. Kaminka, S. Kraus, J. Rosenschein, Ad Hoc Autonomous Agent Teams: Collaboration without Pre-Coordination, Proceedings of the AAAI Conference on Artificial Intelligence 24 (1) (2010) 1504–1509. doi:10.1609/aaai. v24i1.7529. [5] P. Tisnikar, G. Canal, M. Leonetti, Probabilistic inference of human capabilities from passive observations, in: 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE, 2024, pp. 8779–8785. 38

[6] H. Hu, A. Lerer, A. Peysakhovich, J. Foerster, “Other-play” for zero-shot coordination, in: H. D. III, A. Singh (Eds.), Proceedings of the 37th International Conference on Machine Learning, Vol. 119 of Proceedings of Machine Learning Research, PMLR, 2020, pp. 4399–4410. [7] A. Shih, A. Sawhney, J. Kondic, S. Ermon, D. Sadigh, On the Critical Role of Conventions in Adaptive Human-AI Collaboration, ICLR 2021 (2021). arXiv: 2104.02871. URL http://arxiv.org/abs/2104.02871 [8] J. Heinrich, D. Silver, Deep reinforcement learning from self-play in imperfectinformation games, arXiv preprint arXiv:1603.01121 (2016). [9] D. Strouse, K. R. McKee, M. Botvinick, E. Hughes, R. Everett, Collaborating with humans without human data, in: Proceedings of the 35th International Conference on Neural Information Processing Systems, NIPS ’21, Curran Associates Inc., Red Hook, NY, USA, 2021, pp. 14502–14515. [10] A. Lupu, B. Cui, H. Hu, J. Foerster, Trajectory diversity for zero-shot coordination, in: M. Meila, T. Zhang (Eds.), Proceedings of the 38th International Conference on Machine Learning, Vol. 139 of Proceedings of Machine Learning Research, PMLR, 2021, pp. 7204–7213. [11] A. Szot, U. Jain, D. Batra, Z. Kira, R. Desai, A. Rai, Adaptive coordination in social embodied rearrangement, in: Proceedings of the 40th International Conference on Machine Learning, ICML’23, JMLR.org, 2023, pp. 33365–33380. [12] D. Xing, Q. Liu, Q. Zheng, G. Pan, Learning with Generated Teammates to Achieve Type-Free Ad-Hoc Teamwork, in: Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, International Joint Conferences on Artificial Intelligence Organization, 2021, pp. 472–478. doi: 10.24963/ijcai.2021/66. [13] B. Sarkar, A. Shih, D. Sadigh, Diverse conventions for human-ai collaboration, in: Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Curran Associates Inc., Red Hook, NY, USA, 2023, pp. 23115–23139. [14] C. Yu, J. Gao, W. Liu, B. Xu, H. Tang, J. Yang, Y. Wang, Y. Wu, Learning zero-shot cooperation with humans, assuming humans are biased, in: The 39

Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023, OpenReview.net, 2023, pp. 1–10. URL https://openreview.net/forum?id=TrwE8l9aJzs [15] X. Lou, J. Guo, J. Zhang, J. Wang, K. Huang, Y. Du, Pecan: Leveraging policy ensemble for context-aware zero-shot human-ai coordination, in: Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems, AAMAS ’23, International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, 2023, p. 679–688. [16] M. Zhao, R. Simmons, H. Admoni, Coordination With Humans Via Strategy Matching, in: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022, pp. 9116–9123. doi:10.1109/IROS47612.2022. 9982277. [17] O. C. Görür, B. Rosman, S. Albayrak, Anticipatory bayesian policy selection for online adaptation of collaborative robots to unknown human types, in: Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems, 2019, pp. 77–85. [18] S. Barrett, P. Stone, Cooperating with Unknown Teammates in Complex Domains: A Robot Soccer Case Study of Ad Hoc Teamwork, Proceedings of the AAAI Conference on Artificial Intelligence 29 (1) (2015). doi:10.1609/aaai. v29i1.9428. [19] H. Li, T. Ni, S. Agrawal, F. Jia, S. Raja, Y. Gui, D. Hughes, M. Lewis, K. Sycara, Individualized Mutual Adaptation in Human-Agent Teams, IEEE Transactions on Human-Machine Systems 51 (6) (2021) 706–714. doi:10.1109/THMS.2021. 3107675. [20] A. Ghosh, S. Tschiatschek, H. Mahdavi, A. Singla, Towards Deployment of Robust Cooperative AI Agents: An Algorithmic Framework for Learning Adaptive Policies, New Zealand (2020). [21] C. Wang, C. Pérez-D’Arpino, D. Xu, L. Fei-Fei, K. Liu, S. Savarese, Co-gail: Learning diverse strategies for human-robot collaboration, in: Conference on Robot Learning, PMLR, 2022, pp. 1279–1290. [22] S. V. Albrecht, P. Stone, Autonomous agents modelling other agents: A comprehensive survey and open problems, Artificial Intelligence 258 (2018) 66–95. doi:10.1016/j.artint.2018.01.002. 40

[23] M. Carroll, R. Shah, M. K. Ho, T. Griffiths, S. Seshia, P. Abbeel, A. Dragan, On the utility of learning about humans for human-ai coordination, Advances in neural information processing systems 32 (2019). [24] L. M. Hiatt, C. Narber, E. Bekele, S. S. Khemlani, J. G. Trafton, Human modeling for human–robot collaboration, The International Journal of Robotics Research 36 (5-7) (2017) 580–596. doi:10.1177/0278364917690592. [25] R. Raileanu, E. L. Denton, A. Szlam, R. Fergus, Modeling others using oneself in multi-agent reinforcement learning, in: International Conference on Machine Learning, 2018, pp. 4257–4266. URL https://api.semanticscholar.org/CorpusID:3622509 [26] S. V. Albrecht, J. W. Crandall, S. Ramamoorthy, Belief and truth in hypothesised behaviours, Artificial Intelligence 235 (2016) 63–94. doi:10.1016/j. artint.2016.02.004. [27] S. Barrett, A. Rosenfeld, S. Kraus, P. Stone, Making friends on the fly: Cooperating with new teammates, Artificial Intelligence 242 (2017) 132–171. doi:10.1016/j.artint.2016.10.005. [28] S. V. Albrecht, P. Stone, Reasoning about Hypothetical Agent Behaviours and their Parameters, in: Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems, 2019, p. 547–555. arXiv:1906.11064, doi:10.48550/ arXiv.1906.11064. [29] E. Shafipour Yourdshahi, M. A. Do Carmo Alves, A. Varma, L. Soriano Marcolino, J. Ueyama, P. Angelov, On-line estimators for ad-hoc task execution: Learning types and parameters of teammates for effective teamwork, Autonomous Agents and Multi-Agent Systems 36 (2) (2022) 45. doi:10.1007/ s10458-022-09571-9. [30] A. Czechowski, F. A. Oliehoek, Decentralized MCTS via Learned Teammate Models, in: Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, International Joint Conferences on Artificial Intelligence Organization, 2020, pp. 81–88. doi:10.24963/ijcai.2020/12. [31] F. Wu, S. Zilberstein, X. Chen, Online planning for ad hoc autonomous agent teams, in: Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence - Volume One, IJCAI’11, AAAI Press, 2011, p. 439–445. 41

[32] J. Felip, D. Gonzalez-Aguirre, L. Nachman, Intuitive & efficient human-robot collaboration via real-time approximate bayesian inference, in: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022, pp. 3093–3099. doi:10.1109/IROS47612.2022.9982251. [33] J. G. Ribeiro, M. Faria, A. Sardinha, F. S. Melo, Helping People on the Fly: Ad Hoc Teamwork for Human-Robot Teams, in: G. Marreiros, F. S. Melo, N. Lau, H. Lopes Cardoso, L. P. Reis (Eds.), Progress in Artificial Intelligence, Vol. 12981, Springer International Publishing, 2021, pp. 635–647. doi:10.1007/ 978-3-030-86230-5_50. [34] J. G. Ribeiro, L. M. Henriques, S. Colcher, J. C. Duarte, F. S. Melo, R. L. Milidiú, A. Sardinha, HOTSPOT: An ad hoc teamwork platform for mixed humanrobot teams, PLOS ONE 19 (6) (2024) e0305705. doi:10.1371/journal.pone. 0305705. [35] C.-M. Huang, B. Mutlu, Anticipatory robot control for efficient human-robot collaboration, in: 2016 11th ACM/IEEE International Conference on HumanRobot Interaction (HRI), IEEE, 2016, pp. 83–90. doi:10.1109/HRI.2016. 7451737. [36] C. Laidlaw, E. Bronstein, T. Guo, D. Feng, L. Berglund, J. Svegliato, S. Russell, A. Dragan, Assistancezero: Scalably solving assistance games, in: Forty-second International Conference on Machine Learning, 2025, pp. 1–10. URL https://openreview.net/forum?id=b9hVMJi0t2 [37] S. Nikolaidis, R. Ramakrishnan, K. Gu, J. Shah, Efficient Model Learning from Joint-Action Demonstrations for Human-Robot Collaborative Tasks, in: Proceedings of the Tenth Annual ACM/IEEE International Conference on HumanRobot Interaction, HRI ’15, Association for Computing Machinery, 2015, pp. 189–196. doi:10.1145/2696454.2696455. [38] A. Bestick, R. Pandya, R. Bajcsy, A. D. Dragan, Learning Human Ergonomic Preferences for Handovers, in: 2018 IEEE International Conference on Robotics and Automation (ICRA), IEEE, 2018, pp. 3257–3264. doi:10.1109/ICRA. 2018.8461216. [39] M. D. Zhao, R. Simmons, H. Admoni, Learning human contribution preferences in collaborative human-robot tasks, in: J. Tan, M. Toussaint, K. Darvish (Eds.), Proceedings of The 7th Conference on Robot Learning, Vol. 229 of Proceedings of Machine Learning Research, PMLR, 2023, pp. 3597–3618. 42

[40] M. Trivedi, P. Doshi, Inverse Learning of Robot Behavior for Collaborative Planning, in: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2018, pp. 1–9. doi:10.1109/IROS.2018.8593745. [41] S. Nikolaidis, J. Shah, Human-robot cross-training: Computational formulation, modeling and evaluation of a human team training strategy, in: 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2013, pp. 33–40. doi:10.1109/HRI.2013.6483499. [42] S. Nikolaidis, P. Lasota, R. Ramakrishnan, J. Shah, Improved human–robot team performance through cross-training, an approach inspired by human team training practices, The International Journal of Robotics Research 34 (14) (2015) 1711–1730. doi:10.1177/0278364915609673. [43] A. Narcomey, N. Tsoi, R. Desai, M. Vázquez, Learning Human Preferences Over Robot Behavior as Soft Planning Constraints, arXiv:2403.19795 [cs.RO] (2024). arXiv:2403.19795. URL http://arxiv.org/abs/2403.19795 [44] G. Canal, G. Alenyà, C. Torras, Adapting robot task planning to user preferences: An assistive shoe dressing example, Autonomous Robots 43 (6) (2019) 1343–1356. doi:10.1007/s10514-018-9737-2. [45] T. A. Nguyen, M. Do, A. E. Gerevini, I. Serina, B. Srivastava, S. Kambhampati, Generating diverse plans to handle unknown and partially known user preferences, Artificial Intelligence 190 (2012) 1–31. doi:10.1016/j.artint.2012. 05.005. [46] Y. Emam, S. Mayya, G. Notomista, A. Bohannon, M. Egerstedt, Adaptive Task Allocation for Heterogeneous Multi-Robot Teams with Evolving and Unknown Robot Capabilities, in: 2020 IEEE International Conference on Robotics and Automation (ICRA), IEEE, 2020, pp. 7719–7725. doi:10.1109/ICRA40945. 2020.9197283. [47] A. Ali, H. Azevedo-Sa, D. M. Tilbury, L. P. Robert, Heterogeneous human–robot task allocation based on artificial trust, Scientific Reports 12 (1) (2022) 15304. doi:10.1038/s41598-022-19140-5. [48] R. Liu, M. Natarajan, M. C. Gombolay, Coordinating Human-Robot Teams with Dynamic and Stochastic Task Proficiencies, ACM Transactions on HumanRobot Interaction 11 (1) (2022) 1–42. doi:10.1145/3477391. 43

[49] S. Izquierdo-Badiola, G. Canal, C. Rizzo, G. Alenyà, Improved task planning through failure anticipation in human-robot collaboration, in: 2022 International Conference on Robotics and Automation (ICRA), 2022, pp. 7875–7880. [50] S. Zhang, Y. Chen, J. Zhang, Y. Jia, Real-Time Adaptive Assembly Scheduling in Human-Multi-Robot Collaboration According to Human Capability, in: 2020 IEEE International Conference on Robotics and Automation (ICRA), IEEE, 2020, pp. 3860–3866. doi:10.1109/ICRA40945.2020.9196618. [51] B. Fu, W. Smith, D. M. Rizzo, M. Castanier, M. Ghaffari, K. Barton, Robust Task Scheduling for Heterogeneous Robot Teams Under Capability Uncertainty, IEEE Transactions on Robotics 39 (2) (2023) 1087–1105. doi:10.1109/TRO. 2022.3216068. [52] L. Orlov-Savko, A. Jain, G. M. Gremillion, C. E. Neubauer, J. D. Canady, V. Unhelkar, Factorial agent markov model: Modeling other agents’ behavior in presence of dynamic latent decision factors, in: Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems, AAMAS ’22, International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, 2022, p. 982–990. [53] V. Unhelkar, S. Li, J. A. Shah, Semi-supervised learning of decision-making models for human-robot collaboration, in: Conference on Robot Learning, 2019, pp. 192–203. URL https://api.semanticscholar.org/CorpusID:208175027 [54] A. Hallak, D. Di Castro, S. Mannor, Contextual Markov Decision Processes, arXiv:1502.02259 [stat.ML] (2015). arXiv:1502.02259. URL http://arxiv.org/abs/1502.02259 [55] D. V. Pynadath, M. Tambe, The communicative multiagent team decision problem: Analyzing teamwork theories and models, Journal of artificial intelligence research 16 (2002) 389–423. [56] M. A. Beaumont, Approximate bayesian computation, Annual review of statistics and its application 6 (1) (2019) 379–403. [57] L. Benke, T. Miller, M. Papasimeon, N. Lipovetzky, Diverse, Top-k, and Top-Quality Planning Over Simulators, in: K. Gal, A. Nowé, G. J. Nalepa, R. Fairstein, R. Rădulescu (Eds.), Frontiers in Artificial Intelligence and Applications, IOS Press, 2023, pp. 1–10. doi:10.3233/FAIA230275. 44

[58] B. Srivastava, T. A. Nguyen, A. Gerevini, S. Kambhampati, M. B. Do, I. Serina, Domain independent approaches for finding diverse plans, in: Proceedings of the 20th International Joint Conference on Artifical Intelligence, IJCAI’07, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 2007, p. 2016–2022. [59] M. Fox, D. Long, Pddl2. 1: An extension to pddl for expressing temporal planning domains, Journal of artificial intelligence research 20 (2003) 61–124. [60] S. A. Wu, R. E. Wang, J. A. Evans, J. B. Tenenbaum, D. C. Parkes, M. KleimanWeiner, Too Many Cooks: Bayesian Inference for Coordinating Multi-Agent Collaboration, Topics in Cognitive Science 13 (2) (2021) 414–432. doi:10. 1111/tops.12525. [61] Y. Xiao, W. Tan, J. Hoffman, T. Xia, C. Amato, Asynchronous multi-agent deep reinforcement learning under partial observability, The International Journal of Robotics Research 44 (8) (2025) 1257–1286. doi:10.1177/02783649241306124. [62] A. Coles, A. Coles, M. Fox, D. Long, Forward-chaining partial-order planning, in: Proceedings of the International Conference on Automated Planning and Scheduling, Vol. 20, 2010, pp. 42–49. [63] D. Silver, J. Schrittwieser, K. Simonyan, I. Antonoglou, A. Huang, A. Guez, T. Hubert, L. baker, M. Lai, A. Bolton, Y. Chen, T. P. Lillicrap, F. Hui, L. Sifre, G. van den Driessche, T. Graepel, D. Hassabis, Mastering the game of go without human knowledge, Nature 550 (2017) 354–359. URL https://api.semanticscholar.org/CorpusID:205261034

Record · ID 411067 · SHA-256 c6ae49eb954acfb1
Retrieved via Conceptio — every document is proof-bundled with source, license, and retrieval metadata.