arXiv:2604.24663v1 [math.OC] 27 Apr 2026
Dual Control of Linear Systems from Bilinear Observations with Belief Space Model Predictive Control Daniel Cao∗ Cornell [email protected]
Beixi Du∗ Cornell [email protected]
Andrew Lowitt∗ Cornell [email protected]
Sunmook Choi∗ Cornell [email protected]
Sarah Dean Yahya Sattar Cornell Cornell [email protected] [email protected]
Abstract We study finite-horizon quadratic control of linear systems with bilinear observations, in which the control input affects not only the state dynamics but also the partial observations of the state. In this setting, the separation principle can fail because control inputs influence the future quality of state estimates. State estimation requires an input-dependent Kalman filter whose gain and error covariance evolve as functions of the control inputs. To address this challenge, we propose a belief-space model predictive control (B-MPC) method that plans directly over both the estimated state and its error covariance. In particular, B-MPC plans with a deterministic surrogate of the belief evolution defined by the input-dependent Kalman filter. Through numerical experiments in two synthetic settings, we show that B-MPC can outperform both the separation-principle controller and its MPC variant in favorable regimes, and that these gains are accompanied by lower estimation covariance and more uncertainty-aware action choices.
1
Introduction
Linear dynamical systems with bilinear observations (LDS-BO) are a simple example of the observer effect, in which measurements about a system’s state also alter that state. Observer effects are present in applications ranging from quantum systems [PY10] to electronic circuits [LK24]. From the measurement perspective, estimating an unknown state is complicated by the fact that the measurement mechanism may itself influence the system. Motivated by this measurement question, several recent papers study system identification of unknown dynamics in the LDS-BO setting, either through estimating the input-output behavior [CSJ+ 25], a state space realization [LK24], or both [SJD25]. The presence of observer effects also complicates control design, even when the dynamics are known. This is because control inputs play a dual role of both affecting and observing the state. This trade-off is evident in high-performance robotics applications such as drone racing, where accounting for uncertainty during planning can be critical for performance [KBL+ 23]. The dual role of control inputs can be made explicit in the LDS-BO setting with Gaussian noise: the state is linearly affected through the control matrix, while the estimation covariance is nonlinearly affected through the (input-dependent) measurement matrix. Perhaps the simplest optimal control problem that captures the dual control phenomena is quadratic control of LDS-BO with Gaussian noise (BO-LQG). This is a slight variation on the classical linear quadratic Gaussian (LQG) problem, which is solved by the separation principle. Eschewing uncertainty, the optimal LQR controller is linear state feedback treating the state estimate as if it were true. In contrast, separation-principle controller is not optimal for BO-LQG. A recent work [SCJ+ 25] shows that ∗
Equal contribution.
1
the optimal controller is not linear in the estimated state and that the separation principle controller may locally maximize the cost. Solving the BO-LQG problem is nontrivial. It is a specific instance of a partially observed Markov decision process, which are known to be computationally hard in general [PT87], largely due to the complexity of reasoning about long sequences of actions and observations [KLC98]. However, the BO-LDS setting admits a finite dimensional belief state: the distribution of the unobserved state is completely characterized by the state estimation mean and covariance. Belief space planning is a common strategy in robotics to handle practical issues like path planning under perception uncertainties [NS24, ZRTAm22, ZPT23]. We take inspiration from this perspective, which allows us to reformulate the original optimal control problem into a belief space control problem. This transforms the challenge from reasoning about partial observation to synthesizing a controller for nonlinear dynamics, which arise from the nonlinear covariance evolution. A popular approach for addressing nonlinear optimal control problems is model predictive control (MPC) [GP16, May14, RMD24]. Instead of synthesizing a controller directly, an open loop planning problem is repeatedly solved, and each time only the first input is applied. This strategy is usually an approximation: the planning horizon is shortened and surrogate deterministic dynamics may be used (though a variety of approaches exist [RMD24]). Indeed, MPC strategies have also been used for control of bilinear dynamical systems [KKR22, XBSA25], however, these methods differ from our setting in that they consider the full state observation with the bilinearity appearing in the state dynamics rather than the observation equation. Contributions: In this paper, we make the following contributions. First, we formulate linear quadratic control of linear systems with bilinear observations as a belief-space control problem, where the belief state is given by the Kalman filter state estimate and error covariance. This formulation makes the dual role of control explicit in the planning problem. Second, we propose a belief-space model predictive control method, B-MPC, that plans over a deterministic surrogate of the belief dynamics induced by the input-dependent Kalman filter. Lastly, through numerical experiments on two synthetic systems, we show that B-MPC can substantially outperform the separation-principle controller and its MPC variant in favorable parameter regimes. The experiments further show that these gains are associated with lower estimation covariance and improved observability, supporting the dual-control interpretation of the problem. Organization: The rest of the paper is organized as follows. In §2, we formally introduce BO-LQG, and discuss the sub-optimality of the separation-principle controller [SCJ+ 25]. In §3, we develop the belief-space MPC formulation and present the B-MPC algorithm. In §4, we describe the experimental setup and evaluate the performance of B-MPC against baseline controllers. In §5, we further interpret the empirical behavior of B-MPC relative to the baselines through analyses of state estimation and control behavior. Lastly, §6 concludes the paper. Notation: For a symmetric matrix M , we write M ⪰ 0 (or M ≻ 0) for positive semidefinite (or positive definite). For an arbitrary square matrix N , we write tr(N ) for the trace and ρ(N ) for the spectral radius. We use In to denote the n × n identity matrix. For a vector u, (u)i denotes the i-th element. We use the shorthand for sequences ut:t+k = (ut , ..., ut+k ).
2
Problem Settings
In this paper, we consider the following linear dynamical system with bilinear observations: xt+1 = Axt + But + wt , p X yt = C0 + (ut )k Ck xt + zt .
(2.1)
k=1
Here, xt ∈Rn , ut ∈Rp , and yt ∈Rm are the state, input, and the output at time t with the initial state distribution x0 ∼N (x̂0 , Σ0 ). The matrices A ∈ Rn×n and B ∈ Rn×p define the linear dynamics, 2
and C0 , C1 , . . . , Cp ∈ Rm×n define the partial observation model. In this work, we assume that the matrices A, B, C0 , C1 , . . . , Cp are known. We assume that the process noise wt ∈ Rn and the i.i.d.
i.i.d.
measurement noise zt ∈ Rm are independent Gaussian processes, i.e., wt ∼ N (0, Σw ) and zt ∼ N (0, Σz ). p Given an input Pp vector ut ∈R , the matrix C(ut ) denotes the input dependent observation matrix, i.e., C(ut )=C0 + k=1 (ut )k Ck . Letting Θ = (A, B, C0 , C1 , . . . , Cp ), we use LDS-BO(Θ) to refer to the state-space representation (2.1). Under the partially observed system, we want to solve the following finite-horizon optimal control problem, which we refer to as BO-LQG: " # T −1 X ⊤ ⊤ ⊤ (2.2) min E xT QT xT + xt Qxt + ut Rut s.t. LDS-BO(Θ), u0:T −1
t=0
where QT , Q ⪰ 0 are known state-cost matrices and R ≻ 0 is a known input-cost matrix. One can observe that, if the matrices C1 = · · · = Cp = 0, it reduces to the classical LQG setting. In classical LQG, the separation principle states that state estimation and control design can be solved independently: the state is estimated via Kalman filtering [Kal60], and the control input is obtained by applying the optimal linear feedback gain to the state estimate. Sep A natural baseline in BO-LQG is the separation-principle controller, i.e., ut = Lt x̂t|t−1 where Lt is the finite-horizon state-feedback gain computed from the standard Riccati recursion. Here, x̂t|t−1 denotes the state estimate, i.e., the conditional mean of xt given the information It = {u0 , . . . , ut−1 , y0 , . . . , yt−1 } available at time t. Since the observation model depends on the input through C(ut ), this estimate is obtained via an input-dependent Kalman filter. In particular, the state estimate and the corresponding prediction error covariance evolve according to x̂t+1|t = Ax̂t|t−1 + But − L(ut ) yt − C(ut )x̂t|t−1 (2.3) Σt+1|t = AΣt|t−1 A⊤ + L(ut )C(ut )Σt|t−1 A⊤ + Σw where L(ut ) = −AΣt|t−1 C(ut )⊤ (C(ut )Σt|t−1 C(ut )⊤ + Σz )−1 is the input-dependent Kalman gain. Prior work [SCJ+ 25] showed that the separation principle does not hold in BO-LQG. In particular, it showed that the optimal controller for (2.2) is not affine in the estimated state, and provided a setting in which the separation principle controller locally maximizes the cost function. The input-dependent Kalman filter also helps explain this suboptimality. Since the observation matrix C(ut ) depends on the applied input, the quality of the state estimate depends on the control sequence itself. If C(ut ) is weakly informative, then the resulting observation provides little information about the latent state, and the error covariance Σt|t−1 remains large. Thus, an input that appears favorable for immediate regulation may simultaneously degrade future state estimation. The separation-principle controller does not explicitly account for this effect of control on future uncertainty, which is why it can be suboptimal in the bilinear observation setting. The key distinction from classical LQG is that the same input ut appears both in the state transition and in the observation matrix. Thus, choosing ut affects not only the next state distribution but also the information obtained about the current state.
3
Model Predictive Control Formulation
We now formulate model predictive control (MPC) for the bilinear observation system LDS-BO(Θ) and a finite planning horizon H ≥ 1. A natural formulation of the MPC problem is to minimize the following objective over ut:t+H−1 : " # t+H−1 X ⊤ E x⊤ x⊤ It (3.1) τ Qxτ +uτ Ruτ t+H QT xt+H + τ =t
Let JtMPC,H be the minimum of (3.1). In this section, we formulate an MPC problem in the belief space, which is equivalent to minimizing (3.1), induced by the input-dependent Kalman filtering (2.3). 3
3.1
Belief-State Cost and Finite-Horizon MPC Objective
We define the belief state as the mean and covariance of the posterior distribution of the state given past observations and controls: bt := (x̂t|t−1 , Σt|t−1 ) where x̂t|t−1 and Σt|t−1 are given by the input-dependent Kalman filter in (2.3). Note that the Kalman filter provides the exact Bayesian posterior due to the Gaussian noise and linear dynamics. The posterior distribution of the state xt given It remains Gaussian, and is fully characterized by the mean and covariance in bt . Then, the original finite-horizon quadratic cost (2.2) can be rewritten in terms of the belief state using ⊤ E x⊤ t Qxt | It = x̂t|t−1 Qx̂t|t−1 + tr(QΣt|t−1 ) ⊤ E x⊤ T QT xT | IT = x̂T |T −1 QT x̂T |T −1 + tr(QT ΣT |T −1 ). This motivates the following belief-space stage and terminal costs: ⊤ ℓ(bt , ut ) := x̂⊤ t|t−1 Qx̂t|t−1 + tr(QΣt|t−1 ) + ut Rut ,
ϕ(bT ) := x̂⊤ T |T −1 QT x̂T |T −1 + tr(QT ΣT |T −1 ).
(3.2)
From these costs, we construct the finite-horizon MPC objective with planning horizon H in the belief space which is equivalent to (3.1). Given the current belief bt , we have that JtMPC,H is equal to " # t+H−1 X min E ϕ(bt+H ) + ℓ(bτ , uτ ) bt (3.3) ut:t+H−1
τ =t
where the belief trajectory (bτ )t+H τ =t is generated by the input-dependent Kalman filter (2.3) under the chosen open-loop control sequence (uτ )t+H−1 and the bilinear observation model yτ =C(uτ )xτ +zτ . τ =t When H=T −t, JtMPC,H coincides with the optimal cost-to-go of the full dynamic program.
3.2
Deterministic MPC Planning Objective
In practice, planning with the objective (3.3) is infeasible because the belief state update is stochastic and depends on the bilinear observation yτ =C(uτ )xτ +zτ . Instead, we consider its deterministic surrogate based on the available information: yτ =C(uτ )x̂τ . Therefore, given an input ut , we consider the following deterministic update for the planning: x̄t+1 = Ax̄t + But Σ̄t+1 = AΣ̄t A⊤ + L̄(ut )C(ut )Σ̄t A⊤ + Σw
(3.4)
−1 where L̄(ut ) = −AΣ̄t C(ut )⊤ C(ut )Σ̄t C(ut )⊤ + Σz . Under the nominal observation yτ =C(uτ )x̄τ the innovation term is zero, so the deterministic mean update reduces to the open-loop prediction x̄τ +1 = Ax̄τ +Buτ while the covariance still evolves according to the input-dependent Kalman covariance recursion. Let b̄t = (x̄t , Σ̄t ), then (3.4) defines a function F such that b̄t+1 = F (b̄t , ut ).
(3.5)
Thus, the MPC objective corresponding to the deterministic planning is the following: J¯tMPC,H =
min ϕ(b̄t+H ) +
ut:t+H−1
t+H−1 X τ =t
4
ℓ(b̄τ , uτ )
(3.6)
where the trajectory (b̄τ )t+H τ =t is generated by (3.5) starting from b̄t = (x̂t|t−1 , Σt|t−1 ), the true belief state. We emphasize that (3.6) is an approximation to the stochastic belief-space objective (3.3). Replacing future observations by their nominal values is a standard certainty-equivalent approach in nonlinear MPC. Our contribution is a tractable receding-horizon approximation for BO-LQG rather than an exact solution or a closed-loop stability guarantee, though it does account for the dual role of control inputs. In Section 5 and 4, we demonstrate its performance against separation-principle baselines and provide experiments that support the dual-control interpretation of the belief-space controller. Unlike Sep-MPC, the B-MPC objective contains the trace terms tr(QΣt|t−1 ) and tr(QT Σt|t−1 ), and the covariance trajectory depends on the planned inputs through C(u). Therefore, B-MPC can select inputs that are suboptimal for immediate state regulation but beneficial for future estimation. This is the mechanism by which the controller captures the dual role of control.
3.3
Belief-Space MPC Algorithm
We state the receding-horizon MPC algorithm in belief space for any total horizon T and planning horizon H ≥ 1. Given the system LDS-BO(Θ), cost matrices (Q, QT , R), noise covariances (Σw , Σz ), a look-ahead window H ≥ 1, we define a belief-space MPC controller as follows. The controller maintains a belief state bt = (x̂t|t−1 , Σt|t−1 ) at each time t, where x̂t|t−1 and Σt|t−1 are the mean and covariance of xt conditioned on the information It . Starting from the prior belief b0 = (x̂0 , Σ0 ), the mean and covariance of the initial state distribution, the controller repeats the following steps for t = 0, 1, . . . , T − 1: 1. Planning step. Given the current belief bt , the controller solves the finite-horizon optimal control problem t+H−1 X min ϕ(b̄t+H ) + ℓ(b̄τ , uτ ) ut:t+H−1
τ =t
where the stage cost ℓ and terminal cost ϕt+H are defined in (3.2), and the belief trajectory (b̄τ )t+H τ =t is generated by (3.5). 2. Control. Let u⋆t:t+H−1 denote an optimal solution of the above problem. The MPC controller applies only the first input, ut := u⋆t , to the true system. 3. Belief update. After applying ut , the controller observes yt and updates the belief to bt+1 = (x̂t+1|t , Σt+1|t ), using the bilinear Kalman filtering recursion (2.3). We remark that due to the belief state covariance update, the planning problem is nonconvex. However, it is differentiable. This motivates the use of gradient-based local optimization rather than dynamic programming or Riccati-style recursions. We present further discussion on numerical optimization in the experiments section. The algorithm for the belief-space MPC controller, called B-MPC, is summarized in Algorithm 1.
4
Experiments: Setting & Performance
In this section, we demonstrate the performance of our method by comparing it with two baseline controllers on two benchmark systems. We first define the benchmark systems and the controllers in Section 4.1 and Section 4.2. In Section 4.3, we present settings in which B-MPC, our proposed belief-space MPC controller, achieves better performance than the baselines, and we analyze the resulting control behavior. 5
Algorithm 1 B-MPC Require: Trajectory length T , planning horizon H, mean x̂0 and covariance Σ0 of the initial state distribution 1: Initialize x̂0|−1 = x̂0 and Σ0|−1 = Σ0 . 2: Construct the belief state b0 = (x̂0|−1 , Σ0|−1 ). 3: for t = 0, 1, 2, . . . , T − 1 do 4: From bt , define a function J by J(ut , . . . , ut+H−1 , bt ) =
t+H−1 X
ℓ(b̄τ , uτ ) + ϕ(b̄t+H )
τ =t
where (b̄τ )t+H τ =t is generated by (3.5) given the inputs ut , . . . , ut+H−1 , starting from b̄t = bt . 5: Minimize J over (ut , . . . , ut+H−1 ) via L-BFGS optimization, and let (u⋆t , . . . , u⋆t+H−1 ) denote a minimizer. 6: Apply u⋆t in the true system, and observe yt . 7: Update the belief state bt+1 = (x̂t+1|t , Σt+1|t ) via (2.3) using (u⋆t , yt ).
4.1
Bilinear observation systems
We consider two bilinear observation systems: a multi-block double integrator system and a randomly generated system. In both systems, the initial state distribution is set to N (0, I). We describe the system parameters in the following. 4.1.1
Random bilinear observation system
We sample a random bilinear observation system with state dimension n=6, input dimension p=3, and output dimension m=3. Each entry of A is drawn independently from N (0, 1), and the matrix A is rescaled so that ρ(A) matches the target spectral radius. We draw B entrywise from i.i.d. N (0, 1/n), the observation matrix C0 entrywise from N (0, c20 /m), and each matrix Ck , k = 1, . . . , p, entrywise 2 I and Σ = σ 2 I , and the cost matrices from N (0, 1/m). The noise covariances are chosen to Σw = σw n z z m are Q = QT = In and R = Rscale · Ip . 4.1.2
Multi-block double integrator style system
The multi-block double integrator system consists of three identical, dynamically decoupled blocks. The matrices A and B are defined to be ρ h 0 0 0 0 0 0 0 0 ρ 0 0 0 0 h 0 0 0 0 ρ h 0 0 0 0 0 A= , B = 0 h 0 0 0 0 ρ 0 0 0 0 0 0 ρ h 0 0 0 0 0 0 0 0 ρ 0 0 h where ρ is set to the target spectral radius and h = 0.3 is the discretization step size. The observation matrix C0 ∈ R3×6 is defined by (C0 )i, 2i−1 = c0 , i = 1, 2, 3, with all other entries equal to zero. Equivalently, c0 0 0 0 0 0 C0 = 0 0 c0 0 0 0 . 0 0 0 0 c0 0 For k=1, 2, 3, the matrix Ck ∈R3×6 is defined so that its (k, 2k−1)-entry is c1 and all other entries are zero. Throughout the experiments, we set c1 =3. As for the random system, we consider isotropic 6
2 I and Σ = σ 2 I . For the quadratic costs, we set Q = Q = I and noise processes, i.e., Σw = σw 6 z 6 T z 3 R = Rscale · I3 .
4.2
Controllers
We consider three controllers to evaluate their performance on the benchmark systems. The first is the sub-optimal separation principle controller, referred to as Sep, defined as follows: Sep
ut
= Lt x̂t|t−1
where Lt = −(B ⊤ Kt+1 B + R)−1 B ⊤ Kt+1 A.
(4.1)
Here, the matrix Kt is computed recursively through Riccati equation, starting from KT = QT : Kt = A⊤ Kt+1 A − Pt + Q, ⊤
where
⊤
Pt = A Kt+1 B(B Kt+1 B + R)−1 B ⊤ Kt+1 A.
(4.2)
The second is the controller Sep-MPC, which is an MPC version of separation principle controller. At each time t, it solves the finite-horizon deterministic linear-quadratic problem: min x̄⊤ H QT x̄H +
H−1 X
u0:H−1
⊤ (x̄⊤ τ Qx̄τ + uτ Ruτ )
τ =0
s.t. x̄τ +1 = Ax̄τ + Buτ ,
x̄0 = x̂t|t−1 .
Equivalently, this problem is solved via the finite-horizon Riccati recursion MPC KtMPC = A⊤ Kt+1 A − Pt + Q, ⊤
Pt = A
where
MPC MPC MPC Kt+1 B(B ⊤ Kt+1 B + R)−1 B ⊤ Kt+1 A
MPC = Q . Then it applies the action u = LMPC x̂ that starts with Kt+H t T t|t−1 where t MPC MPC LMPC = −(B ⊤ Kt+1 B + R)−1 B ⊤ Kt+1 A. t
Finally, the third controller is B-MPC, our proposed belief-space MPC controller. The algorithm is described in Section 3.3 and Algorithm 1. At each planning step, B-MPC approximately solves the nonconvex finite-horizon optimization problem. We use limited-memory BFGS (L-BFGS), a quasi-Newton optimization algorithm, implemented in PyTorch. We initialize each entry of the H-length control sequence randomly i.i.d. from N (0, 1/H) at every time step, and the optimizer is run for 20 outer iterations with step size 0.8. We use a single initialization per planning step. Unless noted otherwise, the experiments use T =300 time steps, and the same sampled initial condition and noise realization across controllers within each trial.
4.3
Improved performance with B-MPC Table 1: System Parameter Configuration System Random System Double Integrator
ρ(A) 0.95 0.95
c0 0.01 0.01
Rscale 1 1
σw 0.1 0.1
σz 0.1 1.0
We first demonstrate that B-MPC can achieve substantial cost reductions over both Sep and Sep-MPC, for the parameter settings shown in Table 1. These parameters reflect a setting where the system is not very stable, the input-independent observation has small magnitude, and the state and input costs are comparable. 7
Figure 1: Total cost versus look-ahead horizon H ∈ {5, 10, 15, 20, 25, 30} for the three controllers, averaged over 10 trials. Shaded regions denote 95% confidence intervals across trials. Left: random bilinear observation system. Right: multi-block double integrator system. Under the parameter setting in Table 1, Figure 1 shows that B-MPC significantly outperforms the controllers Sep and Sep-MPC for all planning horizons H ∈ {5, 10, 15, 20, 25, 30}. For random bilinear system, the cost reduction from B-MPC is 16.9% compared to the cost from Sep at H=10, and for the double integrator system, the cost reduction is 38.8% at H=15. On the other hand, Sep-MPC performs worse than Sep for double integrator systems when the planning horizon is too low, and otherwise performs comparably. To better understand the source of this improvement, we decompose the total cost into state and control components. The same parameters are used as in Table 1 with fixed look-ahead horizon H. We use H = 10 and H = 15 for random bilinear system and double integrator system, respectively. Table 2: Cost decomposition for each controller in two systems. The average cost over 10 trials is shown. H Controller Sep Random System 10 Sep-MPC B-MPC Sep Double 15 Sep-MPC Integrator B-MPC System
State-cost 124.67 124.68 87.06 876.22 875.05 422.64
Input-cost 2.38 2.38 18.57 53.93 56.14 146.30
Total-cost 127.05 127.06 105.64 930.15 931.20 568.94
Table 2 provides extra information in addition to Figure 1. At a fixed look-ahead window H for each system, it shows that B-MPC achieves substantially lower state cost compared to Sep and Sep-MPC, while it incurs slightly larger control cost. This describes where B-MPC reduces the cost throughout the trajectory.
5
Experiments: Interpretation & Discussion
In this section, we interpret the performance differences observed in Section 4.3. Our goal is to understand why B-MPC outperforms Sep and Sep-MPC in the favorable regime of Figure 1. In particular, we examine the controllers from three complementary perspectives: estimation quality in Section 5.1, the control actions selected along a trajectory in Section 5.2, and how the discrepancy between B-MPC and Sep-MPC changes with the level of uncertainty in Section 5.3. The results illustrate how the main 8
advantage of B-MPC arises due to the fact that planning is carried out in the belief space. For the experiments, we use the same parameter configuration as in Table 1 with fixed planning horizon H=10 and H=15 for random bilinear system and double integrator system, respectively.
5.1
Estimation improves through covariance reduction
We examine trajectories of the multi-block double integrator style system, averaged over 10 trials. Figure 2 shows that the performance gain of B-MPC is closely related to improved state estimation and lower error covariance. On the left, the estimation error ∥xt − x̂t|t−1 ∥2 is generally lower for B-MPC than for Sep and Sep-MPC over the trajectory. On the right, the panel shows an even stronger separation in the estimation covariance trace tr(Σt|t−1 ), with B-MPC maintaining substantially smaller uncertainty throughout the trajectory. Because the covariance update depends on the chosen input in the bilinear observation model, this gap indicates that B-MPC selects actions that make the system state easier to estimate. Hence, Figure 2 provides an empirical illustration that B-MPC leverages the dual role of control, namely state regulation and improvement of future observability.
Figure 2: Kalman filter diagnostics on the multi-block double integrator system with H=15. Left: Kalman filter state estimation error ∥xt − x̂t|t−1 ∥2 . Right: the trace of Kalman filter estimation covariance tr(Σt|t−1 ). Curves are averages over 10 trials under matched initial conditions and noise realizations. The shaded region denotes 95% confidence intervals across trials.
5.2
Differences in control actions
We compare a Sep-MPC and B-MPC through a counterfactual action-matching experiment. First, we −1 generate a single trajectory using Sep-MPC controller. This produces a sequence of belief states (bt )Tt=0 Sep-MPC T −1 and inputs (ut )t=0 along the trajectory. At each time step t, we solve B-MPC controller from the same belief state bt , obtaining uB-MPC . By repeating this for all time steps, we obtain a sequence t −1 B-MPC of counterfactual input sequence (ut )Tt=0 . The goal of this section is to compare the two input sequences. Figure 3 shows the result. From the left to right, each panel corresponds to the i-th element of the action. From the two input sequences obtained above, we plot Sep-MPC (ut )i , (uB-MPC )i t Sep-MPC
so that the x-axis is the i-th coordinate of ut and y-axis is the corresponding coordinate of uB-MPC . t Points on the diagonal indicate agreement between the two controllers, while deviations from the diagonal indicate differences in the selected action coordinate. In both systems, we can see that the actions from Sep-MPC are near zero while the ones from B-MPC are more spread. Interestingly, B-MPC tends to choose inputs that move away from zero, but not so far that the input penalty dominates. This is consistent with dual-control interpretation of B-MPC: the controller explicitly trades off immediate regulation and future information. Since c0 =0.01, near-zero 9
Figure 3: Counterfactual action comparison between Sep-MPC and B-MPC. A trajectory from Sep-MPC is generated first, and at each time step, B-MPC is solved from the same belief state. Each point shows the corresponding Sep-MPC and B-MPC action coordinates. The dashed line indicates equality, uB-MPC = uSep-MPC . actions make the input-dependent observation matrix C(ut ) close to zero, which can substantially reduce observability. This results in worse state estimation and high uncertainty as in Figure 2.
5.3
Controller disagreement increases with uncertainty
The scatter plots in Figure 3 show that B-MPC and Sep-MPC disagree, but they do not reveal when the disagreement is largest. Since B-MPC accounts for covariance evolution in its planning objective, we expect the action difference to grow with the level of state uncertainty. To test this directly, we construct synthetic beliefs: 10 random state estimates x̂ ∼ N (0, 0.25I), each paired with Σ = αI for 20 log-spaced values of α (tr(Σ) ∈ [0.06, 190]). At each synthetic belief state, we solve both B-MPC and Sep-MPC and record ∥uB-MPC − uSep-MPC ∥2 .
Figure 4: Action difference ∥uB-MPC − uSep-MPC ∥2 versus tr(Σ) for synthetic belief states. Results are computed from 10 sampled state estimates combined with 20 log-spaced covariance scales. The black line shows the median across synthetic beliefs, and the shaded region indicates the interquartile range. Figure 4 shows the result on a log-log scale. For both systems, the gap between B-MPC and
10
Sep-MPC increases monotonically as tr(Σ) grows. This confirms that B-MPC deviates from Sep-MPC when uncertainty is high. This behavior aligns with the fundamental trade-off of dual control, demonstrating that the belief-space controller explicitly weighs the long-term value of uncertainty reduction against the immediate penalty of cost minimization.
5.4
Regimes where B-MPC provides limited benefit
Separation principle remains a popular heuristic, and may perform reasonably in practice. Indeed, we find that belief space planning does not always provide benefits: B-MPC does not uniformly outperform the baselines. To map out the boundary, we compute the percentage cost reduction of B-MPC over Sep across a grid of Rscale ∈ {1, 10, 100} and c0 ∈ {0.01, 0.1, 1.0}, selecting for each combination the horizon H ∈ {5, 10, 15, 20, 25, 30} that minimizes B-MPC cost.
Figure 5: Percentage cost improvement of B-MPC over Sep across Rscale and c0 settings for the random bilinear and double-integrator systems, with ρ(A) = 0.95. For each (Rscale , c0 ) pair, we select the horizon H that minimizes the mean B-MPC cost over 10 trials, and report the relative gain 100×(JSep −JB-MPC )/JSep (%). Each cell is annotated with the corresponding percentage. In Figure 5, we can observe a trend that B-MPC tends to perform better as both c0 and Rscale are smaller. If c0 is small, then the input-dependent observation matrix C(ut ) highly depends on the current input ut . Due to the quadratic input cost, Sep finds inputs that are close to zero, resulting in high chance of observability loss. On the other hand, as B-MPC accounts for state uncertainty through the belief space, it outperforms Sep when Rscale =1. However, as Rscale increases, the performance gap between B-MPC and Sep decreases, since large control penalties restrict the controller’s ability to choose exploratory actions that reduce uncertainty. Table 3: System Parameter Configuration System Random System Double Integrator
H 10 15
c0 0.01 0.01
Rscale 1 1
σw 0.1 0.1
σz 0.1 1.0
For an additional view of the regime in which belief-space planning is less favorable, we vary the stability of the system. The system parameters are set according to Table 3. Figure 6 shows that the performance gap between B-MPC and the baseline controllers narrows as the dynamics become more stable. When the spectral radius is small, the system is easier to regulate, and the benefit of dual control is reduced. The marginal value of improving observability is smaller, so explicitly planning over the belief covariance provides less benefit. Consequently, B-MPC performs similarly to the Sep and Sep-MPC.
11
Figure 6: Total cost versus spectral radius for the three controllers, where ρ(A) ∈ {0.85, 0.9, 0.95, 1.0, 1.05, 1.1}. Curves are averaged over 10 trials, and shaded regions denote 95% confidence intervals across trials. Left: random bilinear system. Right: multi-block double integrator.
6
Conclusion
We studied quadratic control of linear dynamical systems from bilinear observations, a setting in which the control input affects both state evolution and observation quality, and where the separation principle no longer applies. To address this, we proposed a belief-space MPC controller that plans over both the estimated state and the estimation error covariance. Our experiments show that B-MPC can outperform both the separation principle controller and an MPC variant of separation control. The empirical results further show that the gain of B-MPC is accompanied by low estimation error covariance and by action choices that differ most from the baseline when uncertainty is high, supporting the dual-control interpretation of the method. Overall, these findings suggest that explicitly modeling uncertainty evolution is an important design principle for control under bilinear observations. Future work includes establishing rigorous closed-loop stability guarantees for the receding-horizon approximation, developing specialized solvers to efficiently handle the nonconvex planning problem, and extending the policy space to neural-network architectures trained via deep reinforcement learning.
7
Acknowledgments
S.D. was partly supported by NSF CCF 2312774, NSF OAC-2311521, NSF IIS-2442137, a gift to the LinkedIn-Cornell Bowers CIS Strategic Partnership, and an AI2050 Early Career Fellowship program at Schmidt Sciences.
References [CSJ+ 25] Sunmook Choi, Yahya Sattar, Yassir Jedra, Maryam Fazel, and Sarah Dean. Explore-then-commit for nonstationary linear bandits with latent dynamics. arXiv preprint arXiv:2510.16208, 2025. (Cited on page 1.) [GP16] Lars Grüne and Jürgen Pannek. Nonlinear model predictive control. In Nonlinear model predictive control: Theory and algorithms, pages 45–69. Springer, 2016. (Cited on page 2.) [Kal60] Rudolph Emil Kalman. A new approach to linear filtering and prediction problems. 1960. (Cited on page 3.)
12
[KBL+ 23] Elia Kaufmann, Leonard Bauersfeld, Antonio Loquercio, Matthias Müller, Vladlen Koltun, and Davide Scaramuzza. Champion-level drone racing using deep reinforcement learning. Nature, 620(7976):982–987, 2023. (Cited on page 1.) [KKR22] Sahand Hadizadeh Kafash, Justin Koeln, and Justin Ruths. Model predictive control of bilinear systems as uncertain linear systems. In 2022 IEEE Conference on Control Technology and Applications (CCTA), pages 562–567. IEEE, 2022. (Cited on page 2.) [KLC98] Leslie Pack Kaelbling, Michael L. Littman, and Anthony R. Cassandra. Planning and acting in partially observable stochastic domains. Artificial Intelligence, 101(1–2):99–134, 1998. (Cited on page 2.) [LK24] Diyou Liu and Mohammad Khosravi. System identification for linear dynamics with bilinear observation models: An expectation–maximization approach. In 2024 IEEE 63rd Conference on Decision and Control (CDC), pages 7190–7195, 2024. (Cited on page 1.) [May14] David Q Mayne. Model predictive control: Recent developments and future promise. Automatica, 50(12):2967–2986, 2014. (Cited on page 2.) [NS24] Keiko Nagami and Mac Schwager. State estimation and belief space planning under epistemic uncertainty for learning-based perception systems. IEEE Robotics and Automation Letters, 9(6):5118–5125, 2024. (Cited on page 2.) [PT87] Christos H. Papadimitriou and John N. Tsitsiklis. The complexity of markov decision processes. Mathematics of Operations Research, 12(3):441–450, 1987. (Cited on page 2.) [PY10] Panos M Pardalos and Vitaliy A Yatsenko. Optimization and control of bilinear systems: theory, algorithms, and applications, volume 11. Springer Science & Business Media, 2010. (Cited on page 1.)
[RMD24] J.B. Rawlings, D.Q. Mayne, and M. Diehl. Model Predictive Control: Theory, Computation, and Design. Nob Hill Publishing, LLC, 2024. (Cited on page 2.) [SCJ+ 25] Yahya Sattar, Sunmook Choi, Yassir Jedra, Maryam Fazel, and Sarah Dean. Sub-optimality of the separation principle for quadratic control from bilinear observations. In 2025 IEEE 64th Conference on Decision and Control (CDC), pages 3862–3867. IEEE, 2025. (Cited on pages 1, 2, and 3.)
[SJD25] Yahya Sattar, Yassir Jedra, and Sarah Dean. Learning linear dynamics from bilinear observations. In 2025 American Control Conference (ACC), pages 3109–3115. IEEE, 2025. (Cited on page 1.)
[XBSA25] Yifan Xie, Julian Berberich, Robin Strässer, and Frank Allgöwer. Bilinear data-driven min-max mpc: Designing rational controllers via sum-of-squares optimization. In 2025 IEEE 64th Conference on Decision and Control (CDC), pages 1042–1047. IEEE, 2025. (Cited on page 2.)
[ZPT23] Vrushabh Zinage, Ali Reza Pedram, and Takashi Tanaka. Optimal sampling-based motion planning in gaussian belief space for minimum sensing navigation, 2023. (Cited on page 2.) [ZRTAm22] Dongliang Zheng, Jack Ridderhof, Panagiotis Tsiotras, and Ali-akbar Agha-mohammadi. Belief space planning: A covariance steering approach. In 2022 International Conference on Robotics and Automation (ICRA), pages 11051–11057. IEEE, 2022. (Cited on page 2.)
13
A
Additional Computational Experiments
In this appendix, we provide additional experiments from an optimization and implementation perspective. The goal is to clarify the computational cost of the proposed belief-space MPC controller and to examine the effect of initialization in the L-BFGS solver used for B-MPC.
A.1
Wall-clock Runtime
We compare the wall-clock time of Sep, Sep-MPC, and B-MPC across planning horizons H. The reported time corresponds to the computation time required to generate one full closed-loop trajectory of length T = 300, averaged over 10 trials. The system parameter configuration follows Table 1 in Section 4.3. For Sep-MPC, we consider two implementations. The first implementation solves the finite-horizon deterministic linear-quadratic planning problem through the Riccati recursion, which is described in Section 4.2. This implementation is numerically stable and expected to have nearly the same computational cost as Sep. In the main experiments, we use the Riccati-based implementation of Sep-MPC. The second implementation solves the same deterministic planning problem by directly optimizing over the H-step planning input sequence using L-BFGS. This is equivalent to ignoring the covariance component of the belief-space objective in the B-MPC optimization problem while keeping the deterministic state component.
Figure 7: Computation time versus planning horizon H for the random bilinear and double-integrator systems. The y-axis reports wall-clock time in seconds for one full closed-loop trajectory (T = 300), averaged over 10 trials. Controllers compared are Sep, Sep-MPC, B-MPC. For Sep-MPC controller, it is implemented in two different ways: one is solving the Riccati equation and the other is obtained via L-BFGS optimizer by ignoring the covariance in the belief state. Figure 7 shows the wall-clock comparison. Although the Riccati-based and L-BFGS-based implementations of Sep-MPC solve the same deterministic planning problem, their runtime difference shows the computational cost of using L-BFGS in place of the finite-horizon Riccati recursion. The runtime gap between L-BFGS-based Sep-MPC and B-MPC is also substantial, even though the two methods use the same optimization procedure. The only difference is that B-MPC additionally propagates the nonlinear input-dependent covariance recursion within each objective evaluation. Thus, the additional computational cost of B-MPC comes from explicitly optimizing over the belief dynamics, especially the covariance evolution.
A.2
Effect of L-BFGS Initialization
We next study the effect of initialization on the L-BFGS optimization used by B-MPC. At each planning step, B-MPC solves a nonconvex finite-horizon optimization problem over the planned control sequence. 14
A natural question is whether initializing this optimization problem at the Sep-MPC control sequence improves the resulting solution compared to random initialization. The system parameter configuration is follows Table 1 in Section 4.3. We compare two initialization schemes. The first is the random initialization used in the main experiments, where the entries of the planned control sequence are initialized independently from N (0, 1/H). The second is Sep-MPC initialization, where the planned control sequence is initialized using the open-loop solution of Sep-MPC computed from the same estimated state x̂t|t−1 , without using the covariance component Σt|t−1 . We then vary the maximum number of L-BFGS iterations and compare the resulting closed-loop rollout cost.
Figure 8: Total rollout cost versus the maximum number of L-BFGS iterations for B-MPC under two initialization schemes. We compare random initialization with Sep-MPC initialization, where the planned control sequence is initialized using the Sep-MPC solution from the same belief state. The experiments use planning horizon H = 15 and trajectory length T = 100, averaged over 10 trials. The shaded regions denote 95% confidence intervals across the trials. Figure 8 shows that Sep-MPC initialization does not consistently improve the B-MPC solution. This effect is especially noticeable in double integrator system, where Sep-MPC initialization can lead to substantially higher cost than random initialization, even after several L-BFGS iterations. Although Sep-MPC initialization provides a natural initial point, the resulting L-BFGS solution can remain in a more suboptimal region of the nonconvex belief-space objective. Therefore, we use random initialization in the main experiments.
15