arXiv CSopen access

Clock-state olfactory search in turbulent flows using Q-learning: The geometry of plume recovery

Clock-state olfactory search in turbulent flows using Q-learning: The geometry of plume recovery Marco Rando Université Côte d’Azur, Inria, CNRS, Laboratoire J.A. Dieudonné, 28 avenue Valrose, 06108, Nice, France

Robin A. Heinonen Machine Learning Genoa Center & Department of Civil, Chemical and Environmental Engineering, University of Genova Via Montallegro 1, 16145 Genoa, Italy

arXiv:2605.15938v1 [physics.bio-ph] 15 May 2026

Yujia Qi Machine Learning Genoa Center & Department of Civil, Chemical and Environmental Engineering, University of Genova, Via Montallegro 1, 16145 Genoa, Italy, now at University of California, Dept Mechanical Engineering, Engineering II, Santa Barbara, CA 93106-5070, USA

Agnese Seminara Machine Learning Genoa Center & Department of Civil, Chemical and Environmental Engineering, University of Genova, Via Montallegro 1, 16145 Genoa, Italy (Dated: May 18, 2026) Finding an odor source in a turbulent flow requires effectively leveraging the history of olfactory observations into a robust navigation strategy. In this work, we use tabular Q-learning to train an olfactory search agent with a minimal memory of past observations: only a running clock since the last whiff. This agent learns an interpretable strategy to recover the plume which combines wellknown behaviors observed in insects: surging, casting, and a return downwind. While achieving good performance on data from direct numerical simulations of turbulence, the agent is limited by an inability to adapt its strategy to the local intermittency level; we show that providing more flexibility improves robustness.

I.

INTRODUCTION

Insects frequently track sources of odors to locate mates and food. In the well-studied and relevant setting of a turbulent flow (such as the atmospheric boundary layer), this “olfactory search” behavior can be remarkably complex, having evolved to cope with the steep challenges posed by stochastic odor encounters, odor sparsity, and the absence of useful spatial gradients, all of which are endemic to turbulence [8, 31, 36]. It is widely recognized that recovering the plume is key to efficient navigation both in biology and robotics, as agents often encounter extended odor-free regions in turbulent conditions, e.g. [7, 9, 14, 15, 30]. Evolved search strategies bear certain similarities across species. After detection of an attractive odor cue, insects frequently begin flying or walking upwind for several seconds [1, 10, 11, 20, 39]. If after this brief surge they do not detect odor, they typically engage in a variety of exploratory behaviors to recover the plume, for example looping [43], crosswind zigzags (possibly performed while drifting upwind or downwind) [3, 25], or direct downwind motion [20, 45]. The precise sequence of behaviors depends on the species; for instance, casting is highly stereotyped in flying moths but can occur with variable timing, speed, and structure in flying Drosophila and walking insects such as cockroaches [36]. The properties of the plume and the turbulent flow help shape these recovery behaviors, which have been

observed to vary when the plume structure is changed [35]. For example, from a Bayesian perspective, the castand-surge behavior may be understood as a means to exhaustively explore the support of the prior on the source location, itself induced by the plume shape [4, 14, 33]. Recovery behavior is also influenced and constrained by the structure of the searcher’s memory. Memory is necessary both to chain distinct behaviors together and to accumulate information about the source, which odor cues grant only noisily due to turbulence. More generally, memory is essential to any partially observable decision task [18]. Behavioral and neural studies suggest that insects integrate information from odor detected along their path [1, 11, 17, 19]. The first relay of olfactory processing shows that insects are able to measure complex features of odor time traces [6, 28], but the extent to which these complex features are used for navigation is unknown. Several models of the agent’s internal memory state have been explored in previous work, ranging from complex, high-dimensional representations such as a Bayesian posterior [12, 23, 42] or the hidden state of a recurrent neural network [34], to simple discrete representations, such as a finite-time window of the recent odor observation history [30], a small handful of learned, abstract memory states (each encoding a distinct behavior) [41], or a clock that resets at each odor detection [4]. This last memory structure may be motivated by experiments on walking flies which suggest that the time since

2 odor was last detected is an important feature driving behavior [11, 17]. In this work, we ask what shapes the recovery strategy of efficient turbulent navigation. To this end, we develop an algorithm that learns how to navigate a realistic turbulent odor plume, using only a clock state that encodes time since the last odor encounter, thereby isolating how much of the navigation problem can be captured by a minimal recovery-oriented memory. This generalizes previous work hard-coding a heuristic, bio-inspired cast-and-surge strategy [4]; here, the recovery strategy is optimized using Q-learning. It is also a simplification of Ref. [30], which learned an additional in-plume strategy depending on the recent odor observation history. As in [30], we assume agents are aware of the direction of the mean wind, which allows them to learn policy of actions labeled as upwind, downwind and crosswind; the more realistic case where agents can only rely on instantaneous wind speeds is investigated elsewhere [29]. We find that remembering only the time since last detection is sufficient for efficient navigation, showing that a highly compressed memory can capture much of the performance of more elaborate strategies which include more information about the odor observation history. This agrees with previous work finding that recovery is by far the most important component of the search strategy [14]. We show that a clock-state Q-learning agent learns a structured recovery policy consisting of upwind surge, lateral exploration, and delayed downwind return. Importantly, the details of the policy vary in different realizations of the training, confirming that a multitude of recovery strategies reach similar performance. We find that the geometric characterization of these behaviors has a systematic dependence on plume sparsity. We argue that the main drawback of this agent is a lack of adaptability to its location within the plume, and we compare our results to a slightly richer two-mode extension which improves generalization to distinct plumes. The remainder of this paper is organized as follows. We present our results in Sec. II: first, we benchmark the performance of the Q-learning strategy agent; second, we define metrics characterizing the geometry of trajectories, and by studying their values in Q-agents, we show how sparsity shapes the recovery strategy; third, we introduce a more sophisticated agent with two independent recovery strategies, and show these strategies adapt to different regions of the plume; and fourth, we demonstrate that a quasi-optimal Bayesian agent can adapt its strategy depending on its observation history, a flexibility which the Q-learning approach lacks. We then summarize our results and suggest future avenues of research in Sec. III. Finally, in Sec. IV we conclude with a detailed presentation of our methodology.

FIG. 1. Problem setting. (a) We perform simulations of a turbulent flow with a source of odors (blue circle). An agent (purple circle) is sensitive to odors and can move in the four cardinal directions. (b) The agent’s internal state s is a clock measuring the time since the last odor detection, running from s = 0 up to s = H, where H is the duration of an episode. Odor detections (black circles) reset the clock, while odor blanks (white circles) increment it. (c) We define four environments with different sparsity levels, which we set by varying the odor detection threshold. (d) Episodic reward G as a function of number of training episodes. The Q-learning algorithm converges within 3 × 106 episodes. (e) Mask showing possible initial positions of the agent within the plume. The position is drawn uniformly from this set, and the starting simulation snapshot is selected randomly.

II.

RESULTS

Reinforcement learning [37] provides a natural framework for olfactory search and other problems involving decision-making under uncertainty. Q-learning [44] is particularly easy to train and is adapted to scenarios where the agent’s internal state space and action space are both discrete. To explore how recovery strategies adapt to the environment, we simplify the Q-learning approach developed in Ref. [30]. In that work, we crafted a small set of olfactory states from a fixed temporal window of concentration measurements; in the event where all measurements in memory are zero, a separate recovery strategy was learned using the time since the last odor whiff as the state. Presently, the olfactory states are dropped, and the agent remembers only the time s since the last odor “whiff”, defined as odor being above a

3 dense

sparse

sparser

f+

⟨Tmin /T⟩

⟨G⟩

denser

mized by the algorithm. The result is plotted in Fig. 2; we compare the results to both a quasi-optimal Bayesian agent and a cast-and-surge heuristic whose characteristic opening angle has been tuned to the setting [2, 4]. The Q-learning agent significantly outperforms cast-andsurge, highlighting the inefficiency of a hard-coded recovery strategy and the value of optimizing the timing and geometry of recovery behaviors. Note that the learned policy and its performance varied significantly from training run to training run; hence, in Fig. 2 and elsewhere we will generally show box-and-whisker plots to illustrate the distribution of learned policies. The upper end of the whisker corresponds to the best learned policy. Somewhat surprisingly, the best Q learning policy nears performance of the Bayesian algorithm, while using a much simpler memory structure. To rationalize the meaning of G, we represent additionally the normalized arrival time Tmin /T (where Tmin is the minimal time of arrival for the episode), and the success rate f + . In all sparsity settings, the agent typically arrives to the source with a 90% success rate or greater, approaching 100% for the least sparse settings.

B. FIG. 2. Performance of the agents presented in this paper. For each strategy, we optimize the mean episodic reward ⟨G⟩ (top); to interpret the meaning of G we additionally quantify the mean normalized time ⟨Tmin /T ⟩ (center) and the success rate f + (bottom). The single-Q, two-Q and heuristic two-Q strategies are shown as box-and-whisker plots over an ensemble of twenty different trained agents (outliers are indicated by white circles). Two baselines, a quasi-optimal Bayesian agent and an optimized one-parameter cast-and-surge heuristic, are presented as horizontal lines

fixed concentration threshold. The agent therefore learns a fixed sequence of moves as a response to plume loss. The methodology, presented in more detail in Sec. IV, is illustrated in Fig. 1. We train and test our approach on high-quality direct numerical simulation of a turbulent channel flow, previously used in Refs. [30, 33]. The flow has an embedded point source of contaminants and represents similar conditions to those of the lower atmospheric boundary layer. We test four different detection thresholds on the concentration, allowing us to tune the sparsity of encounters with the odor. For consistency, the agent’s initial time and position in the simulation is drawn from the same distribution for each choice of threshold.

A.

Performance

We first test the raw performance of the Q-learning algorithm. We measure this using the average episodic reward G, which is the quantity that is directly opti-

Geometry of recovery

After a detection, trained Q-learning agents first surge straight upwind; then, they engage in a complex recovery strategy aimed at re-entering the plume, primarily comprising an oscillating forward motion and eventually a downwind return. We find that the relative prevalence of different recovery behaviors depends strongly on plume sparsity. To study this phenomenon, we define several metrics which characterize the geometry of a trajectory, as illustrated in Fig. 3. Additional examples of learned trajectories are shown in the Supplemental Material (see Appendix A). To wit, these are (1) the upwind distance traversed during the initial surge, (2) the crosswind breadth traversed during casting, (3) the total upwind distance traversed before returning downwind, (4) the total downwind distance traversed during the return, and (5) the time spent searching before the beginning of the downwind return. First, we note that the upwind surge becomes markedly stronger as the sparsity of the plume increases. This corresponds with the intuition that the surge corresponds to a memory of the last odor detection, and that the typical time between successive detections increases with sparsity. On the other hand, while the crosswind cast width and downwind return distance appear to increase and decrease, respectively, with plume sparsity, this dependence is quite weak; the strongest signal is that the downwind returns are very short at the highest sparsity level. In Ref. [14], it was argued that these quantities are connected closely to the wind speed, which was held fixed in our tests.

4

FIG. 3. Geometry of the search strategy depends on sparsity of the plume. (a) Trajectories of (single-)Q agent in absence of odor for different plume sparsity levels, color coded as indicated. Increasing values of the clock state are represented from bright colors (s = 0) to dark colors (s = H). (b) Five different metrics are defined to characterize the shape of a recovery strategy: (1) the upwind surge length, (2) the crosswind cast width, (3) the total upwind search length before returning downwind, (4) the downwind return length, and (5) the downwind return initiation time. (c) Geometry of the learned policies in the different environments. The five metrics are shown as box-and-whisker plots across an ensemble of twenty trained agents. The plots show an increasing tendency to travel upwind and decreasing tendency to return downwind as plume sparsity increases. (d) Probability distribution of the downwind position of the agent at its last detection, conditioned on failing to find the target. The last detection occurs relatively close to the source in all environments, suggesting that failures occur when the agent overshoots the source. The peak moves downwind with increasing sparsity, indicating that agents get lost at larger downwind distances in sparser plumes.

Finally, in sparse plumes, agents spend more time and cover more distance searching upwind before initiating their downwind return. We reason that this is to avoid becoming trapped far downwind of the source in a region that is particularly poor of information within sparse plumes. On the other hand, in dense plumes, the agent begins returning downwind significantly earlier. Drifting downwind will recover the plume when agents accidentally overshoot the source, which is the primary failure mode in dense plumes. Indeed, as sparsity increases, agents get lost increasingly further downwind from the source, as shown by the distributions of downwind distances where the last detection occurred, conditioned on the agent failing to find the source (Fig. 3(d)). Thus, agents fail mainly when they either get trapped far downwind of the plume or overshoot the source; the relative

FIG. 4. A Q-agent’s generalization to different sparsity levels. Upper panel: violin plots showing distributions of the downwind positions of the last odor detection before failure, when an agent is trained in one environment (indicated in x-axis labels) and tested in another (indicated by plot color). Lower panel: box-plots showing episodic rewards under generalization, for both single-Q (solid boxes) and two-Q (empty boxes) strategies. Using two-Q substantially increases generalization when trained in sparse conditions.

FIG. 5. A more flexible algorithm with two separate Qmatrices. The size of the state space has been doubled to S = S + ∪ S − . Both sets of states are ordered by an integer value, s+ ∈ [0, H] and s− ∈ [0, H]. Odor detections (black circles) reset the agent from any state s± > τ to state s+ = 0 and from any state s± ≤ τ to s− = 0. In the absence of odor detection (white circles), the clock state is incremented: s+ → s+ + 1 and s− → s− + 1. The Q-matrix Q(s, a) is split as Q+ = Q(s+ , a) and Q− = Q(s− , a).

importance of these two failures depends on sparsity of the plume. By the same logic, agents trained in dense environments generalize poorly to those sparse environments (Fig. 4), and vice-versa, as they are unable to execute the appropriate strategy to avoid the failure mode that is more common in the new environment. The poor generalization performance can be attributed to an increase in failure rate (see Supplemental Material in Appendix A). As shown in the upper panel of Fig. 4, agents trained in dense environments fail especially often far downwind of

5 the source when they are tested in sparse environments.

C.

Two-Q agent

Avoiding overshooting the source and avoiding getting lost in the back of the plume are ultimately conflicting goals, and an agent with a single recovery trajectory cannot simultaneously solve both problems; instead it will simply prioritize whichever problem it is exposed to more often in training. The agent is unable to adapt its behavior to the local sparsity level of the plume. To test if a single agent may learn to solve both problems, we introduce a more flexible algorithm, with two parallel Q-matrices Q± (Fig. 5). In this setup, the time since the last detection no longer suffices to specify the agent’s action; instead, a crude measure of the local sparsity level is also used. In particular, we fix a threshold time τ . Matrix Q+ is employed when the last detection occurred t > τ after the previous detection, and matrix Q− is employed when it occurred t ≤ τ after the previous detection. We found that the performance of this two-Q agent did not depend strongly on τ , and a good strategy can be learned and adapted to essentially any threshold (see Supplemental Material in Appendix A). We chose to select τ so that both Q matrices are used and thus optimized as close to evenly as possible during training; see the Supplementary Material (Appendix A) for more details. The performance of the two-Q agent is compared to the single-Q agent in Fig. 2. The benefit of this extra flexibility becomes clear only under sparse conditions, where overshooting and becoming lost far downwind are both important issues to solve; in dense plumes, the latter problem does not occur frequently. For the same reason, using the two-Q strategy also significantly improves the agent’s ability to generalize, specifically in the case where the agent was trained in a sparse environment, as shown in Fig. 4 (lower panel). In Fig. 6, we show that Q− is more active in the core of the plume close to the source and the centerline, while Q+ is more active in sparse regions at the edges of the plume. Consistent with this observation, we find that Q− specializes in avoiding overshooting, whereas Q+ specializes in avoiding being trapped at the back of the plume. Indeed, upwind search is far more prevalent in Q+ than in Q− ; and it increases with sparsity, similar to trends observed with single Q. Conversely, the downwind return occurs much sooner in Q− (see Fig. 7(a)). Q− also features a more prominent surge, since it relies on detections more than Q+ . Typical trajectories in Q− and Q+ are compared in Fig. 7(b). We also show in Fig. 7(d) the downwind positions of the last detection before failure when using two-Q, only Q+ , and only Q− . Using Q− alone leads to frequent failures in the back of sparse plumes, but sees no significant increase in failure close to the source, again emphasizing its specialization towards

downwind returns. Conversely, using Q+ alone leads to increased failures close to the source, confirming that it does not learn to avoid overshooting. Finally, since we expect Q+ to be more active in sparse regions and Q− more active in dense regions, we tested a heuristic policy that uses the Q matrix trained under dense (sparse) conditions as Q− (Q+ ) (Fig. 7). As seen in Fig. 2, although this policy is fixed and not re-trained in each new environment, it either outperforms or nearly matches the optimized single-Q agent.

D.

A quasi-optimal agent

The primary shortcoming of the Q-learning approach is its inflexibility; the agent cannot adapt its strategy to its history of odor observations, since it only recalls the time since the most recent detection. To illustrate that an optimal agent does not have this drawback, we trained a quasi-optimal Bayesian agent using the POMDP formalism [18]. This approach retains memory of its observation history in the form of a spatial map, called the belief b, assigning probability to possible source locations, which is updated using a model for the detection probability. This form of memory is lossless if the model is perfect (in fact, we have neglected that observations are correlated in time [13]). The agent’s policy maps beliefs to actions, so that in principle the entire observation history may influence its behavior. As shown in Fig. 8, a single agent’s trajectory after losing the plume may indeed exhibit significantly different geometry depending on its detection history. However, certain trends persist; for one, the typical surge length and time until downwind return continue to increase with sparsity, which are realized primarily as lengthenings in the tails of the distributions of those quantities. This maximum flexibility affords greater performance (Fig. 2), but at the cost that vastly greater computational resources are required to execute the policy.

III.

DISCUSSION

Despite the notorious difficulty of the olfactory search problem in turbulent settings, we have shown that a simple discrete memory specifying the time since the last detection suffices to develop a high-performing agent. The learned policy specifies only how to recover from loss of the plume, underlining that deciding what to do in the absence of odors is the most difficult and most important part of the problem, as previously noted (see e.g. Refs. [14, 30, 41]). The present work also shows that the classical cast-and-surge heuristic, which shares the same set of internal states, can be improved significantly by optimizing the precise shape of the casting and surging and additionally learning to return downwind. We have also shown how plume sparsity influences the geometry of recovery strategies. In dense plumes, re-

6

FIG. 6. Top: colormap of the average Eulerian blank time (the typical time between successive odor detections) at increasing sparsity level. Contours at τ = 5, 10, 25, 50, 100, 200 are shown. Bottom: usage of Q+ , measured as the empirical fraction of test agents using Q+ at each location. Q− is mostly used within the core of the plume, where detections are frequent, whereas Q+ is mostly used in the back and on the edges, where detections are rare. For the denser plume, Q− is mostly used, i.e., most agents are in s ∈ S − .

FIG. 7. In the two-Q agent, Q+ and Q− learn distinct recovery strategies which target different regions of the plume. (a) Characterization of recovery strategy geometry, as in Figure 3(c), for Q+ and Q− . The single-Q result is reproduced for comparison. (b) Sketch of representative policies for both Q+ and Q− in the four environments and (c) of the heuristic policy 2Qh (bottom). (d) Downwind positions of the last detection before failure for two-Q, Q− only, and Q+ only. When using Q− only, agents fail in the back of the plume as Q− is not trained to avoid losing the plume there. The reverse happens for Q+ , which has not learned to prevent overshooting.

turning downwind is essential to avoid overshooting the source; while this is also important under sparse conditions, sparse plumes also require more significant upwind motion to avoid trapping at the sparse rear of the plume. The inability to reconcile both of these behaviors is a key weakness of the single-Q approach, which can be partially mitigated by extending the internal memory in a minimal way, as demonstrated with the two-Q approach. A Bayesian agent, on the other hand, is capable of quasi-optimal performance by adapting its strategy to any arbitrary sequence of odor observations; in addition to much larger computational resources, executing such a policy requires an accurate model of the environment. It should be noted that how exactly the agent negotiates the tradeoffs discussed here depends to some extent on the initial condition. Our agents are optimized for efficient search starting from a fixed area that is sampled uniformly and kept identical in all environments. If instead the search starts from a region that adapts to sparsity of the plume, or if a detection is forced to initiate the search, trends are qualitatively preserved while details vary (see Supplementary information in Appendix A). The range of initial positions must be appropriately considered, as it can vary for a variety of reasons connected to prior information both for animals [7] and in robotic applications [15]. Here, we labeled actions relative to the mean wind, which is supposed to be known in advance by the agents. Measuring the mean wind is however unrealistic because it requires long integration which may be impractical in robotics and unrealistic in biology. In reality, animals rely on local measures of flow speed, using e.g. antennas for insects [5, 26, 32, 38], whiskers for rodents [46] or the lateral line for marine organisms [22]. Forthcoming research will study anemometric agents that align their actions relative to a shifting wind axis which they estimate in real time [29]. Our results also suggest a concrete way to test how much plume recovery is governed by a fixed internal program vs adapts to recent olfactory history. Note that a history dependence in navigating insects has been observed, including variable upwind surge lengths, but the

Surge upwind

7 histories can produce markedly different recovery trajectories. This distinction suggests an experimental protocol: condition the searcher on different recent odor histories, then impose an identical prolonged blank, and compare the ensuing recovery trajectories. In animals, such tests could be implemented in virtual-odor (i.e., optogenetic) experiments, similar to, for example, Ref. [17]. In robots, our results suggest that controllers with highly compressed memory may already be sufficient for robust plume tracking, depending on the acceptable tolerance for failure probability.

20

Downwind length

Upwind search

Crosswind cast

0

10

0 40

ACKNOWLEDGMENTS 0

This research was supported by grants to AS from the European Research Council under the European Union’s Horizon 2020 research and innovation programme (grant agreement number 101002724 RIDING), the National Institutes of Health under award number R01DC018789 and by a France 2030 support managed by the Agence Nationale de la Recherche, under the reference ANR-23PEIA-0004 (PDE-AI project). This work represents only the views of the authors; the European Research Council Executive Agency and the other funding agencies are not responsible for any use that may be made of the information it contains.

20

0

Sparser

Sparse

Dense

0 Denser

Downwind init time

400

IV. FIG. 8. Geometry of a single quasi-optimal Bayesian recovery strategy varies from trial to trial. Distributions of the five trajectory metrics are shown as violin plots; these are obtained by tracking a single agent’s behavior during a long sequence of odor blanks, for an ensemble of prior observation histories. The metrics can take broadly different values depending on the history, reflecting the flexibility granted by the Bayesian agent’s sophisticated, high-dimensional memory. Consistent with the trends observed across Q-agents, the distributions of upwind surge length, upwind search length, and downwind initiation time exhibit increasingly long tails as sparsity increases.

origin of this variation is poorly understood [7, 20, 27, 40]. In the clock-state agent, once odor is lost, the recovery trajectory is determined by a single scalar variable, the time elapsed since the last detection, producing a single stereotyped sequence comprising surge, lateral exploration, and eventual downwind return. By contrast, the Bayesian agent shows that even under the same immediate condition of odor loss, different prior detection

METHODS

Data description

Data used to train and test the agents are set of 2598 123×27 matrices {Dt }2598 t=0 where every matrix Dt ∈ R contains the odor intensity in every position (i, j) i.e. (Dt )i,j represents the odor intensity in position (i, j) at timestep t ∈ N. The odor source is located at position xsource = [0, 13]; positions in a circle of radius 2 centered at the source (source region) are considered terminal states. The odor dataset is a downsampled version of odor simulation used in [30, Simulation 1] where spatial dimensions are reduced by a factor of 10. Data information are summarized in Table I. The four environments (Denser, Dense, Sparse, Sparser) are obtained by thresholding the data, setting all values below the noise threshold nthr ∈ R+ to zero. In particular, for the Denser environment nthr = 10−6 , for Dense nthr = 3 × 10−6 , for Sparse nthr = 5 × 10−6 , and for Sparser nthr = 7 × 10−6 . The dataset and code will be shared shortly.

8 TABLE I. Summary of dataset details Feature

Value

# Grid points 123 × 27 # Time slices 2598 Agent speed (grid point/ time slice) 1 Source position [0, 13] Source region radius 2

Agent design and state representation

In this section, we describe the internal state representations used by the single-Q and two-Q agents, and how these states are updated over time. Single-Q agent. For a single-Q agent, the internal state is a clock variable s ∈ N that counts the number of consecutive time steps during which the observed signal remains below a fixed sensitivity threshold cthr > 0. At each time step t ∈ N, the agent receives an observation ωt ∈ R+ and updates its clock accordingly. Specifically, if ωt ≥ cthr , the agent detects odor and the clock is reset to st = 0. If ωt < cthr , no odor is detected and the agent is considered to be in a void state; in that case, the clock is incremented according to st = st−1 + 1. At time step t = 0, the agent receives an initial observation ω0 ∈ R+ and initializes its clock as ( 0, if ω0 ≥ cthr , s0 = 1, otherwise. For each subsequent time step t > 0, the clock is updated as ( 0, if ωt ≥ cthr , st = st−1 + 1, otherwise. In both training and testing, agents are allowed to execute at most H ∈ N+ steps (the horizon). This implies that this class of agents has H states, with state 0 corresponding to odor detection and the remaining H − 1 states corresponding to void states. Two-Q agent. For a two-Q agent, the internal state is again based on a clock, but the agent additionally records the duration of the previous odor-free interval. More precisely, at each time step t ∈ N, the agent stores the length of the most recent completed blank interval in a variable τt , and compares it with a fixed threshold τ . Unlike the single-Q agent, the two-Q agent uses two distinct families of void states. State st = 0 again indicates that odor has just been detected, i.e. ωt ≥ cthr . When a void observation is encountered, i.e. ωt < cthr , the subsequent evolution depends on whether the previous blank interval was shorter or longer than τ . If τt ≥ τ , the agent uses one family of void states, indexed by 1, . . . , H −1; otherwise it uses a second family, indexed by H, . . . , 2H −1. In total, the two-Q agent therefore uses 2(H − 1) + 1 states.

If an odor detection occurs, the variable τt is updated to the length of the blank interval that has just ended, namely st−1 , and the clock is reset to st = 0. If instead a void observation occurs, then the update depends on the previous clock value. If st−1 = 0, the new state is set to either 1 or H, depending on whether τt < τ or τt ≥ τ . If st−1 > 0, the clock is simply incremented. We initialize the previous blank length as τ0 = τ + 1, and define the initial clock state by ( 0, if ω0 ≥ cthr , s0 = H, otherwise. For each t > 0, the previous blank length is updated as ( st−1 , if ωt ≥ cthr , τt = τt−1 , otherwise, and the clock state is then updated according to   0, if ωt ≥ cthr ,   H, if ωt < cthr ∧ st−1 = 0 ∧ τt ≥ τ, st =  1, if ωt < cthr ∧ st−1 = 0 ∧ τt < τ,    st−1 + 1, if ωt < cthr ∧ st−1 > 0. At initialization, τ0 is set above the threshold τ . This choice makes the agent initially behave as though it were in a sparse region far from the source, thereby triggering the corresponding exploratory strategy. If the agent instead begins near the source, where blank intervals are typically shorter, the value of τt adjusts rapidly after a few detections. Characterization of trajectories in the void. To characterize behavior in the absence of odor, we analyze the deterministic trajectory generated by an agent when it starts at position (0, 0) and thereafter receives only blank observations. From this trajectory we extract five geometric features: surge length, upwind search, backtracking length, cast width, and backtracking initiation time. These quantities are defined as follows: 1. Surge length. This is the number of initial upwind steps (leftward moves) taken by the agent before it leaves the surge phase. To avoid counting isolated deviations caused by optimization noise, we use a tolerance of two steps: the surge is deemed to end only after two consecutive non-leftward actions. 2. Cast width. This quantifies the lateral extent of the largest cast and is approximated by the difference between the maximum and minimum y positions reached by the agent. 3. Upwind search. This measures the extent of the trajectory in the upwind direction and is computed as the minimum x position attained by the agent.

9 4. Backtracking length. This is the extent of the trajectory in the downwind direction and is approximated by the absolute difference between the largest and smallest x values reached. 5. Backtracking initiation time. This is the number of steps executed before the beginning of sustained downwind motion. Operationally, we define it as the number of actions taken before the agent performs three consecutive rightward moves. The values of the upwind search, backtracking length, and cast width depend on how much of the void trajectory is included. Because the final part of the trajectory may be noisy—the deepest void states may have been visited only rarely during training—we evaluated these metrics under several truncation criteria: (i) the full trajectory; (ii) the largest void state visited during the last 500 training episodes (the criterion used in the main text); (iii) the average void state visited during the last 500 training episodes; and (iv) the maximum duration of a blank during the last 500 training episodes; and (v) the mean duration of a blank during the last 500 training episodes (a blank goes from one detection to the next detection, so it excludes instances where the agent gets lost). All choices lead to qualitatively similar patterns (see Supplemental Material A).

Agent behavior and policies

We now describe how the agents interact with the environment to solve the olfactory navigation problem. At each time step t ∈ N, the agent receives an odor observation ωt ∈ R+ , updates its internal state st , and chooses an action at . The action space consists of the four cardinal directions:

denote location of the odor source. The source region S is defined as follows. S = {x : ∥x − xsource ∥2 ≤ 2}. In our experiments, the odor source is xsource = [0, 13]. At each time step, the agent selects an action using an ε-greedy tabular Q-learning policy. Let Q denote the agent’s Q-matrix. Then ( a ∈ arg maxa∈A Q(st , a), at = a ∼ U(A),

with probability 1 − ε, with probability ε,

where U (A) denotes the uniform distribution on A. During training, ε is gradually decreased across episodes. During testing, we set ε = 0, so that the policy is deterministic. Training procedure

Each episode begins at a random location and a random time slice of the DNS data. The initial position is sampled uniformly from the set of grid points at which there is a nonzero probability of detecting odor in the sparsest environment, i.e. from the set of points for which the concentration exceeds 7 × 10−6 at least once. The same initialization rule is used for all threshold values, to ensure consistency across environments. At each time step t, the agent receives an observation ωt ∈ R+ , updates its state st , and selects and executes an action at . After the action is executed, the agent receives a reward rt and a new observation ωt+1 ∈ R+ . The reward is defined by the position reached after the action: if the new position lies outside the source region, then

A := {e1 , e2 , −e1 , −e2 },

rt = −(1 − γ),

where ei ∈ R2 denotes the ith canonical basis vector. Agents are allowed to leave the data grid; once outside the grid, they continue to receive zero observations. If the agent reaches the source region, it receives a distinguished observation ωt = −1. Formally, let xsource ∈ R2

where γ ∈ (0, 1) is the discount factor; if the new position lies inside the source region, then rt = 1. The initial Q-matrix is set to Q0 (s, a) = −1 for all states and actions. For each episode k, the Q-matrix is updated at every step t = 0, . . . , H according to

 Qk+1 (st , at ) = (1 − αk ) Qk (st , at ) + αk

Here st is the state at time step t, at is the action selected by the εk -greedy policy, and st+1 is the state obtained from the new observation ωt+1 .

 rt + γ max Qk (st+1 , a) . a∈A

The learning rate is scheduled as   αdecay − k − 1 αk = αend + (αinit − αend ) max ,0 , αdecay with αinit = 0.1, αend = 10−4 , and αdecay = 2.5 × 106 .

10 Similarly, the exploration parameter is scheduled as   εdecay − k − 1 ,0 , εk = εend + (εinit − εend ) max εdecay with εinit = 1.0, εend = 10−4 , and εdecay = 2.5 × 106 . Thus, at episode k = 0 we have α0 = 0.1 and ε0 = 1.0, while after 2.5 × 106 episodes both parameters remain fixed at 10−4 . With this choice, we strike a balance between speed and accuracy. Note that tabular Q-learning is only guaranteed to converge asymptotically anyways. Additionally, there is no guarantee that the optimal policy is deterministic under partially observability [16]. The horizon is fixed at H = 500 steps, the discount factor is γ = 0.998, and training is run for 3,000,001 episodes. The sensitivity threshold cthr is set to 10−6 for the denser plume, 3 × 10−6 for the dense plume, 5 × 10−6 for the sparse plume, and 7×10−6 for the sparser plume. For each setting, we train 20 single-Q agents and 20 two-Q agents using the parameters above.

Evaluation

To evaluate performance, we consider three metrics: the cumulative reward G, the normalized arrival time, and the success fraction f + . These quantities are averaged over all valid initial positions. Valid initial positions are the points (x, y) ∈ Dinit , where Dinit is the set of grid points outside the source region for which there is a nonzero probability of observing an odor concentration greater than 7 × 10−6 , corresponding to the sparsest environment. For (x, y) ∈ Dinit , let T (x, y) be the number of steps required for the policy to reach the source, and let Tmin (x, y) be the shortest possible arrival time from (x, y). We define the normalized arrival time as

Bayesian POMDP agent

To implement and solve the POMDP, we follow the procedure used previously in [12–14, 23, 24]. The agent’s partially observed state is its position x relative to the source, evolving on a two-dimensional grid of size (2M − 1) × (2N − 1), where M × N are the dimensions of the simulation box. The observation space consists of three symbols: whiff, blank, and source. The likelihoods Pr(whiff | x) and Pr(blank | x) are given by the empirical probabilities that the concentration at x exceeds or falls below, respectively, the threshold cthr . The observation source occurs if and only if the agent is in the source region, which is treated as a single absorbing state. Because the observation model is assumed known, the Bayesian POMDP approach is model-based, whereas Q-learning is model-free. The agent maintains a belief state bt (x), i.e. a posterior distribution over positions. With a slight abuse of notation, we treat the source region as a single state. At each time step, the agent receives an observation ωt , selects an action at ∈ A, and updates its physical state deterministically via x 7→ x + at . The belief is then updated by Bayes’ rule: bt (x − at ) Pr(ωt | x − at ) . ′ ′ x′ bt (x − at ) Pr(ωt | x − at )

bt+1 (x | ωt , at ) = P

The initial belief is uniform on the initialization region Dinit . The agent receives reward 1, discounted at rate γPOMDP , upon entering the source region, and 0 otherwise. Thus, ( 1, R(x, a) = 0,

Tmin (x, y) T (x, y) if the trajectory reaches the source region, and as 0 otherwise. The success fraction f + is the fraction of valid initial positions from which the policy reaches the source. Finally, G denotes the cumulative reward averaged over all valid initial positions.

V [bt ] = max a∈A

( X

if x + a ∈ S, otherwise.

The optimal expected return can be expressed as a functional of the current belief b, namely the value function V [b]. This function satisfies the Bellman optimality equation [18]

) bt (x)R(x, a) + γPOMDP

x

The unique solution specifies the optimal policy by selecting, at each belief state, an action that maximizes the right-hand side. Because this Bellman equation is computationally in-

X

Pr(ω | bt , a) V [bt+1 (· | ω, a)] .

(1)

ω

tractable except through approximation, we use the SARSOP algorithm [21] to compute a quasi-optimal policy for each value of cthr . In these computations we use γPOMDP = 0.98 and terminate SARSOP after a fixed

11 runtime of 4000 seconds. To obtain an ensemble of recovery trajectories for the quasi-optimal agent, we use a constrained Monte Carlo tree procedure. The agent executes a search trial in the DNS data, and each time it makes a whiff observation we create a branch point. Along one branch, the search continues normally; along the other, all subsequent observations are forcibly set to blank for H = 500 time steps. The trajectory generated during this forced-blank sequence is recorded. To downweight highly implausible forced-blank continuations, each such trajectory is assigned the weight

adapted to a discrete gridworld and augmented with a free parameter controlling the opening angle of the search cone. The algorithm is given in Algorithm 1. The upwind distance traveled between successive casts was fixed at one grid unit. For each sparsity level, we tuned the half-opening angle in increments of 5◦ so as to maximize the episodic reward G. Algorithm 1 Angle-parameterized cast-and-surge policy on a grid 1: Input: half-opening angle θ ∈ (0, π/2) and observation

ω ∈ {0, 1} 2: Internal state: x (upwind distance since last detection),

w=

TY back

Pr(blank | xt ),

t=toff

where toff is the time at which the forced-blank sequence begins, and Tback is the time at which the agent completes its downwind return. Each search trial is terminated when the agent reaches the source region or when the initial search time reaches 100 steps. We repeat this procedure 100 times for each threshold value.

Cast-and-surge heuristic

As a baseline, we implemented a variant of the cast-and-surge heuristic of Balkovsky and Shraiman [4],

[1] Efrén Álvarez-Salvado, Angela M Licata, Erin G Connor, Margaret K McHugh, Benjamin MN King, Nicholas Stavropoulos, Jonathan D Victor, John P Crimaldi, and Katherine I Nagel. Elementary sensory-motor transformations underlying olfactory navigation in walking fruitflies. Elife, 7:e37815, 2018. [2] T. C. Baker. Upwind flight and casting flight: complementary and tonic systems used for location of sex pheromone sources by male moths. Proc. 10th Intl Symposium on Olfaction and Taste, 13:18, 1990. [3] TC Baker and Kenneth F Haynes. Manoeuvres used by flying male oriental fruit moths to relocate a sex pheromone plume in an experimentally shifted wind-field. Physiological Entomology, 12(3):263–279, 1987. [4] Eugene Balkovsky and Boris I Shraiman. Olfactory search at high Reynolds number. Proceedings of the national academy of sciences, 99(20):12589–12593, 2002. [5] W. J. Bell and E. Kramer. Search and anemotaxis in insects. J. Insect Physiol, 25:631–640, 1979. [6] Sofia C Brandão, Marion Silies, and Carlotta Martelli. Adaptive temporal processing of odor stimuli. Cell and Tissue Research, 383(1):125–141, 2021. [7] Ring T. Cardé’. Navigation along windborne plumes of pheromone and resource-linked odors. Annual Review of Enthomology, 66:317–336, 2021. [8] Ring T Cardé and Mark A Willis. Navigational strategies used by insects to find distant, wind-borne sources of

y (lateral displacement), s ∈ {−1, +1} (casting side) 3: σ ← tan θ 4: if ω = 1 then 5: (x, y, s) ← (1, 0, +1) 6: return (−1, 0) 7: end if 8: y ⋆ ← s⌈σx⌉ 9: if y < y ⋆ then 10: y ←y+1 11: return (0, 1) 12: else if y > y ⋆ then 13: y ←y−1 14: return (0, −1) 15: else 16: s ← −s 17: x←x+1 18: return (−1, 0) 19: end if

▷ surge upwind

odor. Journal of chemical ecology, 34(7):854–866, 2008. [9] Antonio Celani, Emmanuel Villermaux, and Massimo Vergassola. Odor landscapes in turbulent environments. Physical Review X, 4(4):041015, 2014. [10] Cassandra T David, JS Kennedy, and AR Ludlow. Finding of a sex pheromone source by gypsy moths released in the field. Nature, 303(5920):804–806, 1983. [11] Mahmut Demir, Nirag Kadakia, Hope D Anderson, Damon A Clark, and Thierry Emonet. Walking drosophila navigate complex plumes using stochastic decisions biased by the timing of odor encounters. Elife, 9:e57524, 2020. [12] Robin A Heinonen, Luca Biferale, Antonio Celani, and Massimo Vergassola. Optimal policies for Bayesian olfactory search in turbulent flows. Physical Review E, 107(5):055105, 2023. [13] Robin A Heinonen, Luca Biferale, Antonio Celani, and Massimo Vergassola. Exploring Bayesian olfactory search in realistic turbulent flows. Physical Review Fluids, 10(6):064614, 2025. [14] Robin A Heinonen, Luca Biferale, Antonio Celani, and Massimo Vergassola. Optimal trajectories for Bayesian olfactory search in turbulent flows: The low information limit and beyond. Physical review fluids, 10(4):044601, 2025. [15] Hiroshi Ishida, Hidenao Tanaka, Haruki Taniguchi, and Toyosaka Moriizumi. Mobile robot navigation using vi-

12 sion and olfaction to search for a gas/odor source. Autonomous Robots, 20(3):231–238, 2006. [16] Tommi Jaakkola, Satinder Singh, and Michael Jordan. Reinforcement learning algorithm for partially observable markov decision problems. In G. Tesauro, D. Touretzky, and T. Leen, editors, Advances in Neural Information Processing Systems, volume 7. MIT Press, 1994. [17] Viraaj Jayaram, Aarti Sehdev, Nirag Kadakia, Ethan A Brown, and Thierry Emonet. Temporal novelty detection and multiple timescale integration drive drosophila orientation dynamics in temporally diverse olfactory environments. PLoS computational biology, 19(5):e1010606, 2023. [18] Leslie Pack Kaelbling, Michael L Littman, and Anthony R Cassandra. Planning and acting in partially observable stochastic domains. Artificial intelligence, 101(1-2):99–134, 1998. [19] Nicholas D Kathman, Aaron J Lanz, Jacob D Freed, and Katherine I Nagel. Neural dynamics for working memory and evidence integration during olfactory navigation in drosophila. bioRxiv, pages 2024–10, 2025. [20] LPS Kuenen and Ring T Carde. Strategies for recontacting a lost pheromone plume: casting and upwind flight in the male gypsy moth. Physiological Entomology, 19(1):15–29, 1994. [21] Hanna Kurniawati, David Hsu, Wee Sun Lee, et al. Sarsop: Efficient point-based pomdp planning by approximating optimally reachable belief spaces. In Robotics: Science and systems, volume 2008. Zurich, Switzerland, 2008. [22] James C. Liao. The role of the lateral line and vision on body kinematics and hydrodynamic preference of rainbow trout in turbulent flow. Journal of Experimental Biology, 209(20):4077–4090, 10 2006. [23] Aurore Loisy and Christophe Eloy. Searching for a source without gradients: how good is infotaxis and how to beat it. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 478(2262), 2022. [24] Aurore Loisy and Robin A Heinonen. Deep reinforcement learning for the olfactory search pomdp: a quantitative benchmark. The European Physical Journal E, 46(3):17, 2023. [25] A Mafra-Neto and RT Cardé. Dissection of the pheromone-modulated flight of moths using single-pulse response as a template. Cellular and Molecular Life Sciences, 52(4):373–379, 1996. [26] T. S. Okubo, P. Patella, I. D’Alessandro, and R. I. Wilson. A neural network for wind-guided compass navigation. Neuron, 107:924–940, 2020. [27] Rich Pang, Floris Van Breugel, Michael Dickinson, Jeffrey A Riffell, and Adrienne Fairhall. History dependence in insect flight decisions during odor tracking. PLoS computational biology, 14(2):e1005969, 2018. [28] Mainak Patel and Aaditya Rangan. Olfactory encoding within the insect antennal lobe: The emergence and role of higher order temporal correlations in the dynamics of antennal lobe spiking activity. Journal of theoretical biology, 522:110700, 2021. [29] Lorenzo Piro, Maurizio Carbone, Luca Biferale, Massimo Cencini, Robin A. Heinonen, Marco Rando, and Agnese Seminara. Smart strategies to navigate turbulent odor plumes reorienting to local wind. preprint, 209:4077– 4090, 2026.

[30] Marco Rando, Martin James, Alessandro Verri, Lorenzo Rosasco, and Agnese Seminara. Q-learning with temporal memory to navigate turbulence. Elife, 13:RP102906, 2025. [31] Gautam Reddy, Venkatesh N Murthy, and Massimo Vergassola. Olfactory sensing and navigation in turbulent environments. Annual Review of Condensed Matter Physics, 13(1):191–213, 2022. [32] A. M. Reynolds, D. R. Reynolds, A. D. Smith, and J.W. Chapman. Orientation cues for high-flying nocturnal insect migrants: Do turbulence-induced temperature and velocity fluctuations indicate the mean wind flow? Plos ONE, 5:e15758, 2010. [33] Nicola Rigolli, Gautam Reddy, Agnese Seminara, and Massimo Vergassola. Alternation emerges as a multimodal strategy for turbulent odor navigation. Elife, 11:e76989, 2022. [34] Satpreet H Singh, Floris Van Breugel, Rajesh PN Rao, and Bingni W Brunton. Emergent behaviour and neural dynamics in artificial agents tracking odour plumes. Nature machine intelligence, 5(1):58–70, 2023. [35] Shuchita Soman, Sree Subha Ramaswamy, and Sanjay P Sane. Odor tracking in insects: a multisensory behavior. Journal of Experimental Biology, 229(Suppl 1):jeb250945, 2026. [36] Theresa J Steele, Aaron J Lanz, and Katherine I Nagel. Olfactory navigation in arthropods. Journal of Comparative Physiology A, 209(4):467–488, 2023. [37] Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction, volume 1. MIT press Cambridge, 1998. [38] M. P. Suver, A. M. Matheson, S. Sarkar, M. Damiata, D. Schoppik, and K. I. Nagel. Encoding of wind direction by central neurons in drosophila. Neuron, 102:828–842, 2019. [39] Floris Van Breugel and Michael H Dickinson. Plumetracking behavior of flying drosophila emerges from a set of distinct sensory-motor reflexes. Current Biology, 24(3):274–286, 2014. [40] van Breugel F and Dickinson MH. Plume-tracking behavior of flying drosophila emerges from a set of distinct sensory-motor reflexes. Curr Biol, 24:274, 2014. [41] Kyrell Vann B Verano, Emanuele Panizon, and Antonio Celani. Olfactory search with finite-state controllers. Proceedings of the National Academy of Sciences, 120(34):e2304230120, 2023. [42] Massimo Vergassola, Emmanuel Villermaux, and Boris I Shraiman. ‘infotaxis’ as a strategy for searching without gradients. Nature, 445(7126):406–409, 2007. [43] NJ Vickers and TC Baker. Latencies of behavioral response to interception of filaments of sex pheromone and clean air influence flight track shape in Heliothis virescens (F.) males. Journal of Comparative Physiology A, 178(6):831–847, 1996. [44] Christopher JCH Watkins and Peter Dayan. Q-learning. Machine learning, 8(3):279–292, 1992. [45] Mark A Willis and Edmund A Arbas. Odor-modulated upwind flight of the sphinx moth, manduca sexta l. Journal of Comparative Physiology A, 169(4):427–440, 1991. [46] Yan S. W. Yu, Matthew M. Graff, Chris S. Bresee, Yan B. Man, and Mitra J. Z. Hartmann. Whiskers aid anemotaxis in rats. Science Advances, 2(8):e1600716, 2016.

13 Appendix A: Supplementary Material

FIG. 9. Generalization of single Q agents (left) and two-Qs agents (right), measured by the cumulative reward G (top tow) and its projections on the normalized time to reach the target (mid row) and the fraction of successes (bottom row). Training in sparseALTRI plumes provides particularly visible benefits for success rates in generalization. MODI DI DEFINIRE this is the portion of the trajectory that’s visited on average

this is the portion of the trajectory up to a number of voids that corresponds to the max blank (closed). So it corresponds to portion of trajectory that appears successful. Beyond this, agents wondered outside of the plume, so one may argue that that’s a failure, and the portion of trajectory up to max voids “successful”

trajectory in void up to mean blank experienced by trained agent. Initial portion of trajectory

mean state

max void

mean void

GEOMETRIA.

one may argue recovery is shaped also by trying to come back after loss of contact w plume -> the whole trajectory is meaningful until the state that you experienced. probably this is the easiest to justify. Also shows crosswind shows no clear trend.

max state

Downwind length

Upwind search

Crosswind cast

entire trajectory

FIG. 10. Cast width, upwind search and downwind length, defined in the main text, depend on the duration of the trajectory that is considered. Left to right: results using truncation criteria described in Materials and Methods, Characterization of trajectories in the void, (i) to (v). Second column: results with criterion used throughout the main text.

14

FIG. 11. Results of the 2Q algorithms are robust to the choice of the threshold τ that dictates which Q-matrix is used (Right). We choose τ so that both Q-matrices are used as evenly as possible (left).

SUPPLEMENTARY — 1Q TRAIETTORIE

FIG. 12. Policies of the 20 single Q agents, trained in the four environments with increasing sparsity from top to bottom. The red dot marks the position for state s = 0, successive steps indicate policy for increasing values of the state variable s > 0, with darker and darker colors

15 Init: uniformly sampled on the native cone. Starts at time with detection 20

Init: uniformly sampled on the native cone. Starts any time. 20

Paper: Init uniformly sampled on the sparsest cone. Starts any time.

16

patterns are robust to initial condition

Crosswind cast

Surge upwind

17.5

Init: uniformly sampled on the sparsest cone. Starts at time with detection

0

0

0

80

80

70

1.

0 80

2.

20

10 90

90

5

10

10

80

70

100

100

0 500

Downwind length

Upwind search

30

0

0

400

500

25

50

Downwind init time

0 200

100

80

3.

10

4.

5.

100

FIG. 13. Supplementary. Optimizing agents starting with different initial conditions yield qualitatively similar patterns in geometry of trajectory in the void. First two columns from left: Agents start at a timestamp within the simulation when they detect odor; Last two columns on the right: agents start at a random timestamp within the simulation, when they may or may not detect. In all cases, agents start at a location that is sampled uniformly from a region that encompasses all points where the odor is above threshold at any time. First and third column from left: the threshold is the same that defines the training environment, thus increases with sparsity. Note that for this choice, denser plumes are optimized to reach the target from a region that is larger than dense plumes. Second and fourth column: threshold is fixed at the highest value so that the initial condition is the same for all training environments. Rightmost column is used throughout main paper. All patterns are robust except for downwind initiation and length for initial condition that changes with the environment and forcing a detection at time 0.