Promoting Simple Agents: Ensemble Methods for Event-Log Prediction
arXiv:2604.21629v1 [cs.LG] 23 Apr 2026
Benedikt Bollig1 [0000-0003-0985-6115] , Matthias Függer1 [0000-0001-5765-0301] , Thomas Nowak1,2 [0000-0003-1690-9342] , and Paul Zeinaty1,3 [0009-0003-2885-0410] 1
Université Paris-Saclay, CNRS, ENS Paris-Saclay, LMF, Gif-sur-Yvette, France {bollig,mfuegger,pzeinaty}@lmf.cnrs.fr, [email protected] 2 Institut Universitaire de France, Paris, France 3 Direction Générale de l’Armement, Paris, France
Abstract. We compare lightweight automata-based models (n-grams) with neural architectures (LSTM, Transformer) for next-activity prediction in streaming event logs. Experiments on synthetic patterns and five real-world process mining datasets show that n-grams with appropriate context windows achieve comparable accuracy to neural models while requiring substantially fewer resources. Unlike windowed neural architectures, which show unstable performance patterns, n-grams provide stable and consistent accuracy. While we demonstrate that classical ensemble methods like voting improve n-gram performance, they require running many agents in parallel during inference, increasing memory consumption and latency. We propose an ensemble method, the promotion algorithm, that dynamically selects between two active models during inference, reducing overhead compared to classical voting schemes. On real-world datasets, these ensembles match or exceed the accuracy of non-windowed neural models with lower computational cost.
Keywords: event-log prediction, n-gram, LSTM, Transformer, ensemble methods, agents
1
Introduction
Many modern information systems produce continuous event logs as time-ordered activities associated with concrete cases. Examples include a patient admitted to a hospital, the issuing of a service ticket, or the expiration of a certificate. The logs often contain information that allows one to answer central questions about the corresponding process. The discipline of process mining [1] studies the systematic extraction of models from these logs to describe and predict process behavior. Thereby, classical approaches operate on offline data, e.g., evaluating
2
B. Bollig, M. Függer, T. Nowak, and P. Zeinaty
and suggesting an optimized process. By contrast, recent approaches also target streaming (or online) questions about the process while it is generating the event log, thereby enabling new types of application fields that were not feasible with offline methods [9]. One such streaming-enabled question is the prediction of the next activities from the event log before they happen. Applications are in health monitoring of patients and infrastructure, improved response-times by predictive resource allocations, and targeted countermeasures to threats, among many others. Activity prediction in discrete, case-based event logs. In this work, we focus on logs of finitely many discrete activities. An event is a (case ID, activity) pair. A case is a sequence of events with the same case ID, representing a single process instance such as a patient treatment, user session, or server operation. Examples for cases are subjects like patients, user sessions, or servers. An event log is a collection of events from multiple cases. For such logs, we study the task of next-activity prediction: given a case’s prefix, predict its most likely next activity, including a distinguished symbol stop to mark the end of a sequence. We restrict attention to predictors whose output for a case depends only on that case’s history, that is, where cross-case features can be neglected. This allows us to consider the projection onto the subsequence related to a case, individually. Many such language models have been proposed in the literature, ranging from automata-based methods [36,20,38] to deep learning methods including recurrent architectures [21] and attention-based models [39]. While such language models constantly increase in complexity, alternatives to monolithic language models via agent-based designs have shown promising results [43,34], where specialized agents act as oracles called by a central, high-parameter LLM. While such setups yield high accuracy, the complex LLM remains a bottleneck for throughput and latency, rendering this approach impractical for real-time processing of large event logs. Stream prediction with simple language models. An interesting alternative to such resource-intensive LLMs are ensemble methods [45], where the language model is obtained from simpler agents that operate in parallel on the log file, with their predictions being merged into a typically simple and fast aggregation function like a majority vote. Similar strategies have been analyzed in the context of distributed computing to tolerate a certain fraction of misbehaving agents [24,14,13], though typically motivated by hardware or software faults. Inspired by these approaches, we study whether ensembles of simple language models that act in parallel perform as well as or outperform larger monolithic language models while achieving better throughput. We propose a novel Promotion Algorithm that requires only 2 agents active in parallel during inference, substantially reducing overhead compared to classical ensemble methods like majority voting. We evaluate this approach along with classical ensemble methods and baseline models on synthetic log streams that test specific properties such as counting and interleaving of events, as well as on 5 well-known real-life event-log
Promoting Simple Agents: Ensemble Methods for Event-Log Prediction
3
datasets [27,15,37,16,17], ranging from 15,214 to 2,514,266 events. While long short-term memory (LSTM) networks [21] can outperform individual n-gram models, ensembles of simple n-gram agents match or exceed the accuracy of these neural models stably over a large range of window sizes, with the Promotion Algorithm having only little computational overhead over a simple n-gram. All experiments are conducted within the logicsponge framework [5], a Python library for next-activity prediction algorithms. Related work. Our work touches upon several fields including automata learning and grammatical inference [20,38], process mining [1], and various contexts within machine learning. In automata learning, the Alergia algorithm [11] was proposed to learn stochastic regular languages from a set of words in a language. The work in [28] focuses on learning deterministic probabilistic automata from batches, building on variants of the Alergia algorithm. Compared to classical automata learning in batch mode, grammatical inference for streaming data has received considerably less attention. Notable exceptions include [3,35,4], which propose efficient algorithms for probabilistic automata to address computational complexities in streaming settings. Nonquantitative incremental automata learning was explored in [18]. Event-log prediction in process mining is an active area of research. Recent works include [32,12,31,7,2]. Beyond generic sequence models, predictive monitoring work proposes architectures tailored to event-log data. Transformer-based approaches adapt self-attention to event logs; for example, ProcessTransformer uses event embeddings with positional encodings and is trained over all prefix lengths to predict the next activity, event time, and remaining time [8]. To complement sequence-only encodings, some approaches exploit graph structure: they either combine graph representations with Transformers [19] or apply GNNs (including hybrids with recurrent models) on directly-follows graphs to better capture loops and control-flow dependencies [33,26,41]. Process mining in streaming settings has also been studied (e.g., [10,9,23,44]). Note that [23] suggest using ensemble methods in the presence of noise, though not resorting to n-grams. The use of n-grams dates back to [36] and has been widely applied to a multitude of problems including business processes [7]. Apart from prediction, another interesting application domain of automata learning is verification [28,29,38,25]. For example, [28] focuses on learning deterministic probabilistic automata from batches, building on variants of the Alergia algorithm [11]. However, targeting reactive systems, the focus of this work is on infinite words rather than making predictions based on finite historical data. Finally, several general-purpose automata learning libraries have been developed, including [40,30,6,22]. Outline. We introduce general language models in Section 2. In Section 3, we study a central parameter for many streaming algorithms: the size of the window of historical events it has access to. Section 4 introduces ensemble methods for
4
B. Bollig, M. Függer, T. Nowak, and P. Zeinaty
streaming event-log prediction, including our novel promotion algorithm, and presents experimental results on synthetic and real-world datasets.
2
Model for Next-Activity Prediction
We start with some basic notation. We fix a nonempty finite set Σ of activities, also referred to as actions. The set is not necessarily known in advance to a language model, though an algorithm (e.g., a neural network) may require specifying an upper bound on its size (as part of the embedding dimension in case of the neural network). Activities are referred to as A, B, etc. We write Σ ∗ for the set of finite sequences of activities. In particular, the set includes the empty sequence ε. Given L ⊆ Σ ∗ , we let ∗ L denote the set of all finite sequences obtained by concatenating zero or more words from L. Note that L∗ includes ε. We denote by stop a distinguished symbol not in Σ that marks the end of a sequence. The extended alphabet Σstop = Σ ∪ {stop} contains all activities plus the stop symbol. In the following, we let σ range over Σstop . Activities occur in the context of cases, where each case represents a single execution of the process. Case IDs are usually drawn from a countably infinite set (strings or natural numbers). Since our prediction functions depend only on individual case histories, we abstract from case IDs and view an event log as a multiset over (Σstop )∗ , where each element is a complete case trace. Every position in any word of this multiset corresponds to an event. Note that in a streaming setting with interleaved cases, case IDs would be needed to separate concurrent executions; here, we assume traces are provided separately. A language model can take several forms. Examples are models based on ngrams, bags, probabilistic automata, and recurrent neural networks. In essence, every such language model defines a probabilistic prediction function or, simply, prediction function p : Σ ∗ → ∆(Σstop ), where ∆(Σstop ) is the set of probability distributions over Σstop . Given a sequence w ∈ Σ ∗ , applying p yields a probability distribution over the set of possible next activities, including stop. In analogy to conditional probabilities, one usually writes p(σ | w) for the probability p(w)(σ). The focus in this work is on predicting the most likely next activity, which is readily obtained from a probabilistic prediction function. n-Grams. For n ≥ 1, an n-gram model defines a prediction function p where the probability p(σ | w) depends only on the last n−1 activities of the sequence w. We call this suffix the history. For sequences shorter than n−1 activities, the entire sequence serves as the history. For example, in a 3-gram model, the sequences AB and AAAB share the same suffix AB of length n−1 = 2. The model thus assigns the same probability distribution over next activities in both cases: p(σ | AB) = p(σ | AAAB) for all σ ∈ Σstop . During training, these probabilities are estimated from the frequency distribution of activities that followed the history AB in the training data. In our implementation, n-grams are represented as probabilistic deterministic finite automata (PDFAs) where each state corresponds to a history (a suffix of
Promoting Simple Agents: Ensemble Methods for Event-Log Prediction
5
length up to n−1). This representation enables efficient streaming operation: state transitions during inference require only updating the current automaton state, which implicitly encodes the relevant suffix of length up to n − 1. Training consists of incrementing frequency counters associated with each state. These operations are computationally inexpensive, contributing to the low prediction and training times observed in our experiments. Neural Networks on log sequences: LSTM and Transformer. Recurrent neural networks (RNNs) are a class of neural networks designed to process sequential data by maintaining a hidden state that captures information from previous time steps. Like n-grams, RNNs define a prediction function p : Σ ∗ → ∆(Σstop ), but compute it through learned transformations of sequential hidden states rather than explicit frequency counting. Long Short-Term Memory (LSTM) networks [21] are a specific type of RNN that addresses the vanishing gradient problem, allowing them to capture long-term dependencies in sequences effectively. LSTMs achieve this through the use of memory cells and gating mechanisms that regulate the flow of information. Thus, LSTMs are a natural candidate for predicting next activities in log streams. Another frequently used architecture to handle long histories is the Transformer [39]. They employ self-attention mechanisms that weigh the importance of different parts of the input sequence when computing p(σ | w), enabling them to capture global dependencies effectively.
3
A Central Parameter: Impact of Window Size on Accuracy
While n-grams and LSTMs are inherently suitable for streaming due to their sequential nature (with incremental state transitions), Transformer-based models require modifications to handle streaming data effectively. Since Transformers process entire sequences at once, they require reprocessing the entire sequence every time a new token arrives, inducing arbitrarily long context sizes and rapidly increasing computational cost with sequence length. A common solution is to use sliding windows to truncate the input sequence to a fixed length (also referred to as context size), similarly to n-grams. Although this approach is close to padding techniques used in batch learning, it brings new challenges in streaming, such as determining the appropriate window size and learning positional embeddings that accurately reflect the temporal order of events within the window. Intuitively, a too small window size will not allow the language model to incorporate old, but important actions into its prediction, resulting in a low overall accuracy. Likewise, a too large window size is expected to lead to large parameter numbers that are not trained with the available data, and thus, again a degradation of accuracy. Further, overly complex language models have large latencies and low throughput, invalidating their real-time use for event logs.
6
B. Bollig, M. Függer, T. Nowak, and P. Zeinaty
To better understand how state-of-the-art models behave with bounded windows in streaming settings, we evaluated the impact of different window sizes on their performance, comparing them to n-grams of similar context lengths. Towards that goal, we designed synthetic benchmarks to exhibit specific properties and phenomena relevant to real-life event logs, as well as to highlight the window-size impact on the performance of different models. For each tested pattern, we provide as reference the best theoretical accuracy achievable by any language model with window size n, for varying values of n. We compare this to the accuracy of LSTM- and Transformer-based models with equal window sizes to assess how well they capture the underlying patterns in the data. Finally, we validate our results on real-world datasets. In the following, we describe the different datasets, defined over the alphabet Σ = {A, B}. 3.1
Synthetic Dataset I: Deterministic periodic patterns
A motif that is expected to be of central importance in many datasets, is counting. Counting languages, including periodic patterns, have been studied extensively in the context of neural network learning capabilities [42]. We start with the simplest such sequences, namely deterministic sequences, where the next activity is fully defined by the previous activities, except for the final stop. An ideal language model with a large enough window size can thus obtain an accuracy of asymptotically (within the sequence length) 100%; limited only by wrong predictions of the final stop. For a set L ⊆ Σ ∗ , let pre(L) denote the set of prefixes of sequences in L, concatenated with a final stop symbol. We consider two types of synthetic periodic patterns: – 6-step symmetric, denoted as AAABBB: sequences from pre({A3 B 3 }∗ ). An example is AAABBBAstop. – 5-step asymmetric, denoted as AAABB: sequences from pre({A3 B 2 }∗ ). An example is AAABBAAABstop. By design, an optimal model with a context length of 3 or above (e.g., a sufficiently trained n-gram model with n ≥ 4), should be able to capture the entire pattern and make accurate predictions, while models with shorter context lengths achieve limited accuracy. 3.2
Synthetic Dataset II: Randomized periodic patterns
After evaluating deterministic periodic patterns, we consider event logs that exhibit randomized patterns, to assess the generalization capacity of the language model. Naturally, for such patterns, optimal language models cannot asymptotically achieve 100% accuracy. This is immediately seen from a process that uniformly at random produces A or B until it stops. An optimal model can achieve at most 50% accuracy asymptotically.
Promoting Simple Agents: Ensemble Methods for Event-Log Prediction
7
The selected patterns are slightly more complex to incorporate effects of the history, repeating uniform random picks over a set of predefined sequences of activities, so-called dictionaries D. The sequences are then from the set pre(D∗ ). – 3-step full-discriminative, denoted as xxx̄ and defined by D = {AAB, BBA}. Examples are AABBBAstop and AABAABBBAAAstop. – 4-step half-discriminative, denoted as xAxB and defined by the dictionary D = {AAAB, BABB}. An example is BABBBABBAAstop. – 6-step tier-discriminative, denoted as xABxBA and defined by the dictionary D = {AABABA, BABBBA}. For example: BABBBAAABABAstop. In each pattern, certain positions act as markers that indicate which dictionary word was selected. The patterns combine two common characteristics of process data: periodic repetition of activities (loops) and the selection of different variants within these repetitions. The full-discriminative pattern xxx̄ has all three positions differing between the dictionary words (AAB vs. BBA). To model this pattern, it is necessary to determine which 3-step variant is repeated, as every position provides discriminative information. The half-discriminative pattern xAxB contains markers only at positions 1 and 3 (both A or both B), while positions 2 and 4 are fixed (A and B, respectively). Here, models must identify which 4-step variant is active while separating discriminative positions from routine activities, i.e., activities that occur in all variants. The tier-discriminative pattern xABxBA has markers at positions 1 and 4, with fixed activities in between. Identifying the active 6-step variant requires maintaining information across longer stretches of non-discriminative activities. 3.3
Methods
We evaluated the performance of standard language models for their dependency on window sizes. This includes an LSTM, a Transformer, and an n-gram, each with access to historical events through sliding windows of different sizes. While n-grams and fixed-context-length Transformers inherently require bounded histories, we capped the history for the LSTM to compare its accuracy to the other models. We also ran experiments for non-windowed versions of the Transformer (with increasing context-size) and the LSTM. General experimental setup. All experiments were performed on an 11th Gen Intel Core i9-11900K architecture (3.50GHz, 8 cores, 24GB RAM) with an NVIDIA GeForce RTX 3090 (24GB) running Ubuntu (24.04.1) and Python (3.12.3). For training and inference of the LSTM and Transformer models, we used PyTorch with CUDA. All models are trained with Adam optimizer, cross-entropy loss, learning rate 10−3 , batch size 8, for up to 20 epochs, using early stopping on validation accuracy with patience 3 and gradient clipping. Each dataset was split into a training set (70%), a validation set (15%), and a test set (15%).
8
B. Bollig, M. Függer, T. Nowak, and P. Zeinaty
All inputs use explicit one-hot encoding (no embedding). The LSTM model consists of two stacked LSTM layers (batch-first) with hidden size 128, followed by a linear output layer mapping to vocabulary size. The Transformer is a singlelayer, single-head causal encoder with model dimension equal to the vocabulary size, using Rotary Positional Embeddings (RoPE), which simplify generation of the input sequence by appending and removing actions without changing the positional encoding for the remaining actions, and followed by a linear output layer mapping to vocabulary size. Fixed context-window (prefix) training. In the windowed setting, each training sequence x1 . . . xT ∈ (Σstop )∗ is expanded into all prefixes x1 . . . xk ∈ Σ ∗ , with k < T , each predicting a single target token xk+1 ∈ Σstop . Prefixes are truncated to the last k tokens and left-padded to a fixed window length k, inducing additional processing time for windowed LSTM and Transformer. The model outputs logits for all prefix positions, but the loss is computed only at the final timestep. In contrast, the non-windowed setting uses full sequences, predicts all next tokens in parallel, and applies the loss at every non-padding position, allowing unbounded context. Theoretical best accuracies. For each synthetic dataset and window size, we compute the theoretically best achievable accuracy by listing all possible subsequences of the given window size (with Python, and manually for short ones to check) and computing the accuracy of the optimal predictor. When this best achievable accuracy reaches the proportion of random activities in the pattern, it cannot increase further; we therefore fix this value for all larger window sizes.
3.4
Results
Results of the experiments are shown in Figure 1 for the Synthetic Dataset I (deterministic periodic) and in Figure 2 for the Synthetic Dataset II (randomized periodic). We evaluated model accuracies independently for each pattern. Accuracies for the first quarter (25 cases, 50,000 events) and the full dataset (100 cases, 200,000 events) are shown to assess how the accuracy improves with the dataset size. 6-step symmetric pattern (AAABBB). For this simple pattern, all models match exactly the best theoretical accuracy, at least when window sizes remain under a certain range. Interestingly, the Transformer’s accuracy starts to degrade when the window size exceeds 16, whereas LSTM remains stable. This specific behavior of the Transformer can be explained by its attention mechanism, which may struggle to keep focus on the relevant parts of the input sequence when the window size becomes too large compared to its attention capacity (dependent on single attention head(s)), whereas the LSTM’s recurrent structure allows it to maintain a more consistent representation of the sequence.
Promoting Simple Agents: Ensemble Methods for Event-Log Prediction
9
(a) AAABBB with 50,000 events
(b) AAABBB with 200,000 events
(c) AAABB with 50,000 events
(d) AAABB with 200,000 events
Fig. 1: Window-size impact on next-activity prediction accuracies for deterministic periodic patterns, with varying dataset sizes. The left side shows training on the first quarter and the right side training on the full dataset.
5-step asymmetric pattern (AAABB). In order to slightly increase the difficulty of the task, we tested the asymmetric version of the previous pattern, thus requiring activity-specific model awareness. For pattern AAABB, we first observe a more pronounced performance degradation for the Transformer. This is likely due to the fact that the asymmetry in the pattern requires the model to capture more specific dependencies between activities, which appears more and more challenging for the Transformer as the window size grows beyond what is minimally required to capture the pattern. Another interesting observation is that LSTMs show occasional accuracy drops at certain high window sizes, though these are less severe and systematic than the Transformer’s degradation. On the one hand, this indicates greater architectural robustness to increasing window sizes. 3-step full-discriminative pattern (xxx̄). For the simple randomized pattern, ngrams exhibit a severe accuracy drop when the window size becomes too large. This is expected due to data sparsity: randomness in the sequences means that longer prefixes are each observed fewer times in training, leading to poor probability estimates and eventual accuracy collapse. In accordance with this explanation, an improvement of the n-gram for the full dataset is observed – shifting the observed decline of accuracy to the right from Figure 2(a) to 2(b). The performance of LSTM and Transformer is relatively stable, with a slight performance
10
B. Bollig, M. Függer, T. Nowak, and P. Zeinaty
degradation for the Transformer at very large window sizes, and exceptional learning failures (50% accuracy) for the LSTM at window sizes 7 and 16. 4-step half-discriminative (xAxB) and 6-step tier-discriminative (xABxBA) patterns. For these randomized patterns, as expected, the same statistical effect can be witnessed for n-grams with high window sizes. Similarly, the decline is pushed to the right when comparing the quarter and full dataset. LSTMs are again consistently accurate independently of the window size, whereas the Transformer’s accuracy degrades as previously observed. Interestingly, the window size at which this degradation starts appears to be about the same as for the n-grams. Comparing the Transformer’s accuracy across the three selected randomized periodic patterns, we observe a gradation in terms of slope and consistency of the accuracy degradation, as follows (by increasing consistency, and decreasing slope): half discriminative, then tier-discriminative, and finally fulldiscriminative. This ordering may be linked to the availability of discriminative markers, and thus “how difficult it is to choose the right position for the attention head of the Transformer”.
4
Robustness via Ensemble Algorithms
The previous section highlighted the impact of window sizes on the performance of state-of-the-art models for event-log prediction. Over a large range of window sizes, n-grams demonstrate both good accuracy and computational efficiency due to their algorithmic simplicity. Importantly, in contrast to the generally high accuracies of LSTM and Transformer, the experiments on synthetic datasets showed that n-grams are relatively robust with respect to window size: while performance does degrade at very large windows, n-grams maintain stable accuracy over a broad plateau, whereas LSTM and Transformer exhibit sudden performance breakdowns at specific window sizes within this range. Selecting an appropriate window size for the n-gram within the plateau remains a challenge. We next studied if this challenge can be addressed by running an ensemble of agents with different sizes and aggregating their decision. All base models in an ensemble are trained independently and in parallel on the same training data. The ensemble methods differ only in their evaluation strategy: Soft voting. A natural choice for aggregation is combining probability distributions from multiple n-grams with different window sizes by averaging them and selecting the activity with highest average probability. The overhead of this technique remains non-negligible, however, since all agents have to actively run on the stream. Adaptive voting. This method tracks the accuracy of each model during evaluation, updating accuracy scores after each prediction, and selects the most accurate model from the ensemble for the next prediction.
Promoting Simple Agents: Ensemble Methods for Event-Log Prediction
11
(a) xxx̄ with 50,000 events
(b) xxx̄ with 200,000 events
(c) xAxB with 50,000 events
(d) xAxB with 200,000 events
(e) xABxBA with 50,000 events
(f) xABxBA with 200,000 events
Fig. 2: Window-size impact on next-activity prediction accuracies for randomized periodic patterns, with varying dataset sizes. The left side shows training on the first quarter and the right side training on the full dataset.
As with the previous method, this technique requires running a potentially large set of agents with different window sizes in parallel.
Promotion algorithm. While the previous ensemble methods can increase robustness and avoid the window-size selection problem, they require running many agents in parallel, which comes with high computational overhead. Moreover, the ensemble must cover a wide range of window sizes: our experiments showed that the optimal window size depends both on the inherent process structure (seen in the theoretical accuracy plateau in Figures 1 and 2) and on the amount of training data (seen in the accuracy decline beyond the plateau for large window sizes with limited data).
12
B. Bollig, M. Függer, T. Nowak, and P. Zeinaty
Crucially, we observed that n-grams shift this decline rightward as more training data becomes available. This observation motivates an ensemble algorithm that requires only 2 agents running in parallel. The idea of the Promotion Algorithm (Algorithm 1) is to predict with an n-gram while simultaneously tracking an n′ -gram with n′ > n. If the n′ -gram achieves better accuracy than the ngram for τ ≥ 1 predictions, the algorithm promotes the n′ -gram to become the active model. The promotion threshold τ prevents premature promotion due to short-term performance fluctuations. Requiring τ consecutive confirmations adds robustness: if Mi is truly better, Mi+1 is unlikely to accumulate enough wins to trigger promotion; if Mi+1 is superior, it will reliably do so. In our experiments, we set τ = 20 based on manual empirical calibration during preliminary tests. Smaller values made the decision process unstable and more sensitive to data perturbations, whereas larger values reduced reactivity and delayed promotion, both leading to lower predictive accuracy.
Parameters: Base models M = (M1 , . . . , Mℓ ) with window sizes s1 < s2 < · · · < sℓ ; threshold τ ∈ N. i←1; // start with smallest window size c←0; // promotion counter Acc(Mj ) ← 0 for all j ∈ {1, . . . , ℓ}; while true do obtain predictions âi from Mi and âi+1 from Mi+1 (if i < ℓ); output âi as the current prediction ; // use active model for prediction observe new incoming activity a; update Acc(Mi ) and Acc(Mi+1 ) by comparing âi and âi+1 with a update internal states of Mi and Mi+1 with activity a if i < ℓ and Acc(Mi+1 ) > Acc(Mi ) then c←c+1 ; // increment promotion counter if c ≥ τ then i←i+1 ; // promote the next model c←0; // reset the promotion counter reset accuracy counter for Mi ; // for fair comparison between Mi and Mi+1 end
Algorithm 1: Promotion algorithm with two active language models.
Experimental validation. We tested all these ensemble algorithms together with the previously described monolithic language models on the synthetic datasets as well as real-world datasets; see Table 1. The results for accuracies, average prediction-times, and average trainingtimes are summarized in Table 2 for the synthetic datasets and Table 3 for the real-world datasets. Window sizes for windowed versions of the LSTM and
Promoting Simple Agents: Ensemble Methods for Event-Log Prediction
13
Table 1: Central statistics of the synthetic and real-world datasets used in the validation of next-activity prediction. Synthetic datasets I and II are split into a quarter and the full dataset. Real-world datasets are from [27,15,37,16,17].
#Activities #Cases Avg. case length #Events
Synthetic (Quarter)
Synthetic (Full)
Sepsis (2016)
BPI 2012
BPI 2013
BPI 2017
BPI 2018
2 25 500 50,000
2 100 2,000 200,000
16 1,050 14.49 15,214
24 13,087 20.04 262,200
13 7,554 8.68 65,533
26 31,509 38.16 1,202,267
41 43,809 57.39 2,514,266
Table 2: Results on synthetic datasets. ∆tpred and ∆ttrain denote the average per-activity prediction- and training-time. Best values are in bold.
5-gram LSTM[win:4] Transformer[win:4] LSTM Transformer Soft voting (3,4,5,6) Adaptive voting (3,4,5,6) Promotion (3,5,7,9,13,17,25,33)
A3 B 3
A3 B 2
xxx̄
xAxB
xABxBA
∆tpred (µs)
∆ttrain (µs)
99.95 99.95 99.95 50.02 66.67 99.95 99.95 99.82
99.95 99.95 99.00 59.89 62.22 99.95 99.95 99.88
83.31 83.04 83.09 50.11 50.08 83.12 83.11 83.27
81.11 81.08 81.09 49.68 49.59 87.23 87.23 87.33
83.40 83.27 83.27 50.29 66.62 83.32 83.32 83.35
24 932 2647 11 20 32⋆ 43⋆ 26
4 2133 5213 43 43 9⋆ 5⋆ 10⋆
the Transformer are annotated, as well as ni ’s for the ni -grams used in the ensembles. For the ensemble methods, these sets of individual n-grams were selected by manual empirical calibration in preliminary experiments, retaining the configurations that yielded the best overall predictive performance. Non-windowed versions of Transformers and LSTMs are seen to suffer from accuracy drops in the synthetic dataset, already. Among the windowed language models (with a history length of 4), the 5-gram achieves high accuracy despite its simplicity, and is even surpassed by the Promotion algorithm for more complex synthetic patterns. As expected, prediction and training times are low for the 5-gram, and good for the Promotion algorithm. For the real-world data, LSTMs achieve the highest accuracy. This is in contrast to the synthetic dataset. However, the 5-gram is not far below, demonstrating again a consistently high accuracy across a wide range of datasets. It is even surpassed by ensemble methods, in particular soft voting. Again, the Promotion algorithm achieves comparable accuracy, without requiring the choice of a window size as a hyperparameter, and is faster in prediction than more complex ensemble methods.
⋆
: Latencies for ensemble models include parallelization of the (active) base models.
14
B. Bollig, M. Függer, T. Nowak, and P. Zeinaty
Table 3: Results on real-world datasets. ∆tpred and ∆ttrain denote the average per-activity prediction- and training-time. Best values are in bold.
5
Sepsis (2016)
BPI 2012
BPI 2013
BPI 2017
BPI 2018
∆tpred (µs)
∆ttrain (µs)
5-gram 62.46 LSTM[win:4] 51.50 Transformer[win:4] 57.24 LSTM 64.91 Transformer 62.86 Soft voting (3,4,5,6) 65.68 Adaptive voting (3,4,5,6) 62.55 Promotion (3,5,7,9,13,17,25,33) 61.60
84.83 73.35 73.47 85.87 85.78 85.31 85.58 85.50
72.61 63.27 56.87 73.89 73.69 72.54 72.66 72.40
86.99 81.38 81.07 88.42 88.40 87.16 86.90 87.39
71.90 69.51 68.68 82.07 79.91 74.57 72.71 75.80
57 1106 2696 123 196 97⋆ 129⋆ 62
6 3370 8308 556 781 9⋆ 9⋆ 50⋆
Conclusion
Ensembles of agents have been assessed for their performance when compared to classical monolithic language models in the context of predicting next activities in log streams. A particularly promising algorithm, the Promotion algorithm, has been identified for obtaining high accuracy at low total prediction and training latency. Future work will be devoted to online learning of such algorithms, with emphasis on (i) automatic calibration of the promotion threshold τ and automatic selection of the n-gram sets used by ensemble methods, (ii) extending the Promotion algorithm with a demotion mechanism to improve adaptability under concept drift, and (iii) improving the promotion procedure to reuse the learned structure and parameters of lower-order n-grams to initialize higher-order models. Acknowledgments. The work was supported by the French National Research Agency (ANR) projects DREAMY (ANR-21-CE48-0003) and COSTXPRESS (ANR-23-CE450013). It was also supported by the SAIF project, funded by the “France 2030” government investment plan managed by ANR, under the reference ANR-23-PEIA-0006. We thank the anonymous reviewers for their thoughtful and thorough feedback. Disclosure of Interests. The authors have no competing interests to declare that are relevant to the content of this article.
References 1. van der Aalst, W.M.P.: Process Mining - Data Science in Action, Second Edition. Springer (2016). https://doi.org/10.1007/978-3-662-49851-4, https://doi.org/10. 1007/978-3-662-49851-4 2. van der Aalst, W.M.P., Schonenberg, M.H., Song, M.: Time prediction based on process mining. Inf. Syst. 36(2), 450–475 (2011). https://doi.org/10.1016/J.IS. 2010.09.001, https://doi.org/10.1016/j.is.2010.09.001 3. Balle, B., Castro, J., Gavaldà, R.: Adaptively learning probabilistic deterministic automata from data streams. Mach. Learn. 96(1-2), 99–127 (2014). https://doi. org/10.1007/S10994-013-5408-X, https://doi.org/10.1007/s10994-013-5408-x
Promoting Simple Agents: Ensemble Methods for Event-Log Prediction
15
4. Baumgartner, R., Verwer, S.: Learning state machines from data streams: A generic strategy and an improved heuristic. In: Coste, F., Ouardi, F., Rabusseau, G. (eds.) International Conference on Grammatical Inference, ICGI 2023, 10-13 July 2023, Rabat, Morocco. Proceedings of Machine Learning Research, vol. 217, pp. 117–141. PMLR (2023), https://proceedings.mlr.press/v217/baumgartner23a.html 5. Bollig, B., Függer, M., Nowak, T., Zeinaty, P.: logicsponge-processmining: A library for process-mining tasks and next activity prediction in business processes., https: //github.com/innatelogic/logicsponge-processmining.git, accessed: 2026-02-13 6. Bollig, B., Katoen, J., Kern, C., Leucker, M., Neider, D., Piegdon, D.R.: libalf: The automata learning framework. In: Touili, T., Cook, B., Jackson, P.B. (eds.) Computer Aided Verification, 22nd International Conference, CAV 2010, Edinburgh, UK, July 15-19, 2010. Proceedings. Lecture Notes in Computer Science, vol. 6174, pp. 360–364. Springer (2010). https://doi.org/10.1007/978-3-642-14295-6_32, https://doi.org/10.1007/978-3-642-14295-6_32 7. Breuker, D., Matzner, M., Delfmann, P., Becker, J.: Comprehensible predictive models for business processes. MIS Q. 40(4), 1009–1034 (2016). https://doi.org/ 10.25300/MISQ/2016/40.4.10, https://doi.org/10.25300/misq/2016/40.4.10 8. Bukhsh, Z.A., Saeed, A., Dijkman, R.M.: Processtransformer: Predictive business process monitoring with transformer network (2021). https://doi.org/10.48550/ arXiv.2104.00721, https://arxiv.org/abs/2104.00721 9. Burattin, A.: Streaming process mining. In: van der Aalst, W.M.P., Carmona, J. (eds.) Process Mining Handbook, Lecture Notes in Business Information Processing, vol. 448, pp. 349–372. Springer (2022). https://doi.org/10.1007/ 978-3-031-08848-3_11, https://doi.org/10.1007/978-3-031-08848-3_11 10. Burattin, A., Sperduti, A., van der Aalst, W.M.P.: Control-flow discovery from event streams. In: Proceedings of the IEEE Congress on Evolutionary Computation, CEC 2014, Beijing, China, July 6-11, 2014. pp. 2420– 2427. IEEE (2014). https://doi.org/10.1109/CEC.2014.6900341, https://doi.org/ 10.1109/CEC.2014.6900341 11. Carrasco, R.C., Oncina, J.: Learning stochastic regular grammars by means of a state merging method. In: International Colloquium on Grammatical Inference. pp. 139–152. Springer (1994) 12. Ceci, M., Lanotte, P.F., Fumarola, F., Cavallo, D.P., Malerba, D.: Completion time and next activity prediction of processes using sequential pattern mining. In: Dzeroski, S., Panov, P., Kocev, D., Todorovski, L. (eds.) Discovery Science - 17th International Conference, DS 2014, Bled, Slovenia, October 8-10, 2014. Proceedings. Lecture Notes in Computer Science, vol. 8777, pp. 49–61. Springer (2014). https://doi.org/10.1007/978-3-319-11812-3_5, https://doi.org/ 10.1007/978-3-319-11812-3_5 13. Dolev, D.: The byzantine generals strike again. Journal of algorithms 3(1), 14–30 (1982) 14. Dolev, D., Fischer, M.J., Fowler, R., Lynch, N.A., Strong, H.R.: An efficient algorithm for byzantine agreement without authentication. Information and Control 52(3), 257–274 (1982) 15. van Dongen, B.: BPI Challenge 2012. https://doi.org/10.4121/uuid: 3926db30-f712-4394-aebc-75976070e91f (2012). https://doi.org/10.4121/UUID: 3926DB30-F712-4394-AEBC-75976070E91F 16. van Dongen, B.: BPI Challenge 2017. https://doi.org/10.4121/uuid: 5f3067df-f10b-45da-b98b-86ae4c7a310b (2017). https://doi.org/10.4121/UUID: 5F3067DF-F10B-45DA-B98B-86AE4C7A310B
16
B. Bollig, M. Függer, T. Nowak, and P. Zeinaty
17. van Dongen, B., Borchert, F.: BPI Challenge 2018. https://doi.org/10. 4121/uuid:3301445f-95e8-4ff0-98a4-901f1f204972 (2018). https://doi.org/10.4121/ UUID:3301445F-95E8-4FF0-98A4-901F1F204972 18. Dupont, P.: Incremental regular inference. In: Miclet, L., de la Higuera, C. (eds.) Grammatical Inference: Learning Syntax from Sentences, 3rd International Colloquium, ICGI-96, Montpellier, France, September 25-27, 1996, Proceedings. Lecture Notes in Computer Science, vol. 1147, pp. 222–237. Springer (1996). https: //doi.org/10.1007/BFB0033357, https://doi.org/10.1007/BFb0033357 19. Elyasi, K.A., van der Aa, H., Stuckenschmidt, H.: Pgtnet: A process graph transformer network for remaining time prediction of business process instances (2024). https://doi.org/10.48550/arXiv.2404.06267, https://arxiv.org/abs/2404.06267 20. de la Higuera, C.: Grammatical Inference: Learning Automata and Grammars. Cambridge University Press, USA (2010) 21. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Computation 9(8), 1735–1780 (1997) 22. Isberner, M., Howar, F., Steffen, B.: The open-source learnlib - A framework for active automata learning. In: Kroening, D., Pasareanu, C.S. (eds.) Computer Aided Verification - 27th International Conference, CAV 2015, San Francisco, CA, USA, July 18-24, 2015, Proceedings, Part I. Lecture Notes in Computer Science, vol. 9206, pp. 487–495. Springer (2015). https://doi.org/10.1007/ 978-3-319-21690-4_32, https://doi.org/10.1007/978-3-319-21690-4_32 23. Krawczyk, B., Cano, A.: Online ensemble learning with abstaining classifiers for drifting and noisy data streams. Appl. Soft Comput. 68, 677–692 (2018). https:// doi.org/10.1016/J.ASOC.2017.12.008, https://doi.org/10.1016/j.asoc.2017.12.008 24. Lamport, L., Shostak, R., Pease, M.: The byzantine generals problem. ACM Trans. Program. Lang. Syst. 4(3), 382–401 (1982) 25. Leucker, M.: Learning meets verification. In: de Boer, F.S., Bonsangue, M.M., Graf, S., de Roever, W.P. (eds.) Formal Methods for Components and Objects, 5th International Symposium, FMCO 2006, Amsterdam, The Netherlands, November 7-10, 2006, Revised Lectures. Lecture Notes in Computer Science, vol. 4709, pp. 127–151. Springer (2006). https://doi.org/10.1007/978-3-540-74792-5_6, https:// doi.org/10.1007/978-3-540-74792-5_6 26. Lischka, A., Rauch, S., Stritzel, O.: Directly follows graphs go predictive process monitoring with graph neural networks (2025). https://doi.org/10.48550/arXiv. 2503.03197, https://arxiv.org/abs/2503.03197 27. Mannhardt, F.: Sepsis Cases - Event Log. https://doi.org/10.4121/uuid: 915d2bfb-7e84-49ad-a286-dc35f063a460 (2016). https://doi.org/10.4121/UUID: 915D2BFB-7E84-49AD-A286-DC35F063A460 28. Mao, H., Chen, Y., Jaeger, M., Nielsen, T.D., Larsen, K.G., Nielsen, B.: Learning deterministic probabilistic automata from a model checking perspective. Mach. Learn. 105(2), 255–299 (2016). https://doi.org/10.1007/S10994-016-5565-9, https: //doi.org/10.1007/s10994-016-5565-9 29. Mayr, F., Yovine, S., Carrasco, M., Pan, F., Vilensky, F.: A congruence-based approach to active automata learning from neural language models. In: Coste, F., Ouardi, F., Rabusseau, G. (eds.) International Conference on Grammatical Inference, ICGI 2023, 10-13 July 2023, Rabat, Morocco. Proceedings of Machine Learning Research, vol. 217, pp. 250–264. PMLR (2023), https://proceedings.mlr. press/v217/mayr23a.html 30. Muskardin, E., Aichernig, B.K., Pill, I., Pferscher, A., Tappler, M.: AALpy: an active automata learning library. Innov. Syst. Softw. Eng. 18(3),
Promoting Simple Agents: Ensemble Methods for Event-Log Prediction
17
417–426 (2022). https://doi.org/10.1007/S11334-022-00449-3, https://doi.org/10. 1007/s11334-022-00449-3 31. Pegoraro, M., Uysal, M.S., Georgi, D.B., van der Aalst, W.M.P.: Text-aware predictive monitoring of business processes. In: Abramowicz, W., Auer, S., Lewanska, E. (eds.) 24th International Conference on Business Information Systems, BIS 2021, Hannover, Germany, June 15-17, 2021. pp. 221–232 (2021). https: //doi.org/10.52825/BIS.V1I.62, https://doi.org/10.52825/bis.v1i.62 32. Polato, M., Sperduti, A., Burattin, A., de Leoni, M.: Time and activity sequence prediction of business process instances. Computing 100(9), 1005–1031 (2018). https://doi.org/10.1007/S00607-018-0593-X, https://doi.org/ 10.1007/s00607-018-0593-x 33. Rama-Maneiro, E., Vidal, J.C., Lama, M.: Embedding graph convolutional networks in recurrent neural networks for predictive monitoring (2021). https://doi. org/10.48550/arXiv.2112.09641, https://arxiv.org/abs/2112.09641 34. Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Hambro, E., Zettlemoyer, L., Cancedda, N., Scialom, T.: Toolformer: Language models can teach themselves to use tools. In: Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., Levine, S. (eds.) Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023 (2023) 35. Schmidt, J., Kramer, S.: Online induction of probabilistic real-time automata. J. Comput. Sci. Technol. 29(3), 345–360 (2014). https://doi.org/10.1007/ S11390-014-1435-8, https://doi.org/10.1007/s11390-014-1435-8 36. Shannon, C.E.: A mathematical theory of communication. The Bell system technical journal 27(3), 379–423 (1948) 37. Steeman, W.: BPI Challenge 2013, incidents. https://doi.org/10.4121/uuid: 500573e6-accc-4b0c-9576-aa5468b10cee (2013). https://doi.org/10.4121/UUID: 500573E6-ACCC-4B0C-9576-AA5468B10CEE 38. Vaandrager, F.W.: Model learning. Commun. ACM 60(2), 86–95 (2017). https: //doi.org/10.1145/2967606, https://doi.org/10.1145/2967606 39. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. p. 6000–6010. NIPS’17, Curran Associates Inc., Red Hook, NY, USA (2017) 40. Verwer, S., Hammerschmidt, C.A.: flexfringe: A passive automaton learning package. In: 2017 IEEE International Conference on Software Maintenance and Evolution, ICSME 2017, Shanghai, China, September 17-22, 2017. pp. 638–642. IEEE Computer Society (2017). https://doi.org/10.1109/ICSME.2017.58, https: //doi.org/10.1109/ICSME.2017.58 41. Wang, F., Damiani, E.: Time-aware and transition-semantic graph neural networks for interpretable predictive business process monitoring (2025). https://doi.org/10. 48550/arXiv.2508.09527, https://arxiv.org/abs/2508.09527 42. Weiss, G., Goldberg, Y., Yahav, E.: On the practical computational power of finite precision rnns for language recognition. In: Gurevych, I., Miyao, Y. (eds.) Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 2: Short Papers. pp. 740–745. Association for Computational Linguistics (2018) 43. Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K.R., Cao, Y.: ReAct: Synergizing reasoning and acting in language models. In: The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net (2023)
18
B. Bollig, M. Függer, T. Nowak, and P. Zeinaty
44. van Zelst, S.J., van Dongen, B.F., van der Aalst, W.M.P.: Event stream-based process discovery using abstract representations. Knowl. Inf. Syst. 54(2), 407– 435 (2018). https://doi.org/10.1007/S10115-017-1060-2, https://doi.org/10.1007/ s10115-017-1060-2 45. Zhou, Z.H.: Ensemble Methods: Foundations and Algorithms. Chapman & Hall/CRC, 1st edn. (2012)