Schema-Agnostic Process Trace Construction: From Raw Tables to Execution Behavior Zhi Quan Joel Lim1[0009−0000−3756−7439] , Kar Way Tan1[0000−0002−2707−6588] , and Hoong Chuin Lau1[0000−0002−5326−411X]
arXiv:2606.14775v1 [cs.DB] 10 Jun 2026
1
School of Computing and Information Systems, Singapore Management University, Singapore {joellim.2022, kwtan, hclau}@smu.edu.sg
Abstract. Traditional information systems (IS) engineering assumes stable schemas, explicit keys, and curated event logs. In modern OLTP environments, schemas drift, keys are sparse, and execution traces are dispersed across loosely connected tables, making manual process trace construction costly and error prone. We propose a schema-agnostic pipeline that automatically reconstructs process execution traces directly from raw relational data. The pipeline (i) identifies columns that function like keys or timestamps, (ii) discovers table-to-table connections using statistical signals rather than predefined schemas, (iii) assembles and orders events for each case while accommodating multiple date fields, and (iv) learns likely ordering and flow relations across systems using a Temporal Convolutional Network which models long-range dependencies and patterns. Evaluations on TPC-H/E benchmarks, synthetic corpora, and a real industry dataset show that our pipeline reconstructs highfidelity event traces and accurate trace orderings, correctly predicting the next event with 85% accuracy and recovering about 82% of ground-truth precedence relations. By eliminating dependence on predefined schemas, ER diagrams and domain templates, our work offers a generalizable and scalable pathway for automated reconstruction of execution behaviour in dynamic and continuously evolving IS environments. 1 Keywords: Process Mining · Event Log Generation · Temporal Convolutional Network · Financial Information Systems · Data Profiling
1
Introduction
Modern information systems (IS) increasingly rely on complex, rapidly evolving OLTP infrastructures where business activities are executed across loosely coupled relational tables. In these environments, however, schemas drift over time, surrogate keys are introduced inconsistently and are refactored into lookup tables or omitted, and execution traces are scattered across loosely coupled systems. As a result, reconstructing system behaviour requires manual joins over key-sparse tables and ad hoc timestamp rules, which is costly to build and fragile to maintain. This bottleneck has motivated the search for scalable, domain-agnostic solutions that can automatically infer execution behaviour or operational traces
2
Lim et al.
from raw OLTP data by identifying inter-table relationships and data flows without relying on predefined schemas or domain expertise [10,24,9]. Recovering these execution traces is crucial for a broad range of IS tasks including auditing, compliance monitoring, system modernization, and the generation of event-centric data for downstream applications such as process mining. However, these possibilities are hindered by the substantial effort required to assemble high-quality execution sequences at scale. Evidence of process execution is dispersed across systems. Suitable case identifiers are difficult to determine and multiple timestamp fields often coexist within and across tables, often with ambiguous semantics. As organizations introduce new applications and integrations, the effort required to maintain execution-trace reconstruction pipelines grows disproportionately, underscoring the need for scalable and schema-agnostic approaches. Problem Statement: Given a collection of heterogeneous, key-sparse relational tables from evolving OLTP systems, construct high-fidelity execution sequences, including cases, events, and their temporal ordering, but without relying on predefined schemas, reliable foreign keys, or domain-specific templates. Existing approaches generally rely on explicitly declared and reliable PK/FK structures, well-defined timestamp semantics, or domain knowledge. Such assumptions rarely hold, limiting applicability in continuously changing environments. In this paper, we propose a schema-agnostic pipeline that reconstructs process events directly from raw relational data. Our approach is designed for evolving, multi-system landscapes, as it relies on the data itself rather than predefined schemas, tolerates sparsity and noise, and learns likely precedence and flow relations information when timestamps are ambiguous, partially missing or insufficient to determine the process execution order. Concretely, the pipeline (see Figure 1): (i) identifies columns that function as case identifiers or timestamps, (ii) discovers relationships between tables using statistical indicators rather than explicit foreign keys, (iii) assembles and orders events for each case while accommodating multiple timestamp fields, and (iv) learns likely precedence and flow relations across systems using a Temporal Convolutional Network (TCN). The resulting traces can be consumed directly by standard process-mining tools and can be lifted to object-centric representations when object relations are available, ensuring compatibility without relying on domain-specific templates. Through this work, we address the following research questions (RQ): 1. RQ1 (Event Recovery): How accurately can our pipeline recover the correct set of execution events from heterogeneous, key-sparse relational databases? 2. RQ2 (Precedence / Flow Learning): Can a TCN learn cross-table precedence relations more accurately than rule based approaches? 3. RQ3 (Change Resilience and Scalability): How resilient is our approach to noise, missing data, timestamp-semantics variation, and data-volume growth that characterize continuously changing information environments? In our approach, we learn cross table precedence relations without relying on fixed schemas while maintaining performance under drift. Across synthetic, bench-
Schema-Agnostic Process Trace Construction
3
Fig. 1: Overview of our end-to-end methodology for reconstructing event flows from heterogeneous, schema-poor databases.
mark, and industry datasets, the approach consistently recovers the large majority of events that belong to each process (typically over 90%), reconstructs event sequences with high fidelity, and captures cross-table flow relations with accuracy exceeding 80%. These results demonstrate the robustness and generalizability of the proposed approach. Our contribution is both methodological and operational. Methodologically, we avoid the traditional requirement for complete schemas, explicit primary and foreign keys, or ontological mappings before log construction. Instead, we infer only the high confidence links needed to assemble reliable traces, and we use learned temporal patterns to order events when timestamps are insufficient. Operationally, we provide an end-to-end pathway from raw and evolving relational data to provenance-rich event logs that remain robust to missing timestamps, homonyms, and partial keys, and that can be consumed directly by existing process mining tools. By replacing schema-first assumptions with data-driven similarity and learned precedence, the approach enables automated and reproducible log construction in environments where structures and data meaning change continuously.
2
Related Work
A recent systematic review of 260 process-mining studies highlights that most effort remains focused on pre-analysis and data acquisition, while scalable endto-end log construction from heterogeneous databases is still underexplored [21]. This motivates automated approaches for key-sparse, legacy warehouses of the type addressed in this work. Artifact-centric methods reconstruct primary/foreign-key structure before mapping artifacts to logs [20]. Automated variants [18] infer column types, keys, foreign keys, and timestamp attributes, then cluster tables into artifact schemas.
4
Lim et al.
These techniques assume that PK/FK-like patterns and a suitable timestamp can be reliably recovered. When schemas are clean and stable, [18] is appropriate, otherwise, schema-agnostic pipelines that tolerate key sparsity and multiple candidate timestamp remain necessary. Extensions mine cross-artifact interactions via graph searches [15], while semi-automated or constraint-based extractors additionally rely on domain inputs, predefined keys, or ontology-guided mappings or process documents [2,7,22]. Hierarchy-discovery methods require existing logs and deterministic key mining [8], and meta-model approaches depend on explicit inter-table dependencies [17]. Legacy warehouses, however, often lack declared keys or exhibit only approximate inclusions [11], making these approaches difficult to operationalize. Specialized pipelines exist for platforms such as SAP-centric OCEL extraction [5], but they do not generalize to heterogeneous environments. Our focus instead is on constructing log-ready linkages directly from raw tables without full schema reconstruction or domain knowledge. Schema profiling research provides strong techniques for discovering UCC, INDs and PK/FK candidates [13], as well as surveys of profiling operators relevant for large-scale relationship inference [1] and pipelines for schema normalization [19]. We draw on these insights but restrict our objective to log-ready linkage discovery: we compute JS/KS-based similarity on sampled data, retain only highconfidence edges, and directly assemble event sequences rather than performing complete schema recovery. Unlike profiling-only or template-based extractors, our method is designed to operate under key sparsity, multiple timestamp fields, and noise. Some techniques assume complete event logs and model uncertainty on top [23], while others infer sequences from raw databases using predefined causal process templates [26]. Synthetic-log frameworks and redo-log mining [6,4] are useful when process models or vendor traces exist, but rely on templates or proprietary infrastructure often unavailable in finance. Complementary work on process-data quality highlights issues such as timestamp gaps, inconsistent labels, and incomplete keys [12], which our pipeline mitigates through column profiling, similarity-based filtering, and learned ordering. Predictive monitoring over existing logs typically employs recurrent networks [25]. Temporal Convolutional Networks (TCNs) provide long receptive fields and causal convolutions suited for irregular enterprise sequences [3]. We instead use TCNs to learn ordering itself across events reconstructed from raw relational data—distinct from work that assumes a ready-made log. In summary, while methodologies such as OCPM2 offer a valuable methodological blueprint for OCED extraction[16], they and prior work still presuppose substantial manual configuration by either reconstructing schemas before logging [18,20], assume reliable keys or existing logs [2,8,17,23], rely on templates or vendor traces [26,6,4], or focus on profiling without end-to-end log generation [13,1,19]. In contrast, we contribute a schema-agnostic pipeline that infers highconfidence inter-table links, selects timestamp fields automatically, and learns cross-system ordering with TCNs, addressing the gap identified in [21] and the
Schema-Agnostic Process Trace Construction
5
quality challenges noted in [12], while complementing platform-specific extractors [5].
3
Methodology
We propose a schema-agnostic pipeline that infers inter table links from statistical signals, assembles temporal fields into coherent event sequences, and uses a causal TCN to learn ordering, as shown in Figure 1. The pipeline consists of four stages that correspond to the subsections below: – S1 Profiling Identifier and Timestamp Columns: Detects identifierlike and temporal attributes through column profiling. – S2 Relationship Discovery: Discovers inter-table links using type-aware statistical similarity measures. – S3 Sequencing: Assembles timestamped rows into ordered event sequences by propagating sampled identifiers. – S4 Sequence Learning: Learns context-dependent precedence relations using a TCN. Stages S1–S3 produce the event traces while S4 summarizes these traces into precedence and flow graphs without modifying their event order. Although several components of our pipeline (e.g., identifier scoring, timestamp ranking, similarity thresholds) use lightweight heuristics, these choices are principled rather than ad hoc. They operate on minimal statistical signals such as distinctness, completeness, stability, and value-overlap, which are the few reliable indicators in schema-poor, key-sparse environments. The TCN learns precedence and flow patterns directly from reconstructed traces, capturing cross-table ordering even when timestamps conflict or exhibit drift. This overall design enables the construction of provenance-rich traces that can be consumed directly by process mining tools and lifted to object-centric representations.
3.1
Profiling Identifier and Timestamp Columns
We first profile each column c in table Ti to estimate its likelihood of acting as an identifier and its suitability as a timestamp (Figure 2). Identifier likelihood combines three simple statistics: spk (c) = u(c) γ(c) λ(c), where u(c) is distinctness γ(c) is completeness and λ(c) measures string-length stability. Columns with high spk are retained as candidate case identifiers. We require all three properties to be high simultaneously as multiplicative combination penalizes columns that are strong in only one dimension. sdt (c) = wγ πγ (c) + wδ πδ (c) + wsd πsd (c) + wIQR πIQR (c) + wH πH (c),
6
Lim et al.
where π∗ denote the per-table percentile ranks of the corresponding dispersion and variation metrics (completeness γ, distinctness δ, standard deviation, interquartile range, and entropy)1 . Per table, we retain the top-k identifier and timestamp columns. These candidate identifiers and timestamp columns form the basis for subsequent link discovery. We detail this in Algorithm 1.
Fig. 2: Profiling Identifier and Timestamp Columns: Computes identifier and datetime likelihood per column.
3.2
Relationship Discovery (Column–Column Links)
We infer table connections by scoring every cross-table column pair using a type-aware similarity measure designed to remain reliable under key sparsity (Figure 3). For any pair consisting of column ci from table Ti and column ck from table Tk , the score is s(ci , ck ) = T C(ci , ck ) · Q(ci , ck ) · U (ci , ck ) · R(ci , ck ), where each factor contributes a different aspect of link quality: – Type compatibility T C: downweights pairs whose data types are weakly compatible (e.g., string vs. numeric, date vs. free text). – Column quality Q: penalizes low-quality columns using completeness and distinctness (Q = min{γ(ci )u(ci ), γ(ck )u(ck )}). – Coverage U : ensures the overlap is not driven by rare or fringe values. – Value similarity R: measures actual value alignment. We use Jaccard Similarity (JS) for symbolic columns and a JS/Kolmogorov-Smirnov (KS) blend for numeric distributions. We retain all column pairs with s(ci , ck ) ≥ τS (optionally keeping only the top-K per table) and form an undirected multigraph of candidate inter-table links. We denote the resulting set of retained column–column links as L. Algorithm 2 provides full details. We also report our thresholds used in Table A.1. 1
In all experiments, we assign equal weights (wγ , wδ , wsd , wIQR , wH ) to the five percentile-based dispersion metrics. This avoids dataset-specific tuning and keeps the profiling stage parameter-light. In practice, we observed that moderate variations in these weights did not materially change the ranking of timestamp candidates.
Schema-Agnostic Process Trace Construction
7
Fig. 3: Illustration of column–column link discovery. Edges arise from highoverlap column pairs (approximate inclusion dependencies). Dashed edges indicate weaker type compatibility. Only pairs with final score s ≥ τS are retained.
Because exact INDs often fail under key sparsity and partial overlaps, we rely on type-aware value similarity (Jaccard, JS/KS) to detect approximate table relationships that remain stable even when declared keys are missing or inconsistently populated.
3.3
Temporalization and Sequencing
Using the retained link set L, we propagate sampled case identifiers across all reachable tables and collect their corresponding rows and non-null timestamp candidates (Figure 4). Each (row, timestamp-column) pair is treated as a provisional event. Events for a case are sorted by timestamp to obtain a total order; ties are broken deterministically using table name and row identifier. Adjacent duplicate labels are collapsed2 , while non-adjacent repetitions are preserved as they typically reflect genuine alternations. When timestamps coincide exactly, we randomize the within-timestamp order and average downstream statistics over five seeds. For efficiency, we subsample up to 100,000 identifiers per dataset (about 10% in our experiments). Optional context features. When business attributes (e.g., region, product, channel) are available, we embed them into a context vector cs (dimension d = 16) and concatenate any normalized numeric attributes. This vector is prepended to the event window passed to the TCN in Stage 4, conditioning precedence learning without changing the underlying event order. Stage 3 outputs provenance-rich event traces in JSON format, exportable to XES or liftable to OCEL via additional identifier fields. Full pseudocode is provided in Algorithm 3. This provides a lightweight conditioning signal for the TCN in §3.4. 2
(e.g., A–A–B–C–C–C–D → A–B–C–D)
8
Lim et al.
Fig. 4: From Stage 2 links (left) we propagate sampled primary keys to collect timestamped rows, assemble per-key event traces (center), and emit case traces with optional context (right)
3.4
Sequence learning (ordering across tables)
In Stage 4, we learn cross-table precedence relations using a causal TCN that predicts the next event given a window of past events and an optional context vector (Figure 5). Importantly, Stage 4 preserves the event sequences constructed in Stages S1-S3 and learns precedence relations on top of them. The introduction of context-conditioned transition modelling sharpens precedence predictions. For a trace e1:T , where et ∈ V is the t-th event, the TCN receives the window et−w+1:t = (et−w+1 , . . . , et ), together with cs , and estimates pΘ (et+1 | cs , et−w+1:t ). Aggregating next-event predictions. For any context set C, we form a transition matrix by averaging next-event probabilities over all prefixes whose last token is i: Pij (C) =
1 |Pi (C)|
X
pΘ (et+1 = j | cs , et−w+1:t ),
(cs ,et−w+1:t )∈Pi (C)
interpreting Pij (C) as the probability that j follows i under context C.
Extracting precedence and variability. We add a directed edge i → j when Pij − Pji ≥ τθ . If |Pij − Pji | < τc and max(Pij , Pji ) ≥ τmin , we mark the pair as exhibiting bidirectional order variability (BOV), i.e., both orders appear with similar frequency. Pairs that satisfy neither condition receive no precedence relation. Because our event logs contain only a single timestamp per event and no resource-level concurrency information, we report BOV-F1 rather than strict concurrency measures. A final transitive reduction yields a minimal precedence graph. Algorithm 4 provides full details.
Schema-Agnostic Process Trace Construction
9
Fig. 5: TCN-based next-event prediction and precedence extraction.
4
Evaluation
4.1
Evaluation Framework
For RQ1 we measure event recovery via position-independent token accuracy (PITA) and per-case event-set precision/recall/F1; for RQ2 we assess ordering fidelity with edit distance, LCSr, and Kendall’s τb , and precedence quality with edge precision/recall/F1 and BOV-F1. These are described in Table A.2. For comparing sequence models we additionally report standard predictive metrics (perplexity, Top-k, MRR, Brier). RQ3 uses the same metrics under controlled perturbations
4.2
Data
Publicly available datasets rarely preserve the relational and temporal structure needed for database-centric process mining [26,17,14]. We therefore evaluate on five corpora spanning benchmarks, controlled synthetics, and industry data. Namely, TPC-H, TPC-E, Synth-Light, Synth-Heavy, and Industry-1 (Table A.3). Ethics and Threats to Validity. No human subjects or identifiable data are involved, and industry data are fully de-identified. Internal validity risks (e.g., leakage or seed sensitivity) are mitigated through temporal splits, fixed seeds, and early stopping. Construct validity concerns from imperfect ground truth are addressed via released mapping files, sensitivity checks, and multiple metrics. External validity is supported by evaluation across heterogeneous benchmarks, synthetics with induced drift, and one industry dataset. Statistical-conclusion validity is ensured via non-parametric tests with Holm correction and effect-size reporting.
10
Lim et al.
Ground-truth logs and precedence. For all datasets, the data generator provides both databases and reference logs/precedence relations3 . TPC-H/E data were generated using standard TPC tools. Table-level precedence is derived from the business narratives and schema documentation in the official specifications.4 For Industry-1, we derive an initial precedence graph from system specifications and audit trails, validated independently by two domain experts. 4.3
Baselines
Deterministic Pipeline – Minimal Viable Proxy (DP-MVP). We define a minimal deterministic surrogate to provide a lower bound for the add-in analysis. It (i) identifies keys and timestamps via simple name heuristics (e.g., _ID, _NO, _DATE), (ii) links tables through name matches or high Jaccard overlap, (iii) orders events using a single global timestamp or table order, and (iv) derives precedence by the most frequent successor. Deterministic Pipeline: S1–S3 (DP) DP applies Stages S1–S3 exactly as defined but omits the learned precedence module (S4). It reconstructs traces deterministically and isolates the added contribution of learned ordering. Sequence-learning baselines for S4. To evaluate Stage S4, we compare the TCN against a 3-gram language model and an LSTM trained on the same traces and tokenization. This isolates whether gains arise from the TCN architecture versus learned sequence modeling in general. Our ablation details are given in Table A.5. Schema-centric methods. Schema-centric techniques [20,18] assume rediscoverable PK/FK constraints and a single timestamp attribute, or focus on key discovery rather than log or precedence construction. These assumptions do not hold in key-sparse, multi-timestamp datasets, and substantial manual configuration would be required. We therefore treat these as complementary methods rather than numerical baselines. 4.4
Experimental Setup and Statistical Analysis
All experiments use temporal hold-out splits, five random seeds, and shared tokenization. TCN hyperparameters are provided in Table A.4. Optimizers, batch sizes, and hardware configurations (GPUs/CPUs, peak memory) are summarized in the Appendix. For statistical testing, per-case metrics are averaged per dataset and compared against the deterministic baseline using the Wilcoxon signed-rank test Repository available at git clone https://<ACCESS_TOKEN>@github.com/ joellim2022/schema-agnostic-process-trace-construction.git 4 https://www.tpc.org/tpc_documents_current_versions/current_ specifications5.asp 3
Schema-Agnostic Process Trace Construction
11
Table 1: Unified reconstruction and precedence-graph accuracy metrics across all datasets. Dataset
PITA ED LCSr τb Prec Rec F1 BOV-F1 τΘ
TPC-H TPC-E Synth-Light Synth-Heavy Industry-1
1.00 0.88 0.95 0.82 0.84
0 1.00 1.00 0.81 0.78 0.79 27.4 0.82 0.75 0.84 0.80 0.82 18.9 0.85 0.85 0.79 0.76 0.77 31.1 0.72 0.63 0.75 0.73 0.74 26.7 0.78 0.71 0.82 0.79 0.81
0.00 0.77 0.72 0.69 0.75
τc
0.58 0.12 0.60 0.14 0.53 0.11 0.47 0.08 0.56 0.13
with Holm correction. We report non-parametric effect sizes (Cliff’s d) and 95% BCa bootstrap confidence intervals from 10,000 resamples. For reproducibility, we release scripts to regenerate datasets and results. Public and synthetic datasets are archived with DOIs, while industry results are reported only in aggregate form. For conformance experiments (§5.3), we discover models using Inductive Miner–infrequent (γ = 0.2) and compute fitness/precision via ETConformance alignments. 4.5
Analytical Design
End-to-end comparison. We compare the deterministic pipeline (S1–S3) and the full pipeline (S1–S4) against baselines on event recovery (RQ1), ordering fidelity, and precedence accuracy (RQ2). For sequence learning, the TCN is contrasted with n-gram and LSTM baselines using the predictive metrics from RQ2. To isolate each stage’s contribution, we use (i) an add-in ladder that starts from DP-MVP and re-enables S1–S4 sequentially, and (ii) a knock-out analysis that replaces one stage at a time with its surrogate. Importantly, S4 does not modify the event traces produced by S1–S3. Instead, S4 affects only the precedence-learning stage: the deterministic variants (DP-MVP and DP) aggregate successor frequencies across all contexts, whereas the TCN learns context-conditioned transition patterns (e.g., Equity vs Futures), resulting in sharper and more accurate precedence relations. Any observed differences in reconstruction-level metrics between DP and the full pipeline arise from aggregation over context partitions rather than changes to the underlying reconstructed traces. These experimental configurations are listed in Table A.5 (Appendix).
5
Results
We organize the results around the research questions RQ1–RQ3 (§4.1), covering: (i) end-to-end reconstruction quality of the extracted traces (RQ1), (ii) the contribution of each pipeline stage to this quality and to precedence learning (RQ1, RQ2), and (iii) robustness to schema/timestamp drift and noise or missing data (RQ3).
12
Lim et al.
5.1
RQ1 — Event Recovery (Presence and Accuracy)
Across all corpora, Stages S1-S3 achieve high event-set recovery, with PITA ranging from 0.82 (Synth-Heavy) to 1.00 (TPC-H) and strong ordering fidelity (ED, LCSr, τb ; Table 1). Accuracy declines on TPC-E and Synth-Heavy, which exhibit the greatest key sparsity and timestamp ambiguity, but remains sufficient to provide reliable input for learned precedence in Stage S4. The drop from PITA 0.95 (Synth-Light) to 0.82 (Synth-Heavy) mirrors the increase in key sparsity and overlapping timestamp semantics in the generator; in practice this means that under severe drift our pipeline still recovers most events but loses some intra-table granularity.
5.2
RQ1/RQ2 — Stage Attribution (What Each Stage Adds)
Across all datasets, the add-in ladder (Table 2) shows monotonic gains in reconstruction and precedence metrics as S1-S4 are enabled. Knockout results (Table 3) indicate that S2 contributes most to trace completeness and ordering, while replacing S4 with an n-gram model leaves reconstruction unchanged but reduces precedence F1 by 0.11. The FK-only surrogate confirms that schema-level keys alone are insufficient in key-sparse, multi-timestamp environments. Overall, S1-S3 drive reconstruction quality, and S4 provides the learned cross-table ordering. Table 2: Averaged Stepwise add-in across all Dataset Variant Deterministic Pipeline (MVP): + S1 Profiling + S2 Links + S3 Temporalization + S4 TCN (Full)
5.3
PITA ↑
LCSr ↑
Kendall τb ↑
Precedence F1 ↑
— 0.52 0.65 0.72 0.84
— 0.46 0.61 0.69 0.78
— 0.36 0.51 0.59 0.71
— 0.66 0.73 0.78 0.81
RQ2 — Learning to Order Across Systems
The TCN outperforms n-gram and LSTM baselines on next-event prediction and uncertainty metrics (Table 4), and these predictive gains translate into higher conformance under identical mining and alignment settings. Precedence precision/recall/F1 (Table 1) remain in the 0.74–0.82 range with concurrency-F1 between 0.69–0.77. Thus, learning precedence with the TCN improves both predictive quality and the fidelity of recovered precedence and concurrency relations.Although LSTMs perform competitively in our setting, we adopt TCNs because their convolutional structure is well known to handle long-range temporal patterns effectively, which is important for cross-table event sequencing.
Schema-Agnostic Process Trace Construction
13
Table 3: Knockout from Full on Industry-1 (surrogate in parentheses). Variant
∆PITA ∆LCSr ∆Precedence F1 Interpretation
Full – S1 (→ S1-Weak)
— -0.09
— -0.07
— -0.05
– S2 (→ S2-FK-Only)
-0.12
-0.11
-0.08
– S3 (→ S3-STGS)
-0.06
-0.09
-0.07
– S4 (→ n-Gram)
-0.03
-0.06
-0.11
— More spurious links → extra/noisy events Fewer true cross-table joins (keysparsity hurts) Single-time sorting loses intra-table event granularity Loses long-range/contextual precedence
Table 4: Next-event quality & conformance (mean over 5 seeds, Synth-Heavy with 30% Noise). Model n-Gram (k=3) LSTM TCN (Full)
5.4
Perplx. ↓
Top-1 ↑
Top-3 ↑
MRR ↑
Brier ↓
Fit. ↑
Prec. ↑
16000 1.874 1.588
0.08 0.771 0.861
0.149 0.93 0.970
0.258 0.883 0.924
0.118 0.019 0.019
0.63 0.780 0.788
0.07 0.73 0.862
RQ3 — Change Resilience and Robustness
Under controlled drift on Synth-Heavy (Table 5), the full pipeline retains over 80% of its clean-data PITA and LCSr even under substantial attribute swaps and timestamp missingness, whereas the deterministic baseline drops below 50%. All corpora are processed within our hardware budget. These results show that the pipeline remains viable for systems where schemas and designs evolve continuously. Drift scenario (Synth-Heavy)
Full pipeline
DP baseline
PITA ↑ LCSr ↑ PITA ↑ LCSr ↑ 0% attr swap, 0% TS missing 10% attr swap, 5% TS missing 30% attr swap, 15% TS missing
0.82 0.77 0.68
0.72 0.68 0.65
0.70 0.52 0.34
0.55 0.41 0.27
Rel. to clean (worst drift)
0.83
0.83
0.49
0.49
Table 5: Robustness to synthetic schema/timestamp drift on Synth-Heavy. Each row applies controlled attribute renaming and timestamp missingness. Under the highest drift level, the full pipeline retains about 83% of clean-data PITA and LCSr, while the deterministic DP baseline retains below 50%.
6
Conclusion and Future Works
We introduced a schema-agnostic framework for automatically constructing event logs directly from raw relational data. Our results challenge the long-standing assumption in IS engineering that reliable event logs presuppose clean schemas and curated keys. By treating relational contents as the primary object, and learning linkages and precedence from data rather than from ERDs, it is possible to construct high-quality logs even as schemas drift and systems evolve. This suggests a shift from schema-first to data-first log engineering in continuously changing IS
14
Lim et al.
environments. More broadly, it points to an IS engineering paradigm where operational schemas are allowed to evolve freely, while a data-driven log-engineering layer preserves continuity for analysis, auditing, and redesign. Across benchmark, synthetic, and industry datasets, our pipeline consistently reconstructs event sets, produces high-fidelity orderings, and learns accurate cross-table flow relations in key-sparse, multi-timestamp environments. The add-in and knockout analyses further confirm that each stage contributes meaningfully to overall performance, validating our pipeline’s modular design. Limitations and future work Our approach assumes a basic level of data integrity; severe timestamp sparsity, homonymous attributes, or highly parallel workflows can make reconstruction ambiguous. The scarcity of public datasets with groundtruth relational processes also limits cross-study comparability. Identifier profiling currently considers columns in isolation and thus misses composite keys, although this is consistent with our schema-agnostic focus on quasi-identifiers rather than exact PK/FK recovery. Future work includes extending profiling to unique column combinations, stress-testing the pipeline under stronger noise and drift, automatically detecting contextual attributes (e.g., via clustering or LLMbased annotation), and refining sequence learning to better capture parallel and cyclic behavior.
Appendix Algorithm 1: Profiling for all Ti ∈ T do for all c ∈ Ti do Compute distinctness u(c), completeness γ(c), and length stability λ(c) Compute identifier score: spk (c) ← u(c) γ(c) λ(c) Compute basic timestamp feasibility: is_parsable_datetime(c) distinct_time_ratio(c) passes_timestamp_validity_checks(c) if passes_timestamp_validity_checks(c) then Compute temporal dispersion (sd, IQR, H) end if Compute timestamp score: sdt (c) end for PK(Ti ) ← TopK(spk ) DT(Ti ) ← TopK(sdt ) end for return {PK(Ti ), DT(Ti )} Algorithm 2: DiscoverLinks L ← ∅ for all (ci , ck ) from different tables do if T C[type(ci ), type(ck )] = 0 then continue end if Q ← min{γ(ci )u(ci ), γ(ck )u(ck )} if Vi ∩ Vk = ∅ then continue end if U ← min |Vi ∩ Vk |/|Vi |, |Vi ∩ Vk |/|Vk | if ci , ck numeric then R ← JS/KS blend else R ← Jaccard(Vi , Vk ) end if s(ci , ck ) ← T QU R if s(ci , ck ) ≥ τS then add (ci , ck ) to L end if end for return L
Algorithm 3: AssembleSequences for all s in sample do collect timestamped events across link set R εs ← SortByTime(events) serialize εs to JSON end for return {εs } Algorithm 4: TrainAndInferTCN Require: Sequences {εc s }; window w; model fΘ ; thresholds τθ , τc , τmin ; vocabulary V incl. ⟨eos⟩ Ensure: transition matrix P(C); precedence δ(C); concurrency flags tokenize events (table::timefield) and contexts append ⟨eos⟩ to each trace build training pairs ([cs ; et−w+1:t ], et+1 ) with prepadding to length w train fΘ using cross-entropy over causal dilated convolutions for all C do initialize Cij (C) ← 0, Ni (C) ← 0 for all i, j ∈ V for all prefixes in C with last token i do p̂ ← fΘ ([cs ; e1:t ]) Ci· (C) ← Ci· (C) + p̂ Ni (C) ← Ni (C) + 1 end for for all i, j ∈ V do Cij (C) Pij (C) ← max(1, Ni (C)) end for for all i, j ∈ V do δij (C) ← signτ (Pij (C) − Pji (C)) θ end for for all i, j ∈ V do if |Pij − Pji | < τc and max(Pij , Pji ) ≥ τmin then flag (i, j) as concurrent end if end for end for return {P(C), δ(C)}
Schema-Agnostic Process Trace Construction Table A.1: Hyperparameters.
Table A.4: TCN configuration used in all experiments.
Symbol Description
Value
Component
τS τparse τδ kpk kts B α τcov τmin τθ τc
.20 1 .02 3 2 100 .5 .05 .05 .10 .10
Input window length w = 16 events Label / ctx dims 16 / 16 Blocks 3 residual Kernel size 3 Dilations (1, 2, 3) Hidden channels 64 Norm / activation LayerNorm / ReLU Dropout 0.1 Loss / optimizer CE / Adam (1e-3) Batch size 64 Early stopping 10 epochs
Similarity cutoff (S2) Timestamp parse success rate (S1) Distinct-time ratio threshold (S1) Top-k identifier-like columns Top-k timestamp-like columns Histogram bins for JS/KS JS/KS mixture weight Minimum row-coverage for column links Minimum support for precedence edges Precedence asymmetry cutoff (Pij − Pji ) Symmetry window for BOV classification
Table A.2: Similarity and precedence metrics. Metric
Summary
F1-like event multiset similarity where P IT A(A, B) = 2|A ∩ B|/(|A| + |B|) LCSr Normalized longest common subsequence LCSr(A, B) = LCS(A, B)/max(|A|, |B|) Prec. F1 Strict x ≺ y order where Prec.F1 = F1 over pairs (x, y) where x < y in ground truth BOV. F1 F1 over bidirectional precedence: x ≺ y and y ≺ x
Setting
Table A.5: Stage surrogates used in knockouts and DPMVP. Stage
PITA
15
Surrogate
S1 Profiling S1-Weak: noisy profiling S2 Link discovery S2-FK-Only: declared/name FK S3 Temporalization S3-STGS: single-TS global sort S4 Learning S4-n-Gram: k = 3 model
Supplementary Materials
Table A.3: Dataset characteristics. ‘Tbl’ is the number of tables; ‘TS’ is the number of tables containing timestamp-like attributes; ‘Evts’ reports the median (50th percentile) and 90th-percentile number of events The full source code and experimental scripts supportreconstructed per case. ing this paper are provided through two access options: Preferred: Dataset Tbl Rows TS Cases Evts (50/90) TPC-H TPC-E Synth-L Synth-H Ind-1
8 866K 2 32 >1.5M 8 20 >100K 20 50 >600K 46 27 >500K 25
12k 18k 10k 25k 8k
2 / 2 5 / 8 8 / 12 18 / 40 15 / 28
git clone https://github_pat_11B2UDGFQ0rMnEo12iSLrI_kbwy6WbYZm [email protected]/j oellim2022/schema-agnostic-process-trace-construction.git
Public Link: https://github.com/joellim2022/schemaagnostic-process-trace-construction
This public mirror will remain accessible for 30 days following the submission deadline.
References 1. Z. Abedjan, L. Golab, and F. Naumann. Profiling relational data: a survey. The VLDB Journal, 24:557–581, 2015. 2. R. Andrews, C.G.J. van Dun, M.T. Wynn, W. Kratsch, M.K.E. Röglinger, and A.H.M. ter Hofstede. Quality-informed semi-automated event log generation for process mining. Decision Support Systems, 132:113265, 2020. 3. Shaojie Bai, J. Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. CoRR, abs/1803.01271, 2018. 4. D. Bano, T. Lichtenstein, F. Klessascheck, and M. Weske. Database-less extraction of event logs from redo logs. Business Information Systems, pages 73–82, 2021. 5. A. Berti, G. Park, M. Rafiei, and W. M. P. v. d. Aalst. A generic approach to extract object-centric event data from databases supporting sap erp. Journal of Intelligent Information Systems, 61:835–857, 2023. 6. A. Burattin, B. Re, L. Rossi, and F. Tiezzi. A purpose-guided log generation framework. Lecture Notes in Computer Science, pages 181–198, 2022. 7. Diego Calvanese, Marco Montali, Alifah Syamsiyah, and Wil MP Van Der Aalst. Ontology-driven extraction of event logs from relational databases. In Business Process Management Workshops: BPM 2015, 13th International Workshops, Innsbruck, Austria, August 31–September 3, 2015, Revised Papers 13, pages 140–153. Springer, 2016.
16
Lim et al.
8. R. Conforti, M. Dumas, L. García-Bañuelos, and M. L. Rosa. Beyond tasks and gateways: discovering bpmn models with subprocesses, boundary events and activity markers. Lecture Notes in Computer Science, pages 101–117, 2014. 9. S. R. Hawkins, J. S. Pickerd, S. L. Summers, and D. A. Wood. The development of the process mining event log generator (pmelg) tool. Accounting Horizons, 37:85– 95, 2023. 10. D. Heckl and J. Moormann. Operational process management in the financial services industry. Handbook of Research on Complex Dynamic Process Management, pages 529–550, 2010. 11. Jaciel David Hernandez-Resendiz, Edgar Tello-Leal, Ulises Manuel RamirezAlcocer, and Bárbara A. Macías-Hernández. Semi-automated approach for building event logs for process mining from relational database. Applied Sciences, 12(21), 2022. 12. A. H. M. t. Hofstede, A. Koschmider, A. Marrella, R. Andrews, D. Fischer, S. Sadeghianasl, M. T. Wynn, M. Comuzzi, J. D. Weerdt, K. Goel, N. Martin, and P. Soffer. Process-data quality: the true frontier of process mining. Journal of Data and Information Quality, 15:1–21, 2023. 13. L. Jiang and F. Naumann. Holistic primary key and foreign key detection. Journal of Intelligent Information Systems, 54:439–461, 2019. 14. Xixi Lu, M. Nagelkerke, D. V. D. Wiel, and Dirk Fahland. Discovering interacting artifacts from erp systems, 2015. 15. Xixi Lu, Marijn Nagelkerke, Dennis van de Wiel, and Dirk Fahland. Discovering interacting artifacts from erp systems. IEEE Transactions on Services Computing, 8(6):861–873, 2015. 16. Najmeh Miri, Shahrzad Khayatbashi, Jelena Zdravkovic, and Amin Jalali. Ocpm2 extending the process mining methodology for object-centric event data extraction. In Renata Guizzardi, Luise Pufahl, Arnon Sturm, and Han van der Aa, editors, Enterprise, Business-Process and Information Systems Modeling, pages 123–140, Cham, 2025. Springer Nature Switzerland. 17. E. G. L. d. Murillas, H. A. Reijers, and W. M. P. v. d. Aalst. Connecting databases with process mining: a meta model and toolset. Software & Systems Modeling, 18:1209–1247, 2018. 18. E. H. J. Nooijen, B. F. v. Dongen, and D. Fahland. Automatic discovery of datacentric and artifact-centric processes. Lecture Notes in Business Information Processing, pages 316–327, 2013. 19. Thorsten Papenbrock and Felix Naumann. Data-driven schema normalization. In Proceedings of the 20th International Conference on Extending Database Technology (EDBT), volume 17. OpenProceedings, March 2017. 20. Viara Popova, Dirk Fahland, and Marlon Dumas. Artifact lifecycle discovery. International Journal of Cooperative Information Systems, 24(01):1550001, 2015. 21. Shameer K. Pradhan, Mieke Jans, and Niels Martin. Getting the data in shape for your process mining analysis: An in-depth analysis of the pre-analysis stage. ACM Comput. Surv., 57(6), February 2025. 22. Shameer K. Pradhan, Mieke Jans, and Niels Martin. Which tables are mine(able)? In Renata Guizzardi, Luise Pufahl, Arnon Sturm, and Han van der Aa, editors, Enterprise, Business-Process and Information Systems Modeling, pages 107–122, Cham, 2025. Springer Nature Switzerland. 23. Simon Rauch, Christian M. M. Frey, Ludwig Zellner, and Thomas Seidl. Processaware bayesian networks for sequential event log queries. In 2024 6th International Conference on Process Mining (ICPM), pages 161–168, 2024.
Schema-Agnostic Process Trace Construction
17
24. S. Remy, L. Pufahl, J. P. Sachs, E. P. Böttinger, and M. Weske. Event log generation in a health system: a case study. Lecture Notes in Computer Science, pages 505–522, 2020. 25. N. Tax, I. Verenich, M. L. Rosa, and M. Dumas. Predictive business process monitoring with lstm neural networks. Lecture Notes in Computer Science, pages 477–492, 2017. 26. Philipp Waibel, Lukas Pfahlsberger, Kate Revoredo, and Jan Mendling. Causal process mining from relational databases with domain knowledge. ArXiv, abs/2202.08314, 2022.