Conceptio › Archive › arXiv CS

arXiv CSopen access

From Schema to Signal: Retrieval-Augmented Modeling for Relational Data Analytics

2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

Open Source ↗Direct PDF ↓

data-management databases storage

databases, sql, data management, storage

arXiv:2605.14464v1 [cs.DB] 14 May 2026

From Schema to Signal: Retrieval-Augmented Modeling for Relational Data Analytics Lingze Zeng

Shaofeng Cai

Changshuo Liu

[email protected] National University of Singapore

Zhongle Xie

Yuncheng Wu

Beng Chin Ooi

[email protected] Zhejiang University

[email protected] Renmin University of China

[email protected] Zhejiang University

Abstract Relational data stored in RDBMS is foundational to applications across domains such as e-commerce, finance, and social media. While deep neural networks (DNNs) have achieved strong performance on single flattened tables, extending these models to relational databases is challenging due to the normalized multitable structure and complex inter-table relationships. Existing approaches often rely strictly on schema-defined graphs, which overlook implicit semantic signals embedded in tuple attributes and suffer from rigid connectivity. In this work, we propose Retrieval-Augmented Modeling (RAM), a new relational data modeling paradigm that learns from both explicit structure and implicit semantics by integrating information retrieval (IR) with graph-based modeling. RAM treats tuple attributes as terms and uses random walks to construct contextual “documents”, enabling the use of IR techniques to estimate semantic relevance between tuples and generate a dynamic and data-driven graph. Building on this, we introduce two retrievalbased augmentation strategies: ATRA, which leverages intra-table relevance for self-supervised contrastive learning to capture finegrained attribute correlation, and ETRA, which adds inter-table shortcuts between semantically related tuples to enhance graph connectivity and modeling. This dual-augmentation is further processed by a modular framework with attribute embedding, feature integration, and graph aggregation layers that unifies both attribute and structural information for expressive and flexible representation learning. Extensive experiments on five real-world relational databases demonstrate that RAM consistently outperforms existing baselines on diverse prediction tasks, establishing a state-of-the-art for relational data analytics.

Introduction

Relational Database Management Systems (RDBMS) have long been the backbone of data storage and management, widely adopted in industries such as e-commerce, finance, and sociality for their efficiency and reliability [10, 16, 32, 33]. Relational data in RDBMS is organized in a multi-table format, with a well-defined schema capturing inter-table dependencies through primary-foreign key constraints. As relational data carries rich information, there is a growing demand for advanced analytics to uncover patterns and extract knowledge within the database, supporting critical applications like forecasting and personalized services [22, 47]. While advanced deep neural networks (DNNs) [5, 7, 21] have shown great

success in the conventional tabular learning paradigm on single flattened tables, extending them to relational data remains a significant challenge due to the complex multi-table structure. In a relational database, an analytical task typically aims to predict a specific attribute for a tuple in a target table. For instance, the task illustrated in Figure 1 is to predict whether a user will churn based on the user’s profile and the interaction history within the database. The conventional approach for such tasks requires extensive manual preprocessing, most notably, flattening the multitable schema into a single table centered around the target entity [9, 11, 15, 39]. This process compresses relational information into a fixed-size representation applicable for standard tabular models. However, as the complexity of the database schema increases, the flattening approach becomes labor-intensive, error-prone, and often leads to suboptimal predictive performance due to significant information loss [14, 26–28, 46]. Specifically, flattening collapses the inherent relational structure by converting fine-grained data from related entities into coarse aggregations, such as reducing a full transaction history to an average purchase amount, while simultaneously discarding rich contextual links defined by both intra-table relationships, e.g., shared preferences among similar users, and inter-table dependencies, e.g., user-product interactions. Recent advancements in relational analytics have focused on graph-based methods that directly model the complex relationships within a database, thereby circumventing the need for flattening tables [13, 43]. In this paradigm, the relational data is transformed into a heterogeneous graph where tuples become nodes, tables define node types, and primary-foreign key relationships form the edges [6, 12, 14, 34, 38, 46]. Graph Neural Networks (GNNs) are then applied to this schema-defined graph to construct tuple representations for predictive analytics by aggregating information from neighboring nodes. Nonetheless, these GNN-based methods are constrained by their reliance on the explicit schema, leading to two major limitations. First, the schema-defined graph fails to capture implicit semantic relevance embedded in tuple attributes. For instance, users who share similar demographic or behavioral attributes may be highly correlated for a prediction task, yet remain disconnected in the graph if no direct PK-FK link exists. Second, the schema can impose inefficient message-passing routes. For example, an entity-relationship-entity structure, like USER-RATE-BIZ (userrating-business) in Figure 1, forces information to travel along an indirect multi-hop path via RATE, rather than through a direct one-hop link from USER to BIZ. The extended routes increase the effective graph diameter, forcing information to travel more hops

Lingze Zeng, Shaofeng Cai, Changshuo Liu, Zhongle Xie, Yuncheng Wu, and Beng Chin Ooi

Relational Database CATG

USER

RATE

BIZ

AGGREGATE

USER

tuples

RATE

...

Tabular-Based Modeling

sum of rating

Prediction

Feature Engineering TAG

tabular model

positive ratio

BIZ sampled subgraph

schema-defined graph

....

USER-RATE edge

Graph-Based Modeling

MP MP

Tuple

Whether the user will churn in the next month ?

Graph Building

node embedding for prediction

MP MP

graph neural network

message passing

Prediction

Figure 1: Illustration of relational data modeling. to cross the graph, which in turn dilutes its content, increases the risk of over-smoothing [13, 14, 36], and impairs the GNN’s capacity to model the underlying interaction. In this paper, we present Retrieval-Augmented Modeling (RAM), a new relational data modeling paradigm that learns from both explicit structure and implicit semantics in relational data, shifting analytics from schema-bound modeling to a more holistic, dualrepresentation approach. Unlike existing graph-based methods that rely entirely on explicit PK-FK links, RAM integrates Information Retrieval (IR) techniques to perceive relationships beyond predefined schema links. Specifically, RAM constructs a semantic “document” for each tuple by combining its attributes with graph context gathered via random walks over the graph. These documents are then indexed on a per-table basis into an inverted list that maps terms to the documents containing them, creating retrieval indices capable of identifying semantically similar documents (tuples). This enables RAM to dynamically discover and leverage implicit and deep relationships, effectively learning from a data-driven relational graph that complements the static and schema-defined one. Building on these indices, we introduce two retrieval-based augmentation strategies. The first, IntrA-Table Retrieval Augment (ATRA), retrieves semantically similar tuples within the same table, serving as positive pairs for self-supervised contrastive learning. This encourages the model to learn a more structured and discriminative representation. The second, IntEr-Table Retrieval Augment (ETRA), identifies and connects semantically related tuples across tables, even if they are not directly linked by PK-FK constraints. This enriches the schema-defined graph with new semantic edges, improving graph connectivity and facilitating more efficient message passing between distant yet related entities. Further, we design a modular, layer-wise modeling framework for relational data modeling. The framework consists of an attribute embedding layer to encode heterogeneous attribute types, a feature integration layer to capture intra-tuple feature interactions, and a graph aggregation layer to incorporate structural context from the enhanced relational graph. This layered design effectively decouples the modeling of attribute semantics from that of graph structure, allowing each to be captured independently before unifying them into expressive tuple representations for downstream tasks. We summarize our main contributions as follows.

• We present RAM, a novel relational data modeling paradigm that constructs retrieval indices over a database to build a dynamic and data-driven graph for predictive analytics, which learns from both explicit structure and implicit semantics in relational data. • We introduce two RAM strategies: ATRA leverages intratable semantic similarity for self-supervised contrastive learning to capture fine-grained attribute correlations, while ETRA enriches the graph with inter-table shortcuts to enhance graph connectivity and message passing. • We design a modular GNN-based modeling framework tailored for relational data analytics, which effectively integrates attribute-level semantics with graph structure for expressive tuple representation learning. • We conduct extensive experiments on five real-world relational databases across 13 prediction tasks, demonstrating that RAM consistently outperforms 12 baselines, achieving state-of-the-art results. The rest of this paper is structured as follows. We introduce preliminaries in Section 2 and detail our retrieval-augmented modeling framework in Section 3, present the experimental evaluation in Section 4, and review related works in Section 5, followed by the conclusion in Section 6.

Preliminaries

This section outlines foundational concepts, first defining relational databases and predictive analytics performed on them, then introducing how relational data is modeled as graphs and the basics of information retrieval. Relational Data. A relational database is a structured collection 𝐾 . Each table 𝑇 𝑘 repreof data organized into 𝐾 tables, D := {𝑇 𝑘 }𝑘=1 sents a specific entity type, e.g., customers, products, and contains 𝑁 𝑘 rows, or tuples, and 𝑀 𝑘 columns, or attributes. We use 𝑇𝑖:𝑘 and 𝑇:𝑗𝑘 to denote the 𝑖-th row and 𝑗-th column in table 𝑇 𝑘 , respectively. Each row 𝑇𝑖:𝑘 represents a single data instance xi , defined as an ordered set of attribute values xi = (𝑥 1, 𝑥 2, . . . , 𝑥 𝑀 𝑘 ). These attributes can be of heterogeneous types, including numerical, categorical, or textual data. In contrast to conventional tabular learning, which operates on a single flattened table, relational data involves multiple tables interconnected through primary-foreign key (PK-FK)

From Schema to Signal: Retrieval-Augmented Modeling for Relational Data Analytics

constraints. A column 𝑇:𝑗𝑘 that forms a primary key (PK) uniquely identifies each tuple in its table 𝑇 𝑘 , whereas a foreign key (FK) ′ column references the primary key of another table 𝑇 𝑘 , thereby establishing a relationship between tables. Predictive analytics on relational data involves a target table 𝑇 𝑘 and aims to predict a specific attribute for its tuples. For instance, a customer churn prediction task seeks to predict the is_churn attribute for each tuple in the Customers table. To capture the timesensitive nature of predictions on dynamic databases, a database snapshot at a given time 𝑡 is denoted as D (𝑡). Formally, the objective is to learn a predictive model P that maps an input tuple x from the target table 𝑇 𝑘 and its database context D (𝑡) to an outcome 𝑦: P (x, D (𝑡),𝑇 𝑘 ) = 𝑦,

(1)

where the model leverages the tuple’s own attributes x and contextual information from related tables within D (𝑡) to make the prediction. Our goal is to develop a general modeling framework applicable to a wide range of such analytical tasks over relational data, supporting diverse businesses such as behavior modeling, risk assessment, or transaction prediction. Relational Graph Modeling. The interconnected structure of a relational database is naturally suited for a graph-based representation. Modeling the database as a heterogeneous graph translates the complex relationships encoded in the schema into explicit structural information. This enables the application of expressive graph learning techniques to capture the intricate patterns and dependencies within the relational data. Specifically, each tuple is represented as a node, with its source table defining the node type. The primary-foreign key constraints define relationships between tuples, thereby establishing edges between nodes, where each unique constraint corresponds to a distinct edge type. For instance, if a tuple 𝑣𝑖𝑘 in table T 𝑘 contains a foreign ′ ′ key referencing the primary key of a tuple 𝑣 𝑘𝑗 in table T 𝑘 , an ′ undirected edge (𝑣𝑖𝑘 , 𝑣 𝑘𝑗 ) is established. Formally, this schema-based graph can be defined as: G = (V, E, TV , TE ),

(2)

where V = ∪𝑘 V 𝑘 is the set of all nodes (tuples), with V 𝑘 being the set of nodes from table 𝑇 𝑘 ; E ⊆ V × V is the set of edges, defined by PK-FK relationships; TV is the set of node types (tables); and TE is the set of edge types (PF-FK constraints). Learning from such graph structures poses challenges for traditional deep learning methods due to their non-Euclidean geometry and complex connectivity [8]. Graph Neural Networks (GNNs) are a class of models specifically designed for graph learning, which generate a representation for each node by iteratively aggregating information from its local neighborhood. At each layer 𝑙, the representation h𝑢𝑙 for a node 𝑢 is updated based on its own state and messages from its neighbors N (𝑢): h𝑢(𝑙+1) = 𝛾 (𝑙 ) h𝑢(𝑙 ) , 𝜙 (𝑙 ) {h𝑣(𝑙 ) : 𝑣 ∈ N (𝑢)} , (3) where 𝜙 (𝑙 ) is a differentiable and permutation-invariant aggregation function, e.g., mean or sum pooling, that combines neighbor representations h𝑣(𝑙 ) , and 𝛾 (𝑙 ) is an update function, e.g., a multilayer perceptron, that merges this aggregated information with the

node representation h𝑢(𝑙 ) from the preceding layer. This messagepassing mechanism allows GNNs to effectively integrate contextual signals from across the tables and generate rich, structure-aware representations for each tuple. Information Retrieval (IR). IR is the task of finding relevant information within large data collections. Unlike structured database queries requiring exact matches, IR ranks items based on estimated relevance, making them ideal for discovering semantic similarity in applications like search engines and recommender systems. A foundational approach in IR is the vector space model, which often relies on a Bag-of-Words (BoW) representation where documents are treated as unordered collections of terms (words). One of the most effective and widely used IR ranking functions is Best Matching 25 (BM25), a probabilistic model that scores the relevance of a document 𝐷 to a query 𝑄 composed of terms (𝑞 1, 𝑞 2, · · · , 𝑞𝑛 ):

𝑆𝑐𝑜𝑟𝑒 (𝐷, 𝑄) =

𝑛 ∑︁ 𝑖=1

IDF(𝑞𝑖 ) ·

TF(𝑞𝑖 , 𝐷) · (𝑘 1 + 1) |𝐷 | TF(𝑞𝑖 , 𝐷) + 𝑘 1 · (1 − 𝑏 + 𝑏 · avgdl )

, (4)

where TF(qi, D) is the term frequency of 𝑞𝑖 in 𝐷, IDF(qi ) is the inverse document frequency of the term (measuring rarity across the collection), |𝐷 | is the document length, and avgdl is the average document length. The parameter 𝑘 1 and 𝑏 control the term frequency scaling and document length normalization, respectively. While modern systems increasingly adopt dense retrieval methods based on embeddings from pre-trained language models, BM25 remains a highly efficient and effective technique, particularly for large-scale predictive analytics. In this work, we adopt these IR principles to estimate semantic relevance between database tuples. This enables the discovery of implicit signals beyond the rigid database schema, providing a powerful technique for retrieval-based data augmentation.

Retrieval-Augmented Modeling

In this section, we introduce Retrieval-Augmented Modeling (RAM). Our approach builds retrieval indices to estimate the relevance between tuples within an RDBMS, extracts informative signals in relational data for augmentation, and supports a neural architecture specifically tailored for relational data analytics.

3.1

Tuple Retrieval Index

We describe how to construct a retrieval index for tuples first. As discussed in Section 2, we follow the principles of information retrieval (IR) by treating each tuple as a document, and each attribute as a term. To support this, we introduce two key components: (1) graph-aware documentation, which constructs a document that captures both the semantic and structural context of a tuple in the relational database; (2) attribute tokenization, which preprocesses attributes to enable meaningful IR statistics, like term frequency. These components are detailed in the following subsections. Graph-Aware Documentation. We begin by directly taking each tuple as a document, which aligns well with the Bag-of-Word (BoW) assumption in IR, as the attributes are permutation-invariant and agnostic to order and syntax. However, relying solely on the attributes within a tuple is often insufficient, as it lacks the contextual

Lingze Zeng, Shaofeng Cai, Changshuo Liu, Zhongle Xie, Yuncheng Wu, and Beng Chin Ooi

Learning Paradigm Relational Database

Retrieval-Augment Modeling (RAM)

Analytical Tasks

...

Biz

biz-rating

Biz Tuple

Biz

PK: id

Document

biz-ctr

city

Table:Column:Value ...

description ave_starts

Root Node

Target Tables

Entity-Relationship Diagram

Biz

user-ctr

User

Transform

Random Walks

Graph-Aware Documentation

id 002

C1 V1

Biz:id:002 = (1, 3); Biz:C1:V1 = (3,11);

Attribute Set

... ...

Cm Vn

Attribute Token Biz:Cm:Vn Biz:C1:V1 Biz:id:002

Attribute Tokenization

Inverted List token:(#doc, TF)

Table-Specific Retrieval Index

Handle

Relational Graph

Relational Modeling MP MP

MP MP

Node

tuple embedding Node: Tuple in DB table Edge: PrimaryKey-ForeignKey

tuple attribute

BM25

Similar Pair

Added Edges

Graph Augmentation

IntEr-Table Retrieval-Augment (ETRA)

Sample Pair BM25

Similar Pair

Contrastive Learning

IntrA-Table Retrieval-Augment (ATRA)

Figure 2: Overview of Retrieval-Augmented Modeling. information present in the relational database. For example, user profile data alone may not provide a complete representation, while incorporating user-product interactions can enhance prediction tasks such as recommendations. As outlined in Section 2, the database is modeled as a relational graph, where each tuple corresponds to a node. The goal is to aggregate the information from the graph to construct a representative document for the target node (i.e., tuple). Therefore, we sample relevant nodes around the target node to provide contextual information. Since closer nodes are generally more relevant, they should contribute more to the document. To reflect this, nodes nearer to the target should be sampled more frequently. We employ the Random Walk with Restart (RWR) strategy to generate a root-centric node set, which favors nearby nodes, resulting in higher attribute value counts from closer neighbors. In RWR, a random walker starts at the root node 𝑟 , and at each step, either moves to a neighboring node or restarts from 𝑟 with probability 𝛼. This iterative process converges to a stationary distribution, where each node 𝑣 is assigned a visitation probability 𝜋 𝑣 , indicating its relative relevance to 𝑟 . Formally: 𝜋𝑣 = 𝛼

∞ ∑︁

) (1 − 𝛼)𝑘 𝑝𝑟(𝑘→𝑣 ,

(5)

𝑘 ) where 𝑝𝑟(𝑘→𝑣 is the probability that a length-𝑘 pure random walk ) from 𝑟 ends at 𝑣, and 𝑝𝑟(𝑘→𝑣 = 0 for all 𝑘 < 𝑑 (𝑟, 𝑣). This implies 𝜋 𝑣 ≈ 𝛼 (1 − 𝛼)𝑑 𝑝𝑟𝑑→𝑣 , indicating that nodes farther from the root receive exponentially lower visitation probabilities. In other words, structurally closer nodes contribute more prominently to the context of the root in RWR sampling. Given the tuples, we repeatedly apply RWR above the relational graph to generate corresponding documents. These documents capture both local and multi-hop relational contexts, with node frequency implicitly encoding structural relevance. As a result, closer nodes contribute more to the representation. It facilitates more accurate retrieval and improves the identification of semantically and structurally relevant candidates.

Attribute Tokenization. To enable meaningful IR statistics, such as term frequency, attribute values must be discrete. As noted in

Section 2, each tuple is represented as a set of heterogeneous attributes, 𝑇𝑖:𝑘 = (𝑥 1, 𝑥 2, · · · , 𝑥 𝑀 𝑘 ). We focus on three common types: numerical value, categorical value, and text. Among them, only the categorical values are inherently discrete and can be directly treated as terms in the BoW model. Numerical values and text require preprocessing for frequency counting, as illustrated in Figure 3. Specifically, numerical data encompasses continuous values that convey magnitude-based semantics. For example, 8 and 9 are numerically closer than 0 and 9. Treating them as distinct terms ignores this proximity. Directly using raw numerical values as terms may lead to highly sparse representations due to potentially large cardinality. Such sparsity hampers similarity estimation. Therefore, we adopt a binding strategy to discretize numerical values into a smaller set of intervals, preserving local similarity within each bin. For simplicity, we apply equidistant binning, where the number of bins is dynamically determined based on the number of distinct values. Following empirical heuristics from prior studies [37], we define the number of bins 𝑏 as: ( 1 + log2 (𝑛), if 𝑛 < 1000 𝑏= (6) √ otherwise 2 · 3 𝑛. For datasets with fewer than 1,000 distinct values, we apply Sturges’ Rule, which assumes a normal distribution and suits smaller datasets. For larger datasets, we use the Rice Rule, which scales better and reduces overfitting. Once the number of bins is set, the bin boundaries are computed, and numerical values are discretized into terms. Text snippets, such as user reviews, product descriptions, or comments, are common in relational databases and carry semantic information. These unstructured texts are important in various downstream analytical tasks, such as sentiment analysis and user behavior modeling. However, using raw text as a term directly leads to vocabulary sparsity, since it involves a wide range of word choices and combinations. Such text values make the similarity score modeling difficult to calculate the term frequency, reducing retrieval quality. To address this, we apply a topic-based method to condense each text snippet into several representative words that better support term frequency-based retrieval. In detail, we utilize KeyBERT [17], a pre-trained topic language model, to identify salient words from the text. These extracted topic words serve as

From Schema to Signal: Retrieval-Augmented Modeling for Relational Data Analytics

Table: AdsInfo

3.2

AdsID

Location

Category

Price

002

New York Electronics

850

Title

Content

iPhone 13 ... Latest ...

Figure 4: Details of relational modeling.

the set of indexed tables. We focus specifically on table pairs that are indirectly connected in the schema. Specifically, for such a table pair, we treat tuples from one table as queries and compute the BM25 function over the index of the other table. This yields a similarity score matrix, where top-ranked tuples are considered potential neighbors. We create directed edges from retrieved tuples to the query tuples. To avoid introducing noisy or weak connections, we apply a dynamic thresholding rule that retains only edges whose similarity scores exceed the inter-table retrieval threshold 𝜃 𝑒 = 𝜇 + 2𝜎, where 𝜇 and 𝜎 denote the mean and standard deviation of the similarity scores, respectively. Under a normality assumption, this threshold retains approximately the top 4.55% of edges, corresponding to the two-sigma tail. This strategy ensures that only high-confidence links are added. This enhancement shortens effective path lengths, mitigates oversmoothing, and improves the model’s ability to capture cross-table patterns that are not explicitly defined in the original schema.

3.3

Layer-wise Model Architecture

In this subsection, we present the architecture of our model for analytical tasks on relational databases. The relational data is first preprocessed and then passes through three layers: the Attribute Embedding layer, which encodes heterogeneous data into a unified

latent vector space; the Feature Integration Layer, which captures attribute interactions within tuples to generate tuple-level embeddings; and the Graph Aggregation Layer, which refines these embeddings by incorporating structural information in the database through iterative message passing. Figure 4 illustrates the architecture. We detail the preprocessing step and each layer below, along with the procedure for self-supervised contrastive learning and downstream task fine-tuning. Preprocessing. We first encode raw values according to their data types. For categorical and numerical values, we replace each value with a token, with numerical tokens retaining their original values. For text data type, we concatenate all text attributes along with their column names into a single sentence for contextual information. This aggregated sentence replaces the original text attributes and is embedded as a whole in subsequent stages. Additionally, primary and foreign key attributes are removed, as they serve only as identifiers while their information is already represented in the attribute values and relational graph structure. The process is shown at the bottom of Figure 4. Attribute Embedding Layer. As described in Section 2, the processed tuple is represented as a value set x = (𝑥 1, 𝑥 2, · · · , 𝑥 𝑀 ), where each value can be a numerical value, a categorical value, or text. To enable subsequent modeling, any attribute value 𝑥𝑖 is transformed into an embedding vector e𝑖 . We prepare the specific encoding module for each type. Specifically, categorical values are transformed via embedding lookup, i.e., e𝑖 = E𝑖 [𝑥𝑖 ], e𝑖 ∈ R𝑛 , where 𝑛 is the embedding dimension and E𝑖 is the embedding table. Each unique value in E𝑖 is assigned a distinct embedding vector. For numerical value 𝑥 𝑗 , embeddings are obtained via a learnable linear transformation, i.e. e 𝑗 = 𝑥 𝑗 · ê 𝑗 +b 𝑗 , where êj, b 𝑗 ∈ R𝑛 , are learnable parameters shared across all values within the same attribute. For the aggregated sentence 𝑥𝑘 , we employ a pretrained language model, which converts 𝑥𝑘 into a dense vector representation e𝑘′ ∈ R𝑛𝑏 , where 𝑛𝑏 is the fixed output dimension of the pre-trained language model. To adapt this representation for downstream modeling, we apply a learnable linear transformation: e𝑘 = W · e𝑘′ + b , where W ∈ R𝑛×𝑛𝑏 , b ∈ R𝑛 . All data are projected into a unified latent space according to their types. In this way, a fixed-size representation for each tuple is constructed by concatenating the embeddings of all attributes, i.e. e = e1 ⊕ e2 · · · ⊕ e𝑀 . Feature Integration Layer. Given the initial tuple representation e, potential interactions among attributes are not explicitly captured, despite their importance for accurate tabular predictions. To capture the interaction among attributes within the tuple, we incorporate a feature integration layer in our model. Prior studies in tabular learning [5, 7, 21] have explored various model designs such as attention or gating strategies. We adopt a simple yet effective approach based on ResNet [15], leveraging residual connections to facilitate stable learning. Specifically, the tuple representation e ∈ R𝑛𝑖𝑛 is passed through 𝑛𝑟 residual blocks: e 𝑓 = 𝑅𝑒𝑠𝐵𝐿𝐾 (· · · 𝑅𝑒𝑠𝐵𝐿𝐾 (e)) . | {z } 𝑛𝑟

(8)

From Schema to Signal: Retrieval-Augmented Modeling for Relational Data Analytics

Each residual block applies a transformation function 𝐹 (x) and adds a skip connection with linear projection: 𝑅𝑒𝑠𝐵𝐿𝐾 (x) = 𝐹 (x) + Wx,

𝐹 (x) = 𝛿 (W2 · 𝛿 (W1 x + b1 ) + b2 ).

(10)

Here, W2 ∈ R𝑛𝑜𝑢𝑡 ×𝑛ℎ𝑖𝑑 , W2 ∈ R𝑛ℎ𝑖𝑑 ×𝑛𝑖𝑛 . 𝛿 denotes a sequence of a nonlinear activation, normalization, and dropout. The residual mechanism prevents gradient vanishing and enables deeper feature representations. This layer effectively enhances the tuple representation by capturing feature interactions. We heuristically set 𝑛𝑟 = 2, 𝑛ℎ𝑖𝑑 = 𝑛𝑜𝑢𝑡 = 𝑛, and apply ReLU activation along with layer normalization. Graph Aggregation Layer. After generating tuple embeddings e 𝑓 , we introduce the Graph Aggregation Layer to integrate structural information encoded in the relational graph. As introduced in Section 2, the graph is denoted as G = (V, E, TV , TE ). To accommodate the heterogeneous nature of relational data, we extend the GraphSAGE framework [19] for multi-relation settings. Formally, given a node 𝑣 with its embedding at layer 𝑘 as h𝑘𝑣 , the updated embedding at layer 𝑘+1 is computed as: 1 ∑︁ |TE | 𝑟 ∈ T

∑︁

exp(𝜙 (q, q+ )/𝜏) L = − log Í𝑁 , 𝑖=0 exp(𝜙 (q, q𝑖 )/𝜏)

(9)

where W ∈ R𝑛𝑜𝑢𝑡 ×𝑛𝑖𝑛 is a linear projection. In our implementation, 𝐹 is a two-layer fully connected network:

h𝑘+1 = 𝛿 (W𝑘 h𝑘𝑣 + 𝑣

similarity with negative pairs. The loss function is formulated as:

W𝑘𝑟 h𝑢𝑘 ).

(11)

E 𝑢 ∈ N𝑟 (𝑣)

Here, W𝑘 ∈ R𝑛𝑘+1 ×𝑛𝑘 denotes a learnable linear projection applied to the current node, i.e., a skip connection to preserve its original features. Meanwhile, W𝑘𝑟 ∈ R𝑛𝑘+1 ×𝑛𝑘 represents a relation-specific transformation corresponding to edge type 𝑟 . For simplicity, we set 𝑛𝑘+1 = 𝑛𝑘 = 𝑛. The set N𝑟 (𝑣) comprises the sampled neighbors of node 𝑣 connected via relation type 𝑟 . For aggregation, we sum over neighbors within the same relation to fully capture their influence, then average across different relation types to maintain balanced information flow in heterogeneous graphs. The tuple representation e 𝑓 ∈ R𝑛 is passed through the graph aggregation layer to obtain e𝑔 ∈ R𝑛 , which incorporates structural context from the relational graph. The final tuple representation is then formed by concatenating the two vectors: q = e 𝑓 ⊕ e𝑔 , where q ∈ R2𝑛 . Self-supervised Contrastive Learning. Following the above steps, we obtain a comprehensive tuple representation, denoted as q𝑣 for each node 𝑣. To perform self-supervised contrastive learning, we construct positive and negative pairs based on the ATRA introduced in Section 3.2. If node 𝑣 has an associated positive pair under ATRA, we randomly sample one such tuple 𝑣 + , and denote its corresponding representation as q+ = q𝑣 + . If no ATRA is available for node 𝑣, we generate a synthetic positive by perturbing its local subgraph N (𝑣). Such perturbations include masking attributes, removing nodes, or deleting edges. The perturbed subgraph N (𝑣) + is then used to compute a new representation q+ . For negative sampling, we randomly select unrelated tuples from the graph and denote their representations as q− . We employ the InfoNCE loss [31] to maximize the similarity between positive pairs while minimizing

(12)

where the function 𝜙 measures similarity, typically cosine similarity, and 𝜏 is a temperature parameter controlling the sharpness of the similarity distribution. This contrastive objective allows the model to learn discriminative tuple representations. Downstream Task Fine-tuning. For downstream supervised learning tasks, we choose task-specific loss functions. For instance, in binary classification, the objective function is: 𝑁

ˆ 𝑦) = − L (𝑦,

1 ∑︁ {𝑦𝑖 log𝜎 (𝑦ˆ𝑖 ) + (1 − 𝑦𝑖 )log(1 − 𝜎 (𝑦ˆ𝑖 ))}. 𝑁 𝑖

(13)

For regression tasks, we adopt the L1 loss, which is defined as: 𝑁

ˆ 𝑦) = L (𝑦,

1 ∑︁ |𝑦ˆ𝑖 − 𝑦𝑖 |, 𝑁 𝑖=1

(14)

where 𝑦ˆ is the prediction label, 𝑦 is the ground truth label, 𝑁 is the number of training tuples, and 𝜎 (·) is the sigmoid function. The prediction 𝑦ˆ is obtained by applying a prediction head to the tuple embedding: 𝑦ˆ = 𝐻𝐸𝐴𝐷 (q), where 𝐻𝐸𝐴𝐷 is typically a single-layer linear projection.

Experiments

In this section, we evaluate the effectiveness of RAM, using five real-world datasets across different domains. We first introduce the experimental setup and then report the evaluation results.

4.1

Experimental Setup

Datasets. We conduct experiments based on the relbench library [13] on five multi-table datasets with complex relationships, drawn from domains of healthcare, sociology, and e-commerce. The statistics of these datasets are summarized in Table 1, and the corresponding analytical tasks are illustrated in Table 2. (1) Trial [4], a clinical trial database from the AACT initiative that contains detailed records of medical studies for health and treatment research. It involves one classification task to predict whether a trial will succeed (study-outcome) and two regression tasks to estimate the number of affected patients (study-adverse) and the success rate of the trial site (site-success). (2) Avito [3], an online advertisement database capturing interactions between users and product ads. It includes one classification task: predicting whether a user will click on ads (user-clicks) in the next 4 days, and one regression task to estimate the click-through rate of ads (ad-ctr). (3) Stack [2], a Stack Exchange database tracking user activity across Q&A topics. It includes one classification task, predicting whether a user will earn any new badges (user-badge) in the next 3 months, and one regression task to predict the number of votes a post will receive (post-votes). (4) Event [1], a recommendation database derived from user data on a mobile app, tracking social plans, actions, and event details. It includes two classification tasks: predicting if a user will attend the

Lingze Zeng, Shaofeng Cai, Changshuo Liu, Zhongle Xie, Yuncheng Wu, and Beng Chin Ooi

Table 1: Dataset statistics.

Table 2: Prediction tasks statistics.

Dataset

#Tab

#Rel

#Col

#Feat

#Tuple

Domain

Trial Avito Stack Event Beer

15 8 9 5 9

15 26 14 7 12

77 11 51 117 116

5,369 187,162 1,062,015 1,651 8623

5,434,924 20,678,117 4,897,438 326,268 5,388,603

Healthcare E-commerce Sociology E-commerce Sociology

same event again (user-repeat) or ignore event invitations (userignore), and one regression task to estimate how many events a user will respond to in the next 7 days (user-attendance). (5) Beer [30] is a database of beer reviews from users and drinking places. We design two classification tasks: predicting if a user will post more than 10 beer reviews (user-active) and if a place will achieve an average rating above 75 (place-positive) in the next season, and a regression task to predict the positive rating ratio of a beer, where ratings above 3.5 are considered positive (beerpositive). Baseline Methods. We categorize the baseline methods into three groups. The first group is tabular-based methods, which apply advanced tabular learning models to relational data. Specifically, the target table is joined with directly connected tables to form a single flattened table for prediction. This group includes CatBoost[35], LightGBM [24], MLP, ResNet [15] and FT-Trans [15]. The second group, graph-based approaches, transforms relational data into a heterogeneous graph to enable the application of GNNs. It includes Node2Vec [18], R-GCN [38], R-GAT [41] HGT [20]. In this group, Node2Vec notably differs from the others, as it ignores tuple attributes and learns node embeddings solely from graph structure. Since GAT [41] does not natively support heterogeneous graphs, we extend it by averaging attention weights across different edge types. We refer to this variant as R-GAT. Given the large-scale graph in our setting, these methods are implemented with neighborhood sampling to support scalable and inductive representation learning. The third group, graph-pretrain-based methods, utilizes unsupervised learning techniques to capture intrinsic information from the relational graph. These techniques includes DGI [42], GraphCL [45], and BGRL [40]. In our setting, an R-GCN model is pre-trained using these methods and subsequently fine-tuned on downstream prediction tasks to improve performance. Settings. For an fair comparison, we standardize the experimental settings across all models. Specifically, both the feature embedding size and the hidden layer dimension are set to 128. All GNN and DNN models are configured with 2 layers. For classification tasks, we evaluate performance using the AUC-ROC metric, while for regression tasks, we report Mean Absolute Error (MAE). To ensure stable training in regression settings, dropout layers are deactivated. We apply early stopping based on validation performance: if the metric does not improve for 𝑝 consecutive epochs, training is terminated. This helps avoid overfitting to noise and reduces unnecessary computation. Regarding training hyperparameters, we use a learning rate in the range of 1e-3 to 1e-4 and a batch size of 512 for all methods and datasets. All experiments are conducted on a server with a

Task

#Train Instances

#Valid Instances

#Test Instances

Imbalance Ratio (%)

study-outcome study-adverse site-success

11,994 43,335 151,407

960 3,596 19,740

825 3,098 22,617

63.75 -

user-clicks ad-ctr

59,454 5,100

21,183 1,766

47,996 1,816

3.87 -

user-badge post-votes

3,386,276 2,453,921

247,398 15,6216

255,360 160,903

4.81 -

user-repeat user-ignore user-attendance

3,842 19,239 19,239

268 4,185 2,013

246 3,949 1,958

48.98 16.87 -

user-active place-positive beer-positive

16,656 11,337 45,922

2,794 4,570 12,858

3,558 2,869 7,218

53.18 38.64 -

Xeon Silver 4114 CPU @ 2.2GHz (10 cores), 256GB of memory, and 8 GeForce RTX 3090 Ti. Model implementations are based on PyTorch 2.1.0, PyTorch Geometric 2.5.3, and RelBench 1.1.0, with CUDA 11.8.

4.2

Experimental Results and Analysis

Main Study. We compare our method with three groups of baselines with end-to-end experiments across five datasets covering 13 prediction tasks. Results for classification and regression tasks are summarized in Table 3 and Table 4, respectively. From a macro perspective, graph-based approaches consistently outperform tabular-based methods. This gap is due to the limitation of compressing relational data into a single table, which results in the loss of structural information. Even with advanced feature engineering, much of the relational context in the database remains underutilized. In contrast, graph-based methods preserve the relational structure, enabling more comprehensive utilization of the data during modeling. In this conclusion, two specific observations are worth highlighting. First, tabular-based approaches slightly outperform graph-based methods on the study-outcome task in the Trial dataset. This is likely because the target table contains 28 rich, well-defined attributes, sufficient for accurate prediction. When the target table provides ample information, tabular-based models excel at capturing attribute interactions and can achieve strong performance. Second, within the graph-based methods, Node2Vec performs worse than other models, as it relies solely on graph structure and does not consider node attributes. This limits its effectiveness when attribute information is crucial, such as in the study-outcome task. However, in datasets like Avito, where graph topology carries strong signals (e.g., user clicks, user visits), Node2Vec outperforms tabular-based methods. This highlights the varying importance of structure versus attributes across different datasets. Next, we compare standard graph-based approaches with graphpretrained-based methods. We observe in Table 3 that pre-training methods offer no significant advantage in classification tasks. For instance, the performance of R-GCN is comparable to its variants

From Schema to Signal: Retrieval-Augmented Modeling for Relational Data Analytics

Table 3: Classification prediction results (ROC-AUC, higher is better). Method/Result (ROC-AUC ↑) Dataset

Task

Tabular-Based

Graph-Based

Graph-Pretrain-Based

CatBoost

LightGBM

MLP

ResNet

FT-Trans

Node2Vec

R-GCN

R-GAT

HGT

DGI

GraphCL

BGRL

RAM

Trial

study-outcome

0.7003

0.7009

0.7122

0.7021

0.7130

0.5544

0.7007

0.7035

0.7018

0.6946

0.7028

0.7053

0.7292

Avito

user-clicks

0.5545

0.5360

0.5498

0.5435

0.5463

0.6080

0.6640

0.6597

0.6373

0.6531

0.6628

0.6570

0.6743

Stack

user-badge

0.6386

0.6343

0.6004

0.6140

0.6111

0.5317

0.8711

0.8288

0.8710

0.8717

0.8716

0.8748

0.8939

Event

user-repeat user-ignore

0.6904 0.8024

0.6804 0.7993

0.6889 0.7825

0.6657 0.7733

0.6600 0.7499

0.6679 0.8070

0.7659 0.8146

0.7186 0.7993

0.7582 0.8090

0.7422 0.7999

0.7653 0.8151

0.7555 0.8155

0.8011 0.8490

Beer

user-active place-positive

0.8987 0.8007

0.9147 0.8143

0.8952 0.8093

0.8996 0.8067

0.9008 0.8120

0.7996 0.6110

0.9228 0.8140

0.8854 0.7904

0.9246 0.8195

0.9163 0.8130

0.9231 0.8162

0.9248 0.8253

0.9457 0.8560

Table 4: Regression prediction results (MAE, lower is better). Method/Result (MAE ↓) Dataset

Task

Tabular-Based

Graph-Based

Graph-Pretrain-Based

CatBoost

LightGBM

MLP

ResNet

FT-Trans

Node2Vec

R-GCN

R-GAT

HGT

DGI

GraphCL

BGRL

RAM

Trial

site-success study-adverse

0.4227 51.404

0.4250 44.011

0.4387 50.595

0.4267 46.709

0.4279 51.859

0.4547 52.843

0.3999 44.890

0.3913 45.820

0.3899 44.931

0.4100 44.385

0.3990 44.052

0.3863 43.584

0.3481 42.673

Avito

ad-ctr

0.0408

0.0410

0.0426

0.0427

0.0421

0.0417

0.0392

0.0387

0.0381

0.0380

0.0366

Stack

post-votes

0.0676

0.0680

0.0684

0.0682

0.0687

0.0685

0.0673

0.0677

0.0679

0.0662

0.0669

0.0678

0.0642

Event

user-attendance

0.2635

0.2640

0.2642

0.2644

0.2641

0.2733

0.2554

0.2533

0.2528

0.2538

0.2518

0.2530

0.2344

Beer

beer-positive

0.1780

0.1790

0.2115

0.2077

0.2068

0.2447

0.1769

0.2117

0.1782

0.1758

0.1732

0.1697

0.1573

initialized with DGI or GraphCL. However, for regression tasks in Table 4, the graph-pretrained-based methods generally perform better. This discrepancy can be attributed to two main factors. First, regression requires learning fine-grained, continuous mappings, which are more sensitive to the quality of feature representations. Pre-training provides a stronger initialization that facilitates better generalization and convergence. Second, regression tasks are more prone to overfitting, especially when target values contain noise. The self-supervised training acts as an implicit regularizer, enhancing the robustness and stability of the representation. Finally, our proposed method RAM achieves significant improvements over all baseline methods on both regression and classification tasks, demonstrating the effectiveness of our two retrievaldriven augmentation signals. In Inter-Table Retrieval Augment (ETRA), new edges are added between semantically related nodes, facilitating GNNs to capture high-level dependencies and crossentity interactions that standard message passing can not reach due to long-hop limitations. This enriches the graph with meaningful links that are not explicitly present in the original schema. In Intra-Table Retrieval Augment (ATRA), contrastive learning is employed to align the embeddings of tuples that exhibit semantic relevance based on retrieval scores. Unlike existing unsupervised graph pre-training methods that rely on structural perturbations for robustness, ATRA leverages intrinsic semantic similarities as supervised signals, enabling the model to learn more informative representations. This augmentation guides the predictive model to extract generalizable patterns from relational data and helps

alleviate over-smoothing in GNN training, resulting in more robust and discriminative embeddings for downstream tasks. Ablation Study. To better understand the individual contributions of the two augmentations, ETRA and ATRA, we conduct an ablation study on the Trial and Event datasets. By selectively disabling each augmentation in RAM, we evaluate their impact on end-to-end performance across both classification and regression tasks. We use the model without any augmentations as the baseline. The results are summarized in Table 5 and Table 6. We observe that incorporating either augmentation consistently improves the model performance, confirming the effectiveness of our retrieval-driven augmentations. Among these two augmentations, ATRA yields more significant gains than ETRA across most datasets and tasks, suggesting it provides stronger inductive signals for representation learning. ATRA helps the model capture latent patterns and general knowledge inherent in the data while enhancing robustness through perturbation-based contrastive learning. This robustness is especially valuable in real-world relational databases, where missing or incomplete data is common. In comparison, ETRA focused on graph augmentation, is more sensitive to schema sparsity or weak connectivity. While beneficial, its impact is less pronounced. These findings highlight the complementary strengths of both augmentations, with ATRA playing a more prominent role in learning transferable and resilient representations for relational data. Furthermore, we experiment to analyze the effect of ATRA in downstream task fine-tuning. We track the training loss over the

Lingze Zeng, Shaofeng Cai, Changshuo Liu, Zhongle Xie, Yuncheng Wu, and Beng Chin Ooi

ETRA ATRA

% %

% "

" %

" "

Trial

study-outcome

0.7016

0.7189

0.7080

0.7292

Event

user-repeat user-ignore

0.7718 0.8253

0.7879 0.8419

0.7820 0.8250

0.8011 0.8490

user-repeat

0.8

0.7

0.6

0.5 0.4

study-outcome L1Loss

BCELoss

0.8

Method/Result (ROC-AUC)

0.5 0 50 100 150

Steps

0.4

0 50 100 150

Steps

(a) Classification Tasks

user-attendance 1.0 site-success 0.8

0.6

0.4

0.2

0 50 100 150

Steps

0.2

0 50 100 150

Steps

(b) Regression Tasks

Figure 5: Effect of ATRA on training loss trend. first 200 steps, as shown in Figure 5, and record the best validation performance achieved in each epoch, presented in Figure 6. In Figure 5, we smooth the loss curve and calculate its standard deviation within a sliding window to reduce the impact of stochastic fluctuations and highlight overall optimization trends more clearly. Across datasets and tasks, we consistently observe that the model with ATRA begins with lower initial loss compared to the model without ATRA, indicating that ATRA provides a better initialization and allows the model to start closer to an optimal solution in the parameter space. In Figure 6, we further observe that the model with ATRA achieves higher validation performance in early epochs, demonstrating that the initialized representations transfer well and align with downstream objectives. As a result of the early-stopping mechanism, the model with ATRA terminated training earlier due to faster convergence and stabilized validation performance. It confirms that self-supervised contrastive learning within ATRA leads to more effective and robust fine-tuning. Hyperparameter Study. Now, we investigate the effects of three key hyperparameters in RAM: the inter-table retrieval threshold (𝜃 𝑒 ), which determines the number of newly added edges to the relational graph; the intra-table retrieval threshold (𝜃 𝑎 ), which filters high-confidence positive pairs for contrastive learning; and the number of negative samples (𝑁 ) used in the contrastive objective. For 𝜃 𝑒 , we follow the Chebyshev inequality to define a dynamic threshold based on the mean and standard deviation of retrieved scores. Specifically, we set 𝜃 𝑒 = 𝜇 + 𝑘𝛿, where 𝑘 ranges from 0 to 3, and find that 𝜇 + 2𝛿 yields the optimal performance. For 𝜃 𝑎 , which governs the confidence of positive pairs based on normalized retrieved scores, we explore values from 0.5 to 0.8 and identify 0.7 as the best. We also perform a grid search to determine the optimal number of negative samples 𝑁 for contrastive learning. Figure 7 illustrates how the number of newly added edges and positive pairs varies with changes in 𝜃 𝑒 and 𝜃 𝑎 . As 𝜃 𝑒 increases,

% %

% "

" %

" "

Trial

site-success study-adverse

0.3916 44.137

0.3677 43.024

0.3775 43.651

0.3481 42.673

Event

user-attendance

0.2528

0.2474

0.2414

0.2413

study-outcome

50 0

Method/Result (MAE)

ETRA ATRA

user-repeat

1.0 0.8

Augments

Dataset/ Task

Best-AUC (%)

Augments

Dataset/ Task

Table 6: Ablation study on regression tasks.

Epoch

Best-MAE (%)

Table 5: Ablation study on classification tasks.

Epoch

(a) Classification Tasks

user-attendance

site-success

40 0

Epoch

(b) Regression Tasks

Figure 6: Effect of ATRA on best validation metrics.

more low-similarity pairs are filtered out, preserving only highconfidence candidates. This results in an order-of-magnitude reduction in edge count between 𝜇 and 𝜇 + 3𝛿. A similar trend is observed with 𝜃 𝑎 . As 𝜃 𝑎 increases, selected positive pairs decrease sharply, reflecting a stricter selection of relevant tuples. We evaluate the impact of two hyperparameters on model performance using the Trial and Event datasets for both regression and classification tasks. The results are presented in Figure 8 and Figure 9. We observe a performance improvement as 𝜃 𝑒 increases from 𝜇 to 𝜇 + 2𝛿 and as 𝜃 𝑎 increases from 0.5 to 0.7. It can be attributed to the enhanced signal quality: while the index ranks candidates by similarity scores, not all retrieved pairs are semantically reliable. Applying a threshold filters out noisy or weakly related pairs, improving the quality of augmented signals. In contrast, lower thresholds introduce more noise, which can degrade downstream performance. But, increasing 𝜃 𝑒 beyond 𝜇 + 2𝛿 or 𝜃 𝑎 beyond 0.7 causes a performance drop due to a sharp reduction in augmented edges and positive pairs. With fewer learning signals, the model benefits less from pretraining and structural learning. These findings highlight a trade-off in threshold selection: higher thresholds improve the signal quality, while overly strict filtering reduces the diversity and quantity of augmentations. Optimal performance lies in balancing signal quality with the richness of augmentation. We also examine the impact of the number of negative samples 𝑁 in the contrastive learning process, as shown in Figure 10. Unlike the noticeable effect of 𝜃 𝑎 and 𝜃 𝑒 , changing 𝑁 has minimal influence on downstream performance, which remains stable across different values. This indicates that a moderate number of negative samples is sufficient for effective contrastive learning, and that adding more negatives contributes little additional benefit. Such robustness means the RAM does not rely heavily on fine-tuning 𝑁 , making it easier to apply across datasets without extensive parameter tuning.

From Schema to Signal: Retrieval-Augmented Modeling for Relational Data Analytics

105

104

104 + +2 +3

105

Threshold

105

104

+ +2 +3

104 0.5 0.6 0.7 0.8

Threshold

user-repeat 79.8 80.1

80 78 76

0.5 0.6 0.7 0.8

Threshold

(a) Number of Added Edges

Threshold

0.5 0.6 0.7 0.8

78 76

80.1

study-outcome 72.4

72.9

72 71.0

70.8

77.8 70 + +2 +3

Threshold

+ +2 +3

Threshold

(a) Classification Tasks

user-attendance 40 site-success

24 23

25.0

24.9 24.7 24.1

+ +2 +3

37.8

38 37.7 36 34

Threshold

35.3

71.0

70.8 68.9

user-attendance 40 site-success

24.8 24.1 24.3

24 23

0.5 0.6 0.7 0.8

25.1

0.5 0.6 0.7 0.8

38.1 36.3

Threshold

38.2

34.8 0.5 0.6 0.7 0.8

Threshold

(b) Regression Tasks

Figure 9: Effect of threshold 𝜃 𝑎 in ATRA.

26 25

72.9

(a) Classification Tasks

34.8

+ +2 +3

Threshold

(b) Regression Tasks

AUC-ROC (%)

80 79.1 79.6

MAE (%)

AUC-ROC (%)

user-repeat

Threshold

(b) Number of Positive Pairs

study-outcome

78.9 72

76.7

Figure 7: Effect of 𝜃 𝑒 and 𝜃 𝑎 on the number of signals. 82

MAE (%)

105

Trail 106

user-repeat 79.0

80.1

study-outcome 72.9

79.6 79.3 72 71.7

78 76

72.1 72.0

70 10 20 30 40

negative num

10 20 30 40

negative num

(a) Classification Tasks

MAE (%)

Event 106

AUC-ROC (%)

Trial 106

# Positive Pair

# Added Edges

Event 106

user-attendance 40 site-success

25 24.8 24 23

24.1 24.3

24.7 38

10 20 30 40

negative num

36 34

36.3 34.8 35.1

35.5

10 20 30 40

negative num

(b) Regression Tasks

Figure 8: Effect of threshold 𝜃 𝑒 in ETRA.

Figure 10: Effect of the number of negative pairs.

Interpretability Study. To make the effect of RAM more intuitive and straightforward, we directly dive into the effect of ETRA and ATRA using graph-level metrics and visualization methods. For ETRA, we examine how it changes the structural profile of the relational graph in the Event dataset. Specifically, we compute the number of connected components, average degree, average shortest path length, and average clustering coefficient before and after applying ETRA. These metrics, summarized in Table 7, reveal the impact of ETRA on the graph’s topology. First, we observe a decrease in the number of connected components, suggesting that previously isolated or loosely connected subgraphs are now integrated. This indicates that ETRA enhances the global connectivity of the graph by linking semantically related but distant node pairs. Second, the increase in average degree reflects that nodes have gained additional semantically meaningful neighbors, increasing their local connectivity. Third, the average shortest path length decreases from 14.26 to 12.08, indicating that ETRA introduces semantic shortcut edges, which reduce the number of hops required to traverse the graph. This makes the graph more compact and enables more efficient message passing. Lastly, we observe a substantial rise in the average clustering coefficient from 0.0002 to 0.0631. The initially low value suggests that the original graph had a tree-like or chain-like structure. The sharp increase indicates that the added edges encourage the formation of triadic closures, shifting the graph toward a more locally dense and clustered topology. These new connections foster semantically coherent neighborhoods, further enriching the relational structure. Additionally, we plot the shortest path length distribution in the Event dataset from the table users to the table events and table event_interest. To better visualize the change, we fit a normalized distribution to each set of shortest path lengths, shown in Figure 11. We observe a clear leftward shift in the distribution. For the users-events path, the mean and standard deviation decrease from 15.44/4.18 to 11.86/2.86. Similarly, for users-events_interest, the values drop from 14.17/4.2 to 11.50/2.78. These shifts indicate

a consistent reduction in path lengths and variance, meaning the graph becomes not only more compact but also more uniformly connected, highlighting the effect of ETRA in enhancing semantic connectivity and promoting efficient information flow across the relational graph in GNN. For ATRA, we list 5 extracted positive pairs from the Tag Table in the Stack Dataset. Due to the instances not going through desensitization, we can evaluate the effect of the ATRA by directly examining the semantic meaning of the tag name. As shown in Table 8, the identified tag groups capture meaningful intra-table relationships. The pair (bic, aic) consists of model selection criteria commonly used in statistical analysis. Set #2 includes evaluation metrics commonly used in machine learning. The tags in set #3 pertain to experimental design and modeling approaches in both Bayesian and frequentist frameworks. Set #4 groups tags related to survival analysis and event-time modeling. Finally, set #5 includes fundamental concepts and algorithms in reinforcement learning, such as “Q-learning” and “policy-gradient”. These examples demonstrate that ATRA successfully identifies semantically related tuples. Leveraging these positive pairs in contrastive learning helps improve representation learning by enriching tuple embeddings and enhancing both robustness and diversity. To further evaluate the interpretability of ATRA, we conduct an experiment on the Beer dataset to examine whether the extracted positive pairs capture tuples that share implicit common characteristics or belong to the same semantic group, which we refer to as a cohort set [23, 29, 44]. Specifically, we apply ATRA to four entity tables: beers, places, brewers, and countries. We define ground-truth cohort sets based on manually specified patterns. For the beers table, tuples are considered to be of the same cohort if they have the same beer style. For places, cohort tuples are those with the same place type or located in the same state. For brewers, cohorts share the same brewer type and originate from the same place. For countries, cohorts are defined by having the same country code. We compute the cohort set ratio, which measures the proportion of extracted

Lingze Zeng, Shaofeng Cai, Changshuo Liu, Zhongle Xie, Yuncheng Wu, and Beng Chin Ooi

Table 7: Impact of ETRA on the graph profile in global view. Metrics

w/o ETRA

w/ ETRA

#Connected Component Average Degree Average Shortest Path Length Average Cluster Coefficient

8584 3.404 15.436 0.0002

8519 3.611 11.861 0.0631

users - events

0.25

w/o ETRA

Probability

0.20 0.15

users - event_interest

w/ ETRA

w/o ETRA

=15.44 =4.18

=11.86 =2.86

0.10

Shortest Path Length

Tag Name Set in ATRA positive pairs

1 2 3 4 5

(bic, aic) (precision-recall, h-measure, auc) (bambi, mixed-model, anova, counterbalancing) (branching, kaplan-meier, survival) (RL, Q-learning, policy-gradient, contextual-bandit)

beers places brewers countries 0.0

=14.17 =4.20

=11.50 =2.78

30 0

w/ ETRA

0.05 0.00

Shortest Path Length

Figure 11: Effect of ETRA on the distribution of shortest path lengths for specific paths.

positive pairs that fall within the defined cohort sets. As shown in Figure 12, the cohort set ratio exceeds 0.75 for the places, brewers, and countries tables, suggesting that ATRA effectively identifies semantically coherent tuple pairs in these cases. The ratio for the beers table is lower, around 0.38, due to the large number (approximately 300) and fine-grained nature of beer styles. The hierarchical relationships among styles make simple style-level matching overly strict, reducing the cohort overlap. These results demonstrate that ATRA can successfully extract tuple pairs that share implicit semantic similarities or belong to the same latent cohort. Leveraging these pairs in contrastive learning enables the model to construct a more discriminative latent space for tuple embeddings, thereby enhancing the effectiveness of downstream fine-tuning and prediction tasks.

Table 8: Example of ATRA positive pairs with Tag table in the Stack dataset.

Related Work

Deep Tabular Learning. Tabular Learning is designed for data organized in a logical single table, where each instance is a row described by a fixed set of attributes [5, 15, 21, 39]. Classical machine learning approaches such as tree-based models [24, 35] have been widely adopted due to their strong performance and interpretability. Recently, deep learning models for tabular data have emerged, including TabNet [5], FT-Transformer [15], which leverage attention mechanisms or specialized architectures to model complex feature interactions and combinations. However, these methods assume that all relevant information is contained within a flat table, making them ill-suited for multiple interconnected tables in a relational database. When cross-table dependencies are complex and essential for prediction, tabular learning struggles to capture the full semantic information. As a result, these methods are only effective when data has already been aggregated or flattened through careful feature engineering. Deep Graph Learning. Graph learning methods operate on graphstructured data to capture topological information and dependency

38.38% 76.61% 77.55% 75.00%

0.2

0.4

Ratio

0.6

0.8

1.0

Figure 12: Cohort set ratio among ATRA positive pairs in the Beer dataset, with each bar corresponding to a table.

patterns. Early approaches, such as PageRank and DeepWalk [18], leverage random walks or matrix factorization to model node proximity in the graph. Recently, graph neural networks (GNNs) have introduced message-passing mechanisms that allow nodes to aggregate features from their neighbors and learn context-aware embeddings. Some advanced GNNs, such as GCN [25], GAT [41], and GraphSAGE [19] have been successfully applied to domains like social networks and molecular analysis. However, GNNs, primarily designed for graph data, do not explicitly model feature interactions and combinations, which limits their ability to uncover the implicit knowledge in tabular data. In relational data, attribute-level semantics are rich and essential. Properly modeling these interactions is critical for accurate predictive analytics.

Conclusion

In this work, we propose Retrieval-Augmented Modeling (RAM) for relational data analytics. RAM transforms relational data into a heterogeneous graph and represents each tuple as a document enriched with attributes and graph context. We build information retrieval indices on these documents to estimate semantic relevance between tuples. Leveraging these indices, we introduce two augmentations: Intra-Table Retrieval-Augment (ATRA), which selects similar tuples within tables as positive pairs for contrastive learning, and Inter-Table Retrieval-Augment (ETRA), which links semantically similar tuples across tables to enhance graph connectivity. These augmentations can uncover implicit semantic patterns, thereby addressing the rigid connections inherent in the schema-defined graph. Furthermore, we design a layer-wise model architecture for expressive tuple representation learning. We evaluate RAM on five real-world databases across 13 prediction tasks, where RAM consistently outperforms three groups of baselines, achieving stateof-the-art performance in relational data analytics.

From Schema to Signal: Retrieval-Augmented Modeling for Relational Data Analytics

References [1] 2013. Event Recommendation Engine Challenge. https://www.kaggle.com/c/ event-recommendation-engine-challenge. Accessed: 2025-04-16. [2] 2014. Stack Exchange Data Dump. https://archive.org/details/stackexchange. Accessed: 2025-04-16. [3] 2015. Avito Context Ad Clicks. https://www.kaggle.com/c/avito-context-adclicks. Accessed: 2025-04-16. [4] 2016. AACT Clinical Trials.gove. https://aact.ctti-clinicaltrials.org/. Accessed: 2025-04-16. [5] Sercan Ö Arik and Tomas Pfister. 2021. Tabnet: Attentive interpretable tabular learning. In Proceedings of the AAAI conference on artificial intelligence. AAAI Press, Palo Alto, California USA, 6679–6687. [6] Jinze Bai, Jialin Wang, Zhao Li, Donghui Ding, Ji Zhang, and Jun Gao. 2021. ATJNet: Auto-Table-Join Network for Automatic Learning on Relational Databases. In Proceedings of the Web Conference 2021 (Ljubljana, Slovenia) (WWW ’21). Association for Computing Machinery, New York, NY, USA, 1540–1551. doi:10. 1145/3442381.3449980 [7] Shaofeng Cai, Kaiping Zheng, Gang Chen, H. V. Jagadish, Beng Chin Ooi, and Meihui Zhang. 2021. ARM-Net: Adaptive Relation Modeling Network for Structured Data. In Proceedings of the 2021 International Conference on Management of Data (Virtual Event, China) (SIGMOD ’21). Association for Computing Machinery, New York, NY, USA, 207–220. doi:10.1145/3448016.3457321 [8] Wenming Cao, Canta Zheng, Zhiyue Yan, Zhihai He, and Weixin Xie. 2022. Geometric machine learning: research and applications. Multimedia Tools Appl. 81, 21 (Sept. 2022), 30545–30597. doi:10.1007/s11042-022-12683-9 [9] Nadiia Chepurko, Ryan Marcus, Emanuel Zgraggen, Raul Castro Fernandez, Tim Kraska, and David Karger. 2020. ARDA: automatic relational data augmentation for machine learning. Proc. VLDB Endow. 13, 9 (May 2020), 1373–1387. doi:10. 14778/3397230.3397235 [10] E. F. Codd. 1982. Relational database: a practical foundation for productivity. Commun. ACM 25, 2 (Feb. 1982), 109–117. doi:10.1145/358396.358400 [11] Alexis Cvetkov-Iliev, Alexandre Allauzen, and Gaël Varoquaux. 2023. Relational data embeddings for feature enrichment with background information. Machine Learning 112, 2 (2023), 687–720. [12] Milan Cvitkovic. 2020. Supervised Learning on Relational Databases with Graph Neural Networks. arXiv:2002.02046 [cs.LG] https://arxiv.org/abs/2002.02046 [13] Matthias Fey, Weihua Hu, Kexin Huang, Jan Eric Lenssen, Rishabh Ranjan, Joshua Robinson, Rex Ying, Jiaxuan You, and Jure Leskovec. 2024. Position: relational deep learning - graph representation learning on relational databases. In Proceedings of the 41st International Conference on Machine Learning (ICML’24). JMLR.org, Vienna, Austria, Article 544, 16 pages. [14] Quan Gan, Minjie Wang, David Wipf, and Christos Faloutsos. 2024. Graph Machine Learning Meets Multi-Table Relational Data. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (Barcelona, Spain) (KDD ’24). Association for Computing Machinery, New York, NY, USA, 6502–6512. doi:10.1145/3637528.3671471 [15] Yury Gorishniy, Ivan Rubachev, Valentin Khrulkov, and Artem Babenko. 2021. Revisiting deep learning models for tabular data. In Proceedings of the 35th International Conference on Neural Information Processing Systems (NIPS ’21). Curran Associates Inc., Red Hook, NY, USA, Article 1447, 12 pages. [16] Burton Grad. 2013. Relational Database Management Systems: The Business Explosion [Guest editor’s introduction]. IEEE Annals of the History of Computing 35, 2 (2013), 8–9. [17] Maarten Grootendorst. 2020. KeyBERT: Minimal keyword extraction with BERT. doi:10.5281/zenodo.4461265 [18] Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable Feature Learning for Networks. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (San Francisco, California, USA) (KDD ’16). Association for Computing Machinery, New York, NY, USA, 855–864. doi:10. 1145/2939672.2939754 [19] William L. Hamilton, Rex Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Proceedings of the 31st International Conference on Neural Information Processing Systems (Long Beach, California, USA) (NIPS’17). Curran Associates Inc., Red Hook, NY, USA, 1025–1035. [20] Ziniu Hu, Yuxiao Dong, Kuansan Wang, and Yizhou Sun. 2020. Heterogeneous Graph Transformer. In Proceedings of The Web Conference 2020 (Taipei, Taiwan) (WWW ’20). Association for Computing Machinery, New York, NY, USA, 2704–2710. doi:10.1145/3366423.3380027 [21] Xin Huang, Ashish Khetan, Milan Cvitkovic, and Zohar Karnin. 2020. TabTransformer: Tabular Data Modeling Using Contextual Embeddings. arXiv:2012.06678 [cs.LG] https://arxiv.org/abs/2012.06678 [22] Peng Jia, Shaofeng Cai, Beng Chin Ooi, Pinghui Wang, and Yiyuan Xiong. 2023. Robust and Transferable Log-based Anomaly Detection. Proc. ACM Manag. Data 1, 1, Article 64 (May 2023), 26 pages. doi:10.1145/3588918 [23] Dawei Jiang, Qingchao Cai, Gang Chen, H. V. Jagadish, Beng Chin Ooi, Kian-Lee Tan, and Anthony K. H. Tung. 2016. Cohort query processing. Proc. VLDB Endow. 10, 1 (Sept. 2016), 1–12. doi:10.14778/3015270.3015271

[24] Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017. LightGBM: A Highly Efficient Gradient Boosting Decision Tree. In Advances in Neural Information Processing Systems, I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30. Curran Associates, Inc. https://proceedings.neurips.cc/paper_ files/paper/2017/file/6449f44a102fde848669bdd9eb6b76fa-Paper.pdf [25] Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. arXiv:1609.02907 [cs.LG] https://arxiv.org/abs/ 1609.02907 [26] Hoang Thanh Lam, Beat Buesser, Hong Min, Tran Ngoc Minh, Martin Wistuba, Udayan Khurana, Gregory Bramble, Theodoros Salonidis, Dakuo Wang, and Horst Samulowitz. 2021. Automated Data Science for Relational Data . In 2021 IEEE 37th International Conference on Data Engineering (ICDE). IEEE Computer Society, Los Alamitos, CA, USA, 2689–2692. doi:10.1109/ICDE51399.2021.00305 [27] Hoang Thanh Lam, Tran Ngoc Minh, Mathieu Sinn, Beat Buesser, and Martin Wistuba. 2019. Neural Feature Learning From Relational Database. arXiv:1801.05372 [cs.AI] https://arxiv.org/abs/1801.05372 [28] Hoang Thanh Lam, Johann-Michael Thiebaut, Mathieu Sinn, Bei Chen, Tiep Mai, and Oznur Alkan. 2017. One button machine for automating feature engineering in relational databases. arXiv:1706.00327 [cs.DB] https://arxiv.org/abs/1706. 00327 [29] Changshuo Liu, Lingze Zeng, Kaiping Zheng, Shaofeng Cai, Beng Chin Ooi, and James Wei Luen Yip. [n. d.]. NeuralCohort: Cohort-aware Neural Representation Learning for Healthcare Analytics. In Forty-second International Conference on Machine Learning. [30] Julian John McAuley and Jure Leskovec. 2013. From amateurs to connoisseurs: modeling the evolution of user expertise through online reviews. In Proceedings of the 22nd international conference on World Wide Web. 897–908. [31] Andriy Mnih and Koray Kavukcuoglu. 2013. Learning word embeddings efficiently with noise-contrastive estimation. In Advances in Neural Information Processing Systems, C.J. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger (Eds.), Vol. 26. Curran Associates, Inc. https://proceedings.neurips. cc/paper_files/paper/2013/file/db2b4182156b2f1f817860ac9f409ad7-Paper.pdf [32] Boris Motik, Ian Horrocks, and Ulrike Sattler. 2007. Bridging the gap between OWL and relational databases. In Proceedings of the 16th International Conference on World Wide Web (Banff, Alberta, Canada) (WWW ’07). Association for Computing Machinery, New York, NY, USA, 807–816. doi:10.1145/1242572.1242681 [33] Lakshmi Nivas Nalla and Vijay Mallik Reddy. 2020. Comparative Analysis of Modern Database Technologies in Ecommerce Applications. International Journal of Advanced Engineering Technologies and Innovations 1, 2 (2020), 21–39. [34] Jakub Peleška and Gustav Šír. 2024. Transformers Meet Relational Databases. arXiv:2412.05218 [cs.LG] https://arxiv.org/abs/2412.05218 [35] Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Dorogush, and Andrey Gulin. 2018. CatBoost: unbiased boosting with categorical features. In Advances in Neural Information Processing Systems, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (Eds.), Vol. 31. Curran Associates, Inc. https://proceedings.neurips.cc/paper_files/paper/2018/ file/14491b756b3a51daac41c24863285549-Paper.pdf [36] T Konstantin Rusch, Michael M Bronstein, and Siddhartha Mishra. 2023. A survey on oversmoothing in graph neural networks. arXiv preprint arXiv:2303.10993 (2023). [37] Raphael Sahann, Torsten Möller, and Johanna Schmidt. 2021. Histogram binning revisited with a focus on human perception. arXiv:2109.06612 [cs.HC] https: //arxiv.org/abs/2109.06612 [38] Michael Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne van nbsp;den Berg, Ivan Titov, and Max Welling. 2018. Modeling Relational Data with Graph Convolutional Networks. In The Semantic Web: 15th International Conference, ESWC 2018, Heraklion, Crete, Greece, June 3–7, 2018, Proceedings (Heraklion, Greece). Springer-Verlag, Berlin, Heidelberg, 593–607. doi:10.1007/978-3-319-93417-4_38 [39] Ravid Shwartz-Ziv and Amitai Armon. 2022. Tabular data: Deep learning is not all you need. Information Fusion 81 (2022), 84–90. [40] Shantanu Thakoor, Corentin Tallec, Mohammad Gheshlaghi Azar, Mehdi Azabou, Eva L. Dyer, Rémi Munos, Petar Veličković, and Michal Valko. 2023. Large-Scale Representation Learning on Graphs via Bootstrapping. arXiv:2102.06514 [cs.LG] https://arxiv.org/abs/2102.06514 [41] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. In International Conference on Learning Representations. https://openreview.net/forum?id=rJXMpikCZ [42] Petar Veličković, William Fedus, William L. Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. 2018. Deep Graph Infomax. arXiv:1809.10341 [stat.ML] https://arxiv.org/abs/1809.10341 [43] Minjie Wang, Quan Gan, David Wipf, Zhenkun Cai, Ning Li, Jianheng Tang, Yanlin Zhang, Zizhao Zhang, Zunyao Mao, Yakun Song, Yanbo Wang, Jiahang Li, Han Zhang, Guang Yang, Xiao Qin, Chuan Lei, Muhan Zhang, Weinan Zhang, Christos Faloutsos, and Zheng Zhang. 2024. 4DBInfer: A 4D Benchmarking Toolbox for Graph-Centric Predictive Modeling on Relational DBs. arXiv:2404.18209 [cs.LG] https://arxiv.org/abs/2404.18209

Lingze Zeng, Shaofeng Cai, Changshuo Liu, Zhongle Xie, Yuncheng Wu, and Beng Chin Ooi

[44] Fei Xiao, Shaofeng Cai, Gang Chen, H. V. Jagadish, Beng Chin Ooi, and Meihui Zhang. 2024. VecAug: Unveiling Camouflaged Frauds with Cohort Augmentation for Enhanced Detection. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (Barcelona, Spain) (KDD ’24). Association for Computing Machinery, New York, NY, USA, 6025–6036. doi:10.1145/3637528. 3671527 [45] Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. 2020. Graph Contrastive Learning with Augmentations. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.), Vol. 33. Curran Associates, Inc., 5812–5823. https://proceedings.neurips.cc/paper_files/paper/2020/file/

3fe230348e9a12c13120749e3f9fa4cd-Paper.pdf GFS: [46] Han Zhang, Quan Gan, David Wipf, and Weinan Zhang. 2023. Graph-based Feature Synthesis for Prediction over Relational Databases. arXiv:2312.02037 [cs.LG] https://arxiv.org/abs/2312.02037 [47] Kaiping Zheng, Shaofeng Cai, Horng Ruey Chua, Wei Wang, Kee Yuan Ngiam, and Beng Chin Ooi. 2020. TRACER: A Framework for Facilitating Accurate and Interpretable Analytics for High Stakes Applications. In Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data (Portland, OR, USA) (SIGMOD ’20). Association for Computing Machinery, New York, NY, USA, 1747–1763. doi:10.1145/3318464.3389720

From Schema to Signal: Retrieval-Augmented Modeling for Relational Data Analytics

Related documents