Syntax Meets Semantics: Understanding Scientific Formulae

Syntax Meets Semantics: Understanding Scientific Formulae Yuni Susanti

Moritz Schubotz

FIZ Karlsruhe Berlin, Germany yuni.susanti@fiz-karlsruhe.de

FIZ Karlsruhe Berlin, Germany moritz.schubotz@fiz-karlsruhe.de

arXiv:2608.02457v1 [cs.IR] 3 Aug 2026

Abstract Scientific formulae are a fundamental component of scholarly communication, yet their dual nature – as structured syntax and carriers of semantics – remains underexplored in scholarly information retrieval. Although prior studies show that jointly modeling syntactic and semantic modalities improves retrieval performance, the relationship between their underlying representations has not been systematically investigated. In this work, we empirically study cross-modal correspondence between formula syntax and semantics. We find that their native representation spaces exhibit extremely weak observable correspondence despite strong latent correlation, indicating a substantial representation mismatch between the two modalities. We further evaluate whether this mismatch can be reduced using standard representation learning and alignment techniques. We represent syntactic structure using graph-based encoders and semantic information using text-based encoders, then apply contrastive learning to induce a shared representation space. Results show that the learned alignment substantially improves cross-modal retrieval, suggesting that explicit representation learning can recover correspondence absent from the original representation spaces.

CCS Concepts • Information systems → Information retrieval.

Keywords scholarly IR, scientific formulae, cross-modal alignment

1

Introduction

Scientific formulae play a central role in scientific communication, functioning both as precise symbolic expressions and as dense carriers of conceptual meaning. This duality manifests in two fundamentally different representations of formula. The first is its syntax, typically captured using markup languages such as LaTeX or MathML [10, 19, 27]. For example, the physics formula 𝐹 = 𝑚 ·𝑎 has syntactic representation as an equal (=) node whose left child is the symbol 𝐹 and right child is a multiplication node with operands 𝑚 and 𝑎. The second is its semantic, which corresponds to the naturallanguage description or scientific concept expressed by the formula; in the previous example, that force equals mass multiplied by acceleration, defining Newton’s second law of motion. Similar examples exists for mathematical identities (e.g., (𝑎 +𝑏) 2 = 𝑎 2 + 2𝑎𝑏 +𝑏 2 ), and in chemical formulae such as H2 O. Although these two modalities

This work is licensed under a Creative Commons Attribution 4.0 International License.

jointly determine how humans read, and ultimately understand scientific formula, their relationship remains insufficiently understood from a computational perspective. Many downstream tasks in scholarly IR implicitly rely on the assumption that syntactic similarity approximates semantic similarity. For instance, formula retrieval systems typically rely on tree-based representations [3, 4, 31, 32, 35] or text- and substructure-based overlap [14, 20, 21], while embedding-based methods tokenize formulae directly, implicitly assuming that structural representations capture semantic information [23, 27, 28]. Although prior work has shown that jointly modeling syntactic and semantic information improves retrieval effectiveness[15, 16, 28], comparatively little is known about the intrinsic relationship between their underlying representations. In particular, it remains unclear to what extent these modalities correspond in their native representation spaces and whether any mismatch can be systematically characterized. To address this question, we first conduct a quantitative analysis of the relationship between syntactic and semantic representations using alignment measures spanning instance-level, global, and latent-space correspondence. Our analysis reveals a discrepancy: the two modalities exhibit extremely weak observable correspondence (R@10 < 0.02), yet share strong latent correlations (Top-10 CCA = 0.91), indicating that related information is organized differently in their respective representation spaces. Motivated by this observation, we then investigate whether this mismatch can be reduced through learned cross-modal representations. Using graph-based encoders for formula syntax and text-based encoders for semantics, we evaluate standard contrastive learning as a mechanism for inducing a shared embedding space. The resulting representations substantially improve cross-modal retrieval (up to +0.52 Recall), suggesting that explicit representation learning can recover correspondence that is not directly accessible in the original representation spaces. Overall, this work provides a systematic empirical characterization of the relationship between syntactic and semantic representations of scientific formulae and examines how learned alignment can bridge the gap between them. Our dataset and code are publicly available for future research.1

2

Related Work

Research in formula-related information retrieval has traditionally focused on syntactic structure. Early approaches model formulae and their surrounding textual context as sequences of tokens, employing token-based representations for retrieval and matching [13, 33, 34]. Owing to the inherently hierarchical nature of formulae, many subsequent studies adopt tree-structured representations to explicitly capture their structure [3, 4, 31, 32, 35]. More recent

1 https://github.com/susantiyuni/formula-alignment

Y. Susanti et al.

work learns formula embeddings directly from LaTeX or MathML sequences, or from graph-structured operator trees. These approaches range from symbol-level embeddings [2, 13, 17, 18, 30, 33], including symbol2vec and formula2vec [7], to higher-level graph embeddings that encode global formula structure [29]. Semantic information is typically captured via natural-language descriptions, and transformer-based models, such as BERT [5] and Sentence-BERT (SBERT) [25], have become standard tools for encoding scientific text. Several works [22, 23, 28] extend this by pretraining the models on math-intensive corpora. These models and their variants have been applied to a range of scientific tasks, including math-specific such as math question answering [22] and scientific document retrieval and recommendation [1].

3

Formula Representation

Accurate representation of scientific formulae requires capturing both their syntactic and semantic dimension, as purely syntactic representations may miss conceptual meaning, while purely semantic representations may ignore precise structural constraints. To enable a systematic comparison between these two modalities, we construct separate syntactic and semantic representations for every formula. These representations serve as the basis for the experiments in Section 4.

3.1

Syntactic Representation

Scientific formulae can be interpreted as hierarchical symbolic structures in which operators and functions express relationships among identifiers and numerical constants. To capture these patterns, we represent each formula as an operator tree (OPT) derived from its content MathML encoding. This convert a formula into a tree whose nodes correspond to operators, functions, and symbols, capturing semantic via its syntax [15, 23, 35]. Input Construction. From a MathML expression, we construct a normalized OPT by assigning each element to one of three canonical node types: symbolic (mi, mn, mtext), operator (mo), and functional nodes for structured constructs such as superscripts (msup), subscripts (msub), fractions (mfrac), and square roots (msqrt). Formally, for each MathML node 𝑢, we generate an OPT node OPT(𝑢) = (type(𝑢), value(𝑢), children(𝑢)), where type(𝑢) indicates the canonical category, value(𝑢) stores the associated token (e.g., “𝑥”, “+”, “power”), and children(𝑢) is the ordered list of child nodes. Each child is annotated with a predefined semantic role (e.g., base and exp for superscripts, num and den for fractions), preserving both hierarchical and semantic relations of the formula in the symbolic OPT representation. Graph-based Syntactic Encoder. Each OPT tree representing the syntactic form of formula (see §3.1) is converted into a directed graph 𝐺 = (𝑉 , 𝐸). for graph neural network (GNN) processing. For every node 𝑣 ∈ 𝑉 , we assign a canonical node type {symbol, operator, function}, where each encoded as a learned embedding. For every edge (𝑖, 𝑗) ∈ 𝐸 linking a parent node to one of its children, we attach a two-dimensional edge attribute: edge_attr(𝑖, 𝑗) = (role𝑖 𝑗 , pos𝑖 𝑗 ),

where role𝑖 𝑗 is an integer encoding the semantic role (e.g., base, exp, num, den, sub, sup, arg), and pos𝑖 𝑗 ∈ N denotes the child position index. This design allows the model to encode distinctions between common structural relations in scientific formulae, such as numerator vs. denominator or base vs. exponent. To encode the structural information present in 𝐺, we use a GINE [9]-based message-passing network with node-type embeddings and role- and position-aware edge features to encode formula structure. Each node embedding is initialized as type

ℎ𝑖(0) = emblabel (𝑥𝑖label ) + embtype (𝑥𝑖

).

Each edge (𝑖, 𝑗) receives an embedding 𝑒𝑖 𝑗 = embrole (role𝑖 𝑗 ) + embpos (pos𝑖 𝑗 ). We further apply two stacked GINEConv [9] layers followed by global mean pooling to obtain final syntactic representation 𝑥 struct = 𝑊proj · pool(ℎ𝑖(𝐿) ).

3.2

Semantic Representation

We compute semantic representation for each formula by leveraging textual descriptions of the underlying scientific concepts it expresses. The intuition is that a formula reflects domain knowledge through its associated concepts, often documented in external knowledge sources. Embedding these descriptions provides a highlevel semantic signal that is not captured by symbolic structure. Input Construction. To construct a comprehensive semantic representation, all available textual annotations, (e.g., labels, descriptions) are aggregated into a unified text sequence. Duplicate information is removed to preserve only unique semantic content while maintaining the contextual richness of the representation. Finally, for a formula 𝑖, 𝐷𝑖 = {𝑑𝑖1, . . . , 𝑑𝑖𝑘 } denote the final set of distinct textual annotations associated with formula 𝑖. Transformer-based Semantic Text Encoder. The resulting text sequence is subsequently encoded into dense vector embeddings using a pretrained language model (e.g., SentenceTransformer [25]): 𝑒𝑖 𝑗 = 𝑓encoder (𝑑𝑖 𝑗 ) ∈ R𝑑2 . To handle multiple descriptions per formula, embeddings are aggregated. This yields a fixed-dimensional semantic embedding (𝑖 ) 𝑋 sem ∈ R𝑑2 representing the combined semantic meaning of the formula, capturing contributions from each concept’s description as well as the overall formula description, if exists. As an example, consider the formula pH = − log10 [𝐻 + ], linking the concept pH to the concepts hydronium ion (H+ ), molarity ([.]), and common logarithm (log). Its semantic representation is, (pH)

𝑋 sem = 𝑒 pH + 𝑒 H+ + 𝑒 log + 𝑒 [.] where 𝑒 pH, 𝑒 H+ , 𝑒 log are the embeddings of each of the linked concepts. This aggregation ensures that the formula embedding reflects its multi-concept structure and the domain-specific interpretations of each concept. Figure 1 illustrates the syntactic and semantic representation of the 𝑝𝐻 formula.

Syntax Meets Semantics: Understanding Scientific Formulae

CCA: Structural Semantic Latent Correlations

log₁₀

1.00

Mean = 0.816

common logarithm

[OP] neg

0.95 pH

[FUNC] log

H⁺

[FUNC] [.] (arg)

hydronium ion

[FUNC] power [SYM] H

[⋅]

[SYM] +

Figure 1: Formula pH = − log10 tactic (a) and concept-based semantic (b) representation.

Evaluation Experiments

Having obtained structural and semantic representations for each formula, we next investigate the extent to which these modalities naturally correspond before any learned alignment is applied (§4.2: Quantifying Syntax–Semantic Correspondence). This is followed by the second experiment (§4.3: Learning Syntax–Semantic Alignment), investigating whether standard cross-modal alignment techniques can effectively learn shared space between the two modalities.

4.1

Dataset

We construct a gold dataset of 2,405 scientific formulae paired with their associated scientific concepts and descriptions from Wikidata, using formula-bearing properties (P2534:defining formula and P7235:in defining formula). We use Wikidata because it provides structured, high-quality formula–concept links, making it ideal for studying syntax–semantic correspondence and alignment. We publish the full dataset on our repository (see Page 1).

4.2

0.85 0.80

molarity

[𝐻 + ] with its OPT -based syn-

4

Formulas

[SYM] 10 (base)

0.90 Correlation

[SYM] pH

Quantifying Syntax–Semantic Correspondence

We quantify the syntax–semantic correspondence of scientific formulae using the following measures: 1. Instance-Level Correspondence. To quantify instance-level correspondence between the two modalities, we compute cosine similarity between each formula syntactic and semantic representation. We also perform cross-modal retrieval, measuring whether the correct semantic embedding can be retrieved from a structural query representation. Additionally, we evaluate geometric compatibility using Orthogonal Procrustes alignment [26], which finds the optimal rigid transformation from the syntactic to the semantic space. These measures capture how well the two spaces align without any learned transformation. 2. Global Representational Similarity. To assess whether both modalities encode similar overall relationships, we compute Representational Similarity Analysis (RSA) [12] and Centered Kernel Alignment (CKA) [11]. RSA compares the pairwise similarity structures induced by the two representations, while CKA quantifies alignment invariant to linear transformations and scaling. These metrics provide a global view of representational correspondence beyond individual formula pairs.

0.75 0.70

2

4 6 8 Canonical Component

10

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48

Structural Semantic Similarity Gap RSA Spearman = 0.122 0.175 0.150 0.125 0.100 0.075 0.050 0.025

0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48

[OP] =

Formulas

Figure 2: Raw alignment analysis: CCA latent correlations (left); syntax–semantic similarity gap (right). 3. Shared Latent Structure. To evaluate whether the modalities share underlying latent factors, we apply Canonical Correlation Analysis (CCA) [8]. CCA finds linear projections that maximize correlation between the syntactic and semantic spaces, revealing if a shared latent space exists even when the original representation are misaligned. High CCA indicates both representations encode overlapping information in a linearly recoverable form. 4.2.1 Results and Discussion. Table 1 summarizes the alignment analysis results, with selected results shown in Fig. 2. We summarize the key findings, as follows. Table 1: Raw alignment Weak direct alignment. The mean coanalysis metrics results. sine similarity between paired representation is 0.085 (median 0.094), Metric Value indicating limited direct correspon- Mean Cosine .0847 dence at the individual formula level. Median Cosine .0939 .2114 Cross-modal retrieval further con- Std Cosine firms this mismatch with low accura- RSA Spearman 𝜌 .1215 cies (R@10 < 0.02). Similarly, orthog- Procrustes Error .8299 Linear CKA .1566 onal Procrustes alignment yields a Top-10 CCA .85–.91 high alignment error (0.83), suggest- Mean CCA .8161 ing that the two representations are Recall@1 .0006 .0068 not geometrically compatible under Recall@5 .0125 rigid transformations. Nevertheless, Recall@10 Permutation Test .0847 permutation test shows that cosine alignment is significantly above chance (0.08), implying the presence of weak but non-random correspondence. Modest global similarity. Despite poor direct, instance-level alignment, global similarity analyses reveal moderate correspondence between the two representations. RSA yields a Spearman correlation of 𝜌 = 0.12 (p < 0.001), indicating that formulae that are syntactically similar tend to be semantically similar more often than chance. Consistently, linear CKA reports a score of approximately 0.16, reflecting partial representational overlap that is invariant to scaling and rotation. These suggest that structural encoders capture aspects of semantic, albeit in a distributed and indirect manner. Strong shared latent structure. In contrast to direct and global alignment metrics, CCA reveals substantial shared latent structure between the two modalities. The mean canonical correlation across

Y. Susanti et al.

components is 0.82, with the top canonical dimensions exceeding 0.90. This pronounced latent correspondence indicates that semantic information is strongly embedded within structural representations, but expressed in a transformed subspace that is not directly accessible through raw similarity measures. The weak direct alignment indicates that syntactic and semantic representations, despite describing related formulae, organize information differently in their native embedding spaces. This result does not imply that the two modalities are unrelated; rather, the strong latent correlation suggests that shared information exists but is expressed through different geometric structures. This motivates the second half of the paper, where we investigate whether standard cross-modal alignment techniques can recover this hidden correspondence. Importantly, our goal is not to propose a novel joint embedding method, but to empirically examine the extent to which existing representation alignment approaches can reconcile the geometric mismatch between formula syntax and semantics.

4.3

Learning Syntax–Semantic Alignment

The correspondence analysis in the previous section indicates that syntactic and semantic representations exhibit weak observable alignment despite sharing substantial latent structure. We therefore investigate whether this mismatch can be reduced through learned cross-modal representation alignment. In cross-modal alignment, the goal is to learn a shared space such that different modalities are integrated coherently. The joint representation of formula syntax and semantic should then jointly encode both information while maintaining its coherence, i.e., the similarity in either modality is meaningfully reflected in the space. Contrastive learning (CL) has proven effective for aligning heterogeneous modalities, including image–text (e.g., CLIP) [24] and code–documentation pairs [6]. These methods learn shared embedding spaces where semantically corresponding items are brought closer together. Inspired by this line of work, we investigate contrastive alignment framework for aligning syntactic and semantic representations of formulae. Contrastive Cross-Modal Alignment. Given the structural embedding 𝑥 struct and semantic embedding 𝑥 sem introduced in Section 3, we learn modality-specific projection functions, 𝑧 struct = 𝑊struct 𝑥 struct,

𝑧 sem = 𝑊sem 𝑥 sem,

which map both modalities into a shared latent space. The projection heads are optimized using a symmetric InfoNCE objective, following the contrastive learning framework popularized by CLIP [24]. The objective encourages representations of matched syntax–semantic pairs to be close in the shared embedding space while pushing unmatched pairs farther apart. 𝐵 , Formally, for a batch of paired embeddings {(𝑧𝑖struct, 𝑧𝑖sem )}𝑖=1 we employ the loss function as follow:   struct sem ⊤   sem struct ⊤  1 𝑧 (𝑧 ) 𝑧 (𝑧 ) CE + CE , Lcross = 2 𝜏 𝜏 where 𝜏 denotes contrastive temperature. During training, both modalities are ℓ2 -normalized. To support consistency within each modality, we apply additional intra-modal losses: Lstruct = InfoNCE(𝑧 struct, 𝑧 struct ),

Lsem = InfoNCE(𝑧 sem, 𝑧 sem ).

4.3.1 Experiment Settings. We evaluate the learned alignment on cross-modal retrieval task, considering two retrieval directions: (1) Syntactic → Semantic, where the model retrieves the correct semantic representation given a formula’s syntactic form, (2) Semantic → Syntactic, where the model retrieves the correct syntactic representation given its semantic representation. We evaluate the retrieval performance using Recall@k (𝑘 = 1, 5, 10) and Mean Reciprocal Rank (MRR) metrics. For comparison, we consider the following retrieval-based alignment approaches: (a) BM25, a non-neural retrieval model where each formula is linearized into a token sequence and all associated textual information are aggregated into a single corpus against which BM25 scores are computed. (b) Dual Encoder (CL): both formula syntax and semantic are encoded with the same transformer model and trained via constrastive learning (see §4.3), following general design of prior work on formula retrieval [16]. We implement the contrastive learning on two transformer models: • SBERT (all-mpnet-base-v2), • MathBERT [28]. For each, variants without any contrastive learning (w/o CL) are also included. (c) Graph–Text Encoder CL, which applies contrastive learning directly to align graph-based syntactic representations with transformer-based semantic text encoder; also in prior work[15]. In all experiments, we use the dataset described in §4.1. To account for its relatively small size, we evaluate with 5-fold crossvalidation (0.8/0.2 train–test split) to ensure robust evaluation. We release all code and model training hyperparameters on GitHub. 4.3.2 Results and Discussion. Table 2 summarizes the results of all experiments. BM25, as a non-neural baseline, performs poorly on both retrieval directions (R@10 < 0.07, MRR < 0.05). This is expected, as BM25 relies on exact token overlap and treats formulae as bags-of-words, ignoring syntactic and semantic cues. In our dataset, there is minimal lexical√ overlap between syntactic tokens and textual descriptions (e.g., 2 vs. square root of 2), limiting BM25’s ability to identify correct pairs. Using a pretrained neural embedding model (SBERT/MathBERT) without contrastive learning (w/o CL) improves performance over BM25, though the gain remains modest (R@10 < 0.28, MRR < 0.15). In contrast, neural-based contrastive alignment (both Dual Encoders and Graph–Text CL) significantly outperform BM25 (up to +0.52 R@10) and neural baseline without CL (up to +0.47 R@10); all statistically significant (𝑝 < 0.005). These results confirm that the observed mismatch between syntactic and semantic representations can be effectively reduced through learned shared embedding spaces. Importantly, the improvements are consistent across model architectures, suggesting that the gain primarily stems from explicit alignment rather than specific encoder design choices. From a structural perspective, graph-based encoders provide the strongest performance. In graph-based encoding, node types and role- and position-aware edges preserve hierarchical and relational structure of formula components. This suggest that explicitly modeling hierarchical formula structure is beneficial for cross-modal alignment. Nevertheless, the transformer-based dual encoders with

Syntax Meets Semantics: Understanding Scientific Formulae

Table 2: Cross-modal retrieval results and ablation without contrastive learning (w/o CL), averaged over 5-fold CV. methods

R@1

R@5

R@10

MRR

Syntactic → Semantic BM25 (std ±) SBERT [25] Dual Encoder (w/o CL) (std ±) MathBERT[28] (w/o CL) (std ±) SBERT [25] Dual Encoder (CL) (std ±) MathBERT[28] (CL) (std ±) Graph–Text Encoder CL (std ±)

.011

.030

.052

.033

(.003)

(.005)

(.010)

(.003)

.020

.061

.100

.052

(.008)

(.005)

(.009)

(.006)

.015

.051

.090

.046

(.003)

(.004)

(.013)

(.003)

.224

.450

.556

.334

(.016)

(.028)

(.028)

(.015)

.185

.360

.447

.274

(.005)

(.015)

(.020)

(.007)

.238

.444

.574

.345

(.014)

(.014)

(.014)

(.011)

.017

.047

.072

.041

(.006)

(.009)

(.008)

(.006)

SBERT [25] Dual Encoder (CL) (std ±) MathBERT[28] (CL) (std ±) Graph–Text Encoder CL (std ±)

.082

.201

.280

.149

(.016)

(.028)

(.028)

(.015)

.038

.092

.141

.075

(.011)

(.014)

(.018)

(.012)

.230

.454

.557

.340

(.010)

(.014)

(.012)

(.009)

.179

.361

.443

.270

(.006)

(.012)

(.011)

(.007)

.250

.458

.580

.355

(.019)

(.019)

(.017)

(.018)

SBERT/MathBERT also perform competitively, suggesting that sequential representations already capture a substantial portion of the syntactic information relevant for semantic matching. Notably, performance remains largely symmetric across retrieval directions, indicating that the learned representations support bidirectional cross-modal retrieval. This is consistent with our earlier observation that syntactic and semantic representations share strong latent structure despite weak observable correspondence. Overall, these results suggest that explicit representation learning can recover meaningful alignment between the two modalities, even when such correspondence is not directly present in the original embedding spaces.

5

Acknowledgments This research was supported by the LUMEN project, funded by the European Union’s Horizon Europe Research and Innovation Programme (Grant Agreement No. 101187940). Views and opinions expressed are those of the authors only and do not necessarily reflect those of the European Union or the granting authority.

References

Semantic → Syntactic BM25 (std ±) SBERT [25] Dual Encoder (w/o CL) (std ±) MathBERT[28] (w/o CL) (std ±)

Overall, this work provides an empirical analysis of the relationship between formula modalities and a foundation for studying their integration in scholarly information systems. Future work will investigate larger and more diverse scientific corpora, as well as exploring its application to downstream tasks such as neuro-symbolic reasoning and scientific knowledge discovery.

Conclusion

This work investigates the relationship between syntactic and semantic representations of scientific formulae from a cross-modal perspective. Through a multi-level analysis of natural correspondence, we show that the two modalities exhibit weak observable alignment in their native embedding spaces, despite sharing substantial latent structure. These findings reveal a geometric mismatch between symbolic and semantic representations: related information is encoded across both modalities but organized differently. We further examine whether this mismatch can be reduced through learned representation alignment. Using graph-based structural encoders and text-based semantic encoders, we demonstrate that standard contrastive alignment techniques can substantially improve cross-modal retrieval performance. These results suggest that the correspondence between formula syntax and semantics is not directly accessible from independent embedding spaces, but can be recovered through explicit alignment objectives.

[1] Arman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey, and Daniel Weld. 2020. SPECTER: Document-level Representation Learning using Citation-informed Transformers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Online, 2270–2282. doi:10.18653/v1/2020.acl-main.207 [2] Yifan Dai, Liangyu Chen, and Zihan Zhang. 2020. An N-ary Tree-based Model for Similarity Evaluation on Mathematical Formulae. In Proceedings of the 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC). 2578–2584. [3] Kenny Davila and Richard Zanibbi. 2017. Layout and Semantics: Combining Representations for Mathematical Formula Search. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). 1165–1168. [4] Kenny Davila, Richard Zanibbi, Andrew Kane, and Frank Wm. Tompa. 2016. Tangent-3 at the NTCIR-12 MathIR Task. In Proceedings of the 12th NTCIR Conference on Evaluation of Information Access Technologies (NTCIR). 338–345. [5] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL. [6] Zhangyin Feng, Daya Guo, et al. 2020. CodeBERT: A Pre-Trained Model for Programming and Natural Languages. In EMNLP. [7] Liangcai Gao, Zhuoren Jiang, Yue Yin, Ke Yuan, Zuoyu Yan, and Zhi Tang. 2017. Preliminary Exploration of Formula Embedding for Mathematical Information Retrieval: Can Mathematical Formulae be Embedded like A Natural Language? arXiv preprint arXiv:1707.05154 (2017). [8] Harold Hotelling. 1936. Relations between two sets of variates. Biometrika 28, 3/4 (1936), 321–377. [9] Weihua Hu, Tianyan Xie, Yong Yan, Robin Sommer, Yu Jiang, Li Zhang, George Karypis, Jian Sun, and Jure Leskovec. 2020. Strategies for pre-training graph neural networks. In International Conference on Learning Representations (ICLR). https://openreview.net/forum?id=rJe7CCsEx Introduces the Graph Isomorphism Network with Edge features (GINEConv) for message passing. [10] Michael Kohlhase. 2006. OMDoc: An open markup format for mathematical documents. In International Conference on Artificial Intelligence and Symbolic Computation. Springer, 32–52. [11] Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. 2019. Similarity of neural network representations revisited. In International Conference on Machine Learning (ICML). PMLR, 3519–3529. [12] Nikolaus Kriegeskorte, Marieke Mur, and Peter Bandettini. 2008. Representational similarity analysis–connecting the branches of systems neuroscience. Frontiers in Systems Neuroscience 2 (2008). [13] Kriste Krstovski and David M. Blei. 2018. Equation Embeddings. arXiv:1803.09123 [stat.ML] https://arxiv.org/abs/1803.09123 [14] P. Pavan Kumar, Arun Agarwal, and Chakravarthy Bhagvati. 2012. A Structure Based Approach for Mathematical Expression Retrieval. In Proceedings of the 6th Multi-Disciplinary International Workshop on Artificial Intelligence (MIWAI). 23–34. [15] Ruyin Li and Xiaoyu Chen. 2025. SSEmb: A Joint Structural and Semantic Embedding Framework for Mathematical Formula Retrieval. arXiv:2508.04162 [cs.IR] https://arxiv.org/abs/2508.04162 [16] Zichao Li. 2025. Formula-Text Cross-Retrieval: A Benchmarking Study of Dense Embedding Methods for Mathematical Information Retrieval. In Proceedings of The 3rd Workshop on Mathematical Natural Language Processing (MathNLP 2025), Marco Valentino, Deborah Ferreira, Mokanarangan Thayaparan, Leonardo Ranaldi, and Andre Freitas (Eds.). Association for Computational Linguistics, Suzhou, China, 124–133. doi:10.18653/v1/2025.mathnlp-main.9 [17] Behrooz Mansouri, Douglas W. Oard, and Richard Zanibbi. 2022. Contextualized Formula Search Using Math Abstract Meaning Representation. In Proceedings of

Y. Susanti et al.

the 31st ACM International Conference on Information & Knowledge Management (CIKM). 4329–4333. [18] Behrooz Mansouri, Shaurya Rohatgi, Douglas W. Oard, Jian Wu, C. Lee Giles, and Richard Zanibbi. 2019. Tangent-CFT: An Embedding Model for Mathematical Formulas. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). 11–18. [19] Bruce Miller and Deyan Ginev. [n. d.]. LaTeXML: A LaTeX to XML/HTML/MathML Converter. https://math.nist.gov/~BMiller/LaTeXML/ Accessed: 2025Oct-22. [20] Bruce R. Miller and Abdou S. Youssef. 2003. Technical Aspects of the Digital Library of Mathematical Functions. Annals of Mathematics and Artificial Intelligence 38, 1–3 (2003), 121–136. [21] Jozef Mišutka and Leo Galamboš. 2008. Extending Full Text Search Engine for Mathematical Content. Towards Digital Mathematics Library (2008), 55–67. [22] Vít Novotný and Michal Štefánik. 2022. Combining Sparse and Dense Information Retrieval. In Proceedings of the Working Notes of CLEF 2022, Guglielmo Faggioli, Nicola Ferro, Allan Hanbury, and Martin Potthast (Eds.). CEUR-WS, 104–118. http://ceur-ws.org/Vol-3180/paper-06.pdf [23] Shuai Peng, Ke Yuan, Liangcai Gao, and Zhi Tang. 2021. MathBERT: A Pre-Trained Model for Mathematical Formula Understanding. arXiv:2105.00377 [cs.CL] https://arxiv.org/abs/2105.00377 [24] Alec Radford, Jong Wook Kim, Chris Hallacy, et al. 2021. Learning Transferable Visual Models From Natural Language Supervision. In ICML. [25] Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics. https://arxiv.org/abs/1908.10084 [26] P. H. Schönemann. 1966. A generalized solution of the orthogonal Procrustes problem. Psychometrika 31 (1966), 1–10. [27] Moritz Schubotz, André Greiner-Petter, Philipp Scharpf, Norman Meuschke, Howard S. Cohl, and Bela Gipp. 2018. Improving the Representation and Conversion of Mathematical Formulae by Considering their Textual Context. In

Proceedings of the 18th ACM/IEEE on Joint Conference on Digital Libraries (Fort Worth, Texas, USA) (JCDL ’18). Association for Computing Machinery, New York, NY, USA, 233–242. doi:10.1145/3197026.3197058 [28] Jia Tracy Shen, Michiharu Yamashita, Ethan Prihar, Neil T. Heffernan, Xintao Wu, and Dongwon Lee. 2021. MathBERT: A Pre-trained Language Model for General NLP Tasks in Mathematics Education. CoRR abs/2106.07340 (2021). arXiv:2106.07340 https://arxiv.org/abs/2106.07340 [29] Yujin Song and Xiaoyu Chen. 2021. Searching for Mathematical Formulas Based on Graph Representation Learning. In Proceedings of the 14th Conference on Intelligent Computer Mathematics (CICM). 137–152. [30] Abhinav Thanda, Ankit Agarwal, Kushal Singla, Aditya Prakash, and Abhishek Gupta. 2016. A Document Retrieval System for Math Queries. In Proceedings of the 12th NTCIR Conference on Evaluation of Information Access Technologies (NTCIR). 346–353. [31] Zichao Wang, Andrew S. Lan, and Richard G. Baraniuk. 2021. Mathematical Formula Representation via Tree Embeddings. In Proceedings of the Third International Workshop on Intelligent Textbooks (iTextbooks@AIED 2021) (CEUR Workshop Proceedings, Vol. 2895), Sergey Sosnovsky, Peter Brusilovsky, Richard Baraniuk, and Andrew Lan (Eds.). CEUR-WS.org, 121–133. https://ceur-ws.org/Vol2895/paper02.pdf Published online July 04, 2021. [32] Zichao Wang, Mengxue Zhang, Richard G. Baraniuk, and Andrew S. Lan. 2021. Scientific Formula Retrieval via Tree Embeddings. In 2021 IEEE International Conference on Big Data (Big Data). 1493–1503. doi:10.1109/BigData52589.2021. 9671942 [33] Michihiro Yasunaga and John D Lafferty. 2019. Topiceq: A joint topic and mathematical equation model for scientific texts. In AAAI, Vol. 33. 7394–7401. [34] Ke Yuan, Liangcai Gao, Yuehan Wang, Xiaohan Yi, and Zhi Tang. 2016. A mathematical information retrieval system based on RankBoost. In 2016 IEEE/ACM Joint Conference on Digital Libraries (JCDL). 259–260. [35] Wei Zhong and Richard Zanibbi. 2019. Structural Similarity Search for Formulas Using Leaf-Root Paths in Operator Subtrees. In Proceedings of the 41st European Conference on Information Retrieval (ECIR). 116–129.

Related documents

• Reason, Reward, Refine: Step-Level Errors Corrections with Structured Feedback for Physics Reasoning in Small Language Models#343514
• Extraction and evaluation of formulaic expressions used in scholarly papers#239027
• Test-Time Scaling for Scientific Equation Discovery#792910
• Test-Time Scaling for Scientific Equation Discovery#788906
• Second Order Science: Putting the Metaphysics Back Into the Practice of Science#748776