arXiv:2604.15169v1 [cs.LG] 16 Apr 2026
Assessing the Potential of Masked Autoencoder Foundation Models in Predicting Downhole Metrics from Surface Drilling Data 1st Aleksander Berezowski
2nd Hassan Hassanzadeh
Schulich School of Engineering Department of Electrical and Software Engineering University of Calgary Calgary, Canada [email protected]
Schulich School of Engineering Department of Chemical and Petroleum Engineering University of Calgary Calgary, Canada [email protected]
3rd Gouri Ginde Schulich School of Engineering Department of Electrical and Software Engineering University of Calgary Calgary, Canada [email protected]
Abstract—Oil and gas drilling operations generate extensive time-series data from surface sensors, yet accurate real-time prediction of critical downhole metrics remains challenging due to the scarcity of labelled downhole measurements. This systematic mapping study reviews thirteen papers published between 2015 and 2025 to assess the potential of Masked Autoencoder Foundation Models (MAEFMs) for predicting downhole metrics from surface drilling data. The review identifies eight commonly collected surface metrics and seven target downhole metrics. Current approaches predominantly employ neural network architectures such as artificial neural networks (ANNs) and long short-term memory (LSTM) networks, yet no studies have explored MAEFMs despite their demonstrated effectiveness in timeseries modeling. MAEFMs offer distinct advantages through selfsupervised pre-training on abundant unlabeled data, enabling multi-task prediction and improved generalization across wells. This research establishes that MAEFMs represent a technically feasible but unexplored opportunity for drilling analytics, recommending future empirical validation of their performance against existing models and exploration of their broader applicability in oil and gas operations. Index Terms—drilling optimization, downhole metric prediction, masked autoencoder, foundation models, self-supervised learning, transfer learning, neural networks
I. I NTRODUCTION Oil and gas wellbore drilling is a highly data-intensive process that requires continuous monitoring and control of numerous physical parameters to ensure operational efficiency. During drilling operations, vast volumes of time-series data are generated from surface sensors, including weight on bit (WOB), standpipe pressure (SPP), torque (T), and rotational speed (RPM) [1]. These surface measurements are used to estimate downhole conditions that are difficult or costly to measure in real time, such as bottom-hole pressure (BHP) [2],
equivalent circulating density (ECD) [3], and downhole string vibrations [4], [5]. Accurate prediction of these downhole metrics is critical for optimizing drilling performance. In recent years, machine learning (ML) has emerged as a promising approach for inferring downhole metrics solely from surface measurements [6]. A variety of ML models have been developed to map multiple surface features to single downhole targets [2], [4], [5], [7]–[16]. While these approaches have shown utility, they often rely on large labeled datasets for each target metric, struggle to generalize across wells, and are limited in their ability to support multiple predictive tasks simultaneously. Masked Autoencoder Foundation Models (MAEFMs), first introduced by He et al. [17], constitute a fundamentally new class of ML models with capabilities that directly address these limitations. Building on the concept of latentspace representations originally proposed by Rumelhart et al. [18], MAEFMs employ self-supervised pre-training to learn generalizable features from large unlabeled datasets and can subsequently be fine-tuned on relatively small labeled datasets for multiple downstream tasks. This paradigm is particularly well-suited to the oil and gas sector, where drilling operations generate abundant surface sensor data but labeled downhole measurements are scarce and expensive. MAEFMs have demonstrated state-of-the-art performance in both computer vision and time-series forecasting [19], yet their application to downhole metric prediction remains entirely unexplored. The potential to leverage large-scale drilling datasets to learn transferable latent representations, enable multi-task predictions, and reduce dependence on labeled downhole data represents a significant opportunity for research and innovation in the sector.
Given this context, it is important to evaluate the suitability of MAEFMs for oil and gas drilling applications, review existing ML approaches for downhole metric prediction, and identify performance limitations and knowledge gaps. This study aims to investigate the feasibility of MAEFMs for predicting critical downhole metrics, outline directions for future research, and highlight where this emerging technology could offer significant advantages in operational efficiency, safety, and cost reduction. II. R ESEARCH Q UESTIONS The research study is guided by four research questions, each with one or more subquestions, designed to analyze the application of MAEFMs in oil and gas drilling. Q1: What measurements are important to oil and gas drillers? • Q1.1: What surface metrics are collected? • Q1.2: What downhole metrics are important to predict? Drilling is data-intensive, with numerous metrics collected at the surface and a desire to predict many downhole metrics. This question identifies what metrics can be used as inputs to ML models (i.e., metrics collected at the surface) and what metrics are the outputs of ML models (i.e., downhole metrics that are predicted), allowing for greater clarity of the scope of ML in oil and gas drilling. Q2: What methods are used to predict downhole metrics? • Q2.1: What methods are currently used to predict downhole metrics from surface drilling data? This question establishes how ML is currently used to predict downhole metrics, providing background context on current ML applications. Q3: Can MAEFMs be used for oil and gas drilling? • Q3.1: What are MAEFMs? • Q3.2: Are MAEFMs suitable for time-series data such as surface drilling metrics? • Q3.3: Are MAEFMs used in oil and gas drilling currently? This question evaluates the technical feasibility and novelty of MAEFMs in oil and gas drilling. This ensures that MAEFMs are suitable for oil and gas and are able to have an impact on the field. Q4: What should the research direction for MAEFMs in the context of oil and gas drilling be? • Q4.1: What areas could MAEFMs be applied to in oil and gas drilling? This question focuses on defining where MAEFMs could have the most impact, and where research should focus. This helps define the best opportunity for MAEFM application in oil and gas drilling.
III. DATA C OLLECTION M ETHODOLOGY The data collection methodology is divided into three sections: paper search, inclusion and exclusion criteria, and data extraction. The methodology presented is consistent with methodologies presented by Peterson et al. [20], making it a version of a systematic mapping study. A. Paper Search A search query was crafted to find papers that used timeseries surface drilling data to predict time-series downhole metrics using ML. The search query was composed of four parts that were combined with AND operators. The first part ensured domain relevancy, ensuring the search targeted research in the drilling space. The second part narrowed the search to research focused on downhole metrics. The third part further narrowed the search to prediction, ensuring research was focused on predicting downhole metrics. The final part of the search ensured ML was being used to predict the downhole drilling metrics, as opposed to rules based algorithms. This resulted in the following search query: ("drilling data" OR "surface drilling measurements" OR "surface drilling parameters") AND ("downhole metrics" OR "downhole parameters" OR "bottom hole pressure" OR "equivalent circulating density" OR "downhole vibrations" OR "torque and drag") AND ("prediction" OR "estimation" OR "forecasting" OR "time series") AND ("machine learning" OR "neural network" OR "deep learning" OR "regression" OR "artificial intelligence") Additionally, the following search criteria were applied: • Language: Published in English • Date Range: Published in or after January 2015 until November 2025 • Article Type: Article or conference proceedings • Availability: Full text available online The search was conducted across multiple databases, and the following databases returned at least one relevant result: • ScienceDirect • OnePetro • The American Society of Mechanical Engineers Digital Collection • Directory of Open Access Journals • National Library of Medicine The search string and search criteria resulted in 20 results.
B. Inclusion and Exclusion Criteria
•
The retrieved papers were evaluated through a two-stage screening process consisting of a title and abstract review followed by a full-text review. The inclusion and exclusion criteria outlined in Table I were applied at both stages. Papers that did not meet the inclusion criteria or that met any of the exclusion criteria were excluded from the final selection. TABLE I I NCLUSION AND E XCLUSION C RITERIA Inclusion Criteria • •
Exclusion Criteria
Used surface metrics as inputs to an ML model. Predicted one or more downhole metrics as an output for an ML model.
•
•
Focused on non-drilling applications like reservoir modeling or seismic studies. Was a duplicate study.
This review is limited from 2015 to 2025, potentially excluding earlier foundation work or future studies not yet published.
V. Q1: W HAT MEASUREMENTS ARE IMPORTANT TO OIL AND GAS DRILLERS ? Collected data related to drilling metrics are shown in Table III. A. Q1.1: What surface metrics are collected? The collected data on surface metrics used are shown in Figure 1. Metrics that were used in two or fewer papers were considered outliers and discarded.
After applying the criteria, there were 13 papers, as shown in Table II. C. Data Extraction For each included paper, data was systematically extracted into a structured table. The table columns were as followed: • Title: Provides reference point for the paper, ensuring that results are traceable and verifiable. • Surface Metrics Used: Addresses research question 1.1, identifying what surface metrics are collected. • Downhole Metrics Predicted: Addresses research question 1.2, identifying what downhole metrics are predicted. • ML Model Used: Addresses research question 2.1, identifying what techniques are currently used. These columns were selected because they directly align with the research questions that guide the study. Model accuracy was not included because papers used different accuracy metrics (i.e., root mean squared error, mean absolute error, mean squared error, r-squared), predicted different units, and worked with datasets having different statistical characteristics (i.e., mean and standard deviation), making direct comparison of accuracy metrics across papers problematic. The strengths and weaknesses of each method were not included for two reasons. First, assessing methodological strengths and weaknesses requires interpretive judgment, which risks reducing reproducibility. Second, papers tended to focus only on the strengths of their methods and the resulting accuracy, providing few weaknesses to interpret. Furthermore, accuracy and strengths/weaknesses were not included as columns because they do not directly answer any of the proposed research questions. IV. L IMITATIONS This methodology has several limitations: • This review is limited to only research published in English, which excludes non-English research. • Although multiple databases were used, the chosen databases do not cover every possible place that relevant studies may be indexed.
Fig. 1. Bar chart illustrating the frequency of surface value frequencies across the reviewed papers.
There were 8 different surface metrics collected, and they are defined as: • Rotations Per Minute (RPM): The number of rotations the bit makes in a minute. This review does not differentiate between bit RPM and drillstring RPM [21]. • Weight on Bit (WOB): The amount of downward force exerted on the drill bit [21]. • Flowrate (Q): Mud is the fluid that circulates through the drillstring and the wellbore [21]. Flowrate refers to the volume of mud pumped through the drillstring per unit time. It’s usually expressed in gallons per minute or liters per minute [22]. • Rate of Penetration (ROP): Drilling speed, usually expressed in distance per time unit (i.e., feet per hour, meters per second) [21]. • Standpipe Pressure (SPP): The sum of all pressure losses that occur due to fluid friction in the wellbore, which is a sum of loss in annulus pressure, drillstring pressure, bottom hole assembly pressure, and drill bit pressure [23]. • Surface Torque (T): The amount of torque exerted on the drillstring [21]. • Mud Weight (MW): Mud weight refers to the density of the mud [21]. • Depth: The depth of the wellbore, usually in meters or feet [21].
TABLE II S UMMARY OF R EVIEWED PAPERS Paper 1
2
3
4
5
6
7
8
9
10
11
12
13
Title Explainable machine-learningbased prediction of equivalent circulating density using surface-based drilling data [7] Machine Learning Models for Equivalent Circulating Density Prediction from Drilling Data [8] New approach to evaluate the equivalent circulating density (ECD) using artificial intelligence techniques [9] The different member equivalent circulating density prediction model and drilling parameter optimization under narrow density window [10] Bottom hole pressure prediction based on hybrid neural networks and Bayesian optimization [11] Intelligent Model for Predicting Downhole Vibrations Using Surface Drilling Data During Horizontal Drilling [4] Detecting downhole vibrations through drilling horizontal sections: machine learning study [5] A Novel Hybrid Transfer Learning Method for Bottom Hole Pressure Prediction [2] An online hybrid prediction model for mud pit volume in the complex geological drilling process [12] Deep learning approach to prediction of drill-bit torque in directional drilling sliding mode: Energy saving [13] Machine Learning-Based Trigger Detection of Drilling Events Based on Drilling Data [14] Downhole data correction for data-driven rate of penetration prediction modeling [15] Using Trees, Bagging, and Random Forests to Predict Rate of Penetration During Drilling [16]
First and Last Authors Gerald Ekechukwu; Abayomi Adejumo
Author Affiliations Louisiana State University; Oriental Energy Resources Limited
Hany Gamal; Salaheldin Elkatatny
King Fahd University Petroleum & Minerals
Khaled Abdelgawad; Shirish Patil
King Fahd University Petroleum and Minerals
Wanchun Zhao; Peihong Zhai
Chengkai Zhang; Liang Han
Citations 7
Year 2024
of
24
2021
of
50
2019
National Key Laboratory of Green Multi-Resource Collaborative Onshore Shale Oil Exploitation; Northeast Petroleum University China University of Petroleum
0
2025
17
2023
Ramy Saadeldin; Abdulazeez Abdulraheem
King Fahd University Petroleum and Minerals
of
17
2022
Ramy Saadeldin; Salaheldin Elkatatny
King Fahd University Petroleum and Minerals
of
14
2023
Rui Zhang; Chenxing Gong
China University of Petroleum; PetroChina Changqing Oilfield Company China University of Geosciences; Chiba University of Commerce
0
2023
8
2021
Wanpeng Cao; Hamzeh Ghorbani
China Suntien Green Energy Corporation Limited; Islamic Azad University
5
2025
Jie Zhao; Sonny Johnston
Schlumberger
34
2017
Mauro Encinas; Dan Sui
University of Stavanger; Northwestern Polytechnical University University of Texas
24
2022
109
2015
Yang Zhou; Takao Terano
Chiranth Hegde; Ken Gray
RPM was the most frequently used input to ML models, appearing in 12 of the 13 papers. WOB and Q were the next most used, each in 10 papers. ROP and SPP were used in 9 papers, T was used in 6, MW was used in 5, and depth was used in the fewest papers, appearing in just 4. B. Q1.2: What downhole metrics are important to predict? The collected data on predicted downhole metrics are shown in Figure 2. There were 7 different predicted downhole metrics, defined as the following:
Equivalent Circulating Density (ECD): Refers to the effective density exerted by the circulating mud against the formation being drilled. It considers the pressure drop in the annulus above the point being considered. It is important to avoid kicks and losses [24]. • Bottom Hole Pressure (BHP): The summation of all pressures exerted on the bottom of the wellbore. Similar to ECD, it helps to avoid kicks in losses [25]. • Downhole String Vibrations: Refers to vibrations experienced by the drillstring. Too much vibration can lead to damage to the drillstring [26]. • ROP Prediction: Refers to predicting the downhole ROP. •
TABLE III S URFACE M ETRICS AND D OWNHOLE M ETRICS OF R EVIEWED PAPERS # 1
2 3
4
5 6
7 8
9
10
11 12 13
Title Explainable machine-learning-based prediction of equivalent circulating density using surfacebased drilling data Machine Learning Models for Equivalent Circulating Density Prediction from Drilling Data New approach to evaluate the equivalent circulating density (ECD) using artificial intelligence techniques The different member equivalent circulating density prediction model and drilling parameter optimization under narrow density window Bottom hole pressure prediction based on hybrid neural networks and Bayesian optimization Intelligent Model for Predicting Downhole Vibrations Using Surface Drilling Data During Horizontal Drilling Detecting downhole vibrations through drilling horizontal sections: machine learning study A Novel Hybrid Transfer Learning Method for Bottom Hole Pressure Prediction An online hybrid prediction model for mud pit volume in the complex geological drilling process Deep learning approach to prediction of drillbit torque in directional drilling sliding mode: Energy saving Machine Learning-Based Trigger Detection of Drilling Events Based on Drilling Data Downhole data correction for data-driven rate of penetration prediction modeling Using Trees, Bagging, and Random Forests to Predict Rate of Penetration During Drilling
Surface Metrics ROP, WOB, WHO, RPM, T, Q, SPP, MW, TGO
Downhole Metric ECD
Q, ROP, RPM, SPP, WOB, T
ECD
MW, DPP, ROP
ECD
Q, RPM, ROP, WOB, SPP
ECD
Depth, RPM, SPP, Q, TVD, Mud Volume, Sand Content, Back Pressure, Outlet Density, MW Q, SPP, RPM, T, WOB, ROP
BHP
Q, SPP, RPM, T, WOB, ROP
Downhole String Vibrations BHP
Depth, RPM, SPP, Q, Back Pressure, Outlet Flow, TVD, MW, Pool Volume, Outlet Density, Viscosity, Sand Content Depth, RPM, WOB, T, Q, SPP, MW, Conductivity, Temp
Downhole String Vibrations
Mud Pit Volume
ROP, WOB, RPM
Drill Torque
WOB, RPM, Q, ROP, Block Position, Azm, Inc, DiffP Depth, Hookload, WOB, T, RPM, SPP
Drilling Event
WOB, RPM, Q, ROP, Block Position, Azm, Inc, DiffP
ROP Prediction
ROP Prediction
Drilling Event: Any abnormal event in drilling is considered a drilling event, such as when part of the drillstring cannot be rotated or moved vertically (stuck pipe) [24] or an irregular movement caused by a buildup and then sudden drop of force on the drillstring due to it momentarily becoming stuck (stick-slip event) [24] [28]. ECD was predicted in 4 papers, making it the most predicted downhole metric. This is probably due to how important it is to predict accurately and the difficulty in predicting it via rules-based algorithms [3]. BHP, downhole string vibrations, and ROP prediction were all predicted in 2 papers. Finally, mud pit volume, drill torque, and drilling events were each predicted in just 1 paper. •
Fig. 2. Bar chart illustrating the frequency of downhole metric frequencies across the reviewed papers.
Mud Pit Volume: A mud pit is a tank that holds mud on the rig. Mud pit volume refers to how much mud is in that tank. While not technically a downhole metric, the amount of mud currently downhole can be calculated using current mud pit volume [24]. • Drill Torque: Refers to predicting torque of the drillstring at the bottom of the wellbore, while T usually refers to torque at the surface [27].
•
VI. Q2: W HAT METHODS ARE USED TO PREDICT DOWNHOLE METRICS ? Collected data related to prediction methodology are shown in Table IV.. A. Q2.1: What methods are currently used to predict downhole metrics from surface drilling data? The collected data on methods used to predict downhole metrics are shown in Table V. Note that several papers used multiple methods, thus more than 13 total methods are presented.
TABLE IV ML M ODELS U SED IN R EVIEWED PAPERS # 1 2 3 4
5 6 7 8 9 10 11 12 13
Title Explainable machine-learning-based prediction of equivalent circulating density using surface-based drilling data Machine Learning Models for Equivalent Circulating Density Prediction from Drilling Data New approach to evaluate the equivalent circulating density (ECD) using artificial intelligence techniques The different member equivalent circulating density prediction model and drilling parameter optimization under narrow density window Bottom hole pressure prediction based on hybrid neural networks and Bayesian optimization Intelligent Model for Predicting Downhole Vibrations Using Surface Drilling Data During Horizontal Drilling Detecting downhole vibrations through drilling horizontal sections: machine learning study A Novel Hybrid Transfer Learning Method for Bottom Hole Pressure Prediction An online hybrid prediction model for mud pit volume in the complex geological drilling process Deep learning approach to prediction of drill-bit torque in directional drilling sliding mode: Energy saving Machine Learning-Based Trigger Detection of Drilling Events Based on Drilling Data Downhole data correction for data-driven rate of penetration prediction modeling Using Trees, Bagging, and Random Forests to Predict Rate of Penetration During Drilling
ANN, ANFIS ANN, ANFIS ANN
RF, XGBoost, BPNN, CNN, GRU, CNN-GRU ANN ANFIS, RBF, FN, SVM LSTM-DANN SVR-BPNN-LSTM DARN SAX, hierarchical clustering LSTM Boosting, Pruned Tree, Random Forests, Decision Tree
14 of 21 methods, showing them to be preferred in downhole metric prediction.
TABLE V ML M ETHODS AND T HEIR F REQUENCY OF U SE ML Method Used Artificial Neural Network (ANN) Adaptive Neuro-Fuzzy Inference System (ANFIS) Extreme Gradient Boosting (XGBoost) Random Forest (RF) Backpropagation Neural Network (BPNN) Convolutional Neural Network (CNN) Gated Recurrent Unit (GRU) Convolutional Neural Network, Gated Recurrent Unit (CNN-GRU) Radial Basis Function network (RBF) Fuzzy Network (FN) Support Vector Machine (SVM) Long Short-Term Memory (LSTM) Long Short-Term Memory, Domain-Adversarial Neural Network (LSTM-DANN) Support Vector Regression, Backpropagation Neural Network, Long Short-Term Memory (SVR-BPNN-LSTM) Deep Adversarial Neural Network (DARN) Symbolic Aggregate Approximation (SAX) Hierarchical Clustering Boosting Pruned Tree Random Forests Decision Tree
ML Model XGBoost
Times Used 4 3 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
Both conventional ML algorithms (i.e., SVM, Random Forest) and more complex algorithms (i.e., LSTM, CNN) are shown to be used in the literature. Furthermore, a wide range of methods are shown to be used, as only 3 of the 21 methods were used in more than one paper. It is important to note that neural networks of some type (i.e., BPNN, FN) were used in
VII. Q3: C AN MAEFM S BE USED FOR OIL AND GAS DRILLING ? A. Q3.1: What are MAEFMs? The term ”Masked Autoencoder Foundation Model” can be broken down into four parts: model, foundation, autoencoder, and masked [17]. Model refers to MAEFMs being a type of ML model that takes inputs and produces an output based on learned weights. The term ”foundation” in MAEFMs refers to the shared base model that serves as a general representation layer for multiple downstream tasks. These models are pretrained on large-scale datasets to learn broad, domain-agnostic patterns within the data. The resulting representations can then be efficiently fine-tuned on smaller, task-specific datasets, enabling effective adaptation to diverse applications while reducing the need for extensive retraining [29]. An autoencoder forms the foundational training mechanism for MAEFMs and is based on an encoder-decoder architecture. It consists of two main components: an encoder, which compresses high-dimensional input data into a lower-dimensional latent representation, and a decoder, which reconstructs the original input from this latent space. First introduced by Rumelhart et al. [18], the autoencoder is trained to minimize the reconstruction loss between its input and output, thereby learning to preserve the most salient features of the data while discarding redundant or nonessential information.
variate temporal modeling. Since MAEFMs are trained using masked autoencoders, these findings indicate that MAEFMs should also exhibit strong performance on time-series and multivariate data. Consequently, MAEFMs appear not only suitable but potentially advantageous for oil and gas drilling applications, which inherently rely on complex, noisy, and multivariate time-series datasets [33], [34]. C. Q3.3: Are MAEFMs used in oil and gas drilling currently?
Fig. 3. Masked autoencoder training architecture [17].
The term ”masked” refers to the deliberate omission of substantial portions of the input data during training of the autoencoder. By reconstructing the missing components, the model is encouraged to learn underlying structural and temporal patterns within the data, thereby capturing essential features more effectively [30] [17]. A critical aspect not conveyed by the term MAEFM is the use of task-specific headers. After pre-training the foundational model using the autoencoder architecture, the decoder is typically removed and the encoder weights are frozen. Taskspecific layers are then appended to the frozen encoder and trained on the target task, preserving the pretrained foundation while enabling adaptation to multiple downstream tasks. This approach allows efficient utilization of smaller labeled datasets for training the task-specific adapters [31]. B. Q3.2: Are MAEFMs suitable for time-series data such as surface drilling metrics? Li et al. evaluated masked autoencoders across five benchmark datasets (electricity transformer temperature, weather forecasting, exchange rate prediction, influenza case forecasting, and the UCR time-series archive [32]) and found that masked autoencoders consistently outperformed or matched other transformer-based architectures. This demonstrated their robustness and adaptability across diverse time-series domains, suggesting that the architecture can generalize to surface drilling metrics [33]. Tang et al. demonstrated that masked autoencoders are adept at handling multivariate time-series forecasting. Using three benchmark datasets (electricity transformer temperature, electricity consumption load, and weather forecasting), their study showed that multivariate masked autoencoders performed comparably to or better than LSTMs and other transformer models. This indicates that masked autoencoders can capture both temporal and cross-variable dependencies, a key requirement for modeling drilling processes where multiple correlated surface metrics (e.g., RPM, WOB, SPP, and flow rate) interact over time [34]. Li et al. and Tang et al. establish that masked autoencoder architectures are highly effective for time-series and multi-
As shown in Table V, MAEFMs, or more broadly, autoencoders of any type, are not currently used in the reviewed literature for downhole metric prediction. Among the 13 analyzed papers, none used autoencoder-based architectures. As previously stated, the dominant approach was instead neural networks such as ANNs, LSTMs, CNNs. All methods were focused on direct supervised learning for mapping surface metrics (e.g., RPM, WOB, SPP) to downhole targets (e.g., ECD, BHP), rather than the representation learning approaches that autoencoders and MAEFMs provide. VIII. Q4: W HAT SHOULD THE RESEARCH DIRECTION FOR MAEFM S IN THE CONTEXT OF OIL AND GAS DRILLING BE ? A. Q4.1: What areas could MAEFMs be applied to in oil and gas drilling? MAEFMs can be applied to the prediction of any downhole metrics specified in Q1.2, which includes ECD, BHP, downhole string vibrations, ROP prediction, mud pit volume, drill torque, and drilling events. The inputs to such model would be the surface metrics specified in Q1.1, namely RPM, WOB, Q, ROP, SPP, and T. Unlike traditional models that are each trained for a specific prediction task, a MAEFM can leverage large amounts of drilling data to learn generalizable representations of drilling dynamics and predict multiple downhole metrics simultaneously. MAEFMs enable a single generalized model to replace several specialized ones, reducing computational overhead, simplifying deployment, and improving maintainability and reliability on the rig. IX. D ISCUSSION Drilling wellbores for oil and gas generates extensive unlabeled datasets comprised of surface metrics such as RPM, WOB, Q, ROP, SPP, T, MW, and depth. These eight surface metrics are currently utilized as inputs to ML models to calculate seven downhole parameters, namely ECD, BHP, downhole string vibrations, ROP prediction, mud pit volume, drill torque, and drilling events. Traditionally, these downhole metrics have been estimated using deterministic models; however, as shown, ML approaches are increasingly being employed. A wide variety of ML techniques have been applied to downhole metric prediction, with neural networks representing the most frequently used ML models. This diversity highlights the range of modeling strategies currently being explored and indicates that the introduction of novel ML paradigms remains feasible. Nonetheless, the dominance of neural network-based
approaches underscores their suitability and preference for capturing the complex, nonlinear relationships inherent in drilling data. MAEFMs, a neural network-based architecture, have been demonstrated to effectively model time-series data. Given their ability to capture temporal and multivariate dependencies, MAEFMs are well suited for predicting downhole metrics from surface measurements. The underlying masked autoencoder framework enables the model to learn meaningful latent representations even from partially observed or unlabeled sequences, which aligns closely with the characteristics of drilling datasets. A key advantage of MAEFMs is their capacity to leverage large volumes of unlabeled drilling data to develop generalizable representations of drilling dynamics. These representations can then support the prediction of multiple downhole metrics simultaneously. By consolidating several task-specific models into a single generalized model, MAEFMs have the potential to reduce computational requirements, simplify deployment, and enhance maintainability and reliability in operational settings. Overall, MAEFMs are well-positioned to offer a novel approach to downhole metric prediction. They perform effectively when provided with extensive unlabeled surface data alongside smaller labeled datasets containing both surface and downhole measurements. Consequently, MAEFMs can exploit the vast quantities of historical and real-time drilling data already generated in the industry. Furthermore, once pretrained on historical datasets, MAEFMs can be adapted to new wells and emerging drilling tasks, offering a flexible and scalable solution for advanced drilling analytics. X. F UTURE R ESEARCH D IRECTIONS Future research should begin by evaluating whether MAEFMs can predict downhole metrics with accuracy comparable to that of existing ML models. Although current literature suggests that MAEFMs are capable of achieving this, empirical validation remains essential. Initial studies should focus on single-task models to establish a baseline for performance and reliability before progressing to multi-task models that can predict multiple downhole metrics simultaneously. Careful comparisons between MAEFMs and currently employed models, particularly neural networks, will be crucial not only in terms of predictive accuracy but also with regard to generalizability across different wells, formations, and drilling conditions. Additionally, understanding the amount of labeled data required for effective training of MAEFMs relative to other ML approaches will be an important consideration, as these models are expected to require substantially fewer labeled examples due to their ability to leverage large quantities of unlabeled data. If MAEFMs are successfully applied to downhole metric prediction, their applicability may extend to other areas of oil and gas operations where representation learning is advantageous. For example, they could be employed in thermal modeling for oil sands extraction or other processes that are
difficult to monitor in real time. More broadly, any task characterized by limited real-time measurements, a shared set of observable inputs, and challenges for traditional rulebased modeling could potentially benefit from MAEFMs. This positions them as a flexible and scalable tool for predictive analytics across a wide range of complex, data-rich operations in the oil and gas industry. XI. C ONCLUSION This study investigated the potential for MAEFMs to be applied in oil and gas drilling. Thirteen papers from the last 10 years were reviewed to identify commonly used surface metrics, predicted downhole metrics, and existing ML approaches. The review found that RPM, WOB, and Q were the most frequently collected surface metrics, while ECD and BHP were the most frequently predicted downhole metrics. The analysis also revealed that neural network-based approaches dominate the current literature, with methods such as ANNs, LSTMs, and CNNs appearing most frequently. Notably, no reviewed studies utilized autoencoder-based or foundation model architectures. Given the demonstrated success of masked autoencoder models in other time-series domains, MAEFMs represent a technically feasible yet unexplored approach for drilling analytics. Their ability to learn generalized representations across multiple datasets and predict several downhole metrics simultaneously makes them a strong candidate for future research and development in drilling optimization. Future work should focus on pre-training MAEFMs on large-scale drilling datasets, evaluating their performance against established deep learning models, and exploring their ability to generalize across different wells and formations. By highlighting both the current state of ML in drilling and the absence of foundation model research, this study identifies a clear opportunity for innovation at the intersection of subsurface engineering and advanced ML. XII. R EFERENCES [1] F. S. Boukredera, A. Hadjadj, M. R. Youcefi, and H. Ouadi, “Ai-driven optimization of drilling performance through torque management using machine learning and differential evolution,” Processes, vol. 13, no. 5, 2025. [Online]. Available: https://www.mdpi.com/2227-9717/13/5/1472 [2] R. Zhang, X. Song, G. Li, Z. Lv, Z. Zhu, C. Zhang, and C. Gong, “A novel hybrid transfer learning method for bottom hole pressure prediction,” 2023. [3] Drilling Data Based Approach for Equivalent Circulation Density Prediction While Drilling, ser. U.S. Rock Mechanics/Geomechanics Symposium, vol. 57th U.S. Rock Mechanics/Geomechanics Symposium, 06 2023. [Online]. Available: https://doi.org/10.56952/ARMA-20230722 [4] R. Saadeldin, H. Gamal, S. Elkatatny, and A. Abdulraheem, “Intelligent model for predicting downhole vibrations using surface drilling data during horizontal drilling,” Journal of energy resources technology, vol. 144, no. 8, 2022. [5] R. Saadeldin, H. Gamal, and S. Elkatatny, “Detecting downhole vibrations through drilling horizontal sections: machine learning study,” Scientific reports, vol. 13, no. 1, pp. 6204–14, 2023. [6] Z. Tariq, M. S. Aljawad, A. Hasan, M. Murtaza, E. Mohammed, A. El-Husseiny, S. A. Alarifi, M. Mahmoud, and A. Abdulraheem, “A systematic review of data science and machine learning applications to the oil and gas industry,” Journal of Petroleum Exploration and Production Technology, vol. 11, no. 12, p. 4339–4374, Sep 2021.
[7] G. Ekechukwu and A. Adejumo, “Explainable machine-learning-based prediction of equivalent circulating density using surface-based drilling data,” Scientific reports, vol. 14, no. 1, pp. 17 780–9, 2024. [8] H. Gamal, A. Abdelaal, and S. Elkatatny, “Machine learning models for equivalent circulating density prediction from drilling data,” ACS omega, vol. 6, no. 41, pp. 27 430–27 442, 2021. [9] K. Z. Abdelgawad, M. Elzenary, S. Elkatatny, M. Mahmoud, A. Abdulraheem, and S. Patil, “New approach to evaluate the equivalent circulating density (ecd) using artificial intelligence techniques,” Journal of petroleum exploration and production technology, vol. 9, no. 2, pp. 1569–1578, 2019. [10] W. Zhao, Z. Yang, T. Wang, Y. Zhou, W. Song, J. Li, and P. Zhai, “The different member equivalent circulating density prediction model and drilling parameter optimization under narrow density window,” Frontiers in earth science (Lausanne), vol. 13, 2025. [11] C.-K. Zhang, R. Zhang, Z.-P. Zhu, X.-Z. Song, Y.-A. Su, G.-S. Li, and L. Han, “Bottom hole pressure prediction based on hybrid neural networks and bayesian optimization,” Petroleum science, vol. 20, no. 6, pp. 3712–3722, 2023. [12] Y. Zhou, X. Chen, E. F. Fukushima, M. Wu, W. Cao, and T. Terano, “An online hybrid prediction model for mud pit volume in the complex geological drilling process,” Control engineering practice, vol. 111, pp. 104 793–, 2021. [13] W. CAO, D. MEI, Y. GUO, and H. Ghorbani, “Deep learning approach to prediction of drill-bit torque in directional drilling sliding mode: Energy saving,” Measurement : journal of the International Measurement Confederation, vol. 250, pp. 117 144–, 2025. [14] J. Zhao, Y. Shen, W. Chen, Z. Zhang, and S. Johnston, “Machine learning-based trigger detection of drilling events based on drilling data,” 2017. [15] M. A. Encinas, A. T. Tunkiel, and D. Sui, “Downhole data correction for data-driven rate of penetration prediction modeling,” Journal of petroleum science & engineering, vol. 210, pp. 109 904–, 2022. [16] C. Hegde, S. Wallace, and K. Gray, “Using trees, bagging, and random forests to predict rate of penetration during drilling,” 2015. [17] K. He, X. Chen, S. Xie, Y. Li, P. Dollar, and R. Girshick, “Masked autoencoders are scalable vision learners,” 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2022. [18] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,” Nature, vol. 323, no. 6088, p. 533–536, Oct 1986. [19] I. D. Mienye and T. G. Swart, “Deep autoencoder neural networks: A comprehensive review and new perspectives,” Archives of Computational Methods in Engineering, vol. 32, no. 7, p. 3981–4000, Mar 2025. [20] K. Petersen, R. Feldt, S. Mujtaba, and M. Mattsson, “Systematic mapping studies in software engineering,” Proceedings of the 12th International Conference on Evaluation and Assessment in Software Engineering, vol. 17, 06 2008. [21] P. Vertex, “Drilling industry glossary.” [Online]. Available: https://www.pvisoftware.com/drilling-glossary [22] IADC85, “Flow rate,” 12 2010. [Online]. Available: https://iadclexicon.org/flow-rate/ [23] Contact, “What is standpipe pressure?” 06 2024. [Online]. Available: https://contactinstruments.com/what-is-standpipe-pressure/ [24] Schlumberger, “Energy glossary.” [Online]. Available: https://glossary.slb.com/en [25] D. Manual, “Bottom hole pressure in drilling,” 12 2023. [Online]. Available: https://www.drillingmanual.com/bottom-hole-pressure-in-drilling/ [26] N. E. T. Laboratory, “Drilling vibration monitoring and control system,” 2008. [Online]. Available: https://www.netl.doe.gov/node/3788 [27] D. Manual, “Drilling parameters definitions & optimization,” 10 2021. [Online]. Available: https://www.drillingmanual.com/drillingparameters-optimization-performance-oil-gas/ [28] Machine Learning–Based Trigger Detection of Drilling Events Based on Drilling Data, ser. SPE Eastern Regional Meeting, vol. SPE Eastern Regional Meeting, 10 2017. [Online]. Available: https://doi.org/10.2118/187512-MS [29] J. Schneider, C. Meske, and P. Kuss, “Foundation models,” Business & Information Systems Engineering, vol. 66, no. 2, p. 221–231, Jan 2024. [30] R. Bachmann, D. Mizrahi, A. Atanov, and A. Zamir, “Multimae: Multimodal multi-task masked autoencoders,” Lecture Notes in Computer Science, p. 348–367, 2022. [31] N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. de Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly, “Parameter-
efficient transfer learning for nlp,” 2019. [Online]. Available: https://arxiv.org/abs/1902.00751 [32] H. A. Dau, E. Keogh, K. Kamgar, C.-C. M. Yeh, Y. Zhu, S. Gharghabi, C. A. Ratanamahatana, Yanping, B. Hu, N. Begum, A. Bagnall, A. Mueen, G. Batista, and Hexagon-ML, “The ucr time series classification archive,” October 2018. [33] Z. Li, Z. Rao, L. Pan, P. Wang, and Z. Xu, “Ti-mae: Selfsupervised masked time series autoencoders,” 2023. [Online]. Available: https://arxiv.org/abs/2301.08871 [34] P. Tang and X. Zhang, “Mtsmae: Masked autoencoders for multivariate time-series forecasting,” 2022. [Online]. Available: https://arxiv.org/abs/2210.02199