SSH-Net: A Deep Neural Network for Predicting Failure Time Distribution Functions under Competing Risks with arXiv:2606.20451v1 [stat.ML] 18 Jun 2026
Application to GPU Data Jie Min1 , Yueyao Wang2∗, and Mengkun Chen3 1 Department of Mathematics & Statistics, University of South Florida, Tampa, FL 2 School of Statistics and Data Science, Zhejiang Gongshang University, Hangzhou, China 3 Department of Statistics, Virginia Tech, Blacksburg, VA
Abstract Competing risks are commonly observed in engineering fields, and can bring challenges to time-to-event data modeling when the application scenarios are complicated. Recently, deep neural networks have received great attention for prediction with competing risks, due to their flexibility and high learning capability. However, the complexity of neural network structure brings extra difficulty in hyperparameter tuning based on different data inputs. Additionally, when an engineered system has complex physical structures with multiple hierarchical levels, treating all structural levels as a single group of inputs may fail to capture critical information. To address the issues, we propose a Structured Segmented Hazard Deep Neural Network (SSH-Net) for failure time prediction under cause-specific competing risks framework. Our approach associates neural network structure with data structures, and allows different covariate groups to impact the failure prediction through separate sub-networks. The neural network is constructed based on a cause-specific competing risks model. The SSH-Net outputs cause-specific hazard functions, and utilizes the penalized log-likelihood as the loss function. The prediction accuracy of SSH-Net is ∗
Corresponding Author. Email: [email protected]
1
validated through simulation studies by evaluating the Brier score, the area under receiver operating characteristic curves (AUC), and the root mean square error (RMSE) of the predicted cause-specific cumulative incident function. We further demonstrate the model’s ability to predict failure time distribution functions using the Titan GPU failure time data. Key Words: Competing Risks; Deep Neural Networks; Failure Time Distribution; Smoothness Penalty; Survival Analysis
1
Introduction
1.1
Motivation
Failure time modeling and prediction for time-to-event data are commonly seen in medical research including cancer studies (Shahraki et al. 2023) and analysis of Alzheimer’s progression (Sharma et al. 2021), as well as the reliability analysis of engineering systems and their component (Man and Zhou 2018). In such studies, more than one causes of failures may exist, where observing the failure time from one cause prevents observing the failure from other causes, leading to the setting of competing risks. With competing risks, distribution functions of the failure times for different failure types can be used to provide probabilistic predictions. In modeling time-to-event data with competing risks, classical survival models typically rely on parametric assumptions regarding the distribution of failure times or certain structure of covariate effects. During recent years, to relax such parametric assumptions and to leverage the prediction power of deep learning models, several neural networks are proposed for prediction with competing risks. One drawback of using neural networks is the prediction performance can be sensitive to the number of nodes and layers in a general proposed neural network structure, given different data. Selecting the number of nodes and layers are commonly included in hyperparameter tuning based on grid search. When the neural network structure cannot be clearly explained, the searching grid is determined based on past experience, leading to the risk that the best set of nodes and layers are not included in the grid. Additionally, when covariates contain special features, such as a hierarchical structure based on the system, it can be difficult for a general neural network to capture these information. To solve these issues, we propose a Structured
2
Segmented Hazard Deep Neural Network (SSH-Net) for predicting the distribution functions of failure times under cause-specific competing risks framework, in which part of the neural network structure is associated with the data structure. The SSH-Net structure has its explanation based on the data, which can be used in hyperparameter tuning. In addition, covariates with different features are processed separately in SSH-Net with different sub-networks, which can improve the prediction performance. A detailed introduction of the neural network structure is presented in Section 2.2. We illustrate the usage of SSH-Net using the failure time prediction of graphics process units (GPUs) inside supercomputers. As GPUs within supercomputers are intensively used in training models with large data, the reliability of GPUs is of interest during recent years. GPUs in supercomputers can fail due to multiple types of errors, including connection error and memory error, which implies the presence of competing risks. Additionally, the failure times of GPUs can be influenced by the locations of cabinets and the positions of GPUs inside cabinets (Ostrouchov et al. 2020), resulting in covariates with hierarchical levels and spatial location information that require extra attention in prediction. One popular dataset used in GPU reliability analysis is the GPU failure time data in the Cray XK7 Titan supercomputer. The performance of SSH-Net is demonstrated on the Titan GPU data.
1.2
Related Work
It is well known that neural networks have the ability to learn non-linear relationships between covariates and response without parametric assumptions. To leverage this advantage, multiple deep neural networks have been developed to model time-to-event data. For example, Cox regression without parametric assumptions is extended through neural networks (Katzman et al. 2018 and Kvamme et al. 2019). Recurrent neural networks are proposed to learn the probability mass function (pmf) of discretized survival time (Giunchiglia et al. 2018) or the hazard rate of discretized survival time (Ren et al. 2019). Additionally, neural networks using piecewise constant hazard functions as outputs are developed in Fornili et al. (2014) and Kvamme and Borgan (2021) to avoid discretizing continuous time. While the above mentioned methods can handle only one type of failure, deep learning methods are also explored in estimating the distribution functions with competing risks. Among 3
the rich literature, one of the popular neural networks for competing risks is DeepHit, proposed by Lee et al. (2018). The model treats the failure times as discrete random variables and uses log-likelihood loss together with a penalization on incorrect ordering pairs to train the neural network in learning the cause-specific pmf of different failure modes. Also treating failure times as discrete random variables, Gupta et al. (2019) combined Long Short-Term Memory (LSTM) layers with fully connected layers to improve the prediction. On the other hand, by adding extra ordinary differential equation (ODE) solver layers onto a similar neural network structure proposed by Lee et al. (2018), Huang et al. (2021) treated the failure times as continuous random variable, and used log-likelihood loss in training. Also using ODEs and log-likelihood loss, Danks and Yau (2022) proposed to output the cause-specific cumulative incident functions (CIFs), together with the failure probability for different failure types. Additionally, Wang and Sun (2022) modified transformers to learn the cause-specific hazard functions with piecewise constant assumption and applied inverse propensity score to deal with extremely imbalanced censor rate across multiple failure types, with a loss function constructed based on multi-task learning. More recently, Jeanselme et al. (2023) proposed the Neural Fine Gray (NFG), which is a computationally efficient monotonic neural network to predict the CIFs. Other than the neural networks that output values of distribution functions, Rahman et al. (2021) proposed to use pseudo values calculated based on the Aalen-Johansen estimators as neural network outputs. Additionally, Lee et al. (2019) extended the neural network structure in Lee et al. (2018) to account for longitudinal covariates. Hong et al. (2022) utilized contrastive learning in prediction with longitudinal covariates. Compared to the existing neural network models, our proposed SSH-Net has a more interpretable neural network structure associated with the data structure, which can be used to assist hyperparameter tuning. The SSH-Net avoids discretizing continuous time by using piecewise-constant hazard functions. Furthermore, a smoothness penalty term is added to the loss function to avoid overfitting, which was not considered in previous literature using piecewise-constant hazard functions. The high prediction accuracy for SSH-Net is shown using criteria including the Brier score, the area under receiver operating characteristic curves (AUC), and root mean squared errors (RMSE) of CIFs, based on the simulation data and the Titan GPU data introduced in Section 4.1.
4
On the application side, the reliability of GPUs in supercomputers has received increasing attention in recent years due to the importance of GPUs in training models on large-scale datasets. In the current literature, the performance of neural network models are commonly compared based on cancer research data such as the Surveillance, Epidemiology, and End Results Program (SEER) dataset published by National Cancer Institute. In this paper, the performance of neural network prediction is first evaluated based on a GPU failure time data, which have a special covariate structure, including spatial location information and multiple hierarchical levels. In analyzing the reliability of GPUs inside supercomputers, Min et al. (2025) developed an accelerated failure time model with spatially correlated random effects and mixture distribution, assuming Weibull or Lognormal distributions in the failure time modeling. Clark et al. (2025) further associated multiple types of distance functions with the accelerated failure time model. Wang et al. (2025) investigated the usage of computationally efficient variational inference method in the GPU lifetime analysis. Additionally, Nie et al. (2018) evaluated performance of machine learning models such as support vector machine and gradient boosting decision tree to predict the single-bit errors, without competing risks. Other statistical models related to time-to-event analysis with competing risks and spatial location information including Hesam et al. (2018) and Momenyan and Poorolajal (2020), where the location information is treated as random effects and is added to the hazard functions.
1.3
Contributions
The main contributions of this paper are as follows. First, we propose SSH-Net, a novel neural network structure that can accommodate complex data structures, such as hierarchical covariates structure, and associates the network structure with underlying data structure, thereby easing hyperparameter tuning. Also, we implement a log-likelihood loss with a smooth penalty to handle continuous failure time and reduce overfitting. Additionally, we design simulation studies that can provide fair comparison among several candidate neural networks without favoring their assumptions. The simulation study design also enables the usage of RMSE to better evaluate the performance of the neural networks, while existing literature focuses on time-dependent Brier score and the weighted time-dependent AUC. The simulation study results illustrate the advantage of SSH-Net compared to candidate models. Lastly, we apply the 5
SSH-Net to analyze the Titan GPU data, showing the prediction of CIFs, the important factors that influence the GPU failure times, and the relative risk among multiple GPU failure types.
1.4
Overview
The rest of the paper is arranged as follows. Section 2 presents the details of the proposed SSH-Net model, including the data notation and cause-specific model, and the SSH-Net network structure, loss function, and hyperparameter selection. Section 3 demonstrates the performance of SSH-Net using simulation studies. Section 4 applies the SSH-Net to analyze the Titan GPU data. Section 5 summarizes the conclusions and future research areas.
2
Methodology
2.1
Preliminaries
In a time-to-event data analysis, for each unit i, let Ti denote the continuous failure time, Ci denote the censoring time, and ti be the observed value of Tei = min(Ti , Ci ), which is the observed failure time or censoring time. Let xi = (x′ic , x′il )′ = (xic1 , . . . , xicpc , xil1 , . . . , xilpl )′ be the covariates corresponding to unit i. In particular, let xil denote the set of “global” covariates and xic denote the covariates at a lower hierarchical level in a system with a hierarchical structure. The pl and pc are the dimensions of xil and xic , respectively. Let Gi denote the failure type of unit i, where Gi = 1, . . . , K, and K is the total number of types of failures. Let δik denote the event indicator. That is, δik = 1 if unit i fails because of failure mode k, and 0 otherwise. Note that for a censored unit, δik = 0 for all k = 1, . . . , K. In summary, we denote the time-to-event data as t = {ti , xi , δik }, i = 1, 2, . . . , n, where n is the total number of units, and k = 1, 2, . . . , K. In a cause-specific competing risks model, the overall hazard function for unit i is defined as Pr(Ti ≤ t + δ | Ti > t) , δ→0 δ
hi (t) = lim
6
and the cause-specific hazard function for failure mode k is Pr(Gi = k, Ti ≤ t + δ | Ti > t) . δ→0 δ
hi (t, k) = lim
In particular, for a continuous failure time, the cause-specific hazard can be expressed as fi (t, k) , Si (t)
hi (t, k) =
(1)
where fi (t, k) is the cause-specific probability density function (pdf) such that dFi (t, k) , dt
fi (t, k) =
and Fi (t, k) is the CIF such that Fi (t, k) = Pr(Gi = k, Ti ≤ t). The Si (t) in (1) is the overall survival function, and Si (t) = 1 − Fi (t), where Fi (t) = Pr(Ti ≤ t) is the overall cumulative distribution function (cdf). In particular, the relationship between cause-sepcific distribution functions and overall distribution functions are hi (t) =
K X
hi (t, k) , Fi (t) =
K X
Fi (t, k), and Si (t) =
Si (t, k).
k=1
k=1
k=1
K X
Note that the failure probability for failure mode k is P (Gi = k), and Si (t, k) = P (Gi = k) − Fi (t, k). Also, the overall survival can be expressed as Si (t) =
K X
Si (t, k) =
k=1
K X k=1
Z t hi (u, k)du . exp − 0
The loss function for the proposed SSH-Net is constructed based on the cause-sepcific hazard and the overall survival function, and will be discussed in detail in Section 2.3.
2.2
The proposed Neural Network Structure
The structure of the proposed SSH-Net is shown in Figure 1. Specifically, let Pr(Gi = k, T ≤ t + δ | T > t, xi ) δ→0 δ
h(t, k | xi ) = hi (t, k) = lim
(2)
be the cause-specific hazard function for failure mode k of unit i with covariates xi . We assume that h(t, k | xi ) is piecewise constant on the time interval (0, τ ), where τ is the length of the study time. For example, τ can be the maximum observed failure or censor time in the training 7
sub-network 𝑴𝟏
…
𝑶𝟏
…
…
sub-network 𝐄
𝒙𝒊𝒄 shared layer 𝒙𝒊
S
output layer
sub-network 𝑴𝐊 𝒙𝒊𝒍
…
𝑶𝑲
sub-network 𝐋 ReLU
Tanh
Concatenation
Figure 1: Visualization of the neural network structure of SSH-Net, where E represents the subnetwork for xic , L represents the sub-network for xil , S represents the shared layer, M1 , . . . , MK represent the sub-networks for multiple failure modes, and O1 , . . . , OK represent output layers, which are the constant log cause-specific hazards. dataset, or a time that is slightly longer than the maximum observed time. Equally spaced time bins are then built based on the time interval (0, τ ), which are, (0, ∆k ), (∆k , 2∆k ), . . . , ((Jk − 1)∆k , τ ), where ∆k is the length of the bins for failure type k, and Jk is the total number of bins for failure type k. Note that the different number of bins across failure types can lead to extra flexibility in estimation, and can reduce the influence from imbalanced censor rates across the failure types. Although the best Jk needs to be chosen using hyperparameter selection, the range of the Jk grid can be determined based on common knowledge related to the study, as discussed in Section 2.4. With the piecewise constant assumption, the segmented cause-specific hazard function in (2) can be written as h(t, k | xi ) =
Jk X
1{(j − 1)∆k < t ≤ j∆k }hjk|xi , j = 1, 2, . . . , Jk ,
(3)
j=1
where hjk|xi is the constant hazard. We let the logarithm of hjk|xi be the outputs of the neural network. 8
To deal with special covariate features, different sub-networks are built for the “global” covariate xil and the lower hierarchical level covariate xic . In particular, we let L represent the sub-network for xil , and E represents the sub-network for xic . In L and E, the number of nodes are associated with pl and pc , which are the dimensions of xil and xic , respectively. In each sub-network, two layers with the same number of nodes are used to learn both the main effects and the second order interactions among the covariates using fully connected layers. Higher order interactions are not considered as they have weaker impact on the failure times. A common shared layer S is then added on top of E and L. We propose to use relatively small number of nodes in S compared to pc + pl , so that S serves as a method for “data reduction”, and is used to extract useful information from E and L. The information obtained by S is passed to the fully connected sub-networks M1 , . . . , MK , which are sub-networks for the K failure types. In a sub-network Mk , the nodes can be viewed as latent variables that influence the cause-specific hazard functions in (2). With the piecewise constant assumption, we expect the number of latent variables stay in the same scope as the number of bins Jk . If there are many latent variables, it is less likely that the cause-specific hazard remains constant within a bin. On the other hand, if there are few latent variables, a smaller number of bins should be used. Therefore, we propose to use a value close to the median of the grid for Jk as the number of nodes in Mk . The activation functions in E and L are chosen to be ReLU to avoid vanishing gradient problem, and the activation functions in S and M1 , . . . , MK are Tanh, which can potentially reduce the bias in neural network prediction as the output of Tanh is symmetric about zero (Li et al. 2020).
2.3
Loss Function of the SSH-Net
To avoid wiggly behavior of the cause-specific hazard functions, we use the minus penalized log likelihood as the loss function, where the penalty is added to the difference of adjacent constant hazards hjk|xi . This method is also used with P-splines, as illustrated in Marra and Radice (2020). Let oik = (o1k|xi , . . . , oJk k|xi )′ , where oik is the SSH-Net output and ojk|xi = log(hjk|xi ),
9
and let hik = (h1k|xi , . . . , hJk k|xi )′ . The minus penalized log likelihood loss is calculated as l=−
K X n X
[δik log[h(ti , k | xi )] − H(ti , k | xi ) − λk hik ′ Dhik ] ,
(4)
k=1 i=1
where H(t, k | xi ) is the cause-specific cumulative hazard function such that Z t h(u, k | xi )du. H(t, k | xi ) = 0
Based on the expression of (3) for h(ti , k | xi ), ⌊t/∆k ⌋
H(t, k | xi ) =
X
∆k hjk|xi + (t − ⌊t/∆k ⌋∆k ) h⌊t/∆k ⌋k|xi ,
t > ∆k ,
j=1
and H(t, k | xi ) = th1k|xi
0 < t ≤ ∆k ,
where ⌊t/∆k ⌋ denotes the floor of t/∆k . The D matrix in (4) is a Jk × Jk matrix used to penalize the differences among adjacent constant hazard rates. In particular, 1 −1 0 . . . 0 0 −1 2 −1 . . . 0 0 D= .. .. .. .. . .. . . . . . 0 0 0 . . . 1 −1 Additionally, λk is the hyperparameter related to the smooth penalty of failure type k.
2.4
Hyperparameter Selection
The three important sets of hyperparameters in the proposed neural network model are the number of bins Jk , the smooth penalty λk , and the number of layers in each sub-network Mk . The hyperparameter selection can be performed using penalized log likelihood on the P validation set based on grid search. In selecting the λk , we use λk = λ∗k × ( ni=1 δik )1/3 , as the hyperparameter related to the penalty should increase with the number of observations in the dataset. Since failed units include more information related to the censored units, we use the number of failed units when adjusting the λk . In this paper, the grid for λ∗k is {0, 0.5, 1, 1.5}. For the number of layers of the sub-networks for failure modes, we consider a grid of {1, 2, 3}. Additionally, as the structure of SSH-Net associates with the data structure, the choice of grid of Jk can be guided based on the underlying system and the study period τ , which eases the process of hyperparameter tuning. Section 4.2 illustrates the details of hyperparameter tuning of Jk using the Titan GPU data as an example. 10
3
Simulation Studies
We use simulation to illustrate the performance of SSH-Net and show its advantage related to candidate neural networks models. In particular, although we delay the introduction of the Titan GPU data to Section 4.1, we generate simulated data that are similar to the Titan GPU data to provide a realistic simulation setting. Note that the design of simulation provides fair comparison among candidate models and does not favor the SSH-Net, since the data is generated based on a latent failure time model introduced in Appendix A, rather than a cause-specific model. The true distribution functions from the latent failure time model can be transformed to match the outputs of SSH-Net and other neural network models, as shown in Appendix B. Multiple simulation scenarios with different number of units and different censor rates are considered to explore the influence of the number of units and censor rates on the prediction performance. Under each scenario, 100 datasets are generated. A detailed introduction about the simulation setting is presented in Section 3.1. We compare the performance of the proposed SSH-Net with the popular neural network DeepHit (Lee et al. 2018) and the recently proposed NFG model (Jeanselme et al. 2023). Hyperparameter selection is performed based on the validation set for each of the neural network. For DeepHit, the number of layers for cause-specific and shared sub-networks are chosen from {1, 2, 3, 5}, number of nodes is selected from {50, 100, 200, 300}, and the penalty for ranking loss is selected from {0, 0.01, 0.05}. For NFG, the number of hidden layers for the embedding network, sub-distribution network, and balancing network are chosen from {1, 2, 3, 4}, and the number of nodes are chosen from {50, 100}. For SSH-Net, the number of bins is selected from {10, 20, 30, 50}, the number of layers in Mk is selected from {1, 2, 3}, and the number of nodes in Mk and the output layer is fixed at 50 based on the structure of the dataset. A detailed explanation of the chosen grids for SSH-Net is deferred to Section 4.2, which presents the real GPU reliability application, as the simulated data are generated based on the Titan GPU data. Additionally, the smooth penalty is selected from {0, 0.5, 1, 1.5}. Table 2 shows the optimization settings in training all the neural networks. To evaluate the performance of the neural networks, we use three criteria, which are the RMSE of CIFs, the time-dependent weighted Brier score, and the weighted time-dependent AUC that is used to measure the concordance of the ranking between predicted failure times 11
Table 1: Number of GPU units and average censor rates for the two failure types in different simulation scenarios. Shut Down Date
2020-01-01
2019-01-01
2018-01-01
2017-01-01
n = 10,000 Failure Type 1 Censor Rate
84.02 %
89.32 %
93.85%
97.42%
Failure Type 2 Censor Rate
62.37 %
70.53%
81.49 %
92.35 %
n = 5,000 Failure Type 1 Censor Rate
81.50 %
87.07 %
92.32%
96.49%
Failure Type 2 Censor Rate
62.06%
70.24%
80.85 %
91.83%
and the observed times (Antolini et al. 2005). The calculation of evaluation criteria is discussed in Section 3.2.
3.1
Simulation Settings
In the simulation, we study the influence of two factors on the neural network performance, which are the number of units n, and the censor rates of the two types of failures. In particular, 8 scenarios are considered, and Table 1 shows the number of units and the average censor rates of the 100 datasets for each simulation scenario. To generate datasets similar to the Titan GPU data while maintaining a fair comparison among candidate models, we employ the latent failure time model proposed by Min et al. (2025), which does not inherently favor the cause-specific framework used in SSH-Net. A brief introduction about the latent failure time model, the true parameter setup for the simulation, and the relationship between the latent failure time model and the cause-specific model is presented in the Appendix A. In simulation, the covariates xi and insert dates of GPUs are sampled directly from the Titan GPU dataset. The censoring rate is controlled by varying the shutdown date of the supercomputer, where a later shutdown date results in a lower censoring rate, as shown in Table 1. In each simulation run, the dataset is divided into training set, validation set, and test set, with proportions 64%, 16%, and 20%, respectively, stratified by the failure type labels, which are censored, Failure Type 1, and Failure Type 2.
12
Table 2: Optimization setting for SSH-Net, NFG, and DeepHit in training. Neural Network Batch Size
3.2
Optimizer
Learning Rate
Epochs
Early Stop
SSH-Net
100
AdamW
1e − 3
100
10
NFG
100
Adam
{1e − 3, 1e − 4}
200
3
DeepHit
100
Adam
1e − 4
500
5
Evaluation Criteria
To evaluate the prediction performance, the RMSE for failure mode k is calculated as RMSEk (t) =
ntest 1 X
(
ntest i=1
N i2 1 Xh F̂j (t, k | xi ) − F (t, k | xi ) N j=1
)1/2 ,
where ntest is the number of units in the test set, N = 100 is the total number of simulated datasets under each simulation scenario, F̂j (t, k | xi ) is the predicted CIF, and F (t, k | xi ) is the true CIF. The predicted CIFs for neural networks are calculated according to formulas in Appendix B. To show the performance of SSH-Net compared to NFG and DeepHit, the ratio of the RMSEs are calculated, which are, RatioSSH-NFG,k (t) =
RMSESSH,k (t) RMSESSH,k (t) , and RatioSSH-DeepHit,k (t) = . RMSENFG,k (t) RMSEDeepHit,k (t)
A ratio smaller than 1 indicates the SSH-Net has better performance in terms of RMSE. Additionally, the commonly used time-dependent weighted Brier score and time-dependent AUC are calculated. The Brier score is obtained based on Gerds and Schumacher (2006), and is used to estimate the mean squared error (MSE) in predicting CIF for right-censored data when truth is unknown. In particular, for one simulated dataset, the weighted Brier score for failure type k at time t is calculated as, Brierk (t) =
ntest n 1 X
ntest i=1
io2
h
1{ti > t} − 1 − F̂ (t, k | xi )
where ω(ti , t) =
1{ti ≤ t} δik Ŝc (t− i )
+
1{ti > t} Ŝc (t)
ω(ti , t),
,
and Ŝc (t) is the Kaplan-Meier estimate for the survival function of the censoring time Ci based on the test set. The weighted AUC is calculated based on Blanche et al. (2013), and can 13
be used to evaluate the concordance of the ranking between predicted failure times and the observed times. Let ri = F̂ (t, k | xi ), rj = F̂ (t, k | xj ) , and ŵ(t) = 1/F̂c (t), the weighted AUC for failure type k at time t is calculated as AUCk (t) = n test n test X X i=1 j=1 n test X
"
(
1 {ti ≤ t, δik = 1} ŵ(ti ) 1{tj > t} ŵ(t) + 1 tj ≤ t, δik = 0,
K X
) δik′ > 0
#
ŵ(tj ) 1{ri > rj }
k′ =1
! (n " ( ) #) . K test X X 1{ti ≤ t, δik = 1} ŵ(ti ) 1{tj > t} ŵ(t) + 1 tj ≤ t, δik = 0, δik′ > 0 ŵ(tj )
i=1
k′ =1
j=1
The Brier and AUC are evaluated on a time grid (0.3, 0.4, . . . , 0.9, 0.95) × max (ttest ), where ttest consists of all the observed times in the test set.
3.3
Results
The SSH-Net and NFG are applied for all the eight simulation scenarios, and DeepHit is applied only to scenarios with shut down date January 1, 2020, and January 1, 2018, because of its longer computation time. Figure 2 shows the prediction for one test unit from a simulated dataset with n = 10, 000 and shut down date January 1, 2020. The cause-specific hazard predictions are illustrated in Figure 2(a)(b), and the CIF predictions are in Figure 2(c)(d). Because of the piecewise hazard assumption, the predicted hazard function from SSH-Net is a step function. The smooth penalty in the loss function prevents wiggly behavior of the hazard functions. Both SSH-Net and NFG successfully learn the trend of the hazard functions, while DeepHit fails to show the trend because it discretizes the failure time and predicts the failure probabilities at every discrete time point. For Failure Type 1, the prediction from NFG is better at the tail part. For Failure Type 2, SSH-Net predicts the mode of the hazard function more accurately. Additionally, DeepHit predicts the Failure Type 1 CIF more accurately compared to the Failure Type 2 CIF. Figure 3 shows the SSH-NFG RMSE ratio and SSH-DeepHit RMSE ratio. The results indicate that SSH-Net performs better than NFG and DeepHit in terms of RMSE, as the ratio at all evaluation time points for all scenarios are below 1. Additionally, NFG has lower RMSE compared with DeepHit. The SSH-NFG RMSE ratio first increases over time and then decreases slightly, indicating that SSH-Net performs better at smaller time values. The ratio reaches the 14
Table 3: Brier score and AUC averaged on the evaluation time grid and across 100 simulated datasets. The smallest Brier score and the largest AUC are highlighted in bold. The scores are reported in units of 1e-2. Shut Down Date
2020-01-01
2018-01-01 2020-01-01
n = 10,000
2018-01-01
n = 5,000
Failure Type 1
SSH-Net
4.65
1.87
5.60
2.32
Avg Brier (%)
NFG
4.78
1.91
5.82
2.40
DeepHit
5.24
2.61
6.43
3.20
Failure Type 2
SSH-Net
9.71
4.38
10.36
4.62
Avg Brier (%)
NFG
9.90
4.46
10.76
4.79
DeepHit
13.98
9.68
13.70
9.55
Failure Type 1
SSH-Net
82.17
81.46
80.19
81.11
Avg AUC (%)
NFG
79.80
77.29
75.85
73.91
DeepHit
79.39
78.40
76.92
77.54
Failure Type 2
SSH-Net
78.10
77.07
77.31
79.14
Avg AUC (%)
NFG
77.48
76.38
75.51
77.20
DeepHit
71.14
74.34
70.24
74.64
highest value around 4 years, indicating the performance of SSH and NFG are more similar around that time. Figure 4 shows the RMSE of SSH-Net across all the simulation scenarios. The RMSE increases through time, showing that the prediction is less accurate for large time values. Overall, the RMSE are reasonably small as it is calculated based on the test data. Table 3 shows the Brier and AUC for the three models for four simulation scenarios. The scores are averaged across the evaluation time grid mentioned in Section 3.1, the test set, and all N simulated datasets. The results of SSH-Net and NFG for all eight scenarios are provided in Supplementary Section 2. A lower Brier score indicates smaller MSE, and a higher AUC indicates better classification performance. For all the scenarios, SSH-Net has the lowest Brier score and highest AUC, indicating its strong performance. Note that the difference among Brier scores are small, as the CIFs are between 0 and 1.
15
2.0
Failure Type 2 Cause−specific Hazard
Failure Type 1 Cause−specific Hazard
2.0
1.5
1.0
0.5
0.0
1.5
1.0
0.5
0.0 0
2
4
6
0
2
Time
4
6
Time Truth
SSH−Net
DeepHit
(a) Failure Type 1
NFGray
(b) Failure Type 2 0.6
0.4
0.4
Failure Type 2 CIF
Failure Type 1 CIF
0.3
0.2
0.2
0.1
0.0
0.0 0
2
4
6
0
2
Time
4
6
Time Truth
SSH−Net
(c) Failure Type 1
DeepHit
NFGray
(d) Failure Type 2
Figure 2: Predicted cause-specific hazards and CIFs for one test unit based on one simulated dataset with n = 10, 000 and shut down date January 1, 2020.
16
RMSE Ratio
0.75
Shut Down Date 2020−01−01 2019−01−01 2018−01−01 2017−01−01
1.00
0.75
Candidate Model DeepHit NFG
RMSE Ratio
1.00
0.50
0.25
0.50
0.00 0
2
4 Time
6
0
(a) Failure Type 1, n = 10, 000 Shut Down Date 2020−01−01 2019−01−01 2018−01−01 2017−01−01
1.00
0.75
Candidate Model DeepHit NFG
0.50
0.25
0.00
0.00 2
4 Time
4 Time
6
6
Shut Down Date 2020−01−01 2019−01−01 2018−01−01 2017−01−01 Candidate Model DeepHit NFG
0.50
0.25
0
2
(b) Failure Type 2, n = 10, 000
RMSE Ratio
RMSE Ratio
0.75
Candidate Model DeepHit NFG
0.25
0.00
1.00
Shut Down Date 2020−01−01 2019−01−01 2018−01−01 2017−01−01
0
(c) Failure Type 1, n = 5, 000
2
4 Time
6
(d) Failure Type 2, n = 5, 000
Figure 3: The SSH-NFG RMSE ratios and SSH-DeepHit RMSE ratios across all simulation scenarios. A ratio smaller than 1 indicates the proposed SSH-Net has better performance.
17
0.06
Shut Down Date 2020−01−01 2019−01−01 2018−01−01 2017−01−01
0.04
0.04 RMSE
RMSE
0.03
Shut Down Date 2020−01−01 2019−01−01 2018−01−01 2017−01−01
0.02
0.02 0.01
0.00
0.00 0
2
4 Time
6
0
(a) Failure Type 1, n = 10, 000 0.05
6
Shut Down Date 2020−01−01 2019−01−01 2018−01−01 2017−01−01
0.06
0.04
0.03
RMSE
RMSE
4 Time
(b) Failure Type 2, n = 10, 000
Shut Down Date 2020−01−01 2019−01−01 2018−01−01 2017−01−01
0.04
2
0.02 0.02 0.01
0.00
0.00 0
2
4 Time
6
0
(c) Failure Type 1, n = 5, 000
2
4 Time
6
(d) Failure Type 2, n = 5, 000
Figure 4: RMSE of SSH-Net calculated across multiple simulation scenarios.
18
4
Data Application
This section introduces the usage of SSH-Net on GPU reliability analysis inside supercomputers. The Titan GPU data is introduced. In the analysis, the CIFs, together with the overall cdf of GPUs are predicted. The relative risk among multiple GPU failure types are further discussed. The results can also show the influence factors of GPU failure times.
4.1
The Titan GPU Data
The Cray XK7 Titan supercomputer GPU operation data published in Ostrouchov et al. (2020) includes time-to-event data of over 30,000 GPUs collected for nearly 7-year-long period. There are two main reasons that trigger the failure of a GPU. One is the Double Bit Error (DBE), which identifies a double bit flip to prevent the use of corrupted data. The other is the Off the Bus (OTB) that flags the loss of the host CPU connection to the GPU. One GPU node can fail multiple times for various reasons during the Titan supercomputer service time. As the later failures can be triggered by the first failure (Ostrouchov et al. 2020), our analysis focuses on the time to the first failure for each GPU. Specifically, the cleaned version of the dataset as in the study by Min et al. (2025) is used. In the cleaned Titan GPU data, there are 19,319 GPU units, with 1127 OTB failures and 3093 DBE failures, leading to 94.2% OTB censor rate and 84.0% DBE censor rate. There are two sets of covariates that can influence the GPU failure times, and the sets are associated with different levels of the Titan supercomputer’s physical structure. Specifically, the first set of covariates represents the “global” spatial location of GPUs in the server room. The Titan system has 8 rows by 25 columns of cabinets, with all GPUs regularly arranged inside these 200 cabinets. Since GPU reliability has been linked to heat dissipation following the work by Ostrouchov et al. (2020), we use the row and column position of cabinets as the first set of covariates to indicate the spatial location of each GPU. The locations of cabinets serve as a proxy for the regional temperature affecting GPU failure. The second set of covariates is linked to the hierarchical structure of GPUs within each cabinet. Every cabinet has a vertical stack of 3 cages, each with 8 slots. Every slot contains 4 nodes, with a single GPU installed per node. Figure 5 presents the hardware structure of GPUs within each cabinet. We consider the cage,
19
Slot Cage 2
0 1 2 3 4 5 6 7
Cage 1
0 1 2 3 4 5 6 7
Cage 0
0 1 2 3 4 5 6 7
Node
Slot
1
0
2
3
Figure 5: The hierarchical structure of Titan supercomputer within each cabinet. There are 4 GPU nodes in each slot, 8 slots in each cage and 3 cages in each cabinet. slot, and node position of each GPU as the second set of covariates that impact GPU lifetime. Figure 6 and Figure 7 visualize the Kaplan-Meier estimates of the Titan GPU data, based on cage and node positions and 6 different cabinet locations. We can see that for both OTB and DBE failures, GPUs in the top cage have a lower survival probability than those in the bottom cage. The survival probability also varies across different spatial locations. These visualizations justify the inclusion of both within cabinet hierarchies and cabinet locations as inputs to our neural network model.
4.2
Failure Probability Prediction for the Titan GPU Data
In this section, we apply the proposed SSH-Net to analyze the Titan GPU Dataset. In applying SSH-Net, we set pl = 200 based on the number of cabinet locations, and pc = 12 based on onehot encoding of the categorical covariates cage, slot, and node, after eliminating collinearity. To select the hyperparameters, recall that Jk is related to the constant hazard assumption, and the grid can be selected based on the background of the study. For the Titan GPU data, we use a grid {10, 20, 30, 50, 70}. As the Titan supercomputer operated for around 8 years, using 70 bins leads to the assumption that the instantaneous failure risk keeps unchanged during around 41 days, which is reasonable for a stable system. Less number of bins correspond to a 20
1.00
0.75
0.75
DBE Survival probability
OTB Survival probability
1.00
0.50
0.50
0.25
0.25
0
2
4
6
0
2
Time in Years
4
6
Time in Years
Cage=0, Node=0
Cage=0, Node=3
Cage=1, Node=2
Cage=2, Node=1
Cage=0, Node=1
Cage=1, Node=0
Cage=1, Node=3
Cage=2, Node=2
Cage=0, Node=2
Cage=1, Node=1
Cage=2, Node=0
Cage=2, Node=3
Figure 6: Kaplan-Meier estimates and the 95% CIs for the OTB and DBE survival probabilities based on the cage and node covariates in the Titan GPU data.
1.0
DBE Survival probability
OTB Survival probability
1.00
0.95
0.90
0.85
0.80
0.9
0.8
0.7
0.6 0
2
4
6
0
Time in Years
2
4
6
Time in Years
Column 1 Row 0
Column 16 Row 3
Column 18 Row 0
Column 1 Row 7
Column 16 Row 7
Column 25 Row 0
Location
Figure 7: Kaplan-Meier estimates for the OTB and DBE survival probabilities based on the 6 different cabinet locations in the Titan GPU data.
21
Table 4: The selected SSH-Net structures for the 5-fold Titan GPU datasets. |E|
|L|
|S| # of layers in Mk
J1
J2
λ∗1
λ∗2
Fold 1
3
20
30
1
1.5
Fold 2
1
20
20
0
0
1
20
20
0
0.5
Fold 4
1
20
20
1.5
0.5
Fold 5
1
20
30
0
1.5
Fold 3
12
200
30
longer time interval of the constant hazard. Additionally, with the smooth penalty term, the difference between adjacent hazard rates can be penalized to values close to 0, leading to a longer interval with approximately constant hazard. To avoid overfitting, we let Jk ≤ Jk∗ if Pn Pn i=1 δik∗ . That is, the failure type with less observed failures has smaller number i=1 δik ≤ of bins. As suggested in Section 2.2, number of nodes in Mk is related to the grid of Jk , and 50 nodes is used for the Titan GPU data. The NFG and DeepHit are also applied to the dataset, with hyperparameter tuning using the same grids as those in Section 3, and the Brier score and AUC of the three neural networks are compared. Additionally, 5-fold cross validation is performed for model comparison. Let | · | denote the number of nodes in each layer of the sub-networks, Table 4 shows the number of nodes and layers, and other hyperparameter values in the SSH-Net selected for the 5-fold data. Dropout is applied except for the final output layer, and the dropout rate used is 0.4. Figure 8 shows the Brier score and AUC from the three models, evaluated at times presented in Section 3.1. The Brier scores increase through time for all the three models, and the smallest Briers are obtained using SSH-Net. Also, SSH-Net has the highest AUC across most of the evaluation points. The result shows that SSH-Net has the best performance, which aligns with the conclusion in the simulation study. Figure 9 shows the predicted CIFs on different cage positions and cabinet locations for Fold 1 test data. In particular, Figure 9 (a)(b) present the predicted CIFs for GPU units at cage 0, cage 1, and cage 2, all with node position 3 and cabinet column location 7. The slot and cabinet row locations are not fixed. The results indicate the strong influence of cage on the OTB and DBE failure probabilities. With other covariates fixed, GPUs in cage 2 have the
22
highest failure risk, and GPUs in cage 0 have the lowest failure risk. The cage influence on GPU failure times is driven by the cooling system in the Titan’s supercomputer, as introduced in Ostrouchov et al. (2020) and Min et al. (2025). The results indicate SSH-Net successfully captures the cage effect. Figure 9 (c)(d) present the predicted CIFs for GPUs at cabinet column 24 row 8 (location 1), column 15 row 1(location 2), and column 15 row 7 (location 3), with cage positions all fixed at 2. The slot and node positions are not fixed. The results indicate the effect of cabinet locations on the OTB and DBE failure risks. Other than the CIFs, the overall cdf can be used to predict the failure times. Additionally, conditioning on a failure time t, the probability Pr(C = k | T < t) for k = 1, 2 can be used to understand the relative risk among multiple failure types. Figure 10 (a) shows the overall cdf for three different GPU units in Fold 5. In terms of uncertainty quantification, Monte-Carlo dropout (Gal and Ghahramani 2016) with dropout rate 0.4 and number of samples 1,000 is used to calculate the pointwise 95% confidence intervals and the mean of the predictions, shown as the colored areas and colored lines in the figure. The covariates for Unit 1, Unit 2, and Unit 3 are cage 0, slot 7, node 2, column 15, row 8; cage 1, slot 8, node 3, column 11, row 1; and cage 2, slot 1, node 2, column 3, row 4, respectively. The predicted overall cdf is the smallest for Unit 1, and is the largest for Unit 2. Figure 10 (b) shows the relative risk of OTB versus DBE, which is calculated as Pr(C = 1|T < t)/ Pr(C = 2|T < t). The results show that the risk for OTB is higher when the time t is approximately between 2.5 years and 3.5 years, for all three units.
5
Conclusions and Areas for Future Research
The failure time distribution prediction of complex systems with competing risks can be a challenging task. In this study, we propose a deep learning model named SSH-Net to predict failure time distribution functions, including cause-specific hazards and CIFs, with competing risks. The proposed model associates the neural network structure with data structures, providing better interpretability of the network and enabling more efficient hyperparameter tuning informed by the data. Additionally, SSH-Net allows covariates from different system hierarchical levels to influence the underlying failure process separately through dedicated sub-networks.
23
0.15
0.3
0.10 Brier
Brier
0.2
0.05
0.1 Method SSH−Net DeepHit NFG
0.00 2
3
4 Time
5
Method SSH−Net DeepHit NFG
0.0
6
2
3
(a) OTB
4 Time
5
6
(b) DBE
0.9 0.9
0.8
AUC
AUC
0.8
0.7
0.7
0.6 Method SSH−Net DeepHit NFG
0.6
2
3
4 Time
5
Method SSH−Net DeepHit NFG
0.5
0.4
6
2
(a) OTB
3
4 Time
5
6
(b) DBE
Figure 8: The Brier score and AUC for SSH-Net, NFG, and DeepHit, calculated based on 5-fold cross validation.
24
0.25 0.6 0.20
0.15 CIF
CIF
0.4
0.10 0.2 0.05
0.00
0.0 0
2
4
6
0
2
Time Cage
Cage 0
4
6
Time
Cage 1
Cage 2
Cage
(a) OTB
Cage 0
Cage 1
Cage 2
(b) DBE
0.6 0.2
CIF
CIF
0.4
0.1 0.2
0.0
0.0 0
2
4
6
0
2
Time Loc
Location 1
Location 2
4
6
Time Location 3
Loc
(c) OTB
Location 1
Location 2
Location 3
(d) DBE
Figure 9: The predicted CIFs for OTB and DBE failures based on different cage positions and cabinet locations.
25
1.00
3
0.75
Relative Risk
Overall Cdf
2
0.50
1 0.25
0.00
0 0
2
4
6
0
2
Time Unit
1
4
6
Time 2
3
Unit
(a) Overall cdf
1
2
3
(b) Relative risk
Figure 10: The predicted overall cdf and relative risk for three different GPU units. Furthermore, a smooth penalty term is incorporated into the loss function to prevent wiggly behavior in the cause-specific hazard function outputs. We demonstrate that our model outperforms alternatives in most scenarios using both simulation studies and the real-world Titan GPU dataset. There are several contributions in this work. While some neural network models exist for predicting failure times with competing risks, they primarily focus on medical and clinical studies. For complex engineering systems, existing networks lack the flexibility to let different modules or hierarchies impact the failure time separately. Our model addresses this by processing different input groups through sub-networks before their effects are passed to a shared layer, allowing covariate groups to impact the failure separately. Secondly, hyperparameter tuning for neural networks is often time-consuming, especially with large datasets. Since we use piecewise constant hazard functions as outputs, the hyperparameters, bin number Jk and smooth penalty λk for the failure mode k, are associated directly with the duration of constant instantaneous failure rate in a period. Therefore, in real-world applications, researchers can pre-determine reasonable ranges of Jk and λk based on domain knowledge to reduce the computational cost and to improve the prediction accuracy. Thirdly, existing studies rarely evaluate model perfor-
26
mance with simulations with known underlying hazard functions. By designing a simulation that mimics the Titan GPU data, each model’s performance against the true distribution functions are compared. From the simulation study, the proposed SSH-Net outperforms the two candidate models in terms of RMSE, Brier score, and AUC. There are several future research directions. In this study, two connected layers are used to model the spatial location effect inside a supercomputer and their interactions. It would also be interesting to adopt the Gaussian process based neural network structure to capture the spatial dependencies among GPUs at different spatial locations for potential prediction improvement. Besides, as the sensor technology advanced, it is common to have time-varying sensory variables for complex systems. It would be beneficial to extend our work to allow time varying covariate inputs in the neural network model.
Acknowledgments The authors acknowledge Dr. Yili Hong for his valuable suggestions that helped improve the paper. The authors also thank the University of South Florida Research Computing for providing computational resources.
Appendix A
Latent Failure Time Model and Simulation Setting
To generate datasets that are similar to the Titan GPU data without favoring cause-specific model, we utilize the latent failure time model proposed in Min et al. (2025). For a unit j at cabinet location l, the latent failure time model assumes Tlj = min(Tlj1 , Tlj2 ), where Tljk is the latent failure time from failure type k. For notation simplicity, let i denote the unit with location index l and inside location index j. Following Min et al. (2025), the latent failure times in the Titan GPU data are assumed to
27
follow Weibull distributions. In particular, we let 1 log(ti ) − µl1 L f1 (ti |xj , wl1 ) = ϕSEV , ξ1 ti ξ1 1 log(ti ) − µl21 1 log(ti ) − µl22 L f2 (ti |xj , wl2 ) =λ ϕSEV + (1 − λ) ϕSEV , ξ21 ti ξ21 ξ22 ti ξ2 and µl1 = µ1 + x′jL β 1 + wl1 , µl21 = µ2 + x′jL β 2 + wl2 , µl22 = µl21 + η, where l = 1, 2, . . . , nl , and j = 1, 2, . . . , nj . The xjL are levels of categorical variables cage, slot, and node, and is different from xi in the neural network model, which contains additional information of cabinet location. The nl is the number of locations, which is set to be 200 in simulation, and nj is the total number of units at location l. The ϕSEV denotes the pdf of the standard smallest extreme value (SEV) distribution, µ1 , µ21 , and µ22 denote the location parameters in the SEV distribution, ξ1 , ξ21 , and ξ22 denote the scale parameters in the SEV distribution, β 1 and β 2 denote the coefficients correspond to the two different failure types, wl1 and wl2 denote normally distributed random effects that are used to capture the spatial information from cabinet location, λ denotes the mixture probability for Failure type 2, and η denotes the difference between the two location parameters for Failure type 2. The use of the mixture of two Weibull distributions for Failure type 2 is further explained in Min et al. (2025). Additionally, the random effects are assumed to follow a multivariate normal distribution. In particular, let wl = (wl1 , wl2 )′ , and w = (w′i , . . . , w′n )′ . It is assumed that w ∼ MVN(0, Σ ⊗Ω), ! σ12 ρ12 σ1 σ2 dslc dslr Σ= , Ω = (ωsl ) = exp − − , νc νr ρ12 σ1 σ2 σ22 where σ1 and σ2 are standard deviations of the random effects for two failure types, ρ12 is the correlation coefficient between the random effects for the two failure types, dslc and dslr are the column and row distances between location s and location l, and νc > 0 and νr > 0 are the parameters in the exponential correlation function. The values of true parameters are selected based on the estimates in Min et al. (2025) and are shown in Table 5.
28
Table 5: True parameter values used in the simulation. µ1
ξ1
µ2
ξ21
ξ22
λ
η
σ1
σ2
ρ12
νc
νr
2.10
0.19
1.66
0.14
1.50
0.60
9.00
0.13
0.11
0.50
0.60
0.50
β1 (0.67, 0.27, 0.04, 0.03, 0.05, 0.04, 0.07, 0.01, −0.01, −0.29, −0.30, −0.06)′ β2 (0.57, 0.23, 0.04, 0.08, 0.07, 0.09, 0.06, 0.06, 0.03, −0.24, −0.26, −0.06)′
B
Transforming from Latent Failure Time Model to Neural Network Outputs
Let F1L (t | xjL , wl1 ) and F2L (t | xjL , wl2 ) be the cdf for the latent failure times Tlj1 and Tlj2 . Let S L (ti1 , ti2 | xjL , wl ) = Pr(Ti1 > ti1 , Ti2 > ti2 | xjL , wl ) be the joint survival function. According to Theorem 3.1 in Crowder (2001), the cause-specific hazard defined in Section 2.1 is associated with the joint survival function, which is, ∂ log S L (ti1 , ti2 | xjL , wl ) |ti1 =t,ti2 =t ∂tik 1 ∂S L (ti1 , ti2 | xjL , wl ) =− L |ti1 =t,ti2 =t , k = 1, 2, S (ti1 , ti2 | xjL , wl ) ∂tik
hL (t, k | xjL , wl ) = −
where L denotes the latent failure time model, distinguishing it from the predicted cause-specific hazards using neural networks. Note that conditioning on the random effects w, the latent failure times are independent. Therefore, S L (ti1 , ti2 | xjL , wl ) = 1 − F1L (ti1 | xjL , wl1 ) [1 − F2L (ti2 | xjL , wl2 )], ∂S L (ti1 , ti2 | xjL , wl ) = −f1L (ti1 | xjl , wl1 )[1 − F2L (ti2 | xjL , wl2 )], ∂ti1 ∂S L (ti1 , ti2 | xjL , wl ) = −f2L (ti2 | xjl , wl2 )[1 − F1L (ti1 | xjL , wl1 )], ∂ti2
29
and hL (t, 1 | xjL , wl ) =
f2L (t | xjL , wl2 ) f1L (t | xjL , wl1 ) L , h (t, 2 | x , w ) = , jL l 1 − F1L (t | xjL , wl1 ) 1 − F2L (t | xjL , wl2 )
(5)
f L (t, 1 | xjL , wl ) = f1L (t | xjL , wl1 )[1 − F2L (t | xjL , wl2 )],
(6)
f L (t, 2 | xjL , wl ) = f2L (t | xjL , wl2 )[1 − F1L (t | xjL , wl1 )].
(7)
The corresponding CIFs, F L (t, k | xjL , wl ), can be calculated by integrating (6) and (7) numerically. Note that F L (t, k | xjL , wl ) is different from cdfs of latent failure times. In simulation studies, hL (t, k | xjL , wl ) and F L (t, k | xjL , wl ) are the true values calculated from the latent failure time model. For SSH-Net, the outputs are the cause-specific hazards, which can be directly compared to the true values calculated based on (5). We denote the predicted cause-specific hazard as ĥS (t, k | xi ) in this section. The CIF can be calculated by integrating the cause-specific pdf, which is S
Z t
F̂ (t, k | xi ) =
fˆS (u, k | xi )du,
(8)
0
fˆS (t, k | xi ) = ĥS (t, k | xi )
K X
exp[−Ĥ S (t, k | xi )],
k=1
where the calculation of Ĥ S (t, k | xi ) is derived in Section 2.2, and the integration in (8) can be calculated numerically. c i = k | xi ) from the balancing networks and m(t, xi ) For NFG, the outputs are b(xi ) = Pr(G from the sub-distribution networks. The CIF from NFG is calculated as F̂ N (t, k | xi ) = b(xi ){1 − exp[−t × m(t, xi )]}, which can be directly compared to F L (t, k | xjL , wl ). Let q(t, xi ) = −t × m(t, xi ), the causespecific hazard can be calculated as ĥN (t, k | xi ) =
b(xi )exp[q(t, xi )] ∂q(t, xi ) . PK ∂t [1 − k=1 F̂ N (t, k | xi )]
(9)
In simulation studies, the derivative in (9) is calculated using numerical differentiation. For DeepHit, the outputs are the cause-specific pmfs under the discrete failure time assumption. Let (0, δ, 2δ, . . . , (r − 1)δ) 30
be the discretized time grid, where δ is the step size, r is the length of the time grid, and (r − 1)δ = τ , the CIF is therefore calculated as D
F̂ (t, k | xi ) =
r−1 X
c = jδ, Gi = k | xi ), 1 {jδ ≤ t} Pr(t
j=0
and can be compared with F L (t, k | xi ). A continuous analogue of the discrete hazard is calculated as ĥD (jδ, k | xi ) =
c = jδ, Gi = k | xi ) 1 Pr(t , P D δ1− K F̂ (t, k | x ) i k=1 i
and is compared with hL (t, k | xi ). In the simulation studies, we let δ = 1/365, as the original unit of the GPU failure time is year.
C
Additional Simulation Results
Table 6 shows the comparison of the weighted Brier score and AUC between SSH-Net and NFG for all the simulation scenarios. The SSH-Net has smaller Brier and larger AUC, indicating that it outperforms NFG across all scenarios.
31
Table 6: Brier and Concordance score averaged on the time grid and across units in the test set. The scores are reported in units of 1e-2. Shut Down Date
2020-01-01
2019-01-01
2018-01-01
2017-06-01
n = 10,000 Failure Type 1
SSH-Net
4.65
3.20
1.87
0.77
Avg Brier (%)
NFG
4.78
3.28
1.91
0.79
Failure Type 2
SSH-Net
9.71
7.07
4.38
1.85
Avg Brier (%)
NFG
9.90
7.19
4.46
1.88
Failure Type 1
SSH-Net
82.17
82.39
81.46
82.21
Avg Concordance (%)
NFG
79.80
79.87
77.29
70.55
Failure Type 2
SSH-Net
78.10
78.42
77.07
74.12
Avg Concordance (%)
NFG
77.48
78.18
76.38
73.07
n = 5,000 Failure Type 1
SSH-Net
5.60
4.19
2.32
1.10
Avg Brier (%)
NFG
5.82
4.32
2.40
1.15
Failure Type 2
SSH-Net
10.36
7.57
4.62
2.09
Avg Brier (%)
NFG
10.76
7.87
4.79
2.17
Failure Type 1
SSH-Net
80.19
81.29
81.11
80.77
Avg Concordance (%)
NFG
75.87
77.18
73.91
66.67
Failure Type 2
SSH-Net
77.31
78.50
79.141
74.72
Avg Concordance (%)
NFG
75.51
77.11
77.20
71.03
32
References Antolini, L., P. Boracchi, and E. Biganzoli (2005). A time-dependent discrimination index for survival data. Statistics in Medicine 24 (24), 3927–3944. Blanche, P., J.-F. Dartigues, and H. Jacqmin-Gadda (2013). Estimating and comparing timedependent areas under receiver operating characteristic curves for censored event times with competing risks. Statistics in Medicine 32 (30), 5381–5397. Clark, J. M., J. Min, Y. Wang, Y. Hong, and G. Ostrouchov (2025). Modeling spatially correlated failure-time data under two distance functions with an application to titan gpu data. arXiv preprint arXiv:2509.05444 . Crowder, M. J. (2001). Classical competing risks. Chapman and Hall/CRC. Danks, D. and C. Yau (2022). Derivative-based neural modelling of cumulative distribution functions for survival analysis. In International Conference on Artificial Intelligence and Statistics, pp. 7240–7256. PMLR. Fornili, M., F. Ambrogi, P. Boracchi, and E. Biganzoli (2014). Piecewise exponential artificial neural networks (peann) for modeling hazard function with right censored data. In Computational Intelligence Methods for Bioinformatics and Biostatistics: 10th International Meeting, pp. 125–136. Springer. Gal, Y. and Z. Ghahramani (2016). Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In Proceedings of the 33rd International Conference on Machine Learning, Volume 48 of Proceedings of Machine Learning Research, pp. 1050– 1059. PMLR. Gerds, T. A. and M. Schumacher (2006). Consistent estimation of the expected brier score in general survival models with right-censored event times. Biometrical Journal 48 (6), 1029–1040. Giunchiglia, E., A. Nemchenko, and M. van der Schaar (2018). RNN-Surv: A deep recurrent model for survival analysis. In Artificial Neural Networks and Machine Learning–ICANN 2018: 27th International Conference on Artificial Neural Networks, pp. 23–32. Springer. Gupta, G., V. Sunder, R. Prasad, and G. Shroff (2019). CRESA: a deep learning approach to 33
competing risks, recurrent event survival analysis. In Advances in Knowledge Discovery and Data Mining: 23rd Pacific-Asia Conference, pp. 108–122. Springer. Hesam, S., M. Mahmoudi, A. R. Foroushani, M. Yaseri, and M. A. Mansournia (2018). A cause-specific hazard spatial frailty model for competing risks data. Spatial Statistics 26, 101–124. Hong, C., F. Yi, and Z. Huang (2022). Deep-CSA: Deep contrastive learning for dynamic survival analysis with competing risks. IEEE Journal of Biomedical and Health Informatics 26 (8), 4248–4257. Huang, P., Y. Liu, et al. (2021). DeepCompete: A deep learning approach to competing risks in continuous time domain. In AMIA Annual Symposium Proceedings, Volume 2020, pp. 177. Jeanselme, V., C. H. Yoon, B. Tom, and J. Barrett (2023). Neural Fine-Gray: Monotonic neural networks for competing risks. In Conference on Health, Inference, and Learning, pp. 379–392. PMLR. Katzman, J. L., U. Shaham, A. Cloninger, J. Bates, T. Jiang, and Y. Kluger (2018). DeepSurv: personalized treatment recommender system using a cox proportional hazards deep neural network. BMC Medical Research Methodology 18, 1–12. Kvamme, H. and Ø. Borgan (2021). Continuous and discrete-time survival prediction with neural networks. Lifetime Data Analysis 27 (4), 710–736. Kvamme, H., Ø. Borgan, and I. Scheel (2019). Time-to-event prediction with neural networks and cox regression. Journal of Machine Learning Research 20 (129), 1–30. Lee, C., J. Yoon, and M. Van Der Schaar (2019). Dynamic-deephit: A deep learning approach for dynamic survival analysis with competing risks based on longitudinal data. IEEE Transactions on Biomedical Engineering 67 (1), 122–133. Lee, C., W. Zame, J. Yoon, and M. Van Der Schaar (2018). Deephit: A deep learning approach to survival analysis with competing risks. In Proceedings of the AAAI Conference on Artificial Intelligence, Volume 32. Li, X., Z. Hu, and X. Huang (2020). Combine Relu with Tanh. In 2020 IEEE 4th Informa34
tion Technology, Networking, Electronic and Automation Control Conference (ITNEC), Volume 1, pp. 51–55. IEEE. Man, J. and Q. Zhou (2018). Remaining useful life prediction for hard failures using joint model with extended hazard. Quality and Reliability Engineering International 34 (5), 748–758. Marra, G. and R. Radice (2020). Copula link-based additive models for right-censored event time data. Journal of the American Statistical Association. Min, J., Y. Hong, W. Q. Meeker, and G. Ostrouchov (2025). A spatially correlated competing risks time-to-event model for supercomputer GPU failure data. Technometrics 67 (3), 531–545. Momenyan, S. and J. Poorolajal (2020). Modeling of spatio-temporally clustered survival HIV/AIDS data in the presence of competing risks setting. Spatial Statistics 38, 100460. National Cancer Institute. Surveillance, Epidemiology, and End Results (SEER) Program. https://seer.cancer.gov/causespecific/. Accessed November 25, 2025. Nie, B., J. Xue, S. Gupta, T. Patel, C. Engelmann, E. Smirni, and D. Tiwari (2018). Machine learning models for GPU error prediction in a large scale HPC system. In 2018 48th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), pp. 95–106. IEEE. Ostrouchov, G., D. Maxwell, R. A. Ashraf, C. Engelmann, M. Shankar, and J. H. Rogers (2020). GPU lifetimes on Titan supercomputer: Survival analysis and reliability. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 1–14. IEEE. Rahman, M. M., K. Matsuo, S. Matsuzaki, and S. Purushotham (2021). DeepPseudo: Pseudo value based deep learning models for competing risk analysis. In Proceedings of the AAAI Conference on Artificial Intelligence, Volume 35, pp. 479–487. Ren, K., J. Qin, L. Zheng, Z. Yang, W. Zhang, L. Qiu, and Y. Yu (2019). Deep recurrent survival analysis. In Proceedings of the AAAI Conference on Artificial Intelligence, Volume 33, pp. 4798–4805.
35
Shahraki, S. Z., M. A. Looha, M. Aria, A. Akbari, H. Emami, F. Asadi, M. E. Akbari, et al. (2023). Time-related survival prediction in molecular subtypes of breast cancer using time-to-event deep-learning-based models. Frontiers in Oncology 13, 1147604. Sharma, R., H. Anand, Y. Badr, and R. G. Qiu (2021). Time-to-event prediction using survival analysis methods for alzheimer’s disease progression. Alzheimer’s & Dementia: Translational Research & Clinical Interventions 7 (1), e12229. Wang, Y., Y. Hong, L. Freeman, and X. Deng (2025). The use of variational inference for lifetime data with spatial correlations. arXiv preprint arXiv:2507.09559 . Wang, Z. and J. Sun (2022). Survtrace: Transformers for survival analysis with competing events. In Proceedings of the 13th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, pp. 1–9.
36