Predicting Multiple Clinical Outcomes Related to Functional Recovery and Social Isolation Among Older Adults After Lower-Limb Fracture or Hip Replacement
Analyze & explore related research in Conceptio
Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.
Date of publication xxxx 00, 0000, date of current version xxxx 00, 0000. Digital Object Identifier 10.1109/ACCESS.2024.0429000
arXiv:2608.23531v1 [cs.CV] 24 Aug 2026
Predicting Multiple Clinical Outcomes Related to Functional Recovery and Social Isolation Among Older Adults After Lower-Limb Fracture or Hip Replacement SANTOSH RAY1 , PRATIK K. MISHRA2,3 , ALI ABEDI4,5 , CHARLENE H. CHU3,5 , AMIR AHMAD6 , AND SHEHROZ S. KHAN 7 1
Faculty of Information Technology, Liwa University, Abu Dhabi, United Arab Emirates Institute of Biomedical Engineering, University of Toronto, Toronto, ON, Canada 3 KITE Research Institute - Toronto Rehabilitation Institute, University Health Network, Toronto, Canada 4 Peter Munk Cardiac Centre, University Health Network, Toronto, Canada 5 Lawrence Bloomberg Faculty of Nursing, University of Toronto, Toronto, Canada 6 College of Information Technology, United Arab Emirates University, Al Ain, United Arab Emirates 7 College of Engineering and Technology, American University of the Middle East, Egaila, 54200, Kuwait 2
Corresponding author: Santosh Ray (e-mail: santosh.ray@lu.ac.ae). This study is funded by Liwa University, United Arab Emirates (Grant: IRG-ENG-003-2025).
ABSTRACT Older adults recovering after lower-limb fracture or hip replacement may experience complex recovery trajectories. Most of the time, these clinical aspects are studied in isolation, masking their joint impact on recovery. This study used the MAISON-LLF dataset, which contains multimodal sensor and clinical assessment data from 18 older adults recovering in the community after lower-limb fracture or hip replacement. Participants were monitored for up to eight weeks, corresponding to a maximum of 1,008 participant-days of sensor monitoring. Forty-six daily features were extracted from indoor motion, acceleration, step count, heart rate, out-of-home mobility, and sleep data. Five clinical outcomes were assessed every two weeks: the Social Isolation Scale, Oxford Hip Score, Oxford Knee Score, Timed Up and Go test, and 30-second Chair Stand test. We utilize an inherent relationship between multi-modal sensor data and different clinical scores and formulate it as a multi-output regression problem. We tested various machine learning and deep learning single- and multi-output regression algorithms to predict these scores simultaneously. The results showed that predicting clinical scores jointly was better than separately. The tabular DL multi-output regressor, NODE, gave a remarkable performance of MSE=3.96 and MAE=1.02 in comparison to other multi- and single-output regressors. The SHAP feature analysis further showed the importance of including multimodal sensors to provide a good estimate of patients’ recovery trajectory. This work may support the simultaneous assessment of functional recovery and social engagement among community-dwelling older adults and ultimately help improve their care and quality of life. INDEX TERMS Lower limb fracture, older adults, multi-output regression, deep learning.
I. INTRODUCTION
With advancements in modern medicine, lower-limb surgery patients are often discharged from hospitals quickly to recover in the community to reduce infection risks and healthcare costs [1]. Recovery after lower-limb fracture or hip replacement can be complex and may involve limitations in mobility and social engagement within the community [2], [3]. Regular assessment of functional and social engagement is mostly not covered by the primary care services, unless the patient meets the designated clinician. Therefore, recovery VOLUME 11, 2023
patterns, functional mobility, and social isolation of these patients are rarely studied in tandem. Previous studies show that mobility [4], sleep [5], and other physiological indicators (e.g., heart rate) [6] are separately correlated with measures of functional mobility and social isolation in post limb-fracture patients. Clinical literature also suggests that there are correlations between functional mobility and social isolation among older adults [7], [8]. Multimodal sensors provide opportunities to monitor and log various health indicators that are associated with functional 1
Author et al.: Preparation of Papers for IEEE TRANSACTIONS and JOURNALS
mobility and social isolation, such as indoor motion, outdoor movements, sleep quality, heart rate, number of steps, and other physiological data. When studied together, these unique data can provide an overall profile of a patient that can predict the degree of functional decline and social isolation. The MAISON-LLF dataset was collected using the cloudbased Multimodal Artificial Intelligence-based Sensor platform for Older iNdividuals (MAISON) [9], [10]. It includes longitudinal wearable, ambient, sleep, mobility, and clinical assessment data from older adults recovering in the community after lower-limb fracture or hip replacement. Clinical outcomes included the Oxford Hip Score (OHS), Oxford Knee Score (OKS), Timed Up and Go (TUG), 30-second Chair Stand, and Social Isolation Scale (SIS). The MAISONLLF data is made publicly available by the authors [11]. Prior work on using sub-samples of MAISON-LLF shows a correlation between sensor data collected for mobility and sleep and functional decline and social isolation [12]. In another work, different clinical assessments were predicted separately with low MAE, highlighting the predictive ability of the dataset [11]. From clinical literature, it is known that different clinical assessments, e.g., OHS, OKS, TUG, Chair Stand and Social Isolation, are interrelated as mobility and social engagement impact the overall recovery in this population. However, there exists no study that attempts to predict multiple clinical assessments on functional recovery and social isolation simultaneously to harness further clinical value from this dataset. The positive outcomes can have significant implications for improving the care and quality of life of patients, reducing readmissions, and lessening caregiver burden. Furthermore, it could provide an alarm ahead of time to the caregivers/clinicians to take preventive measures. We formulate the problem of predicting multiple clinical scores from multimodal sensor data collected from the MAISON platform as a multi-output regression problem. This formulation enables training joint models to harness the interrelation between these clinical outcomes in predicting them simultaneously and provides a multidimensional recovery trajectory overview in this population. The key contributions of this work are as follows: We formulate the multimodal sensor data collected from post lower-limb surgery as a multi-output regression problem to be able to predict five clinical measures simultaneously (OHS, OKS, TUG, Chair Stand and SIS). We further show that predicting multiple clinical measures can yield low MAE in comparison to predicting them separately. • We investigated the role of tabular deep learning models on the MAISON-LLF dataset to establish their superior performance in comparison to the traditional machine learning multi-output regression methods. • Feature selection and SHAP analysis further show that the multimodal sensor approach is useful, as data from different sensors appear in the top features list.
•
2
II. RELATED WORK
This section reviews prior work on predicting social isolation and functional recovery in older adults using multimodal sensor data and machine learning (ML), followed by studies on multi-output regression and multi-task learning for healthrelated and sensor-based prediction tasks. It then summarizes the remaining gaps in the field and situates our work to address them. A. PREDICTING SOCIAL ISOLATION AND FUNCTIONAL RECOVERY IN OLDER ADULTS
Sensor-based monitoring has increasingly been used to capture behavioural, social, and functional changes in older adults outside clinical environments. Early studies focused on loneliness, social isolation, and social interaction patterns using passive sensing and smart-home technologies. Austin et al. used wireless motion sensors, contact sensors, telephone monitoring, computer-use monitoring, and walkingspeed measures and trained a logistic regression classifier to estimate loneliness status in older adults [13]. Martinez et al. used communication, mobility, and activity features collected through a mobile application and passive infrared (PIR) sensing, and applied a decision tree model with the synthetic minority oversampling technique (SMOTE) to classify social isolation risk [14]. Goonawardene et al. used non-intrusive motion and door-contact sensors to extract going-out behavior, daytime napping, and room-use patterns associated with social isolation dimensions [15]. These studies established that daily behavioral traces can provide objective indicators of psychosocial states in older adults. The scoping review by Khan et al. [16] identified very few studies tackling social isolation using sensors. They found that data from motion sensors and actigraphy were commonly used and correlated with self-report measures in developing objective SI assessments. They reported that variability exists in defining social isolation, feature extraction, use of sensors, and self-report assessments. Other studies inferred social connectedness indirectly through home visits and in-home activity patterns. Hu et al. used ambient and wearable sensors to detect home visits among older adults living alone, treating visit occurrence as a proxy for social contact [17]. Schütz et al. extended this direction using in-home ambient sensors and self-training domain adaptation for visit detection, comparing one-class support vector machine (SVM) baselines with logistic regression and random forest classifiers [18]. Walsh et al. used ambient smart-home sensors, including motion, contact, electricity, water flow, and light-switch sensors, and applied linear discriminant analysis (LDA) to infer health-related metrics such as loneliness and anxiety [19]. These works show how passive home-monitoring data can be transformed into behavioral markers of social connectedness and well-being. In parallel, sensor-based ML has been used for functional decline, frailty, activity monitoring, and rehabilitation-related outcomes. Fan et al. combined wearable gait sensors with comprehensive geriatric assessment data and evaluated ranVOLUME 11, 2023
Author et al.: Preparation of Papers for IEEE TRANSACTIONS and JOURNALS
dom forest, decision tree, naïve Bayes, neural network, and stochastic gradient descent models for frailty classification in older adults [20]. Giggins et al. used smartwatch and inertial sensor data collected by community-dwelling older adults at home, and applied ML classifiers to discriminate frailty status [21]. Palermo et al. introduced the Technology Integrated Health Management (TIHM) dataset for remote dementia monitoring, including in-home sensor and health data that support ML-based detection of health and behavioral changes [22]. These studies demonstrate the use of wearable and ambient sensors for mobility, frailty, and functional-status monitoring. Studies closer to lower-limb fracture recovery and postdischarge functional monitoring remain limited. North et al. used wearable ground reaction force sensors embedded in insoles, extracted step-level loading features, and applied logistic regression to predict fracture healing progression [23]. Within the Multimodal Artificial Intelligencebased Sensor platform for Older iNdividuals (MAISON) platform [9], Abedi et al. introduced a multimodal dataset from older adults recovering from lower-limb fractures, including smartphone, smartwatch, motion, sleep, mobility, heart rate, and clinical assessment data [10]. Supervised ML and deep learning (DL) models were used mainly for technical validation and recovery-related prediction, rather than joint modeling of multiple clinical outcomes [10]. A subsequent MAISON-based work linked the longitudinal sensor and clinical data with neighbourhood-level geospatial information to create GEOFRAIL, which included participant demographics; sensor-derived features; repeated frailty, physical function, and social isolation assessments; temporal location records; neighbourhood amenities; crime rates; and socioeconomic indicators [24]. Its results focused on dataset validation, consistency among geospatial, sensor-derived, and clinical measures, and baseline ML analyses (single output) rather than multi-output regression [24]. Khan et al. later used MAISON features, including acceleration, step count, ambient motion, global positioning system (GPS) location, heart rate, sleep, and clinical scores, with modality-wise multiview clustering and large language model (LLM)-based interpretation to explain recovery trajectories; however, the outputs were recovery clusters and narrative explanations rather than numerical joint predictions of social isolation and functional outcomes [25].
problem using temporal group Lasso regularization [27]. ElSappagh et al. extended this direction using a multimodal MTL model with convolutional neural network and bidirectional long short-term memory components to jointly predict Alzheimer’s disease progression and cognitive scores from longitudinal multimodal data [28]. These studies show that shared representations can be learned across related regression and classification targets in neuroimaging and cognitive progression. Multi-task and multi-output learning have also been applied to electronic health record (EHR)-based prediction. Harutyunyan et al. introduced intensive care unit (ICU) benchmarks from Medical Information Mart for Intensive Care III (MIMIC-III), including mortality, decompensation, length of stay, and phenotype prediction, and evaluated linear and recurrent neural network models in single-task and multitask settings [29]. Shickel et al. proposed multimodal transformers for jointly predicting multiple ICU outcomes [30], while Chan et al. developed a heterogeneous graph neural network (GNN) framework for simultaneous EHR tasks, including drug recommendation, length-of-stay prediction, mortality prediction, and readmission prediction [31]. Other studies used multi-output or multitask formulations for healthcare resource utilization [32], electronic phenotyping [33], and joint disease prediction from EHR data [34]. MTL has also been applied to sensor-based prediction problems, particularly in wearable sensing and human activity recognition (HAR), although some of these works are not strictly health-related. Chen et al. proposed METIER, a deep MTL model using wearable sensors to jointly perform activity recognition and user recognition [35]; Nisar et al. used wearable sensor data with a CNN and long short-term memory based hierarchical MTL framework to jointly recognize state, behavioral, and composite activities of daily living [36]; and Duan et al. developed a multi-task DL model for sensorbased HAR that jointly performed activity segmentation and activity recognition from continuous sensor streams [37]. More health-oriented sensor-based examples include Khan et al., who used gait analysis data and LSTM-based MTL to predict knee and ankle kinematic trajectories after botulinum toxin treatment in gait rehabilitation [38], and Saylam and İncel, who used wearable-derived physical activity, sleep, and social-network-related features from the NetHealth dataset, comparing random forest, XGBoost, LSTM, and MTL models to predict depression, anxiety, and stress [39].
B. MULTI-OUTPUT AND MULTI-TASK LEARNING IN HEALTH-RELATED PREDICTION
C. RESEARCH GAP AND CONTRIBUTIONS
Multi-output learning and multi-task learning (MTL) have been explored in several health-related prediction problems where multiple clinical outcomes share underlying structure. In Alzheimer’s disease research, Zhang and Shen proposed multi-modal multi-task learning to jointly predict regression and classification outcomes from magnetic resonance imaging, positron emission tomography (PET), cerebrospinal fluid, and clinical data [26]. Zhou et al. formulated Alzheimer’s disease progression as a multi-task regression
The reviewed literature shows two related but largely separate research directions. First, sensor-based studies in older adults have used mobile, wearable, ambient, sleep, gait, and geospatial data to monitor or predict loneliness, social isolation, home visits, frailty, activity, fracture healing, and recovery trajectories [10], [13]–[15], [17]–[21], [23]–[25], [40]–[44]. However, these studies usually define the learning problem around one clinical target at a time, such as loneliness status, social isolation risk, social interaction group, visit occurrence,
VOLUME 11, 2023
3
Author et al.: Preparation of Papers for IEEE TRANSACTIONS and JOURNALS
frailty status, activity class, fracture healing status, or recovery cluster. Second, multi-output and MTL studies have shown the value of jointly modelling related targets, but they have primarily focused on neuroimaging, biomedical measurements, EHR data, sensor-based HAR, gait trajectories, or mental health prediction rather than post-discharge recovery in older adults after lower-limb fracture [26]–[39]. Third, newer tabular deep-learning models have received limited evaluation for single-output and multi-output regression in community-based recovery monitoring. Therefore, there remains a gap in applying multi-output regression to multimodal sensor-based prediction of social isolation and functional recovery in community-dwelling older adults. This setting is well suited for multi-output learning because the target outcomes reflect related dimensions of the same post-discharge recovery process. Social Isolation Scale (SIS) captures social participation and isolation risk, while Oxford Hip Score (OHS) and Oxford Knee Score (OKS) capture lower-limb pain and function [45]. Timed Up and Go (TUG) reflects mobility, balance, walking ability, and fall risk, and Chair Stand captures lower-extremity strength and functional capacity [46], [47]. Prior evidence also supports the clinical linkage among these domains: social isolation and loneliness are associated with poorer physical performance, including sit-to-stand ability, balance, and walking speed; social isolation is associated with worse outcomes after hip fracture; and hip-fracture-related movement limitations can disrupt social life and contribute to isolation [8], [48], [49]. These relationships suggest that the outcomes represent related dimensions of recovery and motivate an empirical comparison of independent and joint prediction approaches. Therefore, the present study jointly predicts SIS, OHS, OKS, TUG, and Chair Stand from multimodal sensor-derived features and compares single- and multi-output regression across traditional ML and tabular DL models, following the broader rationale that related clinical targets can benefit from shared representations [10], [24]–[31]. III. METHODS A. MAISON-LLF DATASET
The MAISON-LLF dataset [10] contains longitudinal multimodal sensor and clinical assessment data collected from older adults recovering in the community after lower-limb fracture or hip replacement. Participants were recruited before hospital discharge, and data collection began within the first few days after discharge and continued for eight weeks. The MAISON platform [9] integrated a Google Pixel Watch 2 smartwatch, Motorola Moto G54 smartphone, Proteus M5 in-home motion sensor, and Withings Sleep under-mattress sleep-tracking mat. The smartwatch collected acceleration, heart rate, step count, and GPS-based out-of-home mobility data; the motion sensor captured in-home movement events; and the sleep mat recorded sleep information. Clinical and functional assessments were collected every two weeks, including the SIS [50], in which higher scores correspond to greater social interaction and lower social isolation; the OHS 4
[51], which assesses hip-related pain and function; the OKS [52], which assesses knee-related pain and function; the TUG [53], which measures functional mobility and balance; and the 30-second Chair Stand test [54], which evaluates lowerextremity strength and functional capacity. For each clinical assessment obtained every two weeks, the recorded outcome value was assigned to all daily sensor observations within the 14-day period ending on the assessment date, such that the 14 daily observations in each interval shared the same clinical outcome value. The dataset was collected in the Greater Toronto Area, Canada, from participants recruited at Toronto Rehabilitation Institute, University Health Network. The study was approved by the University Health Network Research Ethics Board (study ID: 20-5113.10). Data collection was conducted between March 2022 and October 2025. The dataset includes daily sensor-derived features (refer to Table 1) from 18 participants over 56 days, resulting in 1,008 days of multimodal sensor monitoring, together with repeated clinical questionnaire and physical performance assessments. The study cohort had a mean age of 76.5 years (SD 8.9; range: 60–94), with 14 female participants (78%), 14 participants living alone (78%), and 15 participants identifying as Caucasian (83%). The index procedure or fracture type included hip replacement in 8 participants (44%), hip fracture in 4 (22%), pelvis fracture in 3 (17%), and femur fracture in 3 (17%). Across all biweekly visits, the mean SIS score was 23.8 (SD 3.9; range: 15–30), OHS was 28.4 (SD 7.9; range: 17–47), OKS was 30.3 (SD 9.7; range: 15–48), TUG was 19.6 seconds (SD 12.6; range: 7–59), and Chair Stand performance was 9.8 repetitions (SD 3.4; range: 1–17). B. EXPERIMENTAL SETUP
To comprehensively evaluate the predictive performance for social isolation and functional decline, the analysis in Section IV used a range of traditional machine learning (ML) and state-of-the-art tabular deep learning (DL) regressors. The traditional ML regressors included Linear Regression (LR), ExtraTrees Regressor (ET), Random Forest Regressor (RF), K-Nearest-Neighbors regressor (KNN), and Support Vector Regressor (SVR), with their implementations from the corresponding sklearn library. The DL regressors included NODE [55], FT-Transformer [56], TabPFN [57], and TabNet [58]. The core objective of the analysis was to investigate the performance of multi-output regression compared to single-output regression when predicting the target clinical outcomes. The target outcomes were the Social Isolation Scale (SIS), Oxford Hip Score (OHS), Oxford Knee Score (OKS), Timed Up and Go (TUG), and Chair Stand. While single-output regression isolated these variables and predicted them individually, multi-output regression simultaneously predicted all five variables, allowing the models to leverage the correlations for enhanced predictive accuracy. The sklearn implementations of the ML models LR, ET, RF, and KNN have in-built support for multi-output regression, while SVR used sklearn RegressorChain wrapper to VOLUME 11, 2023
Author et al.: Preparation of Papers for IEEE TRANSACTIONS and JOURNALS
TABLE 1: The data modalities, feature names, and their description. 1 Feature naming convention in the dataset: ‘data modality’ +‘-’+‘feature’. Data modality1
Feature1 count, entropy, kurtosis, mean, skew, std, and sum coefficient-of-variation
accelerationminutes-with-data hours-with-data movement-events-00to06, movement-events-06to12, movement-events-12to18, movement-events-18to24 movement-events-24h intradaily-variability heartrate-
count, max, mean, min, and std hours-with-data
motion (step)-
count max max-timestamp mean ratio
position-
sleep-
count duration distance-travelled deep, light, rem, snoring, and total duration-to-sleep and duration-to-wakeup heartrate-max, heartrate-mean, and heartrate-min wakeup-count
Description The total count, entropy, kurtosis, mean, skewness, standard deviation, and sum of acceleration data in a day. Measures the relative variability of daily physical activity, indicating how consistently an individual moves throughout the day in relation to their average activity level. Represents the total number of minutes within a 24-hour period (from 12:00 AM to the following 12:00 AM) during which acceleration data was recorded, providing insight into the smartwatch’s active recording time, even if it was not necessarily worn on the wrist. Represents the total number of hours within a 24-hour period (from 12:00 AM to the following 12:00 AM) during which acceleration data was recorded, providing insight into the smartwatch’s active recording time, even if it was not necessarily worn on the wrist. Represents the number of detected movement events between 12:00 AM and 6:00 AM, 6:00 AM and 12:00 PM, 12:00 PM and 18:00 PM, 18:00 PM and 12:00 AM, based on acceleration values exceeding a dynamic threshold, and reflects physical activity or restlessness. Represents the total number of detected movement events over a full 24-hour period (from 12:00 AM to the following 12:00 AM), based on acceleration values exceeding a dynamic threshold, and provides an overall measure of daily physical activity. Quantifies the fragmentation of daily activity patterns by measuring the frequency and extent of transitions between active and inactive periods within a 24-hour day; higher values indicate more erratic or irregular activity rhythms. The count, maximum, mean, minimum, and standard deviation of heartrate in a day. Represents the number of distinct hours within a 24-hour period (from 12:00 AM to the following 12:00 AM) during which at least one heart rate measurement was recorded, reflecting periods of active heart rate monitoring and providing an estimate of smartwatch wear time (user compliance). The total count of motions (steps) per day. The maximum number of motions (steps) in hours of a day. The timestamp (hour of day) in which there has been maximum number of motions (steps) in a day. The average number of motions (steps) in hours of a day with at least one motion (step). The ratio of the number of hours with at least a motion (step) to the number of hours without any motion (step) in a day. The total count of position data in a day. The duration (in hours) of being outside the home in a day. The total distance (in kilometers) travelled outside the home in a day. The duration (in hours) of deep, light, rapid-eye-movement, snoring, and total sleep. The duration (in hours) to sleep and to wake up. The maximum, mean, and minimum heartrate during sleep. The count of wakeups during sleep time.
run multi-output regression. All the ML models used sklearn MultiOutputRegressor wrapper to implement single-output regression. As the range of values varied widely across the target variables, a min-max scaling transformation was done before training the models to bring all the target variables in the same range. The target variable and predicted values were then transformed back to the original range before calculating the evaluation metrics. The evaluation metrics were Mean Squared Error (MSE) and Mean Absolute Error (MAE) between the predicted values and the target variable values. The ML regressors were trained with the default parameters. As the physical health and behavioural routines vary across different individuals and time periods, the models were evaluated using two distinct and rigorous validation setups. •
Leave-one-person-out cross-validation: The dataset was divided by participant folds, and models were iteratively trained on all but one participant, testing on the heldout unseen individual. This tested the model’s ability to safely generalize to completely new older adults without prior exposure to their specific baseline behaviours.
VOLUME 11, 2023
•
Leave-one-week-out cross-validation: This strategy holds out a specific week of data across the entire cohort. This temporal validation tested the models’ robustness against week-to-week behavioural shifts, demonstrating that the model does not memorize time-specific routines.
IV. RESULTS AND DISCUSSION A. CORRELATIONS AMONG CLINICAL OUTCOMES
In previous sections, we mentioned the structural similarity between various functional mobility and social isolation scales as they represent different pathways around the recovery of this population. To verify this in the MAISON-LLF dataset, we performed a correlation analysis. Figure 1 illustrates the correlation matrix for the overall dataset, revealing significant coupled relationships between functional mobility and social isolation. Most notably, the TUG test exhibits a strong negative correlation with both the Chair Stand test (0.49) and SIS (-0.43). Furthermore, SIS demonstrates a positive correlation with Chair Stand performance (0.29). On the joint health side, the OHS and OKS share a clear positive correlation (0.33). These overall metrics indicate that functional 5