Random-Splitting Random Forest with Multiple Mixed-Data Covariates
Published in Journal of Biostatistics and Epidemiology, 2023
Title
Random-Splitting Random Forest with Multiple Mixed-Data Covariates
Authors
Mohammad Fayaz, Alireza Abadi, and Soheila Khodakarim
Journal Information
Journal: Journal of Biostatistics and Epidemiology
Year: 2023
Volume: 9
Issue: 1
Publisher: Tehran University of Medical Sciences
Keywords
- Random Forest
- Bagging
- Functional Data Analysis
- Statistical Learning
- Machine Learning
- Mixed Data
- Hybrid Data
- Variable Importance
- Random Splitting
- R Programming
Abstract
Introduction
Bagging (BG) and Random Forest (RF) are among the most widely used supervised statistical learning methods based on classification and regression trees. These methods can accommodate various response types, including continuous and categorical outcomes. However, many modern datasets contain complex covariates such as functional data, curves, time series, and hybrid structures combining functional and non-functional variables. Existing extensions of RF for functional data often rely on random-splitting techniques to summarize functional observations through informative statistics.
Methods
This study extends the random-splitting framework and introduces the Mixed-Data Bagging (MD-BG) and Mixed-Data Random Forest (MD-RF) algorithms for datasets containing multiple functional and non-functional covariates simultaneously. The proposed methods support mixed and hybrid data structures and provide variable importance plots (VIPs) for assessing the contribution of each covariate.
Results
The primary distinction between MD-BG and MD-RF lies in covariate selection. MD-BG retains all available predictors during model construction, whereas MD-RF randomly samples subsets of covariates at each split. The proposed MD-RF framework successfully identifies the most influential regions of functional predictors while simultaneously evaluating the importance of traditional scalar covariates.
Conclusion
The proposed methods were evaluated using the DTI and Tecator datasets for both regression and classification tasks. Results demonstrate the utility of mixed-data random forests in handling complex functional and hybrid datasets. To facilitate practical implementation, the methods were incorporated into an open-source R package named RSRF.
Full Text
Supplementary Files
Software and Reproducible Research
This article is accompanied by the development of the RSRF R package, which implements the proposed mixed-data random-splitting bagging and random forest algorithms.
R Package
RSRF: Random-Splitting Random Forest for Mixed and Functional Data
GitHub Repository:
The package provides:
- Mixed-Data Bagging (MD-BG)
- Mixed-Data Random Forest (MD-RF)
- Functional and Hybrid Data Support
- Variable Importance Plot (VIP) Estimation
- Classification and Regression Models
- Reproducible Examples and Benchmark Datasets
Researchers can freely download, install, modify, and apply the package in methodological and applied studies involving functional, longitudinal, survival, and mixed-data structures.
Research Impact
This work extends traditional Random Forest methodology to settings involving multiple functional and mixed-data covariates. The proposed framework enables machine learning analysis of complex biomedical, neuroimaging, environmental, and longitudinal datasets where conventional tree-based methods are difficult to apply directly.
The methodological developments introduced in this article formed the foundation for subsequent research on functional random forests, survival forests with functional predictors, and machine learning methods for neuroimaging applications.
Conference Presentations and Scientific Dissemination
The methodological developments presented in this article and the accompanying RSRF package were disseminated through several international conferences in statistics, biostatistics, machine learning, and computational methods.
Oral Presentation
Functional Random Forest for Mixed Data
41st Annual Conference of the International Society for Clinical Biostatistics (ISCB 2020)
Jagiellonian University, Kraków, Poland
23–27 August 2020
- Conference Website: https://iscb.international/
- Book of Abstracts:https://mohammad-fayaz.github.io/publications_files/2023/Y_2023_4_ISCB41.pdf
- Presentation Slides:https://mohammad-fayaz.github.io/publications_files/2023/Y_2023_4_ISCB41_Slides.pdf
Authors: Mohammad Fayaz, Alireza Abadi
Abstract
Functional Data Analysis (FDA) provides a framework in which observations are represented as functions rather than discrete measurements. In many practical applications, datasets contain both functional and non-functional covariates, requiring statistical learning approaches capable of handling mixed-data structures.
This work proposed a Functional Random Forest framework for datasets containing both functional and scalar predictors with continuous or categorical outcomes. Functional observations were transformed through random splitting mechanisms that preserved temporal and structural information. Additional extensions based on basis function representations, Functional Principal Component Analysis (FPCA), and Hybrid FPCA were investigated.
Simulation studies and benchmark datasets, including DTI and EEG-EMG studies, demonstrated improved predictive performance compared with conventional random forest approaches. Variable importance measures were also developed to identify influential functional regions and predictors.
This work represents one of the earliest developments that ultimately evolved into the Mixed-Data Random Forest methodology and the RSRF package.
Poster Presentation
E1319: Random Splitting Random Forest for Survival Analysis with Non-Functional and Functional Covariates in an EEG-fNIRS Trial
The 14th International Conference of the ERCIM Working Group on Computational and Methodological Statistics (CMStatistics 2021) and The 15th International Conference on Computational and Financial Econometrics (CFE 2021)
King’s College London, London, United Kingdom
18–20 December 2021
- Conference Website:https://www.cmstatistics.org/CMStatistics2021/index.php
- Book of Abstracts:https://mohammad-fayaz.github.io/publications_files/2023/Y_2023_4_BoACFECMStatistics2021.pdf
- Poster Presentation:https://mohammad-fayaz.github.io/publications_files/2023/Y_2023_4_Poster_CMStat_2021.png
Presenter: Mohammad Fayaz
Co-authors: Nezhat Shakeri, Alireza Abadi, Soheila Khodakarim
Abstract
Survival analysis methods are widely used for modeling time-to-event outcomes with censoring. In this study, random-splitting Bagging and Random Forest algorithms were extended to accommodate multiple functional covariates in survival analysis settings.
Functional covariates were randomly partitioned using intervals generated from exponential distributions, and summary statistics such as means and medians were incorporated into the tree-building process. A novel variable importance measure was also developed to identify the most influential regions of functional predictors.
The proposed methodology was applied to a publicly available EEG-fNIRS dataset involving a discrimination/selection response task. Event-related potentials derived from EEG recordings served as functional predictors, while fNIRS latency measurements were considered time-to-event outcomes.
The study demonstrated the potential of random-splitting survival forests for analyzing complex neuroimaging datasets containing high-dimensional functional covariates and further expanded the methodological framework underlying the RSRF package.
Citation
Fayaz M, Abadi A, Khodakarim S. Random-Splitting Random Forest with Multiple Mixed-Data Covariates. Journal of Biostatistics and Epidemiology. 2023;9(1).
Recommended citation: Fayaz M, Abadi A, Khodakarim S. Random-Splitting Random Forest with Multiple Mixed-Data Covariates. Journal of Biostatistics and Epidemiology. 2023;9(1).
Download Paper
