Model validation in machine learning: A scenario-based guide from hold-out splits to nested group cross-validation in biomedical and applied research

Model validation estimates the performance of a complete learning procedure on new data. However, an invalid split can produce an optimistic and stable result. This tutorial reviews hold-out validation, train/validation/test designs, repeated random subsampling, k-fold and repeated stratified cross-