The Morgan-Pitman Test of Equality of variances and its application to machine learning model evaluation and selection
Resumen:
Model selection in non-linear models often prioritizes performance metrics over statistical tests, limiting the ability to account for sampling variability. We propose the use of a statistical test to assess the equality of variances in forecasting errors. The test builds upon the classic Morgan-Pitman approach, incorporating enhancements to ensure robustness against data with heavy-tailed distributions or outliers with high variance, plus a strategy to make residuals from machine learning models statistically independent. Through a series of simulations and real-world data applications, we demonstrate the test's effectiveness and practical utility, offering a reliable tool for model evaluation and selection in diverse contexts.
| 2025 | |
|
MACHINE LEARNING STATISTICS THEORY STATISTICAL TEST FORECASTING ERRORS ANALYSIS HETEROSKEDASTIC CONSISTENCY NEURAL NETWORKS NESTED MODELS |
|
| Inglés | |
| Universidad de la República | |
| COLIBRI | |
| https://hdl.handle.net/20.500.12008/54678 | |
| Acceso abierto | |
| Licencia Creative Commons Atribución - No Comercial - Sin Derivadas (CC - By-NC-ND 4.0) |
