The paradoxical rise of backtest-squared

February 1, 2018
Yoshiki Obayashi

Statistical methodologies for measuring and minimizing overfitting in backtests are nice in theory, but I find limited practical value in any test of overfitting that is based on backtested data (backtest-squared!). There is too much missing and asymmetric information to estimate the impact of multiple testing with meaningful precision, and backtests-squared may just take overfitting to the next level.