Allocators must take care not to mistake descriptive analysis for prediction. Just because something involves numbers and metrics does not make it “Moneyball for investors.” Common attribution-style analysis is silent on the degree to which past outcome was achieved by skill or luck, and whether it will persist in the future.
Statistical methodologies for measuring and minimizing overfitting in backtests are nice in theory, but I find limited practical value in any test of overfitting that is based on backtested data (backtest-squared!). There is too much missing and asymmetric information to estimate the impact of multiple testing with meaningful precision, and backtests-squared may just take overfitting to the next level.
One can achieve positive alpha by throwing darts at the market, so how can it be a measure of skill? It's a terribly misused metric. Let us contemplate all the ways in which one should not use alpha, and a couple ways in which it may be marginally useful
There is nothing wrong with paying a high price for rare and valuable talent. Our society happily rewards the likes of Roger Federer, Adele, and Elon Musk with tremendous wealth for their singular craft. In principle, the same should apply to talented investment managers, so what explains the ongoing war on fees?
What distinguishes factors from any other systematic strategy out there? In my opinion, nothing. I'm making a neutral statement here; they're not better or worse. Some factors come with very interesting stories, but that should not exempt them from a rigorous empirical test of the truth.
One question that has sparked a fun debate around our office is whether a manager can possess negative skill. Not just unlucky but truly negative skill that can be exploited by doing the opposite. In our continual search for skilled managers, we also keep an eye out for evidence of the human reverse indicator.