When indexing works and when it doesn't

(MoneyWatch) Myths are often created because there appears to be some data supporting the idea. One of the myths is that active managers outperform in some asset classes.

Nobel Prize winner William Sharpe put this to rest in his famous paper, "The Arithmetic of Active Management," which simply said that, in aggregate, active managers must earn the same returns as passive managers before costs, but underperform them after costs, since they incur greater expenses. Despite the simple mathematical proof, the myth lives on. We'll explore why that's the case and also provide the "antidote."

Dunn's Law (named after a Southern California attorney that provided the insights) explains why sometimes it appears that active managers outperform in some asset classes. Dunn's Law states that when an asset class does well, index funds will outperform active managers in that asset class. However, when an asset class does poorly, active managers have a greater chance to outperform their benchmark index, thanks to active funds drifting into better-performing asset classes.

• Are there times you shouldn't be passive?
• Don't be overconfident in investing
• Is indexing ruining the stock market? 

Many active funds engage in style drift, which means the fund invests in asset classes or other investments that don't align with the fund's mission. Large-cap growth funds may own some small-cap stocks or value stocks, or small-cap value funds may own large-cap or growth stocks.

As an example, let's say the large-cap growth asset class did poorly one year, while small-cap value did very well. Large-cap growth funds that hold small-cap value stocks would receive a boost from those holdings, allowing those funds to do better than the index. On the other hand, small-cap value funds that also held large-cap growth stocks would experience a drag on returns, having them lag the index.

Looking through my files I found a great example of Dunn's Law at work. It's from the 2009 mid-year S&P's Indices Versus Active scorecard. The table below shows the results for the one-year period July 2008-June 2009. I have ranked the asset classes by order of returns.

A few points are worth noting. First, despite the bear market, in five of the six asset classes active managers failed to beat their benchmarks. Second, in the asset class with the best return, almost 77 percent of active managers failed to beat their benchmark. However, in the worst performing asset class over 70 percent beat their benchmark. And there's almost perfect symmetry in the data -- the better the relative performance, the more active managers failed to beat their benchmarks.

William Thatcher took another look, updating a 2009 study he had done. The study, "When Indexing Works and When It Doesn't in U.S. Equities," covered the 14-year period ending in 2011. His findings are consistent with Dunn's Law. For example, during this period the best performing of nine asset class was mid-cap value, returning 8.1 percent. Only 28 percent of active funds beat that benchmark. The worst-performing asset class was large-cap growth, returning just 2.6 percent, and 87 percent of active funds beat that benchmark. The correlation between returns and the percent of active funds beating their benchmark was high. In general, the higher the rank, the higher the percentage of index outperformance versus active funds.

Thatcher also looked at sub periods. For the five-year period ending 2006, the best performing asset class was small-cap value, and the benchmark outperformed 73 percent of active funds. The worst-performing asset class was large-cap growth, and the benchmark outperformed just 21 percent of active funds.

Note that in the next five-year period, ending in 2011, the small-cap value benchmark went from outperforming 73 percent of active funds to outperforming just 35 percent of active funds, while the large-cap growth benchmark went from outperforming just 21 percent of active funds to outperforming 79 percent of the active funds. Did those "dumb" small value managers suddenly get smart and the "clever" large growth managers suddenly become stupid? Of course not. The explanation is that the relative performance reversed. Small-cap value stocks returned -0.3 percent (ranking eighth of nine asset classes), while large growth stocks returned 2.6 percent (ranking second).

Dunn's Law provides us with the winning strategy. If an investor can forecast which asset classes will outperform the others, the best chance of capturing those returns is by purchasing index or other passive funds. You would avoid the asset classes that perform poorly -- where active managers have a better chance of outperformance. If you can't forecast which asset classes will do best, you should choose passive funds anyway, because not only are you highly likely to beat your benchmarks, but it's the only way you can control your asset allocation -- the major determinant of returns and risk of your portfolio.

Image courtesy of Flickr user 401(K) 2013