Bias alert: Why you can't trust hedge fund returns

My Right Financial Plan co-author Kevin Grogan recently passed along a paper, "Out of the Dark: Hedge Fund Reporting Biases and Commercial Databases," that examines the problems with hedge-fund indexes.

Specifically, the paper looks at "self-selection bias." That refers to the skewed performance results that stem from some funds choosing not to report their returns. If funds perform poorly, they may stop -- or never start -- reporting results (for obvious reasons). On the other hand, well-performing funds may not report results to an index because they don't need the marketing benefits they get from being members of the index. These funds will attract investors on the strength of their reputation.

To evaluate the impact of this bias, the researchers compiled returns for more than 1,400 distinct hedge funds between 2004 and 2009. Then, they compared the returns of the funds listed in commercially available databases to those of unlisted funds. Key findings of the study:

-- 95 percent "of a typical fund manager's measured skill can be explained by whether they report to a database."

-- "Much like the vast mutual-fund literature that finds little evidence of managerial skill... our results indicate that when voluntarily reported returns are excluded, the average excess return of hedge funds does not differ markedly from zero."

-- "At a minimum, until performance-reporting for hedge funds becomes mandatory, it will likely be difficult to draw strong conclusions from hedge-fund performance studies that use voluntarily reported data."

Add these findings to the mountain of evidence that shows that self-reported hedge-fund data is riddled with such biases. For example, a Journal of Asset Management paper found that so-called survivorship bias -- excluding companies that no longer exist from performance analyses -- added 4.4 percent per year to hedge-fund returns. A study by Princeton University professor Burton Malkiel that appeared in the Financial Analysts Journal also found that "backfill bias" -- which occurs when hedge-fund managers report their performance retrospectively, presumably after they have had some success, rather than from the inception of the fund -- overstated returns by 5 percent per year.

The problem with these biases is that they are pervasive across hedge-fund indices. They can't be diversified away by combining multiple indexes.