Data Games

Here’s how fund performance can be misrepresented

There are many mathematical and presentational tricks that make it possible to build an illusion of performance. Some themes (notes follow):

  1. Selective choice of time period.

  2. Selective nurturing .

  3. Survivorship bias

  4. Inappropriate comparisons

  5. Data mining (choosing data favourable to your case, ignoring data that isn't)

  6. The hidden rare event

1.Choice of time period

Funds will have periods of both good and bad performance, both in absolute terms and in comparison with a benchmark. So they may only chose to promote when comparisons are in their favour.

The regulator tries to limit this practice by mandating five years as the historical time period to be used. That restricts the data set but doesn’t eliminate the practice (just wait for a good period to turn up).

2. Selective nurturing

This means killing bad funds and promoting good ones (as measured by historical performance). More about this in Deeper

3. Survivorship bias

Do you base the average achievement level of your school contemporaries on the attendees at the annual reunion? Wrong! The failures don't show up.

And do you judge the success of a group of funds by analysing the historical performance of those currently on offer?

4. Inappropriate comparisons

For example: comparing the performance of a surviving high risk fund, which needs to have high returns, with a low risk fund. The special case of comparison with past performance is covered in Deeper.

5. Data mining

You can test a hypothesis by checking it against a range of past data, and using statistical tests to see if the data supports the hypothesis. But suppose you just picked the data that happened to fit the hypothesis? That would be a) cheating and b) called "data mining". It’s very popular in sponsored research, particularly PhD theses.

And do you judge the skill of a fund manager by an analysis of his past performance over a time period picked by him? Or the reliability of a whole fund family based on one fund chosen by the managers?

6. Hidden rare events

These are adverse risks that if you had known about them would have affected your financial decisions. More in Deeper.