Heads it’s pure skill, tails it’s luck – can the markets really be predicted?

Are markets efficient, or are investor actions governed by animal instincts? Is it pointless trying to beat the market, or are stocks prone to systematic pricing errors? Is “buy and hold” good advice, or should investors be wary of bubbles and irrational exuberance?

The recent awarding of the Nobel memorial prize in economics to finance professors Eugene Fama and Robert Shiller raises these questions, and many more. Fama is famous for his belief in investor rationality and the efficient market hypothesis (EMH). Essentially, EMH advocates argue investors quickly price in all relevant information into a stock, that there is a wisdom incorporated into market prices that renders futile attempts to beat the indices.

Shiller, like Prof Richard Thaler and other experts in behavioural finance, argues EMH neglects the role of emotions in investing, that prices are too volatile to conform to some Spock-like vision of investor rationality, and that greed and fear trigger bubbles and crashes, with markets routinely becoming overvalued and undervalued.

Markets may be unpredictable but that doesn’t mean they are efficient, says Shiller; that some investors may be irrational doesn’t mean the market is inefficient, retorts Fama.

“It’s great work,” he once said of Richard Thaler’s behavioural findings, “but there is nothing there.”

Today, that view is a minority one, and Fama’s protestations to the contrary can sometimes be reminiscent of the old joke about the finance professor who refuses to believe his friend’s exclamation that there is a €20 note on the pavement. It can’t be, says the professor – if there was one, someone would have picked it up already.

Bubbles
Fama, Richard Thaler has said, "is the only guy on earth who doesn't think there was a bubble in Nasdaq in 2000" (Fama argues tech prices were based on "reasonable beliefs at the time that the internet would revolutionise business"). The word bubble "drives me nuts", he once said, even denying in 2010 that there had been a housing and credit bubble in the US.

Warren Buffett’s quip that he would be “a bum on the street with a tin cup” if EMH was true does not impress Fama, who says he doesn’t know if Buffett “is just lucky or skilful”. Ironically, Fama’s own research shows that small-cap and value stocks outperform, but he attributes this to their supposed extra risk rather than market inefficiency.

Ideological devotion to EMH, say critics, ultimately led to financial carnage. Former Federal Reserve chief Paul Volcker opined that "an unjustified faith in rational expectations [and] market efficiencies" was one of the causes of the global financial crisis. Or, to use an equation devised by value investor Jeremy Grantham: "Greed + Incompetence + A Belief in Market Efficiency = Disaster."

Unsurprisingly, it is no longer heresy to question the gospel of efficient markets. Deregulator extraordinaire Alan Greenspan last week admitted to being "intellectually shocked" by the events of 2008, admitting he now realised "there is something more systematic about the way people behave irrationally".

Prof Burton Malkiel, the author of A Random Walk on Wall Street, one of the classic EMH tomes, now describes himself as "a random walker with a crutch" who accepts markets "go crazy from time to time".

Achievements
However, EMH's flaws should not obscure its achievements. Most people would be better off not wasting their time looking for €20 notes on the pavement – someone else usually does get there first. Burton Malkiel's assertion that a blindfolded monkey throwing darts at the stock pages would do as well as market professionals has been proved right over the last four decades. Only a very small minority of investors ever beat the market, and it is nearly impossible to predict the best performers in advance.

A 2010 paper co-authored by Fama found only 3 per cent of money managers demonstrated skill, while other research indicates the percentage of managers delivering alpha has shrank dramatically in recent decades, amongst increasingly competitive global markets.

Investors are increasingly realising that cheap index funds trump actively managed funds – index operator Vanguard is now the largest fund company in the US. Even Warren Buffett, the supreme stock-picker, says "the know-nothing investor can actually out-perform most investment professionals" by opting for index funds.

Some of the core tents of EMH – the belief in cheap index funds, the dangers of overtrading, the role of over-confidence and luck in the investment game, the uselessness of most investment analysis – are shared by behavioural experts like Shiller and Daniel Kahneman.

The latter, asked for an investment tip the day after he won his economics Nobel in 2002, replied, “Buy and hold”, and has repeatedly argued that beating the market is “just not going to happen” for investors.

A number of behavioural funds do attempt to capitalise on market inefficiencies, but their performance has been disappointing.

One study found behavioural funds over the 1990-2010 period had outperformed conventional active funds but not passive benchmarks.

Complement
What the Nobel judges realised is that EMH and behavioural finance can complement each other, and that the latter built upon the foundations of the former. The moral, said finance professor Andrew Lo, is that markets "can be efficient most of the time" but occasionally "they break down and then we need to develop better analytics to be able to understand them".

Indexing guru John Bogle says EMH is "sometimes correct and sometimes incorrect" – trite but true. He advises investors to focus on CMH – the "cost matters hypothesis".

That advice is heeded by Robert Shiller, who has popularised investing according to indices’ and sectors’ cyclically adjusted price-earnings ratios (Cape).

That doesn’t mean jumping in and out of stocks – it means periodic rebalancing using cheap exchange-traded funds.

Additionally, all investors can learn from the behavioural thinkers – how even conservative investors resort to gambling to avoid losses (loss aversion), how we seek out information that confirms our opinions (confirmation bias), how the past always seems to have been more predictable than it really was (hindsight bias), how herd behaviour can impact on decision making.

Both EMH and behavioural finance proved revolutionary in different ways. Fama, Malkiel and Co helped shatter the myth of “expert” fund managers; similarly, the commonsensical conclusions of the behavioural camp helped wrestle the academic world away from an outlook that had become divorced from the sometimes messy realities of markets.

The arguments between the two camps have been divisive over the years, but ultimately, investors have reason to be grateful to both.