What Makes A Good Economic Model?

Authored by Frank Shostak via The Mises Institute,

In order to make the data "talk," economists utilize a range of statistical methods that vary from highly complex models to a simple display of historical data. It is generally held that by means of statistical correlations one can organize historical data into a useful body of information, which in turn can serve as the basis for assessments of the state of the economy. It is held that through the application of statistical methods on historical data, one can extract the facts of reality regarding the state of the economy.

Unfortunately, things are not as straightforward as they seem to be. For instance, it has been observed that declines in the unemployment rate are associated with a general rise in the prices of goods and services. Should we then conclude that declines in unemployment are a major trigger of price inflation? To confuse the issue further, it has also been observed that price inflation is well correlated with changes in money supply. Also, it has been established that changes in wages display a very high correlation with price inflation.

So what are we to make out of all this? We are confronted here not with one, but with three competing "theories" of inflation. How are we to decide which is the right theory? According to the popular way of thinking, the criterion for the selection of a theory should be its predictive power. On this Milton Friedman wrote,

The ultimate goal of a positive science is the development of a theory or hypothesis that yields valid and meaningful (i.e., not truistic) predictions about phenomena not yet observed.

So long as the model (theory) "works," it is regarded as a valid framework as far as the assessment of an economy is concerned. Once the model (theory) breaks down, we look for a new model (theory). For instance, an economist forms a view that consumer outlays on goods and services are determined by disposable income. Once this view is validated by means of statistical methods, it is employed as a tool in assessments of the future direction of consumer spending. If the model fails to produce accurate forecasts, it is either replaced, or modified by adding some other explanatory variables.

The tentative nature of theories implies that our knowledge of the real world is elusive.

Since it is not possible to establish "how things really work," then it does not really matter what the underlying assumptions of a model are. In fact anything goes, as long as the model can yield good predictions. According to Friedman,

The relevant question to ask about the assumptions of a theory is not whether they are descriptively realistic, for they never are, but whether they are sufficiently good approximation for the purpose in hand. And this question can be answered only by seeing whether the theory works, which means whether it yields sufficiently accurate predictions.

Why the Predictive Capability for Accepting a Model Is Questionable

The popular view that sets predictive capability as the criterion for accepting a model is questionable. Even the natural sciences, which mainstream economics tries to emulate, don"t validate their models this way. For instance, a theory that is employed to build a rocket stipulates certain conditions that must prevail for its successful launch.

One of the conditions is good weather. Would we then judge the quality of a rocket propulsion theory on the basis of whether it can accurately predict the date of the launch of the rocket? The prediction that the launch will take place on a particular date in the future will only be realized if all the stipulated conditions hold.

Whether this will be so cannot be known in advance. For instance, on the planned day of the launch it may be raining. All that the theory of rocket propulsion can tell us is that if all the necessary conditions will hold, then the launch of the rocket will be successful. The quality of the theory, however, is not tainted by an inability to make an accurate prediction of the date of the launch.

The same logic also applies in economics. We can say confidently that, all other things being equal, an increase in the demand for bread will raise its price. This conclusion is true, and not tentative. Will the price of bread go up tomorrow, or sometime in the future? This cannot be established by the theory of supply and demand. Should we then dismiss this theory as useless because it cannot predict the future price of bread?

Or consider a situation when a stock market is following an "up" trend over several years. As a result, an analyst has established that it is possible to outperform the stock market by following the barking of a dog.

If the dog barks three times it is a buy and if he barks once it is a sell. Should such a framework be accepted as a valid theory because it makes good forecasts?

Contrary to the popular way of thinking the criteria for selecting a model is not how well it worked in the past — i.e. passed the criteria of back testing and a life test — but whether it is theoretically sound.

What Nassim Taleb Can Teach Us

Authored by Jeff Deist via The Mises Institute,

Nassim Nicholas Taleb does not suffer fools gladly. Author of several books including The Black Swan and Antifragile, Taleb is known for his incendiary personality almost as much as his brilliant work in probability theory. Readers of his very active Medium page will experience a formidable mind with no patience for trendy groupthink, a mind that takes special pleasure in lambasting elites with no “skin in the game.”

“Skin in the game” is a central (and welcome) tenet of Taleb’s worldview: that we are increasingly ruled by an intellectual, political, economic, and cultural elite that does not bear the consequences of the decisions it makes on our (unwitting) behalf. In this sense Taleb is thoroughly populist, and in fact he correctly identified trends behind the Crash of ’08, Brexit, and Trump’s election. He understands that globalism is not liberalism, that identity and culture matter, and most of all that elites don’t understand how randomness and uncertainty threaten the inevitability of a global order. 

Thus Taleb argues the intelligentsia are not only haughty when they plan our future, they are also clueless: fragility abounds, and threatens to crash the Party of Davos. Hubris results from unearned wealth and prominence, coupled with a blindness to the Black Swans lying in wait.   

Born in Lebanon to a prominent family, educated at the University of Paris and Wharton, Taleb was poised to become part of the cognitive aristocracy he mocks. But he was never one of them. His hard-nosed persona, enhanced by a dedication to rigorous deadlift workouts, is quickly evident in his notorious interviews and very public Twitter brawls. His willingness to delve into history and and religion sets him apart from the neoliberals who hope to wish them both away. Taleb writes for the intelligent everyman, and this blue-collar approach also extends to his description of himself as a “private intellectual, not a public one.”

Austro-libertarians will find much to admire in his brilliant takedowns of the “pseudo-experts” he identifies in academia, journalism, politics, and science. But Taleb is no Austrian. While he holds a decidedly jaundiced view of most economists—calling for the Nobel in economics to be cancelled— he does not denounce economics as a field of study per se. Nor does he claim heterodox or reactionary inclinations:

“I am as orthodox neoclassical economist as they make them, not a fringe heterodox or something. I just do not like unreliable models that use some math like regression and miss a layer of stochasticity, and get wrong results, and I hate sloppy mechanistic reliance on bad statistical methods. I do not like models that fragilize. I do not like models that work on someone"s computer but not in reality. This is standard economics.”

While he is not averse to using mathematics and statistics in economics, Austrians share his perspective that both are tools for economists. Statistical models are mostly bunk that provide no value to economic forecasters or investors, despite the highly paid Ivy League quants who produce them. In fact, models often have harmful effect of creating a false sense of relative certainty where none exists. It"s refreshing to see Taleb make this claim so effectively from outside the Austrian paradigm of praxeology. But if his view of economics is mainline, his tone is Rothbard meets Hayek:

I"m in favour of religion as a tamer of arrogance. For a Greek Orthodox, the idea of God as creator outside the human is not God in God"s terms. My God isn"t the God of George Bush.

We know from chaos theory that even if you had a perfect model of the world, you"d need infinite precision in order to predict future events. With sociopolitical or economic phenomena, we don"t have anything like that.

Taleb does see a role for government, and supports consumer protection laws against predatory lending as one example. But he also purportedly supported Ron Paul in the 2012 presidential election, and has indeed mentioned Hayek as an influence regarding the dispersal of knowledge in society. He’s also applied special venom to several worthy targets in professional economics, including Paul Krugman, Joseph Stiglitz, and Paul Samuelson. Taleb labels as “Stiglitz Syndrome” the process whereby public intellectuals suffer no financial or career consequences for being spectacularly wrong in their predictions.

This is especially galling to a man who correctly called (and in fact became wealthy as a result of) economic crises in 1987 and 2008. In both instances, Taleb had “skin in the game” as a market trader. His own money and reputation were on the line, unlike the court economists in the New York Times.

For an excellent (albeit indirect) analysis of how Austrians and libertarians can advance their cause from a minority position, Taleb’s recent article The Most Intolerant Wins: The Dictatorship of the Small Minority is a must-read. He reminds us that a small minority with courage—the most important form of skin in the game— can prevail over the slumbering masses. And he also reminds us that courageous individual actors, not 51% mass movements, drive real changes in every society:

The entire growth of society, whether economic or moral, comes from a small number of people. So we close this chapter with a remark about the role of skin in the game in the condition of society. Society doesn’t evolve by consensus, voting, majority, committees, verbose meeting, academic conferences, and polling; only a few people suffice to disproportionately move the needle. All one needs is an asymmetric rule somewhere. And asymmetry is present in about everything.

Economics is lost, mired in a quicksand of predictive models that fail to predict and macro-analysis that fails to analyze.

Democratic politics is lost, ruined by bad actors with perverse incentives to burn capital rather than accumulate it.

And academia is lost, still stuck in a centuries-old model run by hopelessly sheltered PhDs.

Taleb gets all of this, and does an admirable job of explaining it. Austro-libertarians would be wise to see him as a valuable ally and voice in the ongoing fight against states, central banks, and planners of all stripes.

Atlanta Fed vs NY Fed: Whose GDP Forecast Is Right?

Authored by Salil Mehta via Statistical Ideas blog,

There is a 2/3 chance that both competing Federal Reserve 2017 Q1 GDP nowcasts are wrong! That’s an audacious prediction for the storied NY and Atlanta institutions (one of them led by my former big boss Timothy Geithner), and yet there is no way around the current confusion they are in. 

This is also critically important as one is showing a robust 3.2% growth reading, while the other is at 0.9% (the 2^nd lowest reading in nearly 3-years) and essentially indicates that we are descending towards recession. 

[NOTE - NOWCAST just downgraded their forecast dramatically from 3.19% to 2.83%]

Are we descending towards recession?  It"s unlikely but zero-growth is certainly in the cards and not reflected by these two nowcasts, and we certainly think there is only a single digit probability of a >3% GDP.  How could the NY Fed plausibly give such a madly high estimate (which if true would be the second highest in 2-years)?  Yet there you have it, two extreme readings, and a 2.3% (3.2%-0.9%) chasm between them.  We show here that the Federal Reserve’s conclusions are somewhat ridiculous, though shouldn’t be since they impact the open market committee monetary decisions that the world looks to.  And there are humbling lessons from these nascent Big Data, overfit models.

The chart here shows some basic information regarding the current GDP nowcasts.  As we via the two blue bars, we have the Atlanta nowcast on the left (the bar was recently as high as 3.4% earlier this year).  And the NY nowcast on the right (the bar was recently as low as 1.5%).  That’s right, both nowcasts passed each other, while aggressively moving further in the opposite direction!  The large swings in each are also doubtful, given each nowcast’s eventually advertised, margin of error (concordant paradigm in various forecasts by Taleb).  For a good chronology of these nowcast reports, refer to MishTalk.

Each nowcast boasts a margin of error of just ~1%, and this clearly poses an issue since the average of these two nowcasts (shown in orange at 2.1%) is clearly outside of both the Atlanta and the NY stated margin of error!  As supportive reference, we also show (in green) that the current 2016 Q4 GDP is nearby at 1.9%.  Now we should ask some important questions about how we keep getting into more strange nowcasts in the past year that they both have operated.  The first thing to appreciate is that the nowcasts are supposed to predict very tight errors that are uncorrelated to the variance in the actual GDP itself.  And good nowcasts should have errors independent of one another, except since the NY and Atlanta Fed operate independent of one another there is a good chance that there may be some modeling similarities.  We modestly assume this and derive through the variance formula (Variance_Atlanta+Variance_NY+2?_Atlanta?_NY?_Atlanta,_NY) that the margin of error of the difference between the models is just less than 1% (silver vertical interval arrows in chart above).  This is a highly plausible tight expected variance.  Sample size is also trivial here as we don"t have the true expectation to model a limit from.  And with this, the probability of seeing an inadvertent 2.3% difference between the two correct Federal Reserve models is <5%.  Or that their publicized margin of error is awkwardly too low (to the point we’ll show later that randomly guessing the GDP would be safer).

We also have the probability that one of the nowcasts is correct, which due to symmetry means applying one of the nowcast stated margin of errors to the other nowcast value.  Or that the one nowcast is unintentionally correct, which would be like the <5% probability above.  So, in total the probability of one of the Federal Reserve nowcasts being correct and the other being wrong is ~10% (not the >½ that many may generously assume as they critique these divergent outputs).

That leaves us with two other possible outcomes still!  That is to split up the remaining 85% probability that both models are individually wrong into: (a) the average of the two models is still correct, and (b) even the average of both models is wrong.  This leaves us with no choice but to conclude that the probability that there is ~40% probability that the correct 2016 Q1 GDP is nowhere near either nowcasts nor the average of the two, and a >½ probability that the GDP is near the 2.1% average that inappropriately happens to be well outside both two nowcasts’ margin of error.  And between those two we can safely claim that there is a 2/3 chance that both models are total wrong (and merely <5% chance they are still both right).  There is perhaps a 30% chance one can smartly use both models in a deliberate way, though this is conditional on how they use the information and not at their endorsed specified face value.   

Now a more practical assumption of this is that the margins of error should be more than doubled (to 2.6%!), in which case no one would even use such nowcast models.  However, the probability breakdown in such a scenario is this:

• 
  

  <30% chance both models -with their current 2.3% chasm- are correct

  
  


• 
  

  ~60% chance one of the models is wrong and one is correct

  
  


• 
  

  ~10% chance both models are still wrong individually, though the average in rare cases is correct

  
  

In all cases, all three probabilities sum to 100% as they should.  And they lend a healthy sense of respect that incessantly observing each of these GDP nowcasts is commonly a waste of time, and that rarely will one gain insight from it other than from ex post luck.  It’s the same as the more senior open market committee models vainly attempt to forecast other macro-economic variables.  Sometimes simply looking at the most recent quarter’s GDP (in this case ~2%) is as good of a guess as any.  So is giving a little more weight to the near-0 probability most have of an outright contraction this quarter.  As a business CEO, one would want to be arranged for anything at this point, which a genuine 2.6% margin of error about GDP infers.

BNP Risk Indicator Flashes "Love" Warning Signal For US Stocks

While the market itself has exhibited the exuberance we have all seen before (and never seem capable of learning from), BNP has quantified this love-panic relationship (and the news is not great for the bulls). When in "love" mode, the average drop in stocks has been 12% in the next six months. The biggest drivers of this "love" have been investor confidence, CoT positioning, short-interest, relative trading volumes, and sectoral outperformance with fund-flows shifting away from "love" suggesting the short-term top is in. The index itself peaked last week at the highest level of "love" in two years...

h/t @Not_Jim_Cramer

BNP explains their framework:

In our Love Panic model, we try to identify distress and euphoria in an attempt to predict forward market returns. In order to successfully predict the market we have chosen parameters with good predictive capabilities during different market cycles but also those that make qualitative sense. Investment should be dispassionate but not automatic. Some investors solve this problem by hiring a mechanic (or quant) to build a machine to invest on their behalf. This indicator is not for them. Instead, this indicator highlights when market sentiment is either overly depressed or excessively optimistic. This helps one at least adjust for ones mood. So we suggest that when the market has reached a level of distress, it’s a good time to buy. Meanwhile, when investors are euphoric,we advocate a sell. As a result we have developed a contrarian indicator model. When our signal is in panic (negative), it indicates a buy. While when the signal reads positive it’s a sell signal. In our Love Panic model, we try to identify distress and euphoria in an attempt to predict forward market returns. In order to successfully predict the market we have chosen parameters with good predictive capabilities during different market cycles but also those that make qualitative sense.

And the market has not done well once investors fall in "love"...