Can the markets be cracked? “Probably not,” said my Dad a million times – last summer, I tried it anyway. I wrote a computer program to download and organize financial data, and test various trading strategies.

Beginning with a generalized trading strategy, the program would generate thousands of variations and historically back-test each one. The best-performing algorithms and their results were stored. The results were promising: my program found trading algorithms which outperformed the S&P 500 Index for the past 30 years running.

Here’s an example of the functionality. Begin with a simple trading structure: e.g. if a stock goes down by *x* percent in the past *y* days, then buy it and hold for *z *days (actual algorithms were more complicated, and had about ten parameters). Wind the clock back 30 years, throw in the 500 stocks of the S&P, and iterate through time, swapping portfolio allocation like lightning according to the pattern of choice — and rinse and repeat for every permutation of our parameter set we could desire. Each 30-year iteration took about a second. I used to let it run overnight.

So am I rich yet? The markets are more chaotic than I ever realized. Picture the above-described optimization procedure over a 30-year window: I receive perfect results, my computer feels rich, and all is well. Let’s try again over the past 10 years: even more virtual money is piled up; the parameter set this time, however, is entirely different. Optimize over 5 years: another parameter set. No successful strategy, it turns out, remains successful over a different time window; *each time window**, *it turns out, delivers a separate trading strategy. The markets are pure chaos.

Four separate result-algorithms – each tracked after the fact in my personal “horserace” – all took a nosedive as the optimization window ended. Four algorithms ran in close competition for years, until Jan. 1, 2012 – then they began to *lose* money.

Is all hope lost? Probably – but I “learned something”, as they say. Firstly, the markets are chaotic and most attempts at “technical analysis” (i.e., anticipating stock movements from nothing other than the shape of the price curve itself) are dubious at best. Secondly, supplementing my pure price data with actual *accounting data* (earnings, income, etc.) might be a promising next step with some scientific backbone. Finally: computational power is excellently put to work in financial economics. Maybe, some day, the markets will be cracked.

A great, and profoundly relevant, article:

http://www.ams.org/notices/201405/rnoti-p458.pdf

“Regarding the measured performance of a backtested strategy, we have to distinguish between two very different readings:

in-sample(IS) andout-of-sample(OOS). The IS performance is the one simulated over the sample used in the design of the strategy (also known as “learning period” or “training set” in the machine-learning literature). The OOS performance is simulated over a sample not used in the design of the strategy (a.k.a. “testing set”). A backtest isrealisticwhen the IS performance is consistent with the OOS performance.“When an investor receives a promising backtest from a researcher or portfolio manager, one of her key problems is to assess how realistic that simulation is. This is because, given any financial series, it is relatively simple to overfit an investment strategy so that it performs well IS.

“Overfitting is a concept borrowed from machine learning and denotes the situation when a model targets particular observations rather than a general structure… After a few iterations, the researcher will come up with “optimal parameters”, which profit from features that are present in that particular sample but may well be rare in the population.

“Recent computational advances allow investment managers to methodically search through thousands or even millions of potential options for a profitable investment strategy. In many instances, that search involves a pseudo-mathematical argument which is spuriously validated through a backtest.”

I enjoyed reading this. I can see how you might look back at it as being a bit naïve at the time though. I’m no market enthusiast, and I haven’t seen any of the details of the program you’ve written, but I’m pretty sure there is a name for the trading structure you’re looking for: Economics!

It’s probably been done already, but an interesting question to ask would be about ‘threshold effects’ in market behavior. Namely, when (i.e. at what values of our current parameters or at the introduction of which parameters) do our automated statistical analyses of markets break down? Is there, in other words, a threshold of market complexity, at which point behavior becomes chaotic? I suspect the answer should be “no”, because anything simple enough not to exhibit chaotic, unpredictable long-term behavior probably doesn’t deserve to be called a ‘market’. While the idealized statistical models economists call ‘markets’ may be cracked if they are sufficiently ideal, any market, like the stock market, whose roots lie in the chaos of the physical world, are probably incapable of being ‘cracked’, in your sense. (Which is obviously not the same as saying that all betting strategies for markets are equally rational, given the data that might be available.)

Interesting thought. This might come off as a bit crackpot, but I see an interesting analogy in games, which can be solved or not solved.

All games seem chaotic to an untrained observer. But some games are easily solved (checkers, 9×9 go, connect 4), while other games certainly are not solved (chess, 19×19 go, connect6). I might compare a simple “market” to a solved game, and a chaotic market to an unsolved game.

The point of this analogy, then, is to show that your question isn’t as silly as you make it out to be. Many games seem hard to beat, but still, there is in fact a threshold, which is obscure to a game-player, but fairly obvious to a computer scientist.

To your point, again, even if game cannot be solved, there still exist strategies for success, for both a computer scientist and a game-player.