May 6, 2010, is a date that should live in infamy. On that day, the US stock market suffered a trillion-dollar collapse. Do you remember it?
In just 15 minutes from 2:45 pm, the Dow Jones plunged by almost 9 percent. Had it stuck, it would have been among the famous index’s biggest declines in a single day. Hundreds of billions of dollars were wiped off the face value of famous companies in the S&P 500. Stocks in some companies were trading for a single cent.
Almost as soon as the sleepless Wall Street traders had time to adjust to the nightmare scenario unfolding in front of them, the market began to rebound. Within another fifteen minutes, almost all of the losses had been recovered.
It was one of those moments where you can only marvel at the absurdity of what it is to be a modern human. The numbers on the Dow Jones index are just numbers; the value they represent is, more than ever before, based on perceptions: stories we tell each other about companies and the economy.
Yet, had the trillion-dollar wipe-out proved permanent, there would have been real consequences, real layoffs, real suffering. Nothing of value was created or destroyed in the what became known as the “flash crash”; the markets weren’t responding to new information. We, collectively, have designed a system that is beyond our understanding, often beyond our control, and the unseen, intangible hands do not always serve us kindly.
The post-mortem for the flash crash was swift. Some people suspected insider trading, while others suggested technical glitches in the stock market might have been responsible. After five years, one man was charged with market manipulation: Navinder Singh Sarao, a 36-year-old from London who had been betting against the market with a “spoofing algorithm” around the time of the crash. He may have profited to the tune of $12 million. Small change against the trillion-dollar fluctuation.
Various outlets, including Bloomberg, have been skeptical that Sarao, who traded from his childhood bedroom in London, could really be held responsible for the flash crash. After all, while he was clearly no ordinary trader, the volumes of money he was moving around were tiny compared to the overall stakes of the flash crash. As it turns out, Sarao was also defrauded of the vast majority of his profits, and could not post his $5 million bail.
It’s an easier story to tell ourselves that one man’s misdeeds triggered the crash, because it allows us to believe that it’s a fixable problem. But even if you believe Sarao was solely responsible, it’s indicative of how vulnerable and chaotic the system is that his actions could trigger such a massive response.
A big part of the reason for this volatility is the sheer density of automated traders and algorithms that control these financial markets.
The most alert, savvy human is still limited by the speed at which his brain can process signals and move his muscles in response. Not so for an AI. In 2010, when Sarao’s algorithm helped trigger the crash, it was hardly alone: 70 percent of all trading activity was completely automated.
This represents a sea change in how trading is done on Wall Street. The classical idea is that the market tells you about investor confidence, and so what it reflects is information. If it’s widely known or anticipated that, say, Amazon is about to declare a record quarter of earnings, their stock will probably be riding high and reflect the value of that expectation.
In this classic system, the way you “win” and profit is by having access to information—or perhaps an expectation—that others don’t have, or by consistently making better judgement calls.
For an algorithmic trader, it’s different. Stories that contain rumors and hearsay are no good to an AI until they’re converted into a numerical value. At the classical end, algorithms might scan through news stories and the facts and figures of earnings statements to suggest new investment opportunities. They’re essentially doing the classical job of a trader, but many times faster. Natural language processing analyzes the sentiment of news articles and converts it into the AI’s “gut feeling.” If one understood the intricacies of the algorithm and the data it was being fed, one could hope to “game” this system with favorable stories in the media too.
At the more rapid edge, flash traders (sometimes called high-frequency traders, or HFTs), are searching for miniscule advantages. They’re not looking into whether Company A is more likely a better long-term investment than Company B, but instead seeking to profit from fluctuations in price that can take place over seconds or milliseconds, putting in thousands of orders for hundreds of stocks automatically in response to inflows of data. These are the people who built a $300 million cable that was ever-so-slightly straighter than the previous effort, to reduce trading times from 17 to 13 milliseconds—and made a killing in doing so.
In its most controversial form, the flash traders view orders from other market participants fractions of a second before the trades go ahead. The data that they’re feeding on isn’t information about the underlying stock, but about the market fluctuations caused by other people’s valuation of the stock. Predatory traders attempt to decode the signals made by one algorithm, while they attempt to conceal their intentions to avoid being scooped. It is this competition between fast and slow algorithms—and, possibly even attempts by the fastest algorithms to “outwit” the slowest—that helped lead to the flash crash. The algorithms, fed on a diet of each other’s behavior, are quite capable of whipping themselves up into a feedback frenzy without human prompting.
Previously, many of these algorithms used statistical models developed by quants, or quantitative analysts. Many used complex mathematics to convert streams of input data into a prediction of the future price of the stock, and therefore buying and selling instructions.
Now, increasingly, being able to run these models quickly is not enough, and hedge funds are turning to artificial intelligence that can improve its own models using Bayesian reasoning: it updates the model parameters depending on how successful the model has been.
Algorithms are not inherently bad; some have even argued that they stabilized the market during the flash crash more than humans might have done. Indeed, the system is likely so complex already that the only solution we can feasibly give echoes Facebook’s response to concern over use of its platform: smarter algorithms.
But the febrile cauldron of Wall Street is best suited to the worst excesses of algorithmic design and use. There is a huge incentive to make algorithms extremely powerful in decision-making, and avoid any semblance of transparency.
To “beat the market”—which now embodies not only everyone’s knowledge about stocks, but also the activity of the best algorithms that are being deployed—you need to have an edge that no one else has.
This inevitably leads to a lack of transparency. Bernie Madoff seemed to have an edge, generating remarkably consistent returns: it turns out his edge was running a multi-billion-dollar Ponzi scheme.
Every day, more and more impenetrable connections are added to this complex network, adding to the glut of inscrutable decisions. It is part of a broader trend, motivated by the undeniable advantages of efficiency and profit, to place more and more decisions and actions in the hands of black-box algorithms whose behavior is often unpredictable, inexplicable, and unquestionable, even as they become ever more deeply ingrained into the system.
As the flash crash warns us, the consequences of doing this without sufficient understanding can rapidly spiral out of control.
Image Credit: ESB Professional / Shutterstock.com
