Riding Herd on Big Data, It’s the “Algo-Jockey”—on Trading Desks Everywhere

It hasn’t become an official job category yet, but just wait—the “algo-jockey” will be coming to a trading desk or an IB fin-tech department any day now. In fact, chances are if your company deals in high frequency trading or any kind of activity that uses big data, he or she may already be there, but under a different title.

If you’re wondering what an algo-jockey does, it’s simple. They step in whenever too much big data causes the algorithms to run amuck, misinterpret some of this data and create situations that would be impossible—such as a stock losing 90 percent of its value in a 10th of a second, sparking a flash crash.

“All computer systems have inherent capacities and we have to keep remembering that, especially with the buy-side, trying to keep up with the market sprinters,” says David Leinwebber, co-founder of Innovative Financial Technology, author of the book Nerds on Wall Street, and from whom I first heard the term algo-jockey.

New Set of Quantitative and Analytical Skills

“The complexities of the buy-side trading decision require an entirely new set of quantitative and analytical skills that once existed only in the nether regions of sell-side proprietary trading teams and quantitative hedge funds,” says Laurie J. Berke, writing for the Tabb Group, a Massachusetts-based financial markets research and strategic advisory firm focused on capital markets.

“Yet there is an additional component of the buy-side trading process that will increasingly be quantitatively driven,” adds Berke, “that is the closer coordination and alignment of inputs into the investment decision with inputs into the trading strategy.”

According to Leinwebber, today’s trader looks nothing like he or she did 30 years ago. Back then, “if you wanted to hire a trader, you looked for people who played sports, and they tended to be tall so they stood out on the trading floor,” says Leinwebber. “But today you want the quants, the nerds, the geeks. Today, the people who thrive are the people who work well with big data and with machines.”

That’s because traditional asset management firms now execute 48 percent of their U.S. equity orders electronically, noted Berke. “There are over 65 trading venues, over 600 broker-built algorithms, thousands of smart order routing logic solutions and more reams of post-trade TCA data than any analyst could ever learn to love,” she adds.

“What all this means,” says Berke, “is that in order to align the execution of a buy order with the drivers of the buy decision, a buy-side trader must have strong analytical skills across the board. If his firm is big enough to have the resources, he may have his own internal quant team, his own analysts and algorithm programmers to build strategies and routing logic to optimize his trading. He’s got the technology to capture and analyze trade data and the mathematicians to build the algo models. Next-generation head traders are responsible for the success of these efforts, and the larger the asset manager, the more global the initiative.”

Algorithms Misinterpret

Algorithms have been known to, on occasion, misinterpret an event or two. In fact, Leinwebber believes the flash crash was caused by robot high frequency trading programs misinterpreting the data. “Instead of usual markets, the robot traders saw buy high and sell low and reacted by saying let’s get out of here,” he explained.

This is where an “algo-jockey” can step in whenever there’s unusual activity, where new patterns emerge that were not seen before and could not possibly happen.

Big Data Comes in Two Sizes: Unstructured and Structured

Another area that is more amendable for humans is in the area of “unstructured big data” which includes everything on the Web, mainstream media news, blogs and social media. There is software that pulls out certain words, but humans can still do a better job of analyzing this data.

“In the financial markets, big data comes in two flavors: structured and unstructured,” said Leinwebber. “The structured is market data, which has become bigger and so fast that people are hard pressed to keep up with them. Even the machines people use are hard pressed to keep up with them.”

That’s why he believes one way to look at this is as job security, but a different kind of job security than what we had in the past.

Collaborative Intelligence

Some of the skills required today include something he calls collaborative intelligence, which he says is a new role for traders. Being able to exploit information and understand their complexities, while having strong language and conceptual abilities.

When stuff is happening in micro-sections, says Leinwebber, you need what he calls “algo-jockeys” to make sure the machines are working properly. He likens it to putting your car on automatic pilot. The computer is able to control certain things like speed, but you still have to step in and make those quick turns.

In the markets, with the machines streaming huge amounts of data, problems occur such as the flash crash, which happened when trading algorithms ran amok and the prices on some stocks dropped to near zero.

What the machines needed was human participation, someone who could see what the algorithms couldn’t see and that the overwhelming flow of information had created a logjam and a situation similar to what happens when you stream a movie over the Internet. Every now and then the picture goes a little whacky.

Big Data is Like Seawater

Bill Inmon, a data warehouse expert, has a wonderful comparison of big data to seawater on his blog d-eye-network.com. Bill writes: “The truth is there is nothing inherently good or bad about seawater … In many respects seawater is like big data … Like the seawater in the ocean, there is a lot of it. Like seawater, big data is impure. Big data contains every type of data that is imaginable. And all (or at least nearly all) of the data from big data is unstructured.” Inmon adds that just as you can’t drink seawater because there’s too much salt in it, as well as microbes and bacteria, big data needs to be refined to make it useful as well.