COMMENT: The theory that will take artificial intelligence to the trading floor

If you want to make money in finance, you are probably pursuing 'alpha.' But alpha generation is not easy: it requires time series forecasting. It also requires that your (hopefully good) forecasts are turned into profits - and this is where things can get complicated.

When you work on the buy-side in finance, you can realize alpha either by placing orders and trading (aggressing) or by slightly modifying – skewing – the prices that you are quoting to others (known as passive risk management, as opposed to aggressive trading). In each case you leak some information about your forecast to the market – and therefore interact with the very object that you are trying to predict.

This interaction will be key to the application of machine learning in finance. Will the intereraction have no effect? Will it help realise your “prophecy” (in which case it is a self-fulfilling prophecy)?  Or will it thwart it (in which case it is a self-defeating prophecy, both terms having been coined by Robert K. Merton, the father of Robert C. Merton of the Black-Scholes-Merton fame)?

Cybernetic systems and the trading floor 

Trading strategies are prime examples of cybernetic systems. Norbert Wiener introduced cybernetics in 1948 as “the scientific study of control and communication in the animal and the machine”. The word originates from the Greek kubernetes, “steersman” via the 1830s French term cybernétique, “the art of governing”.

Wiener understood the importance of message-driven systems. In The Human Use of Human Beings he wrote: “Messages are themselves a form of pattern and organization. Indeed, it is possible to treat sets of messages as having an entropy like sets of states of the external world. Just as entropy is a measure of disorganization, the information carried by a set of messages is a message of organization... It is possible to interpret the information carried by a message as essentially the negative of its entropy, and the negative logarithm of its probability. That is, the more probable the message, the less information it gives. Clichés, for example, are less illuminating than great poems.”

In cybernetics we are considering the inputs and outputs of a particular system over time, possibly in the presence of feedbacks, which can be positive or negative. We are using the inputs to predict – and hopefully control – the outputs. Cyberneticians postulate: what you can measure, you can (sometimes) forecast; what you can forecast, you can (sometimes) manage; and what you can manage, you can (sometimes) prevent.

In cybernetic systems traders are trying to realise gains and avoid losses in markets where the input time series is used to forecast an output time series in the presence of feed back.

Why applying AI on the trading floor isn't easy

It's not easy to generate alpha as a trader - financial time series are notoriously difficult to deal with. They are non-stationary (their statistical properties change over time), non-Gaussian (often skewed and exhibiting fat tails, making extreme events far more likely than they normally would be), influenced by animal spirits (which Keynes defined as “a spontaneous urge to action rather than inaction” – a property of the human soul), driven by unobservables (or latent variables, such as volatility), affected by human errors (including fat-finger errors), complex and interrelated, often multivariate and high-frequency (consisting of numerous intraday observations arriving at irregular time intervals).

Most of the successes in artificial intelligence (AI) so far have been achieved with images and natural languages. However, financial time series are far more challenging, and so applying AI in finance can be struggle.

Cybernetics suffered from the same issues as AI in the 1970s, 1980s, and 1990s. It was explored first and foremost by academics, rather than by engineers or entrepreneurs. It never became a technology, which Stephen Boyd defines at something that “can be reliably used by many people who do not know, and do not need to know, the details.” The computing power accessible in Wiener’s time was insufficient; the MIT Autocorrelator used by Wiener, Jerome B. Wiesner, and Yuk W. Lee was way off the modern Moore’s Law charts. It didn't help that the mathematical technique for replicating the system - stochastic analysis - is complex and labour-intensive, and better-suited to parsimonious models with few parameters. There was no straightforward way to represent a system with anything but the most trivial feedback loops in software.

The technology available to us today is far more powerful. Message-driven processing, event-driven architectures (EDA), let alone reactive programming, were unheard of in Wiener’s times. Today's technologies mean we can move on from mere cybernetics to neocybernetics. We can turn cybernetics into a technology by using it to create user-friendly processes, algorithms, software libraries and end-user products. We can use high-performance computing (HPC) technology, including cloud computing and potentially, going forward, quantum computing. We can complement stochastic analysis with the simpler mathematical language of deep learning and deep reinforcement learning, which rely on simpler probabilistic ideas to express uncertainty. We can use novel software engineering methodologies, such as the modified Functional Reactive Programming (FRP) incorporating transactions and making a clear expression of feedbacks possible.

We now have the new mathematics that makes neocybernetics accessible; programming languages, such as Python, that simplify the process of data science; numerous libraries for dealing with time series data, such as NumPy, SciPy, Scikit-Learn, Matplotlib, and Pandas; FRP libraries, such as ReactiveX and Sodium; special-purpose databases, such as kdb+/q, suitable for capturing, storing and processing vast amounts of data in real-time; and, using TensorFlow and Keras, more or less any data scientist can calibrate a fairly sophisticated neural net.

Kolmogorov and Wiener both recognised that cybernetics would lead to a different view of human beings and a different appreciation of human life – a new anthropology. Something that Master Yoda summarised as “Luminous beings are we, not this crude matter”. Wiener stated, in The Human Use of Human Beings, that the goal of cybernetics is the age-old struggle of humanity against entropy: “In control and communication we are always fighting nature’s tendency to degrade the organized and to destroy the meaningful; the tendency, as Gibbs has shown us, for entropy to increase.”

It turns out that alpha-generating traders are very well positioned to help out in this quest.

Paul Bilokon is a founder of The Thalesians. The Thalesians are an Artificial Intelligence (AI) company specialising in neocybernetics, digitaleconomy, quantitative finance, education, and consulting. The are experts in (and run courses in)  the application of Machine Learning (ML) techniques to time series data, particularly Big Data and high-frequency data. Our areas of expertise also include the mathematics of ML, Deep Learning (DL), Python, and kdb+/q. A former quant and algorithmic trader at Deutsche Bank, Citi and Nomura, Paul also lectures part time at Imperial College London.

Have a confidential story, tip, or comment you’d like to share? Contact: sbutcher@efinancialcareers.com in the first instance. Whatsapp/Signal/Telegram also available.

Bear with us if you leave a comment at the bottom of this article: all our comments are moderated by human beings. Sometimes these humans might be asleep, or away from their desks, so it may take a while for your comment to appear. Eventually it will – unless it’s offensive or libelous (in which case it won’t.)