High Frequency Statistical Arbitrage

High-frequency statistical arbitrage leverages sophisticated quantitative models and cutting-edge technology to exploit fleeting inefficiencies in global markets. Pioneered by hedge funds and proprietary trading firms over the last decade, the strategy identifies and capitalizes on sub-second price discrepancies across assets ranging from public equities to foreign exchange.

At its core, statistical arbitrage aims to predict short-term price movements based on probability theory and historical relationships. When implemented at high frequencies—microseconds or milliseconds—the quantitative models uncover trading opportunities unavailable to human traders. The predictive signals are then executable via automated, low-latency infrastructure.

These strategies thrive on speed. By getting pricing data faster, determining anomalies faster, and executing orders faster than the rest of the market, you expand the momentary windows to trade profitably.

Seminal papers have delved into the mathematical and technical nuances underpinning high-frequency statistical arbitrage. Zhaodong Zhong and Jian Wang’s 2014 paper develops stochastic models to quantify how market microstructure and randomness influence high-frequency trading outcomes. Samuel Wong’s 2018 research explores adapting statistical arbitrage for the nascent cryptocurrency markets.

Yet maximizing the strategy’s profitability poses an ongoing challenge. Changing market dynamics necessitate regular algorithm tweaking and infrastructure upgrades. It’s an arms race for lower latency and better predictive signals. Any edge gained disappears quickly as new firms implement similar systems. Regulatory attention also persists due to concerns over unintended impacts on market stability.

Nonetheless, high-frequency statistical arbitrage retains a crucial role for leading quant funds. Ongoing advances in machine learning, cloud computing, and execution technology promise to further empower the strategy. Though the competitive landscape grows more challenging, the cutting edge continues advancing profitably. Where human perception fails, automated high-frequency strategies recognize and seize value.

Implementing an Intraday Statistical Arbitrage Model

While HFT infrastructure and know-how are beyond the reach of most traders, it is possible to conceive of a system for pairs trading at moderate frequency, say 1-minute intervals.

We illustrate the approach with an algorithm that was originally showcased by Mathworks some years ago (but which has since slipped off the radar and is no longer available to download).  I’ve amended the code to improve its efficiency, but the core idea remains the same:  we conduct a rolling backtest in which data on a pair of assets, in this case spot prices of Brent Crude (LCO) and West Texas Intermediate (WTI), is subdivided into in-sample and out-of-sample periods of varying lengths.  We seek to identify windows in which the price series are cointegrated in the sense of Engle-Granger and then apply the regression parameters to take long and short positions in the pair during the corresponding out-of-sample period.  The idea is to trade only when there is compelling evidence of cointegration between the two series and to avoid trading at other times.

The critical part of the walk-forward analysis code is as shown below.  Note we are using a function parametersweep to conduct a grid search across a range of in-sample dataset sizes to determine if the series are cointegrated (according to the Engle-Granger test) in that sub-period and, if so, determine the position size according to the regression parameters.  The optimal in-sample parameters are then applied in the out-of-sample period and the performance results are recorded. 

Here we are making use of Matlab’s parallelization capabilities, which work seamlessly to spread the processing load across available CPUs, handling the distribution of variables, function definitions and dependencies with ease.  My experience with trying to parallelize Python, by contrast, is often a frustrating one that frequently fails at the first several attempts.

The results appear promising; however, the data is out-of-date, comes from a source that can be less than 100% reliable and may represent price quotes rather than traded prices.  If we switch to 1-minute traded prices in a pair of stocks such as PEP and KO that are known to be cointegrated over long horizons, the outcome is very different:


Conclusion

High-frequency statistical arbitrage represents the convergence of cutting-edge technology and quantitative modeling to uncover fleeting trading advantages invisible to human market participants. This strategy has proven profitable for sophisticated hedge funds and prop shops, but also raises broader questions around fairness, regulation, and the future of finance.

However, the competitive edge gained from high-frequency strategies diminishes quickly as the technology diffuses across the industry. Firms must run faster just to stand still.

Continued advancement in machine learning, cloud computing, and execution infrastructure promises to expand the frontier. But practitioners and policymakers alike share responsibility for ensuring market integrity and stability amidst this technology arms race.

In conclusion, high-frequency statistical arbitrage remains essential to many leading quantitative firms, with the competitive landscape growing ever more challenging. Realizing the potential of emerging innovations, while promoting healthy markets that benefit all participants, will require both vision and wisdom. The path ahead lies between cooperation and competition, ethics and incentives. By bridging these domains, high-frequency strategies can contribute positively to financial evolution while capturing sustainable edge.

References:

Zhong, Zhaodong, and Jian Wang. “High-Frequency Trading and Probability Theory.” (2014).

Wong, Samuel S. Y. “A High-Frequency Algorithmic Trading Strategy for Cryptocurrency.” (2018).

Glossary

For those unfamiliar with the topic of statistical arbitrage and its commonly used terms and concepts, check out my book Equity Analytics, which covers the subject matter in considerable detail.

Hiring High Frequency Quant/Traders

I am hiring in Chicago for exceptional HF Quant/Traders in Equities, F/X, Futures & Fixed Income.  Remuneration for these roles, which will be dependent on qualifications and experience, will be in line with the highest market levels.

Role
Working closely with team members including developers, traders and quantitative researchers, the central focus of the role will be to research and develop high frequency trading strategies in equities, fixed income, foreign exchange and related commodities markets.

Responsibilities
The analyst will have responsibility of taking an idea from initial conception through research, testing and implementation.  The work will entail:

  • Formulation of mathematical and econometric models for market microstructure
  • Data collation, normalization and analysis
  • Model prototyping and programming
  • Strategy development, simulation, back-testing and implementation
  • Execution strategy & algorithms

Qualifications & Experience

  • Minimum 5 years in quantitative research with a leading proprietary trading firm, hedge fund, or investment bank
  • In-depth knowledge of Equities, F/X and/or futures markets, products and operational infrastructure
  • High frequency data management & data mining techniques
  • Microstructure modeling
  • High frequency econometrics (cointegration, VAR,error correction models, GARCH, panel data models, etc.)
  • Machine learning, signal processing, state space modeling and pattern recognition
  • Trade execution and algorithmic trading
  • PhD in Physics/Math/Engineering, Finance/Economics/Statistics
  • Expert programming skills in Java, Matlab/Mathematica essential
  • Must be US Citizen or Permanent Resident

Send your resume to: jkinlay at systematic-strategies.com.

No recruiters please.

Master’s in High Frequency Finance

I have been discussing with some potential academic partners the concept for a new graduate program in High Frequency Finance.  The idea is to take the concept of the Computational Finance program developed in the 1990s and update it to meet the needs of students in the 2010s.

The program will offer a thorough grounding in the modeling concepts, trading strategies and risk management procedures currently in use by leading investment banks, proprietary trading firms and hedge funds in US and international financial markets.  Students will also learn the necessary programming and systems design skills to enable them to make an effective contribution as quantitative analysts, traders, risk managers and developers.

I would be interested in feedback and suggestions as to the proposed content of the program.

Career Opportunity for Quant Traders

Career Opportunity for Quant Traders as Strategy Managers

We are looking for 3-4 traders (or trading teams) to showcase as Strategy Managers on our Algorithmic Trading Platform.  Ideally these would be systematic quant traders, since that is the focus of our fund (although they don’t have to be).  So far the platform offers a total of 10 strategies in equities, options, futures and f/x.  Five of these are run by external Strategy Managers and five are run internally.

The goal is to help Strategy Managers build a track record and gain traction with a potential audience of over 100,000 members.  After a period of 6-12 months we will offer successful managers a position as a PM at Systematic Strategies and offer their strategies in our quantitative hedge fund.  Alternatively, we will assist the manager is raising external capital in order to establish their own fund.

If you are interested in the possibility (or know a talented rising star who might be), details are given below.

Manager Platform

Systematic Futures Trading

In its proprietary trading, Systematic Strategies primary focus in on equity and volatility strategies, both low and high frequency. In futures, the emphasis is on high frequency trading, although we also run one or two lower frequency strategies that have higher capacity, such as the Futures WealthBuilder. The version of WealthBuilder running on the Collective 2 site has performed very well in 2017, with net returns of 30% and a Sharpe Ratio of 3.4:

Futures C2 oct 2017

 

In the high frequency space, our focus is on strategies with very high Sharpe Ratios and low drawdowns. We trade a range of futures products, including equity, fixed income, metals and energy markets. Despite the current low levels of market volatility, these strategies have performed well in 2017:

HFT Futures Oct 2017 (NFA)

Building high frequency strategies with double-digit Sharpe Ratios requires a synergy of computational capability and modeling know-how. The microstructure of futures markets is, of course, substantially different to that of equity or forex markets and the components of the model that include microstructure effects vary widely from one product to another. There can be substantial variations too in the way that time is handled in the model – whether as discrete or continuous “wall time”, in trade time, or some other measure. But some of the simple technical indicators we use – moving averages, for example – are common to many models across different products and markets. Machine learning plays a role in most of our trading strategies, including high frequency.

Here are some relevant blog posts that you may find interesting:

http://jonathankinlay.com/2016/04/high-frequency-trading-equities-vs-futures/

 

http://jonathankinlay.com/2015/05/designing-scalable-futures-strategy/

 

http://jonathankinlay.com/2014/10/day-trading-system-in-vix-futures/

The New Long/Short Equity

High Frequency Trading Strategies

One of the benefits of high frequency trading strategies lies in their ability to produce risk-adjusted rates of return that are unmatched by anything that the hedge fund or CTA community is capable of producing.  With such performance comes another attractive feature of HFT firms – their ability to make money (almost) every day.  Of course, HFT firms are typically not required to manage billions of dollars, which is just as well given the limited capacity of most HFT strategies.  But, then again, with a Sharpe ratio of 10, who needs outside capital?  This explains why most investors have a difficult time believing the level of performance achievable in the high frequency world – they never come across such performance, because HFT firms generally have little incentive to show their results to external investors.

SSALGOTRADING AD

By and large, HFT strategies remain the province of proprietary trading firms that can afford to make an investment in low-latency trading infrastructure that far exceeds what is typically required for a regular trading or investment management firm.  However, while the highest levels of investment performance lie beyond the reach of most investors and money managers, it is still possible to replicate some of the desirable characteristics of high frequency strategies.

Quantitative Equity Strategy

I am going to use an example our Quantitative Equity strategy, which forms part of the Systematic Strategies hedge fund.  The tables and charts below give a broad impression of the performance characteristics of the strategy, which include a CAGR of 14.85% (net of fees) since live trading began in 2013.

Value $1000
The NewEquityLSFig3

 

 

 

 

 

 

 

 

This is a strategy that is designed to produce returns on a  par with the S&P 500 index, but with considerably lower risk:  at just over 4%, the annual volatility of the strategy is only around 1/3 that of the index, while the maximum drawdown has been a little over 2% since inception.  This level of portfolio risk is much lower than can typically be achieved in an equity long/short strategy  (equity market neutral is another story, of course). Furthermore, the realized information ratio of 3.4 is in the upper 1%-tile of risk-adjusted performance amongst equity long/short strategies.  So something rather more interesting must be going on that is very different from the typical approach to long/short equity.
TheNewEquityLSFig5

 

One plausible explanation is that the strategy is exploiting some minor market anomaly that works fine for small amounts of capital, but which cannot be scaled.  But this is not the case here:  the investment universe comprises more than a hundred of the most liquid stocks in US markets, across a broad spectrum of sectors.  And while single-name investment is capped at 10% of average daily volume, this nonetheless provides investment capacity of several hundreds of millions of dollars.

Nor does the reason for the exceptional performance lie in some new portfolio construction technique:  rather, we rely on a straightforward 1/n allocation.  Again, neither is factor exposure the driver of strategy alpha:  as the factor loading table illustrates, strategy performance is largely uncorrelated with most market indices.  It loads significantly on only large cap value, chiefly because the investment universe is defined as comprising the stocks with greatest liquidity (which tend to be large cap value), and on the CBOE VIX index.  The positive correlation with market volatility is a common feature of many types of trading strategy that tend to do better in volatile markets, when short-term investment opportunities are plentiful.

FactorLoadings

While the detail of the strategy must necessarily remain proprietary, I can at least offer some insight that will, I hope, provide food for thought.

We can begin by comparing the returns for two of the stocks in the portfolio, Home Depot and Pfizer.  The charts demonstrate one of important strategy characteristic: not every stock is traded at the same frequency.  Some stocks might be traded once or twice a month; others possibly ten times a day, or more.  In other words, the overall strategy is diversified significantly, not only across assets, but also across investment horizons.  This has a considerable impact on volatility and downside risk in the portfolio.

Home Depot vs. Pfizer Inc.

HD

PFEOverall, the strategy trades an average of 40-60 times a day, or more.   This is, admittedly, towards the low end of the frequency spectrum of HFT strategies – we might describe it as mid-frequency rather than high frequency trading.  Nonetheless,  compared to traditional long/short equity strategies this constitutes a high level of trading activity which, in aggregate, replicates some of the time-diversification benefits of HFT strategies, producing lower strategy volatility.

There is another way in which the strategy mimics, at least partially, the characteristics of a HFT strategy.  The profitability of many (although by no means all) HFT strategies lies in their ability to capture (or, at least, not pay) the bid-offer spread.  That is why latency is so crucial to most HFT strategies – if your aim is to to earn rebates, and/or capture the spread, you must enter and  exit, passively, often using microstructure models to determine when to lean on the bid or offer price.  That in turn depends on achieving a high priority for your orders in the limit order book, which is a function of  latency – you need to be near the top of the queue at all times in order the achieve the required fill rate.

How does that apply here?  While we are not looking to capture the spread, the strategy does seek to avoid taking liquidity and paying the spread.  Where it can do so,  it will offset the bid-offer spread by earning rebates.  In many cases we are able to mitigate the spread cost altogether.  So, while it cannot accomplish what a HFT market-making system can achieve, it can mimic enough of its characteristics – even at low frequency – to produce substantial gains in terms of cost-reduction and return enhancement.  This is important since the transaction volume and portfolio turnover in this approach are significantly greater than for a typical equity long/short strategy.

Portfolio of Strategies vs. Portfolio of Equities

slide06But this feature, while important, is not really the heart of the matter.  Rather, the central point is this:  that the overall strategy is an assembly of individual, independent strategies for each component stock.  And it turns out that the diversification benefit of a portfolio of strategies is generally far greater than for an equal number of stocks, because the equity processes themselves will typically be correlated to a far greater degree than will corresponding trading strategies.  To take the example of the pair of stocks discussed earlier, we find that the correlation between HD and PFE over the period from 2013 to 2017 is around 0.39, based on daily returns.  By comparison, the correlation between the strategies for the two stocks over the same period is only 0.01.

This is generally the case, so that a portfolio of, say, 30 equity strategies, might reasonably be expected to enjoy a level of risk that is perhaps as much as one half that of a portfolio of the underlying stocks, no matter how constructed.  This may be due to diversification in the time dimension, coupled with differences in the alpha generation mechanisms of the underlying strategies – mean reversion vs. momentum, for example

Strategy Robustness Testing

There are, of course, many different aspects to our approach to strategy risk management. Some of these are generally applicable to strategies of all varieties, but there are others that are specific to this particular type of strategy.

A good example of the latter is how we address the issue of strategy robustness. One of the principal concerns that investors have about quantitive strategies is that they may under-perform during adverse market conditions, or even simply stop working altogether. Our approach is to stress test each of the sub-strategy models using Monte Carlo simulation and examine their performance under a wide range of different scenarios, many of which have never been seen in the historical data used to construct the models.

For instance, we typically allow prices to fluctuate randomly by +/- 30% from historical values. But we also randomize the start date of each strategy by up to a year, which reduces the likelihood of a strategy being selected simply on the strength of a lucky start. Finally, we are interested in ensuring that the performance of each sub-strategy is not overly sensitive to the specific parameter values chosen for each model. Again, we test this using Monte Carlo, assessing the performance of each sub-strategy if the parameter values of the model are varied randomly by up to 30%.

The output of all these simulation tests is compiled into a histogram of performance results, from which we select the worst 5%-tile. Only if the worst outcomes – the 1-in-20 results in the left tail of the performance distribution – meet our performance criteria will the sub-strategy advance to the next stage of evaluation, simulated trading. This gives us – and investors – a level of confidence in the ability of the strategy to continue to perform well regardless of how market conditions evolve over time.

MonteCarlo Stress test

 

An obvious question to ask at this point is: if this is such a great idea, why don’t more firms use this approach?  The answer is simple: it involves too much research.  In a typical portfolio strategy there is a single investment idea that is applied cross-sectionally to a universe of stocks (factor models, momentum models, etc).  In the strategy portfolio approach, separate strategies must be developed for each stock individually, which takes far more time and effort.  Consequently such strategies must necessarily scale more slowly.

Another downside to the strategy portfolio approach is that it is less able to control the portfolio characteristics.  For instance, the overall portfolio may, on average, have a beta close to zero; but there are likely to be times when a majority of the individual stock strategies align, producing a significantly higher, or lower, beta.  The key here is to ask the question: what matters more – the semblance of risk control, or the actual risk characteristics of the strategy?  In reality, the risk controls of traditional long/short equity strategies often turn out to be more theoretical than real.  Time and again investors have seen strategies that turn out to be downside-correlated with the market, regardless of the purported “market-neutral” characteristics of the portfolio.  I would argue that what matters far more is how the strategy actually performs under conditions of market stress, regardless of how “market neutral” or “sector neutral” it may purport to be.  And while I agree that this is hardly a widely-held view, my argument would be that one cannot expect to achieve above-average performance simply by employing standard approaches at every turn.

Parallels with Fund of Funds Investment

So, is this really a “new approach” to equity long/short? Actually, no.  It is certainly unusual.  But it follows quite closely the model of a proprietary trading firm, or a Fund of Funds. There, as here, the task is to create a combined portfolio of strategies (or managers), rather than by investing directly in the underlying assets.  A Fund of Funds will seek to create a portfolio of strategies that have low correlations to one another, and may operate a meta-strategy for allocating capital to the component strategies, or managers.  But the overall investment portfolio cannot be as easily constrained as an individual equity portfolio can be – greater leeway must be allowed for the beta, or the dollar imbalance in the longs and shorts, to vary from time to time, even if over the long term the fluctuations average out.  With human managers one always has to be concerned about the risk of “style drift” – i.e. when managers move away from their stated investment mandate, methodologies or objectives, resulting in a different investment outcomes.  This can result in changes in the correlation between a strategy and its peers, or with the overall market.  Quantitative strategies are necessarily more consistent in their investment approach – machines generally don’t alter their own source code – making a drift in style less likely.  So an argument can be made that the risk inherent in this form of equity long/short strategy is on a par with – certainly not greater than – that of a typical fund of funds.

Conclusions

An investment approach that seeks to create a portfolio of strategies, rather than of underlying assets, offers a significant advantage in terms of risk reduction and diversification, due to the relatively low levels of correlation between the component strategies.   The trading costs associated with higher frequency trading can be mitigated using passive entry/exit rules designed to avoid taking liquidity and generating exchange rebates.  The downside is that it is much harder to manage the risk attributes of the portfolio, such as the portfolio beta, sector risk, or even the overall net long/short exposure.  But these are indicators of strategy risk, rather than actual risk itself and they often fail to predict the actual risk characteristics of the strategy, especially during conditions of market stress.  Investors may be better served by an approach to long/short equity that seeks to maximize diversification on the temporal axis as well as in terms of the factors driving strategy alpha.

 

Disclaimer: past performance does not guarantee future results. You should not rely on any past performance as a guarantee of future investment performance. Investment returns will fluctuate. Investment monies are at risk and you may suffer losses on any investment.