Overnight Trading in the E-Mini S&P 500 Futures

Jeff Swanson’s Trading System Success web site is often worth a visit for those looking for new trading ideas.

A recent post Seasonality S&P Market Session caught my eye, having investigated several ideas for overnight trading in the E-minis.  Seasonal effects are of course widely recognized and traded in commodities markets, but they can also apply to financial products such as the E-mini.  Jeff’s point about session times is well-made:  it is often worthwhile to look at the behavior of an asset, not only in different time frames, but also during different periods of the trading day, day of the week, or month of the year.

Jeff breaks the E-mini trading session into several basic sub-sessions:

  1. “Pre-Market” Between 530 and 830
  2. “Open” Between 830 and 900
  3. “Morning” Between 900 though 1130
  4. “Lunch” Between 1130 and 1315
  5. “Afternoon” Between 1315 and 1400
  6. “Close” Between 1400 and 1515
  7. “Post-Market” Between 1515 and 1800
  8. “Night” Between 1800 and 530

In his analysis Jeff’s strategy is simply to buy at the open of the session and close that trade at the conclusion of the session. This mirrors the traditional seasonality study where a trade is opened at the beginning of the season and closed several months later when the season comes to an end.

Evaluating Overnight Session and Seasonal Effects

The analysis evaluates the performance of this basic strategy during the “bullish season”, from Nov-May, when the equity markets traditionally make the majority of their annual gains, compared to the outcome during the “bearish season” from Jun-Oct.

None of the outcomes of these tests is especially noteworthy, save one:  the performance during the overnight session in the bullish season:

Fig 1

The tendency of the overnight session in the E-mini to produce clearer trends and trading signals has been well documented.  Plausible explanations for this phenomenon are that:

(a) The returns process in the overnight session is less contaminated with noise, which primarily results from trading activity; and/or

(b) The relatively poor liquidity of the overnight session allows participants to push the market in one direction more easily.

Either way, there is no denying that this study and several other, similar studies appear to demonstrate interesting trading opportunities in the overnight market.

That is, until trading costs are considered.  Results for the trading strategy from Nov 1997-Nov 2015 show a gain of $54,575, but an average trade of only just over $20:

Gross PL

# Trades

Av Trade

$54,575

2701

$20.21

Assuming that we enter and exit aggressively, buying at the market at the start of the session and selling MOC at the close, we will pay the bid-offer spread and commissions amounting to around $30, producing a net loss of $10 per trade.

The situation can be improved by omitting January from the “bullish season”, but the slightly higher average trade is still insufficient to overcome trading costs :

Gross PL

# Trades

Av Trade

$54,550

2327

$23.44

SSALGOTRADING AD

Designing a Seasonal Trading Strategy for the Overnight Session

At this point an academic research paper might conclude that the apparently anomalous trading profits are subsumed within the bid-offer spread.  But for a trading system designer this is not the end of the story.

If the profits are insufficient to overcome trading frictions when we cross the spread on entry and exit, what about a trading strategy that permits market orders on only the exit leg of the trade, while using limit orders to enter?  Total trading costs will be reduced to something closer to $17.50 per round turn, leaving a net profit of almost $6 per trade.

Of course, there is no guarantee that we will successfully enter every trade – our limit orders may not be filled at the bid price and, indeed, we are likely to suffer adverse selection – i.e. getting filled on every losing trading, while missing a proportion of the winning trades.

On the other hand, we are hardly obliged to hold a position for the entire overnight session.  Nor are we obliged to exit every trade MOC – we might find opportunities to exit prior to the end of the session, using limit orders to achieve a profit target or cap a trading loss.  In such a system, some proportion of the trades will use limit orders on both entry and exit, reducing trading costs for those trades to around $5 per round turn.

The key point is that we can use the seasonal effects detected in the overnight session as a starting point for the development for a more sophisticated trading system that uses a variety of entry and exit criteria, and order types.

The following shows the performance results for a trading system designed to trade 30-minute bars in the E-mini futures overnight session during the months of Nov to May.The strategy enters trades using limit prices and exits using a combination of profit targets, stop loss targets, and MOC orders.

Data from 1997 to 2010 were used to design the system, which was tested on out-of-sample data from 2011 to 2013.  Unseen data from Jan 2014 to Nov 2015 were used to provide a further (double blind) evaluation period for the strategy.

Fig 2

 

 

  

ALL TRADES

LONG

SHORT

Closed Trade Net Profit

$83,080

$61,493

$21,588

  Gross Profit

$158,193

$132,573

$25,620

  Gross Loss

-$75,113

-$71,080

-$4,033

Profit Factor

2.11

1.87

6.35

Ratio L/S Net Profit

2.85

Total Net Profit

$83,080

$61,493

$21,588

Trading Period

11/13/97 2:30:00 AM to 12/31/13 6:30:00 AM (16 years 48 days)

Number of Trading Days

2767

Starting Account Equity

$100,000

Highest Equity

$183,080

Lowest Equity

$97,550

Final Closed Trade Equity

$183,080

Return on Starting Equity

83.08%

Number of Closed Trades

849

789

60

  Number of Winning Trades

564

528

36

  Number of Losing Trades

285

261

24

  Trades Not Taken

0

0

0

Percent Profitable

66.43%

66.92%

60.00%

Trades Per Year

52.63

48.91

3.72

Trades Per Month

4.39

4.08

0.31

Max Position Size

1

1

1

Average Trade (Expectation)

$97.86

$77.94

$359.79

Average Trade (%)

0.07%

0.06%

0.33%

Trade Standard Deviation

$641.97

$552.56

$1,330.60

Trade Standard Deviation (%)

0.48%

0.44%

1.20%

Average Bars in Trades

15.2

14.53

24.1

Average MAE

$190.34

$181.83

$302.29

Average MAE (%)

0.14%

0.15%

0.27%

Maximum MAE

$3,237

$2,850

$3,237

Maximum MAE (%)

2.77%

2.52%

3.10%

Win/Loss Ratio

1.06

0.92

4.24

Win/Loss Ratio (%)

2.10

1.83

7.04

Return/Drawdown Ratio

15.36

14.82

5.86

Sharpe Ratio

0.43

0.46

0.52

Sortino Ratio

1.61

1.69

6.40

MAR Ratio

0.71

0.73

0.33

Correlation Coefficient

0.95

0.96

0.719

Statistical Significance

100%

100%

97.78%

Average Risk

$1,099

$1,182

$0.00

Average Risk (%)

0.78%

0.95%

0.00%

Average R-Multiple (Expectancy)

0.0615

0.0662

0

R-Multiple Standard Deviation

0.4357

0.4357

0

Average Leverage

0.399

0.451

0.463

Maximum Leverage

0.685

0.694

0.714

Risk of Ruin

0.00%

0.00%

0.00%

Kelly f

34.89%

31.04%

50.56%

Average Annual Profit/Loss

$5,150

$3,811

$1,338

Ave Annual Compounded Return

3.82%

3.02%

1.22%

Average Monthly Profit/Loss

$429.17

$317.66

$111.52

Ave Monthly Compounded Return

0.31%

0.25%

0.10%

Average Weekly Profit/Loss

$98.70

$73.05

$25.65

Ave Weekly Compounded Return

0.07%

0.06%

0.02%

Average Daily Profit/Loss

$30.03

$22.22

$7.80

Ave Daily Compounded Return

0.02%

0.02%

0.01%

INTRA-BAR EQUITY DRAWDOWNS

ALL TRADES

LONG

SHORT

Number of Drawdowns

445

422

79

Average Drawdown

$282.88

$269.15

$441.23

Average Drawdown (%)

0.21%

0.20%

0.33%

Average Length of Drawdowns

10 days 19 hours

10 days 20 hours

66 days 1 hours

Average Trades in Drawdowns

3

3

1

Worst Case Drawdown

$6,502

$4,987

$4,350

Date at Trough

12/13/00 1:30

5/24/00 4:30

12/13/00 1:30

Signal Processing and Sample Frequency

The Importance of Sample Frequency

Too often we apply a default time horizon for our trading, whether it below (daily, weekly) or higher (hourly, 5 minute) frequency.  Sometimes the choice is dictated by practical considerations, such as a desire to avoid overnight risk, or the (lack of0 availability of low-latency execution platform.

But there is an alternative approach to the trade frequency decision that often yields superior results in terms of trading performance.    The methodology derives from signal processing and the idea essentially is to use Fourier transforms to help identify the cyclical behavior of the strategy alpha and hence determine the best time-frames for sampling and trading.  I wrote about this is a previous blog post, in which I described how to use principal components analysis to investigate the factors driving the returns in various pairs trading strategies.  Here I want to take a simpler approach, in which we use Fourier analysis to select suitable sample frequencies.  The idea is simply to select sample frequencies where the signal strength appears strongest, in the hope that it will lead to superior performance characteristics in what strategy we are trying to develop.

Signal Decomposition for S&P500 eMini Futures

Let’s take as an example the S&P 500 emini futures contract. The chart below shows the continuous ES futures contract plotted at 1-minute intervals from 1998. At the bottom of the chart I have represented the signal analysis as a bar chart (in blue), with each bar representing the amplitude at each frequency. The white dots on the chart identify frequencies that are spaced 10 minutes apart.  It is immediately evident that local maxima in the spectrum occur around 40 mins, 60 mins and 120 mins.  So a starting point for our strategy research might be to look at emini data sampled at these frequencies.  Incidentally, it is worth pointing out that I have restricted the session times to 7AM – 4PM EST, which is where the bulk of the daily volume and liquidity tend to occur.  You may get different results if you include data from the Globex session.

Emini Signal

This is all very intuitive and unsurprising: the clearest signals occur at frequencies that most traders typically tend to trade, using hourly data, for example. Any strategy developer is already quite likely to consider these and other common frequencies as part of their regular research process.  There are many instances of successful trading strategies built on emini data sampled at 60 minute intervals.

SSALGOTRADING AD

Signal Decomposition for US Bond Futures

Let’s look at a rather more interesting example:  US (30 year) Bond futures. Unlike the emini contract, the spectral analysis of the US futures contract indicates that the strongest signal by far occurs at a frequency of around 47 minutes.  This is decidedly an unintuitive outcome – I can’t think of any reason why such a strong signal should appear at this cycle length, but, statistically it does. 

US Bond futures

Does it work?  Readers can judge for themselves:  below is an example of an equity curve for a strategy on US futures sampled at 47 minute frequency over the period from 2002.  The strategy has performed very consistently, producing around $25,000 per contract per year, after commissions and slippage.

US futures EC

Conclusion

While I have had similar success with products as diverse as Corn and VIX futures, the frequency domain approach is by no means a panacea:  there are plenty of examples where I have been unable to construct profitable strategies for data sampled at the frequencies with very strong signals. Conversely, I have developed successful strategies using data at frequencies that hardly registered at all on the spectrum, but which I selected for other reasons.  Nonetheless, spectral analysis (and signal processing in general) can be recommended as a useful tool in the arsenal of any quantitative analyst.

Trading Strategy Design

In this post I want to share some thoughts on how to design great automated trading strategies – what to look for, and what to avoid.

For illustrative purposes I am going to use a strategy I designed for the ever-popular S&P500 e-mini futures contract.

The overall equity curve for the strategy is show below.

@ES Equity Curve

This is often the best place to start.  What you want to see, of course, is a smooth, upward-sloping curve, without too many sizable drawdowns, and one in which the strategy continues to make new highs.  This is especially important in the out-of-sample test period (Jan 2014- Jul 2015 in this case).  You will notice a flat period around 2013, which we will need to explore later.  Overall, however, this equity curve appears to fit the stereotypical pattern we hope to see when developing a new strategy.

Let’s move on look at the overall strategy performance numbers.

STRATEGY PERFORMANCE CHARACTERISTICS

@ES Perf Summary(click to enlarge)

 1. Net Profit
Clearly, the most important consideration.  Over the 17 year test period the strategy has produced a net profit  averaging around $23,000 per annum, per contract.  As a rough guide, you would want to see a net profit per contract around 10x the maintenance margin, or higher.

2. Profit Factor
The gross profit divided by the gross loss.  You want this to be as high as possible. Too low, as the strategy will be difficult to trade, because you will see sustained periods of substantial losses.  I would suggest a minimum acceptable PF in the region of 1.25.  Many strategy developers aim for a PF of 1.5, or higher.

3. Number of Trades
Generally, the more trades the better, at least from the point of view of building confidence in the robustness of strategy performance.  A strategy may show a great P&L, but if it only trades once a month it is going to take many many years of performance data to ensure statistical significance.  This strategy, on the other hand, is designed to trade 2-3 times a day.  Given that, and the length of the test period, there is little doubt that the results are statistically significant.

SSALGOTRADING AD

Profit Factor and number of trades are opposing design criteria – increasing the # trades tends to reduce the PF.  That consideration sets an upper bound on the # trades that can be accommodated, before the profit factor deteriorates to unacceptably low levels.  Typically, 4-5 trades a day is about the maximum trading frequency one can expect to achieve.

4. Win Rate
Novice system designers tend to assume that you want this to be as high as possible, but that isn’t typically the case.  It is perfectly feasible to design systems that have a 90% win rate, or higher, but which produce highly undesirable performance characteristics, such as frequent, large drawdowns.  For a typical trading system the optimal range for the win rate is in the region of 40% to 66%.  Below this range, it becomes difficult to tolerate the long sequences of losses that will result, without losing faith in the system.

5. Average Trade
This is the average net profit per trade.  A typical range would be $10 to $100.  Many designers will only consider strategies that have a higher average trade than this one, perhaps $50-$75, or more.  The issue with systems that have a very small average trade is that the profits can quickly be eaten up by commissions. Even though, in this case, the results are net of commissions, one can see a significant deterioration in profits if the average trade is low and trade frequency is high, because of the risk of low fill rates (i.e. the % of limit orders that get filled).  To assess this risk one looks at the number of fills assumed to take place at the high or low of the bar.  If this exceeds 10% of the total # trades, one can expect to see some slippage in the P&L when the strategy is put into production.

6. Average Bars
The number of bars required to complete a trade, on average.  There is no hard limit one can suggest here – it depends entirely on the size of the bars.  Here we are working in 60 minute bars, so a typical trade is held for around 4.5 hours, on average.   That’s a time-frame that I am comfortable with.  Others may be prepared to hold positions for much longer – days, or even weeks.

Perhaps more important is the average length of losing trades. What you don’t want to see is the strategy taking far longer to exit losing trades than winning trades. Again, this is a matter of trader psychology – it is hard to sit there hour after hour, or day after day, in a losing position – the temptation to cut the position becomes hard to ignore.  But, in doing that you are changing the strategy characteristics in a fundamental way, one that rarely produces a performance improvement.

What the strategy designer needs to do is to figure out in advance what the limits are of the investor’s tolerance for pain, in terms of maximum drawdown, average losing trade, etc, and design the strategy to meet those specifications, rather than trying to fix the strategy afterwards.

7. Required Account Size
It’s good to know exactly how large an account you need per contract, so you can figure out how to scale the strategy.  In this case one could hope to scale the strategy up to a 10-lot in a $100,000 account.  That may or may not fit the trader’s requirements and again, this needs to be considered at the outset.  For example, for a trader looking to utilize, say, $1,000,000 of capital, it is doubtful whether this strategy would fit his requirements without considerable work on the implementations issues that arise when trying to trade in anything approaching a 100 contract clip rate.

8. Commission
Always check to ensure that the strategy designer has made reasonable assumptions about slippage and commission.  Here we are assuming $5 per round turn.  There is no slippage, because the strategy executes using limit orders.

9. Drawdown
Drawdowns are, of course, every investor’s bugbear.  No-one likes drawdowns that are either large, or lengthy in relation to the annual profitability of the strategy, or the average trade duration.  A $10,000 max drawdown on a strategy producing over $23,000 a year is actually quite decent – I have seen many e-mini strategies with drawdowns at 2x – 3x that level, or larger.  Again, this is one of the key criteria that needs to be baked into the strategy design at the outset, rather than trying to fix later.

 ANNUAL PROFITABILITY

Let’s now take a look at how the strategy performs year-by-year, and some of the considerations and concerns that often arise.

@ES Annual1. Performance During Downturns
One aspect I always pay attention to is how well the strategy performs during periods of high market stress, because I expect similar conditions to arise in the fairly near future, e.g. as the Fed begins to raise rates.

Here, as you can see, the strategy performed admirably during both the dot com bust of 1999/2000 and the financial crisis of 2008/09.

2. Consistency in the # Trades and % Win Rate
It is not uncommon with low frequency strategies to see periods of substantial variation in the # trades or win rate.  Regardless how good the overall performance statistics are, this makes me uncomfortable.  It could be, for instance, that the overall results are influenced by one or two exceptional years that are unlikely to be repeated.  Significant variation in the trading or win rate raise questions about the robustness of the strategy, going forward.  On the other hand, as here, it is a comfort to see the strategy maintaining a very steady trading rate and % win rate, year after year.

3. Down Years
Every strategy shows variation in year to year performance and one expects to see years in which the strategy performs less well, or even loses money. For me, it rather depends on when such losses arise, as much as the size of the loss.  If a loss occurs in the out-of-sample period it raises serious questions about strategy robustness and, as a result, I am very unlikely to want to put such a strategy into production. If, as here, the period of poor performance occurs during the in-sample period I am less concerned – the strategy has other, favorable characteristics that make it attractive and I am willing to tolerate the risk of one modestly down-year in over 17 years of testing.

INTRA-TRADE DRAWDOWNS

Many trades that end up being profitable go through a period of being under-water.  What matters here is how high those intra-trade losses may climb, before the trade is closed.  To take an extreme example, would you be willing to risk $10,000 to make an average profit of only $10 per trade?  How about $20,000? $50,000? Your entire equity?

The Maximum Average Excursion chart below shows the drawdowns on a trade by trade basis.  Here we can see that, over the 17 year test period, no trade has suffered a drawdown of much more than $5,000.  I am comfortable with that level. Others may prefer a lower limit, or be tolerant of a higher MAE.

MAE

Again, the point is that the problem of a too-high MAE is not something one can fix after the event.  Sure, a stop loss will prevent any losses above a specified size.  But a stop loss also has the unwanted effect of terminating trades that would have turned into money-makers. While psychologically comfortable, the effect of a stop loss is almost always negative  in terms of strategy profitability and other performance characteristics, including drawdown, the very thing that investors are looking to control.

 CONCLUSION
I have tried to give some general guidelines for factors that are of critical importance in strategy design.  There are, of course, no absolutes:  the “right” characteristics depend entirely on the risk preferences of the investor.

One point that strategy designers do need to take on board is the need to factor in all of the important design criteria at the outset, rather than trying (and usually failing) to repair the strategy shortcomings after the event.

 

 

 

High Frequency Trading Strategies

Most investors have probably never seen the P&L of a high frequency trading strategy.  There is a reason for that, of course:  given the typical performance characteristics of a HFT strategy, a trading firm has little need for outside capital.  Besides, HFT strategies can be capacity constrained, a major consideration for institutional investors.  So it is amusing to see the reaction of an investor on encountering the track record of a HFT strategy for the first time.  Accustomed as they are to seeing Sharpe ratios in the range of 0.5-1.5, or perhaps as high as 1.8, if they are lucky, the staggering risk-adjusted returns of a HFT strategy, which often have double-digit Sharpe ratios, are truly mind-boggling.

By way of illustration I have attached below the performance record of one such HFT strategy, which trades around 100 times a day in the eMini S&P 500 contract (including the overnight session).  Note that the edge is not that great – averaging 55% profitable trades and profit per contract of around half a tick – these are some of the defining characteristics of HFT trading strategies.  But due to the large number of trades it results in very substantial profits.  At this frequency, trading commissions are very low, typically under $0.1 per contract, compared to $1 – $2 per contract for a retail trader (in fact an HFT firm would typically own or lease exchange seats to minimize such costs).

Fig 2 Fig 3 Fig 4

 

Hidden from view in the above analysis are the overhead costs associated with implementing such a strategy: the market data feed, execution platform and connectivity capable of handling huge volumes of messages, as well as algo logic to monitor microstructure signals and manage order-book priority.  Without these, the strategy would be impossible to implement profitably.

SSALGOTRADING AD

Scaling things back a little, lets take a look at a day-trading strategy that trades only around 10 times a day, on 15-minute bars.  Although not ultra-high frequency, the strategy nonetheless is sufficiently high frequency to be very latency sensitive. In other words, you would not want to try to implement such a strategy without a high quality market data feed and low-latency trading platform capable of executing at the 1-millisecond level.  It might just be possible to implement a strategy of this kind using TT’s ADL platform, for example.

While the win rate and profit factor are similar to the first strategy, the lower trade frequency allows for a higher trade PL of just over 1 tick, while the equity curve is a lot less smooth reflecting a Sharpe ratio that is “only” around 2.7.

Fig 5 Fig 6 Fig 7

 

The critical assumption in any HFT strategy is the fill rate.  HFT strategies execute using limit or IOC orders and only a certain percentage of these will ever be filled.  Assuming there is alpha in the signal, the P&L grows in direct proportion to the number of trades, which in turn depends on the fill rate.  A fill rate of 10% to 20% is usually enough to guarantee profitability (depending on the quality of the signal). A low fill rate, such as would typically be seen if one attempted to trade on a retail trading platform, would  destroy the profitability of any HFT strategy.

To illustrate this point, we can take a look at the outcome if the above strategy was implemented on a trading platform which resulted in orders being filled only when the market trades through the limit price.  It isn’t a pretty sight.

 

Fig 8

The moral of the story is:  developing a HFT trading algorithm that contains a viable alpha signal is only half the picture.  The trading infrastructure used to implement such a strategy is no less critical.  Which is why HFT firms spend tens, or hundreds of millions of dollars developing the best infrastructure they can afford.

Designing a Scalable Futures Strategy

I have been working on a higher frequency version of the eMini S&P 500 futures strategy, based on 3-minute bar intervals, which is designed to trade a couple of times a week, with hold periods of 2-3 days.  Even higher frequency strategies are possible, of course, but my estimation is that a hold period of under a week provides the best combination of liquidity and capacity.  Furthermore, the strategy is of low enough frequency that it is not at all latency sensitive – indeed, in the performance analysis below I have assumed that the market must trade through the limit price before the system enters a trade (relaxing the assumption and allowing the system to trade when the market touches the limit price improves the performance).

The other important design criteria are the high % of profitable trades and Kelly f (both over 95%).  This enables the investor to employ money management techniques, such a fixed-fractional allocation for example, in order to scale the trade size up from 1 to 10 contracts, without too great a risk of a major drawdown in realized P&L.

The end result is a strategy that produces profits of $80,000 to $100,000 a year on a 10 contract position, with an annual rate of return of 30% and a Sharpe ratio in excess of 2.0.

Furthermore, of the 682 trades since Jan 2010, only 29 have been losers.

Annual P&L (out of sample)

Annual PL

 

Equity Curve

EC

Strategy Performance

Perf 1

What’s the Downside?

Everything comes at a price, of course.  Firstly, the strategy is long-only and, by definition, will perform poorly in falling markets, such as we saw in 2008.  That’s a defensible investment thesis, of course – how many $billions are invested in buy and hold strategies? – and, besides, as one commentator remarked, the trick is to develop multiple strategies for different market regimes (although, sensible as that sounds, one is left with the difficulty of correctly identifying the market regime).

The second drawback is revealed by the trade chart below, which plots the drawdown experienced during each trade.  The great majority of these drawdowns are unrealized, and in most cases the trade recovers to make a profit.  However, there are some very severe cases, such as Sept 2014, when the strategy experienced a drawdown of $85,000 before recovering to make a profit on the trade.  For most investors, the agony of risking an entire year’s P&L just to make a few hundred dollars would be too great.

SSALGOTRADING AD

It should be pointed out that the by the time the drawdown event took place the strategy had already produced many hundreds of thousands of dollars of profit.  So, one could take the view that by that stage the strategy was playing with “house money” and could well afford to take such a risk.

One obvious “solution” to the drawdown problem is to use some kind of stop loss. Unfortunately, the effect is simply to convert an unrealized drawdown into a realized loss.  For some, however, it might be preferable to take a hit of $40,000 or $50,000 once every few years, rather than suffer the  uncertainty of an even larger potential loss.  Either way, despite its many pleasant characteristics, this is not a strategy for investors with weak stomachs!

Trade