Strategy Backtesting in Mathematica

This is a snippet from a strategy backtesting system that I am currently building in Mathematica.

One of the challenges when building systems in WL is to avoid looping wherever possible. This can usually be accomplished with some thought, and the efficiency gains can be significant. But it can be challenging to get one’s head around the appropriate construct using functions like FoldList, etc, especially as there are often edge cases to be taken into consideration.

A case in point is the issue of calculating the profit and loss from individual trades in a trading strategy. The starting point is to come up with a FoldList compatible function that does the necessary calculations:

CalculateRealizedTradePL[{totalQty_, totalValue_, avgPrice_, PL_,
totalPL_}, {qprice_, qty_}] :=
Module[{newTotalPL = totalPL, price = QuantityMagnitude[qprice],
newTotalQty, tradeValue, newavgPrice, newTotalValue, newPL},
newTotalQty = totalQty + qty;
tradeValue =
If[Sign[qty] == Sign[totalQty] || avgPrice == 0, priceqty, If[Sign[totalQty + qty] == Sign[totalQty], avgPriceqty,
price(totalQty + qty)]]; newTotalValue = If[Sign[totalQty] == Sign[newTotalQty], totalValue + tradeValue, newTotalQtyprice];
newavgPrice =
If[Sign[totalQty + qty] ==
Sign[totalQty], (totalQtyavgPrice + tradeValue)/newTotalQty, price]; newPL = If[(Sign[qty] == Sign[totalQty] ) || totalQty == 0, 0, qty(avgPrice - price)];
newTotalPL = newTotalPL + newPL;
{newTotalQty, newTotalValue, newavgPrice, newPL, newTotalPL}]

Trade P&L is calculated on an average cost basis, as opposed to FIFO or LIFO.

Note that the functions handle both regular long-only trading strategies and short-sale strategies, in which (in the case of equities), we have to borrow the underlying stock to sell it short. Also, the pointValue argument enables us to apply the functions to trades in instruments such as futures for which, unlike stocks, the value of a 1 point move is typically larger than 1(e.g.50 for the ES S&P 500 mini futures contract).

We then apply the function in two flavors, to accommodate both standard numerical arrays and timeseries (associations would be another good alternative):

CalculateRealizedPLFromTrades[tradeList_?ArrayQ, pointValue_ : 1] :=
Module[{tradePL =
Rest@FoldList[CalculateRealizedTradePL, {0, 0, 0, 0, 0},
tradeList]},
tradePL[[All, 4 ;; 5]] = tradePL[[All, 4 ;; 5]]pointValue; tradePL] CalculateRealizedPLFromTrades[tsTradeList_, pointValue_ : 1] := Module[{tsTradePL = Rest@FoldList[CalculateRealizedTradePL, {0, 0, 0, 0, 0}, QuantityMagnitude@tsTradeList["Values"]]}, tsTradePL[[All, 4 ;; 5]] = tsTradePL[[All, 4 ;; 5]]pointValue;
tsTradePL[[All, 2 ;;]] =
Quantity[tsTradePL[[All, 2 ;;]], "US Dollars"];
tsTradePL =
TimeSeries[
Transpose@
Join[Transpose@tsTradeList["Values"], Transpose@tsTradePL],
tsTradeList["DateList"]]]

These functions run around 10x faster that the equivalent functions that use Do loops (without parallelization or compilation, admittedly).

Let’s see how they work with an example:

Trade Simulation

Next, we’ll generate a series of random trades using the AAPL time series, as follows (we also take the opportunity to convert the list of trades into a time series, tsTrades):

trades = Transpose@
Join[Transpose[
tsAAPL["DatePath"][[
Sort@RandomSample[Range[tsAAPL["PathLength"]],
20]]]], {RandomChoice[{-100, 100}, 20]}];
trades // TableForm

Trade P&L Calculation

We are now ready to apply our Trade P&L calculation function, first to the list of trades in array form:

TableForm[
Flatten[#] & /@ 
Partition[
Riffle[trades, 
CalculateRealizedPLFromTrades[trades[[All, 2 ;; 3]]]], 2], 
TableHeadings -> {{}, {"Date", "Price", "Quantity", "Total Qty", 
"Position Value", "Average Price", "P&L", "Total PL"}}]

The timeseries version of the function provides the output as a timeseries object in Quantity[“US Dollars”] format and, of course, can be plotted immediately with DateListPlot (it is also convenient for other reasons, as the complete backtest system is built around timeseries objects):

tsTradePL = CalculateRealizedPLFromTrades[tsTrades]

Overnight Trading in the E-Mini S&P 500 Futures

Jeff Swanson’s Trading System Success web site is often worth a visit for those looking for new trading ideas.

A recent post Seasonality S&P Market Session caught my eye, having investigated several ideas for overnight trading in the E-minis.  Seasonal effects are of course widely recognized and traded in commodities markets, but they can also apply to financial products such as the E-mini.  Jeff’s point about session times is well-made:  it is often worthwhile to look at the behavior of an asset, not only in different time frames, but also during different periods of the trading day, day of the week, or month of the year.

Jeff breaks the E-mini trading session into several basic sub-sessions:

  1. “Pre-Market” Between 530 and 830
  2. “Open” Between 830 and 900
  3. “Morning” Between 900 though 1130
  4. “Lunch” Between 1130 and 1315
  5. “Afternoon” Between 1315 and 1400
  6. “Close” Between 1400 and 1515
  7. “Post-Market” Between 1515 and 1800
  8. “Night” Between 1800 and 530

In his analysis Jeff’s strategy is simply to buy at the open of the session and close that trade at the conclusion of the session. This mirrors the traditional seasonality study where a trade is opened at the beginning of the season and closed several months later when the season comes to an end.

Evaluating Overnight Session and Seasonal Effects

The analysis evaluates the performance of this basic strategy during the “bullish season”, from Nov-May, when the equity markets traditionally make the majority of their annual gains, compared to the outcome during the “bearish season” from Jun-Oct.

None of the outcomes of these tests is especially noteworthy, save one:  the performance during the overnight session in the bullish season:

Fig 1

The tendency of the overnight session in the E-mini to produce clearer trends and trading signals has been well documented.  Plausible explanations for this phenomenon are that:

(a) The returns process in the overnight session is less contaminated with noise, which primarily results from trading activity; and/or

(b) The relatively poor liquidity of the overnight session allows participants to push the market in one direction more easily.

Either way, there is no denying that this study and several other, similar studies appear to demonstrate interesting trading opportunities in the overnight market.

That is, until trading costs are considered.  Results for the trading strategy from Nov 1997-Nov 2015 show a gain of $54,575, but an average trade of only just over $20:

Gross PL

# Trades

Av Trade

$54,575

2701

$20.21

Assuming that we enter and exit aggressively, buying at the market at the start of the session and selling MOC at the close, we will pay the bid-offer spread and commissions amounting to around $30, producing a net loss of $10 per trade.

The situation can be improved by omitting January from the “bullish season”, but the slightly higher average trade is still insufficient to overcome trading costs :

Gross PL

# Trades

Av Trade

$54,550

2327

$23.44

SSALGOTRADING AD

Designing a Seasonal Trading Strategy for the Overnight Session

At this point an academic research paper might conclude that the apparently anomalous trading profits are subsumed within the bid-offer spread.  But for a trading system designer this is not the end of the story.

If the profits are insufficient to overcome trading frictions when we cross the spread on entry and exit, what about a trading strategy that permits market orders on only the exit leg of the trade, while using limit orders to enter?  Total trading costs will be reduced to something closer to $17.50 per round turn, leaving a net profit of almost $6 per trade.

Of course, there is no guarantee that we will successfully enter every trade – our limit orders may not be filled at the bid price and, indeed, we are likely to suffer adverse selection – i.e. getting filled on every losing trading, while missing a proportion of the winning trades.

On the other hand, we are hardly obliged to hold a position for the entire overnight session.  Nor are we obliged to exit every trade MOC – we might find opportunities to exit prior to the end of the session, using limit orders to achieve a profit target or cap a trading loss.  In such a system, some proportion of the trades will use limit orders on both entry and exit, reducing trading costs for those trades to around $5 per round turn.

The key point is that we can use the seasonal effects detected in the overnight session as a starting point for the development for a more sophisticated trading system that uses a variety of entry and exit criteria, and order types.

The following shows the performance results for a trading system designed to trade 30-minute bars in the E-mini futures overnight session during the months of Nov to May.The strategy enters trades using limit prices and exits using a combination of profit targets, stop loss targets, and MOC orders.

Data from 1997 to 2010 were used to design the system, which was tested on out-of-sample data from 2011 to 2013.  Unseen data from Jan 2014 to Nov 2015 were used to provide a further (double blind) evaluation period for the strategy.

Fig 2

 

 

  

ALL TRADES

LONG

SHORT

Closed Trade Net Profit

$83,080

$61,493

$21,588

  Gross Profit

$158,193

$132,573

$25,620

  Gross Loss

-$75,113

-$71,080

-$4,033

Profit Factor

2.11

1.87

6.35

Ratio L/S Net Profit

2.85

Total Net Profit

$83,080

$61,493

$21,588

Trading Period

11/13/97 2:30:00 AM to 12/31/13 6:30:00 AM (16 years 48 days)

Number of Trading Days

2767

Starting Account Equity

$100,000

Highest Equity

$183,080

Lowest Equity

$97,550

Final Closed Trade Equity

$183,080

Return on Starting Equity

83.08%

Number of Closed Trades

849

789

60

  Number of Winning Trades

564

528

36

  Number of Losing Trades

285

261

24

  Trades Not Taken

0

0

0

Percent Profitable

66.43%

66.92%

60.00%

Trades Per Year

52.63

48.91

3.72

Trades Per Month

4.39

4.08

0.31

Max Position Size

1

1

1

Average Trade (Expectation)

$97.86

$77.94

$359.79

Average Trade (%)

0.07%

0.06%

0.33%

Trade Standard Deviation

$641.97

$552.56

$1,330.60

Trade Standard Deviation (%)

0.48%

0.44%

1.20%

Average Bars in Trades

15.2

14.53

24.1

Average MAE

$190.34

$181.83

$302.29

Average MAE (%)

0.14%

0.15%

0.27%

Maximum MAE

$3,237

$2,850

$3,237

Maximum MAE (%)

2.77%

2.52%

3.10%

Win/Loss Ratio

1.06

0.92

4.24

Win/Loss Ratio (%)

2.10

1.83

7.04

Return/Drawdown Ratio

15.36

14.82

5.86

Sharpe Ratio

0.43

0.46

0.52

Sortino Ratio

1.61

1.69

6.40

MAR Ratio

0.71

0.73

0.33

Correlation Coefficient

0.95

0.96

0.719

Statistical Significance

100%

100%

97.78%

Average Risk

$1,099

$1,182

$0.00

Average Risk (%)

0.78%

0.95%

0.00%

Average R-Multiple (Expectancy)

0.0615

0.0662

0

R-Multiple Standard Deviation

0.4357

0.4357

0

Average Leverage

0.399

0.451

0.463

Maximum Leverage

0.685

0.694

0.714

Risk of Ruin

0.00%

0.00%

0.00%

Kelly f

34.89%

31.04%

50.56%

Average Annual Profit/Loss

$5,150

$3,811

$1,338

Ave Annual Compounded Return

3.82%

3.02%

1.22%

Average Monthly Profit/Loss

$429.17

$317.66

$111.52

Ave Monthly Compounded Return

0.31%

0.25%

0.10%

Average Weekly Profit/Loss

$98.70

$73.05

$25.65

Ave Weekly Compounded Return

0.07%

0.06%

0.02%

Average Daily Profit/Loss

$30.03

$22.22

$7.80

Ave Daily Compounded Return

0.02%

0.02%

0.01%

INTRA-BAR EQUITY DRAWDOWNS

ALL TRADES

LONG

SHORT

Number of Drawdowns

445

422

79

Average Drawdown

$282.88

$269.15

$441.23

Average Drawdown (%)

0.21%

0.20%

0.33%

Average Length of Drawdowns

10 days 19 hours

10 days 20 hours

66 days 1 hours

Average Trades in Drawdowns

3

3

1

Worst Case Drawdown

$6,502

$4,987

$4,350

Date at Trough

12/13/00 1:30

5/24/00 4:30

12/13/00 1:30

Trading Strategy Design

In this post I want to share some thoughts on how to design great automated trading strategies – what to look for, and what to avoid.

For illustrative purposes I am going to use a strategy I designed for the ever-popular S&P500 e-mini futures contract.

The overall equity curve for the strategy is show below.

@ES Equity Curve

This is often the best place to start.  What you want to see, of course, is a smooth, upward-sloping curve, without too many sizable drawdowns, and one in which the strategy continues to make new highs.  This is especially important in the out-of-sample test period (Jan 2014- Jul 2015 in this case).  You will notice a flat period around 2013, which we will need to explore later.  Overall, however, this equity curve appears to fit the stereotypical pattern we hope to see when developing a new strategy.

Let’s move on look at the overall strategy performance numbers.

STRATEGY PERFORMANCE CHARACTERISTICS

@ES Perf Summary(click to enlarge)

 1. Net Profit
Clearly, the most important consideration.  Over the 17 year test period the strategy has produced a net profit  averaging around $23,000 per annum, per contract.  As a rough guide, you would want to see a net profit per contract around 10x the maintenance margin, or higher.

2. Profit Factor
The gross profit divided by the gross loss.  You want this to be as high as possible. Too low, as the strategy will be difficult to trade, because you will see sustained periods of substantial losses.  I would suggest a minimum acceptable PF in the region of 1.25.  Many strategy developers aim for a PF of 1.5, or higher.

3. Number of Trades
Generally, the more trades the better, at least from the point of view of building confidence in the robustness of strategy performance.  A strategy may show a great P&L, but if it only trades once a month it is going to take many many years of performance data to ensure statistical significance.  This strategy, on the other hand, is designed to trade 2-3 times a day.  Given that, and the length of the test period, there is little doubt that the results are statistically significant.

SSALGOTRADING AD

Profit Factor and number of trades are opposing design criteria – increasing the # trades tends to reduce the PF.  That consideration sets an upper bound on the # trades that can be accommodated, before the profit factor deteriorates to unacceptably low levels.  Typically, 4-5 trades a day is about the maximum trading frequency one can expect to achieve.

4. Win Rate
Novice system designers tend to assume that you want this to be as high as possible, but that isn’t typically the case.  It is perfectly feasible to design systems that have a 90% win rate, or higher, but which produce highly undesirable performance characteristics, such as frequent, large drawdowns.  For a typical trading system the optimal range for the win rate is in the region of 40% to 66%.  Below this range, it becomes difficult to tolerate the long sequences of losses that will result, without losing faith in the system.

5. Average Trade
This is the average net profit per trade.  A typical range would be $10 to $100.  Many designers will only consider strategies that have a higher average trade than this one, perhaps $50-$75, or more.  The issue with systems that have a very small average trade is that the profits can quickly be eaten up by commissions. Even though, in this case, the results are net of commissions, one can see a significant deterioration in profits if the average trade is low and trade frequency is high, because of the risk of low fill rates (i.e. the % of limit orders that get filled).  To assess this risk one looks at the number of fills assumed to take place at the high or low of the bar.  If this exceeds 10% of the total # trades, one can expect to see some slippage in the P&L when the strategy is put into production.

6. Average Bars
The number of bars required to complete a trade, on average.  There is no hard limit one can suggest here – it depends entirely on the size of the bars.  Here we are working in 60 minute bars, so a typical trade is held for around 4.5 hours, on average.   That’s a time-frame that I am comfortable with.  Others may be prepared to hold positions for much longer – days, or even weeks.

Perhaps more important is the average length of losing trades. What you don’t want to see is the strategy taking far longer to exit losing trades than winning trades. Again, this is a matter of trader psychology – it is hard to sit there hour after hour, or day after day, in a losing position – the temptation to cut the position becomes hard to ignore.  But, in doing that you are changing the strategy characteristics in a fundamental way, one that rarely produces a performance improvement.

What the strategy designer needs to do is to figure out in advance what the limits are of the investor’s tolerance for pain, in terms of maximum drawdown, average losing trade, etc, and design the strategy to meet those specifications, rather than trying to fix the strategy afterwards.

7. Required Account Size
It’s good to know exactly how large an account you need per contract, so you can figure out how to scale the strategy.  In this case one could hope to scale the strategy up to a 10-lot in a $100,000 account.  That may or may not fit the trader’s requirements and again, this needs to be considered at the outset.  For example, for a trader looking to utilize, say, $1,000,000 of capital, it is doubtful whether this strategy would fit his requirements without considerable work on the implementations issues that arise when trying to trade in anything approaching a 100 contract clip rate.

8. Commission
Always check to ensure that the strategy designer has made reasonable assumptions about slippage and commission.  Here we are assuming $5 per round turn.  There is no slippage, because the strategy executes using limit orders.

9. Drawdown
Drawdowns are, of course, every investor’s bugbear.  No-one likes drawdowns that are either large, or lengthy in relation to the annual profitability of the strategy, or the average trade duration.  A $10,000 max drawdown on a strategy producing over $23,000 a year is actually quite decent – I have seen many e-mini strategies with drawdowns at 2x – 3x that level, or larger.  Again, this is one of the key criteria that needs to be baked into the strategy design at the outset, rather than trying to fix later.

 ANNUAL PROFITABILITY

Let’s now take a look at how the strategy performs year-by-year, and some of the considerations and concerns that often arise.

@ES Annual1. Performance During Downturns
One aspect I always pay attention to is how well the strategy performs during periods of high market stress, because I expect similar conditions to arise in the fairly near future, e.g. as the Fed begins to raise rates.

Here, as you can see, the strategy performed admirably during both the dot com bust of 1999/2000 and the financial crisis of 2008/09.

2. Consistency in the # Trades and % Win Rate
It is not uncommon with low frequency strategies to see periods of substantial variation in the # trades or win rate.  Regardless how good the overall performance statistics are, this makes me uncomfortable.  It could be, for instance, that the overall results are influenced by one or two exceptional years that are unlikely to be repeated.  Significant variation in the trading or win rate raise questions about the robustness of the strategy, going forward.  On the other hand, as here, it is a comfort to see the strategy maintaining a very steady trading rate and % win rate, year after year.

3. Down Years
Every strategy shows variation in year to year performance and one expects to see years in which the strategy performs less well, or even loses money. For me, it rather depends on when such losses arise, as much as the size of the loss.  If a loss occurs in the out-of-sample period it raises serious questions about strategy robustness and, as a result, I am very unlikely to want to put such a strategy into production. If, as here, the period of poor performance occurs during the in-sample period I am less concerned – the strategy has other, favorable characteristics that make it attractive and I am willing to tolerate the risk of one modestly down-year in over 17 years of testing.

INTRA-TRADE DRAWDOWNS

Many trades that end up being profitable go through a period of being under-water.  What matters here is how high those intra-trade losses may climb, before the trade is closed.  To take an extreme example, would you be willing to risk $10,000 to make an average profit of only $10 per trade?  How about $20,000? $50,000? Your entire equity?

The Maximum Average Excursion chart below shows the drawdowns on a trade by trade basis.  Here we can see that, over the 17 year test period, no trade has suffered a drawdown of much more than $5,000.  I am comfortable with that level. Others may prefer a lower limit, or be tolerant of a higher MAE.

MAE

Again, the point is that the problem of a too-high MAE is not something one can fix after the event.  Sure, a stop loss will prevent any losses above a specified size.  But a stop loss also has the unwanted effect of terminating trades that would have turned into money-makers. While psychologically comfortable, the effect of a stop loss is almost always negative  in terms of strategy profitability and other performance characteristics, including drawdown, the very thing that investors are looking to control.

 CONCLUSION
I have tried to give some general guidelines for factors that are of critical importance in strategy design.  There are, of course, no absolutes:  the “right” characteristics depend entirely on the risk preferences of the investor.

One point that strategy designers do need to take on board is the need to factor in all of the important design criteria at the outset, rather than trying (and usually failing) to repair the strategy shortcomings after the event.

 

 

 

Quant Strategies in 2018

Quant Strategies – Performance Summary Sept. 2018

The end of Q3 seems like an appropriate time for an across-the-piste review of how systematic strategies are performing in 2018.  I’m using the dozen or more strategies running on the Systematic Algotrading Platform as the basis for the performance review, although results will obviously vary according to the specifics of the strategy.  All of the strategies are traded live and performance results are net of subscription fees, as well as slippage and brokerage commissions.

Volatility Strategies

Those waiting for the hammer to fall on option premium collecting strategies will have been disappointed with the way things have turned out so far in 2018.  Yes, February saw a long-awaited and rather spectacular explosion in volatility which completely destroyed several major volatility funds, including the VelocityShares Daily Inverse VIX Short-Term ETN (XIV) as well as Chicago-based hedged fund LJM Partners (“our goal is to preserve as much capital as possible”), that got caught on the wrong side of the popular VIX carry trade.  But the lack of follow-through has given many volatility strategies time to recover. Indeed, some are positively thriving now that elevated levels in the VIX have finally lifted option premiums from the bargain basement levels they were languishing at prior to February’s carnage.  The Option Trader strategy is a stand-out in this regard:  not only did the strategy produce exceptional returns during the February melt-down (+27.1%), the strategy has continued to outperform as the year has progressed and YTD returns now total a little over 69%.  Nor is the strategy itself exceptionally volatility: the Sharpe ratio has remained consistently above 2 over several years.

Hedged Volatility Trading

Investors’ chief concern with strategies that rely on collecting option premiums is that eventually they may blow up.  For those looking for a more nuanced approach to managing tail risk the Hedged Volatility strategy may be the way to go.  Like many strategies in the volatility space the strategy looks to generate alpha by trading VIX ETF products;  but unlike the great majority of competitor offerings, this strategy also uses ETF options to hedge tail risk exposure.  While hedging costs certainly acts as a performance drag, the results over the last few years have been compelling:  a CAGR of 52% with a Sharpe Ratio close to 2.

F/X Strategies

One of the common concerns for investors is how to diversify their investment portfolios, especially since the great majority of assets (and strategies) tend to exhibit significant positive correlation to equity indices these days. One of the characteristics we most appreciate about F/X strategies in general and the F/X Momentum strategy in particular is that its correlation to the equity markets over the last several years has been negligible.    Other attractive features of the strategy include the exceptionally high win rate – over 90% – and the profit factor of 5.4, which makes life very comfortable for investors.  After a moderate performance in 2017, the strategy has rebounded this year and is up 56% YTD, with a CAGR of 64.5% and Sharpe Ratio of 1.89.

Equity Long/Short

Thanks to the Fed’s accommodative stance, equity markets have been generally benign over the last decade to the benefit of most equity long-only and long-short strategies, including our equity long/short Turtle Trader strategy , which is up 31% YTD.  This follows a spectacular 2017 (+66%) , and is in line with the 5-year CAGR of 39%.   Notably, the correlation with the benchmark S&P500 Index is relatively low (0.16), while the Sharpe Ratio is a respectable 1.47.

Equity ETFs – Market Timing/Swing Trading

One alternative to the traditional equity long/short products is the Tech Momentum strategy.  This is a swing trading strategy that exploits short term momentum signals to trade the ProShares UltraPro QQQ (TQQQ) and ProShares UltraPro Short QQQ (SQQQ) leveraged ETFs.  The strategy is enjoying a banner year, up 57% YTD, with a four-year CAGR of 47.7% and Sharpe Ratio of 1.77.  A standout feature of this equity strategy is its almost zero correlation with the S&P 500 Index.  It is worth noting that this strategy also performed very well during the market decline in Feb, recording a gain of over 11% for the month.

Futures Strategies

It’s a little early to assess the performance of the various futures strategies in the Systematic Strategies portfolio, which were launched on the platform only a few months ago (despite being traded live for far longer).    For what it is worth, both of the S&P 500 E-Mini strategies, the Daytrader and the Swing Trader, are now firmly in positive territory for 2018.   Obviously we are keeping a watchful eye to see if the performance going forward remains in line with past results, but our experience of trading these strategies gives us cause for optimism.

Conclusion:  Quant Strategies in 2018

There appear to be ample opportunities for investors in the quant sector across a wide range of asset classes.  For investors with equity market exposure, we particularly like strategies with low market correlation that offer significant diversification benefits, such as the F/X Momentum and F/X Momentum strategies.  For those investors seeking the highest risk adjusted return, option selling strategies like the Option Trader strategy are the best choice, while for more cautious investors concerned about tail risk the Hedged Volatility strategy offers the security of downside protection.  Finally, there are several new strategies in equities and futures coming down the pike, several of which are already showing considerable promise.  We will review the performance of these newer strategies at the end of the year.

Go here for more information about the Systematic Algotrading Platform.

Money Management – the Good, the Bad and the Ugly

The infatuation of futures traders with the subject of money management, (more aptly described as position sizing), is something of a puzzle for someone coming from a background in equities or forex.  The idea is, simply, that one can improve one’s  trading performance through the judicious use of leverage, increasing the size of a position at times and reducing it at others.

MM Grapgic

Perhaps the most widely known money management technique is the Martingale, where the size of the trade is doubled after every loss.  It is easy to show mathematically that such a system must win eventually, provided that the bet size is unlimited.  It is also easy to show that, small as it may be, there is a non-zero probability of a long string of losing trades that would bankrupt the trader before he was able to recoup all his losses.  Still, the prospect offered by the Martingale strategy is an alluring one: the idea that, no matter what the underlying trading strategy, one can eventually be certain of winning.  And so a virtual cottage industry of money management techniques has evolved.

One of the reasons why the money management concept is prevalent in the futures industry compared to, say, equities or f/x, is simply the trading mechanics.  Doubling the size of a position in futures might mean trading an extra contract, or perhaps a ten-lot; doing the same in equities might mean scaling into and out of multiple positions comprising many thousands of shares.  The execution risk and cost of trying to implement a money management program in equities has historically made the  idea infeasible, although that is less true today, given the decline in commission rates and the arrival of smart execution algorithms.  Still, money management is a concept that originated in the futures industry and will forever be associated with it.

SSALGOTRADING AD

Van Tharp on Position Sizing
I was recently recommended to read Van Tharp’s Definitive Guide to Position Sizing, which devotes several hundred pages to the subject.  Leaving aside the great number of pages of simulation results, there is much to commend it.  Van Tharp does a pretty good job of demolishing highly speculative and very dangerous “money management” techniques such as the Kelly Criterion and Ralph Vince’s Optimal f, which make unrealistic assumptions of one kind or another, such as, for example, that there are only two outcomes, rather than the multiple possibilities from a trading strategy, or considering only the outcome of a single trade, rather than a succession of trades (whose outcome may not be independent).  Just as  with the Martingale, these techniques will often produce unacceptably large drawdowns.  In fact, as I have pointed out elsewhere, the use of leverage which many so-called money management techniques actually calls for increases in the risk in the original strategy, often reducing its risk-adjusted return.

As Van Tharp points out, mathematical literacy is not one of the strongest suits of futures traders in general and the money management strategy industry reflects that.

But Van Tharp  himself is not immune to misunderstanding mathematical concepts.  His central idea is that trading systems should be rated according to its System Quality Number, which he defines as:

SQN  = (Expectancy / standard deviation of R) * square root of Number of Trades

R is a central concept of Van Tharp’s methodology, which he defines as how much you will lose per unit of your investment.  So, for example, if you buy a stock today for $50 and plan to sell it if it reaches $40,  your R is $10.  In cases like this you have a clear definition of your R.  But what if you don’t?  Van Tharp sensibly recommends you use your average loss as an estimate of R.

Expectancy, as Van Tharp defines it, is just the expected profit per trade of the system expressed as a multiple of R.  So

SQN = ( (Average Profit per Trade / R) / standard deviation (Average Profit per Trade / R) * square root of Number of Trades

Squaring both sides of the equation, we get:

SQN^2  =  ( (Average Profit per Trade )^2 / R^2) / Variance (Average Profit per Trade / R) ) * Number of Trades

The R-squared terms cancel out, leaving the following:

SQN^2     =  ((Average Profit per Trade ) ^ 2 / Variance (Average Profit per Trade)) *  Number of Trades

Hence,

SQN = (Average Profit per Trade / Standard Deviation (Average Profit per Trade)) * square root of Number of Trades

There is another name by which this measure is more widely known in the investment community:  the Sharpe Ratio.

On the “Optimal” Position Sizing Strategy
In my view,  Van Tharp’s singular achievement has been to spawn a cottage industry out of restating a fact already widely known amongst investment professionals, i.e. that one should seek out strategies that maximize the Sharpe Ratio.

Not that seeking to maximize the Sharpe Ratio is a bad idea – far from it.  But then Van Tharp goes on to suggest that one should consider only strategies with a SQN of greater than 2, ideally much higher (he mentions SQNs of the order of 3-6).

But 95% or more of investable strategies have a Sharpe Ratio less than 2.  In fact, in the world of investment management a Sharpe Ratio of 1.5 is considered very good.  Barely a handful of funds have demonstrated an ability to maintain a Sharpe Ratio of greater than 2 over a sustained period (Jim Simon’s Renaissance Technologies being one of them).  Only in the world of high frequency trading do strategies typically attain the kind of Sharpe Ratio (or SQN) that Van Tharp advocates.  So while Van Tharp’s intentions are well meaning, his prescription is unrealistic, for the majority of investors.

One recommendation of Van Tharp’s that should be taken seriously is that there is no single “best” money management strategy that suits every investor.  Instead, position sizing should be evolved through simulation, taking into account each trader or investor’s preferences in terms of risk and return.  This makes complete sense: a trader looking to make 100% a year and willing to risk 50% of his capital is going to adopt a very different approach to money management, compared to an investor who will be satisfied with a 10% return, provided his risk of losing money is very low.  Again, however, there is nothing new here:  the problem of optimal allocation based on an investor’s aversion to risk has been thoroughly addressed in the literature for at least the last 50 years.

What about the Equity Curve Money Management strategy I discussed in a previous post?  Isn’t that a kind of Martingale?  Yes and no.  Indeed, the strategy does require us to increase the original investment after a period of loss. But it does so, not after a single losing trade, but after a series of losses from which the strategy is showing evidence of recovering.  Furthermore, the ECMM system caps the add-on investment at some specified level, rather than continuing to double the trade size after every loss, as in a Martingale.

But the critical difference between the ECMM and the standard Martingale lies in the assumptions about dependency in the returns of the underlying strategy. In the traditional Martingale, profits and losses are independent from one trade to the next.  By contrast, scenarios where ECMM is likely to prove effective are ones where there is dependency in the underlying strategy, more specifically, negative autocorrelation in returns over some horizon.  What that means is that periods of losses or lower returns tend to be followed by periods of gains, or higher returns.  In other words, ECMM works when the underlying strategy has a tendency towards mean reversion.

CONCLUSION
The futures industry has spawned a myriad of position sizing strategies.  Many are impractical, or positively dangerous, leading as they do to significant risk of catastrophic loss.  Generally, investors should seek out strategies with higher Sharpe Ratios, and use money management techniques only to improve the risk-adjusted return.  But there is no universal money management methodology that will suit every investor.  Instead, money management should be conditioned on each individual investors risk preferences.

Day Trading System in VIX Futures – JonathanKinlay.com

This is a follow up to my earlier post on a Calendar Spread Strategy in VIX Futures (more information on calendar spreads ).

The strategy trades the front two months in the CFE VIX futures contract, generating an annual profit of around $25,000 per spread.

DAY TRADING SYSTEM
I built an equivalent day trading system in VIX futures in Trading Technologies visual ADL language, using 1-min bar data for 2010, and tested the system out-of-sample in 2011-2014. (for more information on X-Trader/ ADL go here).

The annual net PL is around $20,000 per spread, with a win rate of 67%.   On the downside, the profit factor is rather low and the average trade is barely 1/10 of a tick). Note that this is net of Bid-Ask spread of 0.05 ($50) and commission/transaction costs of $20 per round turn.  These cost assumptions are reasonable for online trading at many brokerage firms.

SSALGOTRADING AD

However, the strategy requires you to work the spread to enter passively (thereby reducing the cost of entry).  This is usually only feasible on a  platform suitable for a high frequency trading, where you can assume that your orders have acceptable priority in the limit order queue.  This will result in a reasonable proportion of your passive bids and offers will be executed.  Typically the spread trade is held throughout the session, exiting on close (since this is a day trading system).

Overall, while the trading system characteristics are reasonable, the spread strategy is better suited to longer (i.e. overnight) holding periods, since the VIX futures market is not the most liquid and the tick value is large.  We’ll take a look at other day trading strategies in more liquid products, like the S&P 500 e-mini futures, for example, in another post.

High Freq Strategy Equity Curve(click to enlarge)

 

High Frequency Perf Results

(click to enlarge)

Enhancing Mutual Fund Returns With Market Timing

Summary

In this article, I will apply market timing techniques to several popular mutual funds.

The market timing approach produces annual rates of return that are 3% to 7% higher, with lower risk, than an equivalent buy and hold mutual fund investment.

Investors could in some cases have earned more than double the return achieved by holding a mutual fund investment over a 10-year period.

Hedging strategies that use market timing signals are able to sidestep market corrections, volatile conditions and the ensuing equity drawdowns.

Hedged portfolios typically employ around 12% less capital than the equivalent buy and hold strategy.

Background to the Market Timing Approach

In an earlier article, I discussed how to use marketing timing techniques to hedge an equity portfolio correlated to the broad market. I showed how, by using signals produced by a trading system modeled on the CBOE VIX index, we can smooth out volatility in an equity portfolio consisting of holdings in the SPDR S&P 500 ETF (NYSEARCA:SPY). An investor will typically reduce their equity holdings by a modest amount, say 20%, or step out of the market altogether during periods when the VIX index is forecast to rise, returning to the market when the VIX is likely to fall. An investment strategy based on this approach would have avoided most of the 2000-03 correction, as well as much of the market turmoil of 2008-09.

A more levered version of the hedging strategy, which I termed the MT aggressive portfolio, uses the VIX index signals to go to cash during high volatility periods, and then double the original equity portfolio holdings (using standard Reg-T leverage) during benign market conditions, as signaled by the model. The MT aggressive approach would have yielded net returns almost three times greater than that of a buy and hold portfolio in the SPY ETF, over the period from 1999-2014. Even though this version of the strategy makes use of leverage, the average holding in the portfolio would have been slightly lower than in the buy and hold portfolio because, in a majority of days, the strategy would have been 100% in cash. The result is illustrated in the chart in Fig. 1, which is reproduced below.

Fig. 1: Value of $1,000 – Long-Only Vs. MT Aggressive Portfolio

Source: Yahoo Finance.

Note that this approach does not entail shorting any stock. And for investors who prefer to buy and hold, I would make the point that the MT aggressive approach would have enabled you to buy almost three times as much stock in dollar terms by mid-2014 than would be the case if you had simply owned the SPY portfolio over the entire period.

SSALGOTRADING AD

Market Timing and Mutual Funds

With that background, we turn our attention to how we can use market timing techniques to improve returns from equity mutual funds. The funds selected for analysis are the Vanguard 500 Index Admiral (MUTF:VFIAX), Fidelity Spartan 500 Index Advtg (MUTF:FUSVX) and BlackRock S&P 500 Stock K (MUTF:WFSPX). This group of popular mutual funds is a representative sample of available funds that offer broad equity market exposure, with a high degree of correlation to the S&P 500 index. In what follows, we will focus attention on the MT aggressive approach, although other more conservative hedging strategies are equally valid.

We consider performance over the 10-year period from 2005, as at least one of the funds opened late in 2004. In each case, the MT aggressive portfolio is created by exiting the current mutual fund position and going 100% to cash, whenever the VIX model issues a buy signal in the VIX index. Conversely, we double our original mutual fund investment when the model issues a sell signal in the VIX index. In calculating returns, we make an allowance for trading costs of $3 cents per share for all transactions.

Returns for each of the mutual funds, as well as for the SPY ETF and the corresponding MT aggressive hedge strategies, are illustrated in the charts in Fig. 2. The broad pattern is similar in each case – we see significant outperformance of the MT aggressive portfolios relative to their ETF or mutual fund benchmarks. Furthermore, in most cases the hedge strategy tends to exhibit lower volatility, with less prolonged drawdowns during critical periods such as 2000/03 and 2008/09.

Fig. 2 – Value of $1,000: Mutual Fund Vs. MT Aggressive Portfolio January 2005 – June 2014

Source: Yahoo Finance.

Looking at the performance numbers in more detail, we can see from the tables shown in Fig. 3 that the MT aggressive strategies outperformed their mutual fund buy and hold benchmarks by a substantial margin. In the case of VFIAX and WFSPX, the hedge strategies produce a total net return more than double that of the corresponding mutual fund. With one exception, FUSVX, annual volatility of the MT aggressive portfolio was similar to, or lower than, that of the corresponding mutual fund, confirming our reading of the charts in Fig. 2. As a consequence, the MT aggressive strategies have higher Sharpe Ratios than any of the mutual funds. The improvement in risk adjusted returns is significant – more than double in the case of two of the funds, and about 40% higher in the case of the third.

Finally, we note that the MT aggressive strategies have an average holding that is around 12% lower than the equivalent long-only fund. That’s because of the periods in which investment proceeds are held in cash.

Fig. 3: Mutual Fund and MT Aggressive Portfolio Performance January 2005 – June 2014

Mutual Fund vs. MT Aggressive Portfolio Performance

Source: Yahoo Finance.

Conclusion

The aim of market timing is to smooth out the returns by hedging, and preferably avoiding altogether periods of market turmoil. In other words, the objective is to achieve the same, or better, rates of return, with lower volatility and drawdowns. We have demonstrated that this can be done, not only when the underlying investment is in an ETF such as SPY, but also where we hold an investment in one of several popular equity mutual funds. Over a 10-year period the hedge strategies produced consistently higher returns, with lower volatility and drawdown, while putting less capital at risk than their counterpart buy and hold mutual fund investments.

How Not to Develop Trading Strategies – A Cautionary Tale

In his post on Multi-Market Techniques for Robust Trading Strategies (http://www.adaptrade.com/Newsletter/NL-MultiMarket.htm) Michael Bryant of Adaptrade discusses some interesting approaches to improving model robustness. One is to use data from several correlated assets to build the model, on the basis that if the algorithm works for several assets with differing price levels, that would tend to corroborate the system’s robustness. The second approach he advocates is to use data from the same asset series at different bars lengths. The example he uses @ES.D at 5, 7 and 9 minute bars. The argument in favor of this approach is the same as for the first, albeit in this case the underlying asset is the same.

I like Michael’s idea in principle, but I wanted to give you a sense of what can all too easily go wrong with GP modeling, even using techniques such as multi-time frame fitting and Monte Carlo simulation to improve robustness testing.

In the chart below I have extended the analysis back in time, beyond the 2011-2012 period that Michael used to build his original model. As you can see, most of the returns are generated in-sample, in the 2011-2012 period. As we look back over the period from 2007-2010, the results are distinctly unimpressive – the strategy basically trades sideways for four years.

Adaptrade ES Strategy in Multiple Time Frames

 

How do Do It Right

In my view, there is only one, safe way to use GP to develop strategies. Firstly, you need to use a very long span of data – as much as possible, to fit your model. Only in this way can you ensure that the model has encountered enough variation in market conditions to stand a reasonable chance of being able to adapt to changing market conditions in future.

SSALGOTRADING AD

Secondly, you need to use two OOS period. The first OOS span of data, drawn from the start of the data series, is used in the normal way, to visually inspect the performance of the model. But the second span of OOS data, from more recent history, is NOT examined before the model is finalized. This is really important. Products like Adaptrade make it too easy for the system designer to “cheat”, by looking at the recent performance of his trading system “out of sample” and selecting models that do well in that period. But the very process of examining OOS performance introduces bias into the system. It would be like adding a line of code saying something like:

IF (model performance in OOS period > x) do the following….

I am quite sure if I posted a strategy with a line of code like that in it, it would immediately be shot down as being blatantly biased, and quite rightly so. But, if I look at the recent “OOS” performance and use it to select the model, I am effectively doing exactly the same thing.

That is why it is so important to have a second span of OOS data that it not only not used to build the model, but also is not used to assess performance, until after the final model selection is made. For that reason, the second OOS period is referred to as a “double blind” test.

That’s the procedure I followed to build my futures daytrading strategy: I used as much data as possible, dating from 2002. The first 20% of the each data set was used for normal OOS testing. But the second set of data, from Jan 2012 onwards, was my double-blind data set. Only when I saw that the system maintained performance in BOTH OOS periods was I reasonably confident of the system’s robustness.

DoubleBlind

This further explains why it is so challenging to develop higher frequency strategies using GP. Running even a very fast GP modeling system on a large span of high frequency data can take inordinate amounts of time.

The longest span of 5-min bar data that a GP system can handle would typically be around 5-7 years. This is probably not quite enough to build a truly robust system, although if you pick you time span carefully it might be (I generally like to use the 2006-2011 period, which has lots of market variation).

For 15 minute bar data, a well-designed GP system can usually handle all the available data you can throw at it – from 1999 in the case of the Emini, for instance.

Why I don’t Like Fitting Models over Short Time Spans

The risks of fitting models to data in short time spans are intuitively obvious. If you happen to pick a data set in which the market is in a strong uptrend, then your model is going to focus on that kind of market behavior. Subsequently, when the trend changes, the strategy will typically break down.
Monte Carlo simulation isn’t going to change much in this situation: sure, it will help a bit, perhaps, but since the resampled data is all drawn from the same original data set, in most cases the simulated paths will also show a strong uptrend – all that will be shown is that there is some doubt about the strength of the trend. But a completely different scenario, in which, say, the market drops by 10%, is unlikely to appear.

One possible answer to that problem, recommended by some system developers, is simply to rebuild the model when a breakdown is detected. While it’s true that a product like MSA can make detection easier, rebuilding the model is another question altogether. There is no guarantee that the kind of model that has worked hitherto can be re-tooled to work once again. In fact, there may be no viable trading system that can handle the new market dynamics.

Here is a case in point. We have a system that works well on 10 min bars in TF.D up until around May 2012, when MSA indicates a breakdown in strategy performance.

TF.F Monte Carlo

So now we try to fit a new model, along the pattern of the original model, taking account some of the new data.  But it turns out to be just a Band-Aid – after a few more data points the strategy breaks down again, irretrievably.

TF EC 1

This is typical of what often happens when you use GP to build a model using s short span of data. That’s why I prefer to use a long time span, even at lower frequency. The chances of being able to build a robust system that will adapt well to changing market conditions are much higher.

A Robust Emini Trading System

Here, for example is a GP system build on daily data in @ES.D from 1999 to 2011 (i.e. 2012 to 2014 is OOS).

ES.D EC

A Scalping Strategy in E-Mini Futures

This is a follow up post to my post on the Mathematics of Scalping. To illustrate the scalping methodology, I coded up a simple strategy based on the techniques described in the post.

The strategy trades a single @ES contract on 1-minute bars. The attached ELD file contains the Easylanguage code for ES scalping strategy, which can be run in Tradestation or Multicharts.

This strategy makes no attempt to forecast market direction and doesn’t consider market trends at all. It simply looks at the current levels of volatility and takes a long volatility position or a short volatility position depending on whether volatility is above or below some threshold parameters.

By long volatility I mean a position where we buy or sell the market and set a loose Profit Target and a tight Stop Loss. By short volatility I mean a position where we buy or sell the market and set a tight Profit Target and loose Stop Loss. This is exactly the methodology I described earlier in the post. The parameters I ended up using are as follows:

Long Volatility: Profit Target = 8 ticks, Stop Loss = 2 ticks
Short Volatility: Profit Target = 2 ticks, Stop Loss = 30 ticks

I have made no attempt to optimize these parameters settings, which can easily be done in Tradestation or Multicharts.

What do we mean by volatility being above our threshold level? I use a very simple metric: I take the TrueRange for the current bar and add 50% of the increase or decrease in TrueRange over the last two bars. That’s my crude volatility “forecast”.

SSALGOTRADING AD

The final point to explain is this: let’s suppose our volatility forecast is above our threshold level, so we know we want to be long volatility. Ok, but do we buy or sell the ES? One approach ia to try to gauge the direction of the market by estimating the trend. Not a bad idea, by any means, although I have argued that volatility drowns out any trend signal at short time frames (like 1 minute, for example). So I prefer an approach that makes no assumptions about market direction.

In this approach what we do is divide volatility into upsideVolatility and downsideVolatility. upsideVolatility uses the TrueRange for bars where Close > Close[1]. downsideVolatility is calculated only for bars where Close < Close[1]. This kind of methodology, where you calculate volatility based on the sign of the returns, is well known and is used in performance measures like the Sortino ratio. This is like the Sharpe ratio, except that you calculate the standard deviation of returns using only days in which the market was down. When it’s calculated this way, standard deviation is known as the (square root of the) semi-variance.

Anyway, back to our strategy. So we calculate the upside and downside volatilities and test them against our upper and lower volatility thresholds.

The decision tree looks like this:

LONG VOLATILITY
If upsideVolatilityForecast > upperVolThrehold, buy at the market with wide PT and tight ST (long market, long volatility)
If downsideVolatilityForecast > upperVolThrehold, sell at the market with wide PT and tight ST (short market, long volatility)

SHORT VOLATILITY
If upsideVolatilityForecast < lowerVolThrehold, sell at the Ask on a limit with tight PT and wide ST (short market, short volatility)
If downsideVolatilityForecast < lowerVolThrehold, buy at the Bid on a limit with tight PT and wide ST (long market, short volatility)

NOTE THE FOLLOWING CAVEATS. DO NOT TRY TO TRADE THIS STRATEGY LIVE (but use it as a basis for a tradable strategy)

1. The strategy makes the usual TS assumption about fill rates, which is unrealistic, especially at short intervals like 1-minute.
2. The strategy allows fees and commissions of $3 per contract, or $6 per round turn. Your trading costs may be higher than this.
3. Tradestation is unable to perform analysis at the tick level for a period as long at the one used here (2000 to 2014). A tick by tick analysis would likely show very different results (better or worse).
4. The strategy is extremely lop-sided: the great majority of the profits are made on the long side and the Win Rates and Profit Factors are very different for long trades vs short trades. I suspect this would change with a tick by tick analysis. But it also may be necessary to add more parameters so that long trades are treated differently from short trades.
5. No attempt has been made to optimize the parameters.
6 This is a daytading strategy that will exit the market on close.

So with all that said here are the results.

As you can see, the strategy produces a smooth, upward sloping equity curve, the slope of which increases markedly during the period of high market volatility in 2008.
Net profits after commissions for a single ES contract amount to $243,000 ($3.42 per contract) with a win rate of 76% and Profit Factor of 1.24.

This basic implementation would obviously require improvement in several areas, not least of which would be to address the imbalance in strategy profitability on the short vs long side, where most of the profits are generated.

Scalping Strategy EC

 

Scalping Strategy Perf Report

 

 

The Mathematics of Scalping

NOTE:  if you are unable to see the Mathematica models below, you can download the free Wolfram CDF player and you may also need this plug-in.

You can also download the complete Mathematica CDF file here.

In this post I want to explore aspects of scalping, a type of strategy widely utilized by high frequency trading firms.

I will define a scalping strategy as one in which we seek to take small profits by posting limit orders on alternate side of the book. Scalping, as I define it, is a strategy rather like market making, except that we “lean” on one side of the book. So, at any given time, we may have a long bias and so look to enter with a limit buy order. If this is filled, we will then look to exit with a subsequent limit sell order, taking a profit of a few ticks. Conversely, we may enter with a limit sell order and look to exit with a limit buy order.
The strategy relies on two critical factors:

(i) the alpha signal which tells us from moment to moment whether we should prefer to be long or short
(ii) the execution strategy, or “trade expression”

In this article I want to focus on the latter, making the assumption that we have some kind of alpha generation model already in place (more about this in later posts).

There are several means that a trader can use to enter a position. The simplest approach, the one we will be considering here, is simply to place a single limit order at or just outside the inside bid/ask prices – so in other words we will be looking to buy on the bid and sell on the ask (and hoping to earn the bid-ask spread, at least).

SSALGOTRADING AD

One of the problems with this approach is that it is highly latency sensitive. Limit orders join the limit order book at the back of the queue and slowly works their way towards the front, as earlier orders get filled. Buy the time the market gets around to your limit buy order, there may be no more sellers at that price. In that case the market trades away, a higher bid comes in and supersedes your order, and you don’t get filled. Conversely, yours may be one of the last orders to get filled, after which the market trades down to a lower bid and your position is immediately under water.

This simplistic model explains why latency is such a concern – you want to get as near to the front of the queue as you can, as quickly as possible. You do this by minimizing the time it takes to issue and order and get it into the limit order book. That entails both hardware (co-located servers, fiber-optic connections) and software optimization and typically also involves the use of Immediate or Cancel (IOC) orders. The use of IOC orders by HFT firms to gain order priority is highly controversial and is seen as gaming the system by traditional investors, who may end up paying higher prices as a result.

Another approach is to layer limit orders at price points up and down the order book, establishing priority long before the market trades there. Order layering is a highly complex execution strategy that brings addition complications.

Let’s confine ourselves to considering the single limit order, the type of order available to any trader using a standard retail platform.

As I have explained, we are assuming here that, at any point in time, you know whether you prefer to be long or short, and therefore whether you want to place a bid or an offer. The issue is, at what price do you place your order, and what do you do about limiting your risk? In other words, we are discussing profit targets and stop losses, which, of course, are all about risk and return.

Risk and Return in Scalping

Lets start by considering risk. The biggest risk to a scalper is that, once filled, the market goes against his position until he is obliged to trigger his stop loss. If he sets his stop loss too tight, he may be forced to exit positions that are initially unprofitable, but which would have recovered and shown a profit if he had not exited. Conversely,  if he sets the stop loss too loose, the risk reward ratio is very low – a single loss-making trade could eradicate the profit from a large number of smaller, profitable trades.

Now lets think about reward. If the trader is too ambitious in setting his profit target he may never get to realize the gains his position is showing – the market could reverse, leaving him with a loss on a position that was, initially, profitable. Conversely, if he sets the target too tight, the trader may give up too much potential in a winning trade to overcome the effects of the occasional, large loss.

It’s clear that these are critical concerns for a scalper: indeed the trade exit rules are just as important, or even more important, than the entry rules. So how should he proceed?

Theoretical Framework for Scalping

Let’s make the rather heroic assumption that market returns are Normally distributed (in fact, we know from empirical research that they are not – but this is a starting point, at least). And let’s assume for the moment that our trader has been filled on a limit buy order and is looking to decide where to place his profit target and stop loss limit orders. Given a current price of the underlying security of X, the scalper is seeking to determine the profit target of p ticks and the stop loss level of q ticks that will determine the prices at which he should post his limit orders to exit the trade. We can translate these into returns, as follows:

to the upside: Ru = Ln[X+p] – Ln[X]

and to the downside: Rd = Ln[X-q] – Ln[X]

This situation is illustrated in the chart below.

Normal Distn Shaded

The profitable area is the shaded region on the RHS of the distribution. If the market trades at this price or higher, we will make money: p ticks, less trading fees and commissions, to be precise. Conversely we lose q ticks (plus commissions) if the market trades in the region shaded on the LHS of the distribution.

Under our assumptions, the probability of ending up in the RHS shaded region is:

probWin = 1 – NormalCDF(Ru, mu, sigma),

where mu and sigma are the mean and standard deviation of the distribution.

The probability of losing money, i.e. the shaded area in the LHS of the distribution, is given by:

probLoss = NormalCDF(Rd, mu, sigma),

where NormalCDF is the cumulative distribution function of the Gaussian distribution.

The expected profit from the trade is therefore:

Expected profit = p * probWin – q * probLoss

And the expected win rate, the proportion of profitable trades, is given by:

WinRate = probWin / (probWin + probLoss)

If we set a stretch profit target, then p will be large, and probWin, the shaded region on the RHS of the distribution, will be small, so our winRate will be low. Under this scenario we would have a low probability of a large gain. Conversely, if we set p to, say, 1 tick, and our stop loss q to, say, 20 ticks, the shaded region on the RHS will represent close to half of the probability density, while the shaded LHS will encompass only around 5%. Our win rate in that case would be of the order of 91%:

WinRate = 50% / (50% + 5%) = 91%

Under this scenario, we make frequent, small profits  and suffer the occasional large loss.

So the critical question is: how do we pick p and q, our profit target and stop loss?  Does it matter?  What should the decision depend on?

Modeling Scalping Strategies

We can begin to address these questions by noticing, as we have already seen, that there is a trade-off between the size of profit we are hoping to make, and the size of loss we are willing to tolerate, and the probability of that gain or loss arising.  Those probabilities in turn depend on the underlying probability distribution, assumed here to be Gaussian.

Now, the Normal or Gaussian distribution which determines the probabilities of winning or losing at different price levels has two parameters – the mean, mu, or drift of the returns process and sigma, its volatility.

Over short time intervals the effect of volatility outweigh any impact from drift by orders of magnitude.  The reason for this is simple:  volatility scales with the square root of time, while the drift scales linearly.  Over small time intervals, the drift becomes un-noticeably small, compared to the process volatility.  Hence we can assume that mu, the process mean is zero, without concern, and focus exclusively on sigma, the volatility.

What other factors do we need to consider?  Well there is a minimum price move, which might be 1 tick, and the dollar value of that tick, from which we can derive our upside and downside returns, Ru and Rd.  And, finally, we need to factor in commissions and exchange fees into our net trade P&L.

Here’s a simple formulation of the model, in which I am using the E-mini futures contract as an exemplar.

 WinRate[currentPrice_,annualVolatility_,BarSizeMins_, nTicksPT_, nTicksSL_,minMove_, tickValue_, costContract_]:=Module[{ nMinsPerDay, periodVolatility, tgtReturn, slReturn,tgtDollar, slDollar, probWin, probLoss, winRate, expWinDollar, expLossDollar, expProfit},
nMinsPerDay = 250*6.5*60;
periodVolatility = annualVolatility / Sqrt[nMinsPerDay/BarSizeMins];
tgtReturn=nTicksPT*minMove/currentPrice;tgtDollar = nTicksPT * tickValue;
slReturn = nTicksSL*minMove/currentPrice;
slDollar=nTicksSL*tickValue;
probWin=1-CDF[NormalDistribution[0, periodVolatility],tgtReturn];
probLoss=CDF[NormalDistribution[0, periodVolatility],slReturn];
winRate=probWin/(probWin+probLoss);
expWinDollar=tgtDollar*probWin;
expLossDollar=slDollar*probLoss;
expProfit=expWinDollar+expLossDollar-costContract;
{expProfit, winRate}]

For the ES contract we have a min price move of 0.25 and the tick value is $12.50.  Notice that we scale annual volatility to the size of the period we are trading (15 minute bars, in the following example).

Scenario Analysis

Let’s take a look at how the expected profit and win rate vary with the profit target and stop loss limits we set.  In the following interactive graphics, we can assess the impact of different levels of volatility on the outcome.

Expected Profit by Bar Size and Volatility

Expected Win Rate by Volatility

Notice to begin with that the win rate (and expected profit) are very far from being Normally distributed – not least because they change radically with volatility, which is itself time-varying.

For very low levels of volatility, around 5%, we appear to do best in terms of maximizing our expected P&L by setting a tight profit target of a couple of ticks, and a stop loss of around 10 ticks.  Our win rate is very high at these levels – around 90% or more.  In other words, at low levels of volatility, our aim should be to try to make a large number of small gains.

But as volatility increases to around 15%, it becomes evident that we need to increase our profit target, to around 10 or 11 ticks.  The distribution of the expected P&L suggests we have a couple of different strategy options: either we can set a larger stop loss, of around 30 ticks, or we can head in the other direction, and set a very low stop loss of perhaps just 1-2 ticks.  This later strategy is, in fact, the mirror image of our low-volatility strategy:  at higher levels of volatility, we are aiming to make occasional, large gains and we are willing to pay the price of sustaining repeated small stop-losses.  Our win rate, although still well above 50%, naturally declines.

As volatility rises still further, to 20% or 30%, or more, it becomes apparent that we really have no alternative but to aim for occasional large gains, by increasing our profit target and tightening stop loss limits.   Our win rate under this strategy scenario will be much lower – around 30% or less.

Non – Gaussian Model

Now let’s address the concern that asset returns are not typically distributed Normally. In particular, the empirical distribution of returns tends to have “fat tails”, i.e. the probability of an extreme event is much higher than in an equivalent Normal distribution.

A widely used model for fat-tailed distributions in the Extreme Value Distribution. This has pdf:

PDF[ExtremeValueDistribution[,],x]

 EVD

Plot[Evaluate@Table[PDF[ExtremeValueDistribution[,2],x],{,{-3,0,4}}],{x,-8,12},FillingAxis]

EVD pdf

Mean[ExtremeValueDistribution[,]]

+EulerGamma

Variance[ExtremeValueDistribution[,]]

EVD Variance

In order to set the parameters of the EVD, we need to arrange them so that the mean and variance match those of the equivalent Gaussian distribution with mean = 0 and standard deviation . hence:

EVD params

The code for a version of the model using the GED is given as follows

WinRateExtreme[currentPrice_,annualVolatility_,BarSizeMins_, nTicksPT_, nTicksSL_,minMove_, tickValue_, costContract_]:=Module[{ nMinsPerDay, periodVolatility, alpha, beta,tgtReturn, slReturn,tgtDollar, slDollar, probWin, probLoss, winRate, expWinDollar, expLossDollar, expProfit},
nMinsPerDay = 250*6.5*60;
periodVolatility = annualVolatility / Sqrt[nMinsPerDay/BarSizeMins];
beta = Sqrt[6]*periodVolatility / Pi;
alpha=-EulerGamma*beta;
tgtReturn=nTicksPT*minMove/currentPrice;tgtDollar = nTicksPT * tickValue;
slReturn = nTicksSL*minMove/currentPrice;
slDollar=nTicksSL*tickValue;
probWin=1-CDF[ExtremeValueDistribution[alpha, beta],tgtReturn];
probLoss=CDF[ExtremeValueDistribution[alpha, beta],slReturn];
winRate=probWin/(probWin+probLoss);
expWinDollar=tgtDollar*probWin;
expLossDollar=slDollar*probLoss;
expProfit=expWinDollar+expLossDollar-costContract;
{expProfit, winRate}]

WinRateExtreme[1900,0.05,15,2,30,0.25,12.50,3][[2]]

0.21759

We can now produce the same plots for the EVD version of the model that we plotted for the Gaussian versions :

Expected Profit by Bar Size and Volatility – Extreme Value Distribution

Expected Win Rate by Volatility – Extreme Value Distribution

Next we compare the Gaussian and EVD versions of the model, to gain an understanding of how the differing assumptions impact the expected Win Rate.

Expected Win Rate by Stop Loss and Profit Target

As you can see, for moderate levels of volatility, up to around 18 % annually, the expected Win Rate is actually higher if we assume an Extreme Value distribution of returns, rather than a Normal distribution.If we use a Normal distribution we will actually underestimate the Win Rate, if the actual return distribution is closer to Extreme Value.In other words, the assumption of a Gaussian distribution for returns is actually conservative.

Now, on the other hand, it is also the case that at higher levels of volatility the assumption of Normality will tend to over – estimate the expected Win Rate, if returns actually follow an extreme value distribution. But, as indicated before, for high levels of volatility we need to consider amending the scalping strategy very substantially. Either we need to reverse it, setting larger Profit Targets and tighter Stops, or we need to stop trading altogether, until volatility declines to normal levels.Many scalpers would prefer the second option, as the first alternative doesn’t strike them as being close enough to scalping to justify the name.If you take that approach, i.e.stop trying to scalp in periods when volatility is elevated, then the differences in estimated Win Rate resulting from alternative assumptions of return distribution are irrelevant.

If you only try to scalp when volatility is under, say, 20 % and you use a Gaussian distribution in your scalping model, you will only ever typically under – estimate your actual expected Win Rate.In other words, the assumption of Normality helps, not hurts, your strategy, by being conservative in its estimate of the expected Win Rate.

If, in the alternative, you want to trade the strategy regardless of the level of volatility, then by all means use something like an Extreme Value distribution in your model, as I have done here.That changes the estimates of expected Win Rate that the model produces, but it in no way changes the structure of the model, or invalidates it.It’ s just a different, arguably more realistic set of assumptions pertaining to situations of elevated volatility.

Monte-Carlo Simulation Analysis

Let’ s move on to do some simulation analysis so we can get an understanding of the distribution of the expected Win Rate and Avg Trade PL for our two alternative models. We begin by coding a generator that produces a sample of 1,000 trades and calculates the Avg Trade PL and Win Rate.

Gaussian Model

GenWinRate[currentPrice_,annualVolatility_,BarSizeMins_, nTicksPT_, nTicksSL_,minMove_, tickValue_, costContract_]:=Module[{ nMinsPerDay, periodVolatility, randObs, tgtReturn, slReturn,tgtDollar, slDollar, nWins,nLosses, perTradePL, probWin, probLoss, winRate, expWinDollar, expLossDollar, expProfit},
nMinsPerDay = 250*6.5*60;
periodVolatility = annualVolatility / Sqrt[nMinsPerDay/BarSizeMins];
tgtReturn=nTicksPT*minMove/currentPrice;tgtDollar = nTicksPT * tickValue;
slReturn = nTicksSL*minMove/currentPrice;
slDollar=nTicksSL*tickValue;
randObs=RandomVariate[NormalDistribution[0,periodVolatility],10^3];
nWins=Count[randObs,x_/;x>=tgtReturn];
nLosses=Count[randObs,x_/;xslReturn];
winRate=nWins/(nWins+nLosses)//N;
perTradePL=(nWins*tgtDollar+nLosses*slDollar)/(nWins+nLosses);{perTradePL,winRate}]

GenWinRate[1900,0.1,15,1,-24,0.25,12.50,3]

{7.69231,0.984615}

Now we can generate a random sample of 10, 000 simulation runs and plot a histogram of the Win Rates, using, for example, ES on 5-min bars, with a PT of 2 ticks and SL of – 20 ticks, assuming annual volatility of 15 %.

Histogram[Table[GenWinRate[1900,0.15,5,2,-20,0.25,12.50,3][[2]],{i,10000}],10,AxesLabel{“Exp. Win Rate (%)”}]

WinRateHist

Histogram[Table[GenWinRate[1900,0.15,5,2,-20,0.25,12.50,3][[1]],{i,10000}],10,AxesLabel{“Exp. PL/Trade ($)”}]

PLHist

Extreme Value Distribution Model

Next we can do the same for the Extreme Value Distribution version of the model.

GenWinRateExtreme[currentPrice_,annualVolatility_,BarSizeMins_, nTicksPT_, nTicksSL_,minMove_, tickValue_, costContract_]:=Module[{ nMinsPerDay, periodVolatility, randObs, tgtReturn, slReturn,tgtDollar, slDollar, alpha, beta,nWins,nLosses, perTradePL, probWin, probLoss, winRate, expWinDollar, expLossDollar, expProfit},
nMinsPerDay = 250*6.5*60;
periodVolatility = annualVolatility / Sqrt[nMinsPerDay/BarSizeMins];
beta = Sqrt[6]*periodVolatility / Pi;
alpha=-EulerGamma*beta;
tgtReturn=nTicksPT*minMove/currentPrice;tgtDollar = nTicksPT * tickValue;
slReturn = nTicksSL*minMove/currentPrice;
slDollar=nTicksSL*tickValue;
randObs=RandomVariate[ExtremeValueDistribution[alpha, beta],10^3];
nWins=Count[randObs,x_/;x>=tgtReturn];
nLosses=Count[randObs,x_/;xslReturn];
winRate=nWins/(nWins+nLosses)//N;
perTradePL=(nWins*tgtDollar+nLosses*slDollar)/(nWins+nLosses);{perTradePL,winRate}]

Histogram[Table[GenWinRateExtreme[1900,0.15,5,2,-10,0.25,12.50,3][[2]],{i,10000}],10,AxesLabel{“Exp. Win Rate (%)”}]

WinRateEVDHist

Histogram[Table[GenWinRateExtreme[1900,0.15,5,2,-10,0.25,12.50,3][[1]],{i,10000}],10,AxesLabel{“Exp. PL/Trade ($)”}]

PLEVDHist

 

 

Conclusions

The key conclusions from this analysis are:

  1. Scalping is essentially a volatility trade
  2. The setting of optimal profit targets are stop loss limits depend critically on the volatility of the underlying, and needs to be handled dynamically, depending on current levels of market volatility
  3. At low levels of volatility we should set tight profit targets and wide stop loss limits, looking to make a high percentage of small gains, of perhaps 2-3 ticks.
  4. As volatility rises, we need to reverse that position, setting more ambitious profit targets and tight stops, aiming for the occasional big win.