The post appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>And those he plays never suspect

He doesn’t play for the money he wins

He doesn’t play for respect

.

He deals the cards to find the answer

The sacred geometry of chance

The hidden law of a probable outcome

The numbers lead a dance

.

* – Shape of My Heart, *Sting

The post appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>The post Trading Market Sentiment appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>In the early days of the developing field of market sentiment analysis, the supply of machine readable content was limited to mainstream providers of financial news such as Reuters or Bloomberg. Over time this has changed with the entry of new competitors in the provision of machine readable news, including, for example, Ravenpack or more recent arrivals like Accern. Providers often seek to sell not only the raw news feed service, but also their own proprietary sentiment indicators that are claimed to provide additional insight into how individual stocks, market sectors, or the overall market are likely to react to news. There is now what appears to be a cottage industry producing white papers seeking to demonstrate the value of these services, often accompanied by some impressive pro-forma performance statistics for the accompanying strategies, which include long-only, long/short, market neutral and statistical arbitrage.

For the purpose of demonstration I intend to forego the blandishments of these services, although many are no doubt are excellent, since the reader is perhaps unlikely to have access to them. Instead, in what follows I will focus on a single news source, albeit a highly regarded one: the Wall Street Journal. This is, of course, a simplification intended for illustrative purposes only – in practice one would need to use a wide variety of news sources and perhaps subscribe to a machine readable news feed service. But similar principles and techniques can be applied to any number of news feeds or online sites.

We are going to access the Journal’s online archive, which presents daily news items in a convenient summary format, an example of which is shown below. The archive runs from the beginning of 2012 through to the current day, providing ample data for analysis. In what follows, I am going to make two important assumptions, neither of which is likely to be 100% accurate – but which will not detract too much from the validity of the research, I hope. The first assumption is that the news items shown in each daily archive were reported prior to the market open at 9:30 AM. This is likely to be true for the great majority of the stories, but there are no doubt important exceptions. Since we intend to treat the news content of each archive as antecedent to the market action during the corresponding trading session, exceptions are likely to introduce an element of look-ahead bias. The second assumption is that the archive for each day is shown in the form in which it would have appeared on the day in question. In reality, there are likely to have been revisions to some of the stories made subsequent to their initial publication. So, here too, we must allow for the possibility of look-ahead bias in the ensuing analysis.

With those caveats out of the way, let’s proceed. We are going to be using broad market data for the S&P 500 index in the analysis to follow, so the first step is to download daily price series for the index. Note that we begin with daily opening prices, since we intend to illustrate the application of news sentiment analysis with a theoretical day-trading strategy that takes positions at the start of each trading session, exiting at market close.

From there we calculate the intraday return in the index, from market open to close, as follows:

Next we turn to the task of reading the news archive and categorizing its content. Mathematica makes the importation of html pages very straightforward, and we can easily crop the raw text string to exclude page headers and footers. The approach I am going to take is to derive a sentiment indicator based on an analysis of the sentiment of each word in the daily archive. Before we can do that we must first convert the text into individuals words, stripping out standard stop-words such as “the” and “in” and converting all the text to lower case. Naturally one can take this pre-processing a great deal further, by identifying and separating out proper nouns, for example. Once the text processing stage is complete we can quickly summarize the content, for example by looking at the most common words, or by representing the entire archive in the form of a word cloud. Given that we are using the archive for the first business day of 2012, it is perhaps unsurprising that we find that “2012”, “new” and “year” feature so prominently!

The subject of sentiment analysis is a complex one and I only touch on it here. For those interested in the subject I can recommend *The Text Mining Handbook*, by Feldman and Sanger, which is a standard work on the topic. Here I am going to employ a machine learning classifier provided with Mathematica 11. It is not terribly sophisticated (or, at least, has not been developed with financial applications especially in mind), but will serve for the purposes of this article. For those unfamiliar with the functionality, the operation of the sentiment classification algorithm is straightforward enough. For instance:

We apply the algorithm to classify each word in the daily news archive and arrive at a sentiment indicator based on the proportion of words that are classified as “positive”. The sentiment reading for the archive for Jan-3, 2012, for example, turns out to be 67.4%:

We can automate the process of classifying the entire WSJ archive with just a few lines of code, producing a time series for the daily sentiment indicator, which has an average daily value of 68.5% – the WSJ crowd tends to be bullish, clearly! Note how the 60-day moving average of the indicator rises steadily over the period from 2012 through Q1 2015, then abruptly reverses direction, declining steadily thereafter – even somewhat precipitously towards the end of 2016.

As with most data series in investment research, we are less interested in the level of a variable, such as a stock price, than we are in the changes in level. So the next step is to calculate the daily percentage change in the sentiment indicator and examine the correlation with the corresponding intraday return in the S&P 500 Index. At first glance our sentiment indicator appears to have very little predictive power – the correlation between indicator changes and market returns is negligibly small overall – but we shall later see that this is not the last word.

Thus far the results appear discouraging; but as is often the case with this type of analysis we need to look more closely at the conditional distribution of returns. Specifically, we will examine the conditional distribution of S&P 500 Index returns when changes in the sentiment index are in the upper and lower quantiles of the distribution. This will enable us to isolate the impact of changes in market sentiment at times when the swings in sentiment are strongest. In the analysis below, we begin by examining the upper and lower third of the distribution of changes in sentiment:

The analysis makes clear that the distribution of S&P 500 Index returns is very different on days when the change in market sentiment is large and positive vs. large and negative. The difference is not just limited to the first moment of the conditional distribution, where the difference in the mean return is large and statistically significant, but also in the third moment. The much larger, negative skewness means that there is a greater likelihood of a large decline in the market on days in which there is a sizable drop in market sentiment, than on days in which sentiment significantly improves. In other words, the influence of market sentiment changes is manifest chiefly through the mean and skewness of the conditional distributions of market returns.

We can capitalize on these effects using a simple trading strategy in which we increase the capital allocated to a long-SPX position on days when market sentiment improves, while reducing exposure on days when market sentiment falls. We increase the allocation by a factor – designated the leverage factor – on days when the change in the sentiment indicator is in the upper 1/3 of the distribution, while reducing the allocation by 1/leveragefactor on days when the change in the sentiment indicator falls in lower 1/3 of the distribution. The allocation on other days is 100%. The analysis runs as follows:

It turns out that, using a leverage factor of 2.0, we can increase the CAGR from 10% to 21% over the period from 2012-2016 using the conditional distribution approach. This performance enhancement comes at a cost, since the annual volatility of the news sentiment strategy is 17% compared to only 12% for the long-only strategy. However, the overall net result is positive, since the risk-adjusted rate of return increases from 0.82 to 1.28.

We can explore the robustness of the result, comparing different quantile selections and leverage factors using Mathematica’s interactive Manipulate function:

We have seen that a simple market sentiment indicator can be created quite easily from publicly available news archives, using a standard machine learning sentiment classification algorithm. A market sentiment algorithm constructed using methods as straightforward as this appears to provide the capability to differentiate the conditional distribution of market returns on days when changes in market sentiment are significantly positive or negative. The differences in the higher moments of the conditional distribution appears to be as significant as the differences in the mean. In principle, we can use the insight provided by the sentiment indicator to enhance a long-only day-trading strategy, increasing leverage and allocation on days when changes to market sentiment are positive and reducing them on days when sentiment declines. The performance enhancements resulting from this approach appear to be significant.

Several caveats apply. The S&P 500 index is not tradable, of course, and it is not uncommon to find trading strategies that produce interesting theoretical results. In practice one would be obliged to implement the strategy using a tradable market proxy, such as a broad market ETF or futures contract. The strategy described here, which enters and exits positions daily, would incur substantial trading costs, that would be further exacerbated by the use of leverage.

Of course there are many other uses one can make of news data, in particular with firm-specific news and sentiment analytics, that fall outside the scope of this article. Hopefully, however, the methodology described here will provide a sign-post towards further, more practically useful research.

The post Trading Market Sentiment appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>The post Financial Data in Mathematica appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>The post Selling Volatility appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>The post High Frequency Scalping Strategies appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>- The strategy is highly profitable, with a Sharpe Ratio in excess of 9 (net of transaction costs of $14 prt)
- Performance is consistent and reliable, being based on a large number of trades (10-20 per day)
- The strategy has low, or negative correlation to the underlying equity and volatility indices
- There is no overnight risk

The attractiveness of such strategies is undeniable. So how does one go about developing them?

It is important for the reader to familiarize himself with some of the background to high frequency trading in general and scalping strategies in particular. Specifically, I would recommend reading the following blog posts:

The key to understanding HFT strategies is that execution is everything. With low frequency strategies a great deal of work goes into researching sources of alpha, often using highly sophisticated mathematical and statistical techniques to identify and separate the alpha signal from the background noise. Strategy alpha accounts for perhaps as much as 80% of the total return in a low frequency strategy, with execution making up the remaining 20%. It is not that execution is unimportant, but there are only so many basis points one can earn (or save) in a strategy with monthly turnover. By contrast, a high frequency strategy is highly dependent on trade execution, which may account for 80% or more of the total return. The algorithms that generate the strategy alpha are often very simple and may provide only the smallest of edges. However, that very small edge, scaled up over thousands of trades, is sufficient to produce a significant return. And since the risk is spread over a large number of very small time increments, the rate of return can become eye-wateringly high on a risk-adjusted basis: Sharpe Ratios of 10, or more, are commonly achieved with HFT strategies.

In many cases an HFT algorithm seeks to estimate the conditional probability of an uptick or downtick in the underlying, leaning on the bid or offer price accordingly. Provided orders can be positioned towards the front of the queue to ensure an adequate fill rate, the laws of probability will do the rest. So, in the HFT context, much effort is expended on mitigating latency and on developing techniques for establishing and maintaining priority in the limit order book. Another major concern is to monitor order book dynamics for signs that book pressure may be moving against any open orders, so that they can be cancelled in good time, avoiding adverse selection by informed traders, or a buildup of unwanted inventory.

In a high frequency scalping strategy one is typically looking to capture an average of between 1/2 to 1 tick per trade. For example, the VIX scalping strategy illustrated here averages around $23 per contract per trade, i.e. just under 1/2 a tick in the futures contract. Trade entry and exit is effected using limit orders, since there is no room to accommodate slippage in a trading system that generates less than a single tick per trade, on average. As with most HFT strategies the alpha algorithms are only moderately sophisticated, and the strategy is highly dependent on achieving an acceptable fill rate (the proportion of limit orders that are executed). The importance of achieving a high enough fill rate is clearly illustrated in the first of the two posts referenced above. So what is an acceptable fill rate for a HFT strategy?

I’m going to address the issue of fill rates by focusing on a critical subset of the problem: fills that occur at the extreme of the bar, also known as “extreme hits”. These are limit orders whose prices coincide with the highest (in the case of a sell order) or lowest (in the case of a buy order) trade price in any bar of the price series. Limit orders at prices within the interior of the bar are necessarily filled and are therefore uncontroversial. But limit orders at the extremities of the bar may or may not be filled and it is therefore these orders that are the focus of attention.

By default, most retail platform backtest simulators assume that all limit orders, including extreme hits, are filled if the underlying trades there. In other words, these systems typically assume a 100% fill rate on extreme hits. This is highly unrealistic: in many cases the high or low of a bar forms a turning point that the price series visits only fleetingly before reversing its recent trend, and does not revisit for a considerable time. The first few orders at the front of the queue will be filled, but many, perhaps the majority of, orders further down the priority order will be disappointed. If the trader is using a retail trading system rather than a HFT platform to execute his trades, his limit orders are almost always guaranteed to rest towards the back of the queue, due to the relatively high latency of his system. As a result, a great many of his limit orders – in particular, the extreme hits – will not be filled.

The consequences of missing a large number of trades due to unfilled limit orders are likely to be catastrophic for any HFT strategy. A simple test that is readily available in most backtest systems is to change the underlying assumption with regard to the fill rate on extreme hits – instead of assuming that 100% of such orders are filled, the system is able to test the outcome if limit orders are filled only if the price series subsequently exceeds the limit price. The outcome produced under this alternative scenario is typically extremely adverse, as illustrated in first blog post referenced previously.

In reality, of course, neither assumption is reasonable: it is unlikely that either 100% or 0% of a strategy’s extreme hits will be filled – the actual fill rate will likely lie somewhere between these two outcomes. And this is the critical issue: at some level of fill rate the strategy will move from profitability into unprofitability. The key to implementing a HFT scalping strategy successfully is to ensure that the execution falls on the right side of that dividing line.

One solution to the fill rate problem is to spend millions of dollars building HFT infrastructure. But for the purposes of this post let’s assume that the trader is confined to using a retail trading platform like Tradestation or Interactive Brokers. Are HFT scalping systems still feasible in such an environment? The answer, surprisingly, is a qualified yes – by using a technique that took me many years to discover.

To illustrate the method I will use the following HFT scalping system in the E-Mini S&P500 futures contract. The system trades the E-Mini futures on 3 minute bars, with an average hold time of 15 minutes. The average trade is very low – around $6, net of commissions of $8 prt. But the strategy appears to be highly profitable ,due to the large number of trades – around 50 to 60 per day, on average.

So far so good. But the critical issue is the very large number of extreme hits produced by the strategy. Take the trading activity on 10/18 as an example (see below). Of 53 trades that day, 25 (47%) were extreme hits, occurring at the high or low price of the 3-minute bar in which the trade took place.

Overall, the strategy extreme hit rate runs at 34%, which is extremely high. In reality, perhaps only 1/4 or 1/3 of these orders will actually execute – meaning that remainder, amounting to around 20% of the total number of orders, will fail. A HFT scalping strategy cannot hope to survive such an outcome. Strategy profitability will be decimated by a combination of missed, profitable trades and losses on trades that escalate after an exit order fails to execute.

So what can be done in such a situation?

One approach that will not work is to assume naively that some kind of manual oversight will be sufficient to correct the problem. Let’s say the trader runs two versions of the system side by side, one in simulation and the other in production. When a limit order executes on the simulation system, but fails to execute in production, the trader might step in, manually override the system and execute the trade by crossing the spread. In so doing the trader might prevent losses that would have occurred had the trade not been executed, or force the entry into a trade that later turns out to be profitable. Equally, however, the trader might force the exit of a trade that later turns around and moves from loss into profit, or enter a trade that turns out to be a loser. There is no way for the trader to know, ex-ante, which of those scenarios might play out. And the trader will have to face the same decision perhaps as many as twenty times a day. If the trader is really that good at picking winners and cutting losers he should scrap his trading system and trade manually!

An alternative approach would be to have the trading system handle the problem, For example, one could program the system to convert limit orders to market orders if a trade occurs at the limit price (MIT), or after x seconds after the limit price is touched. Again, however, there is no way to know in advance whether such action will produce a positive outcome, or an even worse outcome compared to leaving the limit order in place.

In reality, intervention, whether manual or automated, is unlikely to improve the trading performance of the system. What is certain, however, is that by forcing the entry and exit of trades that occur around the extreme of a price bar, the trader will incur additional costs by crossing the spread. Incurring that cost for perhaps as many as 1/3 of all trades, in a system that is producing, on average less than half a tick per trade, is certain to destroy its profitability.

For many years I assumed that the only solution to the fill rate problem was to implement scalping strategies on HFT infrastructure. One day, I found myself asking the question: what would happen if we slowed the strategy down? Specifically, suppose we took the 3-minute E-Mini strategy and ran it on 5-minute bars?

My first realization was that the relative simplicity of alpha-generation algorithms in HFT strategies is an advantage here. In a low frequency context, the complexity of the alpha extraction process mitigates its ability to generalize to other assets or time-frames. But HFT algorithms are, by and large, simple and generic: what works on 3-minute bars for the E-Mini futures might work on 5-minute bars in E-Minis, or even in SPY. For instance, if the essence of the algorithm is something as simple as: “buy when the price falls by more than x% below its y-bar moving average”, that approach might work on 3-minute, 5-minute, 60-minute, or even daily bars.

So what happens if we run the E-mini scalping system on 5-minute bars instead of 3-minute bars?

Obviously the overall profitability of the strategy is reduced, in line with the lower number of trades on this slower time-scale. But note that average trade has increased and the strategy remains very profitable overall.

More importantly, the average extreme hit rate has fallen from 34% to 22%.

Hence, not only do we get fewer, slightly more profitable trades, but a much lower proportion of them occur at the extreme of the 5-minute bars. Consequently the fill-rate issue is less critical on this time frame.

Of course, one can continue this process. What about 10-minute bars, or 30-minute bars? What one tends to find from such experiments is that there is a time frame that optimizes the trade-off between strategy profitability and fill rate dependency.

However, there is another important factor we need to elucidate. If you examine the trading record from the system you will see substantial variation in the extreme hit rate from day to day (for example, it is as high as 46% on 10/18, compared to the overall average of 22%). In fact, there are significant variations in the extreme hit rate during the course of each trading day, with rates rising during slower market intervals such as from 12 to 2pm. The important realization that eventually occurred to me is that, of course, what matters is not clock time (or “wall time” in HFT parlance) but trade time: i.e. the rate at which trades occur.

What we need to do is reconfigure our chart to show bars comprising a specified number of trades, rather than a specific number of minutes. In this scheme, we do not care whether the elapsed time in a given bar is 3-minutes, 5-minutes or any other time interval: all we require is that the bar comprises the same amount of trading activity as any other bar. During high volume periods, such as around market open or close, trade time bars will be shorter, comprising perhaps just a few seconds. During slower periods in the middle of the day, it will take much longer for the same number of trades to execute. But each bar represents the same level of trading activity, regardless of how long a period it may encompass.

How do you decide how may trades per bar you want in the chart?

As a rule of thumb, a strategy will tolerate an extreme hit rate of between 15% and 25%, depending on the daily trade rate. Suppose that in its original implementation the strategy has an unacceptably high hit rate of 50%. And let’s say for illustrative purposes that each time-bar produces an average of 1, 000 contracts. Since volatility scales approximately with the square root of time, if we want to reduce the extreme hit rate by a factor of 2, i.e. from 50% to 25%, we need to increase the average number of trades per bar by a factor of 2^2, i.e. 4. So in this illustration we would need volume bars comprising 4,000 contracts per bar. Of course, this is just a rule of thumb – in practice one would want to implement the strategy of a variety of volume bar sizes in a range from perhaps 3,000 to 6,000 contracts per bar, and evaluate the trade-off between performance and fill rate in each case.

Using this approach, we arrive at a volume bar configuration for the E-Mini scalping strategy of 20,000 contracts per bar. On this “time”-frame, trading activity is reduced to around 20-25 trades per day, but with higher win rate and average trade size. More importantly, the extreme hit rate runs at a much lower average of 22%, which means that the trader has to worry about maybe only 4 or 5 trades per day that occur at the extreme of the volume bar. In this scenario manual intervention is likely to have a much less deleterious effect on trading performance and the strategy is probably viable, even on a retail trading platform.

(Note: the results below summarize the strategy performance only over the last six months, the time period for which volume bars are available).

We have seen that is it feasible in principle to implement a HFT scalping strategy on a retail platform by slowing it down, i.e. by implementing the strategy on bars of lower frequency. The simplicity of many HFT alpha generation algorithms often makes them robust to generalization across time frames (and sometimes even across assets). An even better approach is to use volume bars, or trade-time, to implement the strategy. You can estimate the appropriate bar size using the square root of time rule to adjust the bar volume to produce the requisite fill rate. An extreme hit rate if up to 25% may be acceptable, depending on the daily trade rate, although a hit rate in the range of 10% to 15% would typically be ideal.

Finally, a word about data. While necessary compromises can be made with regard to the trading platform and connectivity, the same is not true for market data, which must be of the highest quality, both in terms of timeliness and completeness. The reason is self evident, especially if one is attempting to implement a strategy in trade time, where the integrity and latency of market data is crucial. In this context, using the data feed from, say, Interactive Brokers, for example, simply will not do – data delivered in 500ms packets in entirely unsuited to the task. The trader must seek to use the highest available market data feed that he can reasonably afford.

That caveat aside, one can conclude that it is certainly feasible to implement high volume scalping strategies, even on a retail trading platform, providing sufficient care is taken with the modeling and implementation of the system.

The post High Frequency Scalping Strategies appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>The post Stochastic Calculus in Mathematica appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>Financial Engineering has long been a staple application of Mathematica, an area in which is capabilities in symbolic logic stand out. But earlier versions of the software lacked the ability to work with Ito calculus and model stochastic processes, leaving the user to fill in the gaps by other means. All that changed in version 9 and it is now possible to provide the complete framework of modern financial theory within the context of the Wolfram Language.

The advantages of this approach are considerable. The capabilities of the language make it easy to provide interactive examples to illustrate theoretical concepts and develop the student’s understanding of them through experimentation. Furthermore, the student is not limited merely to learning and applying complex formulae for derivative pricing and risk, but can fairly easily derive the results for themselves. As a consequence, a course in stochastic calculus taught using Mathematica can be broader in scope and go deeper into the theory than is typically the case, while at the same time reinforcing understanding and learning by practical example and experimentation.

The post Stochastic Calculus in Mathematica appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>The post Applications of Graph Theory In Finance appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>

In this example I have selected a universe of the Dow 30 stocks, together with a sample of commodities and bonds and compiled a database of daily returns over the period from Jan 2012 to Dec 2013. If we want to look at how the assets are correlated, one way is to created an adjacency graph that maps the interrelations between assets that are correlated at some specified level (0.5 of higher, in this illustration).

Obviously the choice of correlation threshold is somewhat arbitrary, and it is easy to evaluate the results dynamically, across a wide range of different threshold parameters, say in the range from 0.3 to 0.75:

The choice of parameter (and time frame) may be dependent on the purpose of the analysis: to construct a portfolio we might select a lower threshold value; but if the purpose is to identify pairs for possible statistical arbitrage strategies, one will typically be looking for much higher levels of correlation.

Reverting to the original graph, there is a core group of highly inter-correlated stocks that we can easily identify more clearly using the Mathematica function FindClique to specify graph nodes that have multiple connections:

We might, for example, explore the relative performance of members of this sub-group over time and perhaps investigate the question as to whether relative out-performance or under-performance is likely to persist, or, given the correlation characteristics of this group, reverse over time to give a mean-reversion effect.

An obvious application might be to construct a replicating portfolio comprising this equally-weighted sub-group of stocks, and explore how well it tracks the Dow index over time (here I am using the DIA ETF as a proxy for the index, for the sake of convenience):

The correlation between the Dow index (DIA ETF) and the portfolio remains strong (around 0.91) throughout the out-of-sample period from 2014-2016, although the performance of the portfolio is distinctly weaker than that of the index ETF after the early part of 2014:

Another application might be to construct robust portfolios of lower-correlated assets. Here for example we use the graph to identify independent vertices that have very few correlated relationships (designated using the star symbol in the graph below). We can then create an equally weighted portfolio comprising the assets with the lowest correlations and compare its performance against that of the Dow Index.

The new portfolio underperforms the index during 2014, but with lower volatility and average drawdown.

Graph theory clearly has a great many potential applications in finance. It is especially useful as a means of providing a graphical summary of data sets involving a large number of complex interrelationships, which is at the heart of portfolio theory and index replication. Another useful application would be to identify and evaluate correlation and cointegration relationships between pairs or small portfolios of stocks, as they evolve over time, in the context of statistical arbitrage.

The post Applications of Graph Theory In Finance appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>The post High Frequency Futures Scalping Strategies Update appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>

The post High Frequency Futures Scalping Strategies Update appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>The post Metal Logic appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>Precious metals have been in free-fall for several years, as a consequence of the Fed’s actions to stimulate the economy that have also had the effect of goosing the equity and fixed income markets. All that changed towards the end of 2015, as the Fed moved to a tightening posture. So far, 2016 has been a banner year for metal, with spot prices for platinum, gold and silver up 26%, 28% and 44% respectively.

So what are the prospects for metals through the end of the year? We take a shot at predicting the outcome, from a quantitative perspective.

Source: Wolfram Alpha. Spot silver prices are scaled x100

One of the key characteristics of metals is the very high levels of price-correlation between them. Over the period under investigation, Jan 2012 to Aug 2016, the estimated correlation coefficients are as follows:

A plot of the join density of spot gold and silver prices indicates low- and high-price regimes in which the metals display similar levels of linear correlation.

Levels of correlation that are consistently as high as this over extended periods of time are fairly unusual in financial markets and this presents a potential trading opportunity. One common approach is to use the ratios of metal prices as a trading signal. However, taking the ratio of gold to silver spot prices as an example, a plot of the series demonstrates that it is highly unstable and susceptible to long term trends.

A more formal statistical test fails to reject the null hypothesis of a unit root. In simple terms, this means we cannot reliably distinguish between the gold/silver price ratio and a random walk.

Along similar lines, we might consider the difference in log prices of the series. If this proved to be stationary then the log-price series would be cointegrated order 1 and we could build a standard pairs trading model to buy or sell the spread when prices become too far unaligned. However, we find once again that the log-price difference can wander arbitrarily far from its mean, and we are unable to reject the null hypothesis that the series contains a unit root.

We can hope to do better with a standard linear model, regressing spot silver prices against spot gold prices. The fit of the best linear model is very good, with an R-sq of over 96%:

A trader might look to exploit the correlation relationship by selling silver when its market price is greater than the value estimated by the model (and buying when the model price exceeds the market price). Typically the spread is bought or sold when the log-price differential exceeds a threshold level that is set at twice the standard deviation of the price-difference series. The threshold levels derive from the assumption of Normality, which in fact does not apply here, as we can see from an examination of the residuals of the linear model:

Given the evident lack of fit, especially in the left tail of the distribution, it is unsurprising that all of the formal statistical tests for Normality easily reject the null hypothesis:

However, Normality, or the lack of it, is not the issue here: one could just as easily set the 2.5% and 97.5% percentiles of the empirical distribution as trade entry points. The real problem with the linear model is that it fails to take into account the time dependency in the price series. An examination of the residual autocorrelations reveals significant patterning, indicating that the model tends to under-or over-estimate the spot price of silver for long periods of time:

As the following chart shows, the cumulative difference between model and market prices can become very large indeed. A trader risks going bust waiting for the market to revert to model prices.

How does one remedy this? The shortcoming of the simple linear model is that, while it captures the interdependency between the price series very well, it fails to factor in the time dependency of the series. What is required is a model that will account for both features.

Rather than modeling the metal prices individually, or in pairs, we instead adopt a multivariate vector autoregression approach, modeling all three spot price processes together. The essence of the idea is that spot prices in each metal may be influenced, not only by historical values of the series, but also potentially by current and lagged prices of the other two metals.

Before proceeding we divide the data into two parts: an in-sample data set comprising data from 2012 to the end of 2015 and an out-of-sample period running from Jan-Aug 2016, which we use for model testing purposes. In what follows, I make the simplifying assumption that a vector autoregressive moving average process of order (1, 1) will suffice for modeling purposes, although in practice one would go through a procedure to test a wide spectrum of possible models incorporating moving average and autoregressive terms of varying dimensions.

In any event, our simplified VAR model is estimated as follows:

The chart below combines the actual, in-sample data from 2012-2015, together with the out-of-sample forecasts for each spot metal from January 2016.

It is clear that the model projects a recovery in spot metal prices from the end of 2015. How did the forecasts turn out? In the chart below we compare the actual spot prices with the model forecasts, over the period from Jan to Aug 2016.

The actual and forecast percentage change in the spot metal prices over the out-of-sample period are as follows:

The VAR model does a good job of forecasting the strong upward trend in metal prices over the first eight months of 2016. It performs exceptionally well in its forecast of gold prices, although its forecasts for silver and platinum are somewhat over-optimistic. Nevertheless, investors would have made money taking a long position in any of the metals on the basis of the model projections.

Another way to apply the model would be to implement a relative value trade, based on the model’s forecast that silver would outperform gold and platinum. Indeed, despite the model’s forecast of silver prices turning out to be over-optimistic, a relative value trade in silver vs. gold or platinum would have performed well: silver gained 44% in the period form Jan-Aug 2016, compared to only 26% for gold and 28% for platinum. A relative value trade entailing a purchase of silver and simultaneous sale of gold or platinum would have produced a gross return of 17% and 15% respectively.

A second relative value trade indicated by the model forecasts, buying platinum and selling gold, would have turned out less successfully, producing a gross return of less than 2%. We will examine the reasons for this in the next section.

If we re-estimate the VAR model using all of the the available data through mid-Aug 2016 and project metal prices through the end of the year, the outcome is as follows:

While the positive trend in all three metals is forecast to continue, the new model (which incorporates the latest data) anticipates lower percentage rates of appreciation going forward:

Once again, the model predicts higher rates of appreciation for both silver and platinum relative to gold. So investors have the option to take a relative value trade, hedging a long position in silver or platinum with a short position in gold. While the forecasts for all three metals appear reasonable, the projections for platinum strike me as the least plausible.

The reason is that the major applications of platinum are industrial, most often as a catalyst: the metal is used as a catalytic converter in automobiles and in the chemical process of converting naphthas into higher-octane gasolines. Although gold is also used in some industrial applications, its demand is not so driven by industrial uses. Consequently, during periods of sustained economic stability and growth, the price of platinum tends to be as much as twice the price of gold, whereas during periods of economic uncertainty, the price of platinum tends to decrease due to reduced industrial demand, falling below the price of gold. Gold prices are more stable in slow economic times, as gold is considered a safe haven.

This is the most likely explanation of why the gold-platinum relative value trade has not worked out as expected hitherto and is perhaps unlikely to do so in the months ahead, as the slowdown in the global economy continues.

We have shown that simple models of the ratio or differential in the prices of precious metals are unlikely to provide a sound basis for forecasting or trading, due to non-stationarity and/or temporal dependencies in the residuals from such models.

On the other hand, a vector autoregression model that models all three price processes simultaneously, allowing both cross correlations and autocorrelations to be captured, performs extremely well in terms of forecast accuracy in out-of-sample tests over the period from Jan-Aug 2016.

Looking ahead over the remainder of the year, our updated VAR model predicts a continuation of the price appreciation, albeit at a slower rate, with silver and platinum expected to continue outpacing gold. There are reasons to doubt whether the appreciation of platinum relative to gold will materialize, however, due to falling industrial demand as the global economy cools.

The post Metal Logic appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>The post Dynamic Time Warping appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>You certainly wouldn’t know it from a reading of the CBOE S&P500 Volatility Index (CBOE:VIX), which printed a low of 11.44 on Friday, but there is a great deal of uncertainty about the prospects for the market as we move further into the third quarter, traditionally the most challenging period of the year. Reasons for concern are not hard to fathom, with the Fed on hold and poised to start raising rates, despite anemic growth in the economy; gloom over the “earnings recession”; and an abundance of political risk factors in play, not least of which is the upcoming presidential election.

At times like these investors need a little encouragement to stay the course – and where better to look for it than in the history books. More specifically, the question is whether the past has anything to teach us about the prospects for the market, going forward. Academic theory says no; but Wall Street traders controlling trillions of dollars of investments believe that, on the contrary, history contains valuable information that can be helpful in predicting the likely future outcome for the market.

There are several difficulties in making historical comparisons. Firstly, while it maybe intuitively obvious what one means when speaking about a “comparable” historical period, it is not as easy as one might think to translate that concept into mathematical form. And it does need to be translated, because unless you plan to study market history comprising tens or hundreds of thousands of historical patterns by hand (eye), you are going to have to rely on computers to do the heavy lifting.

Perhaps the most well-known measure of “similarity” is correlation; but this is entirely unsuited to the task at hand. The reason is that correlation removes the very thing that traders are most interested in: the trend, or drift. Let me illustrate the point with a simple example. The following chart illustrates two, perfectly correlated series (technically, Wiener processes) that simulate the evolution of returns in two stocks.

Source: Wolfram Research

Even though the processes are perfectly correlated, they end up producing very different outcomes, chiefly because the red process has a strongly positive drift, or trend, while the green process has a strongly negative drift. It is highly implausible that the path realized by the green process would be regarded as similar to that of the red process, despite their correlation. The moral is simple: whatever else we mean by “similar”, it must entail a comparable result in terms of the overall appreciation (or depreciation) of the stock; i.e. the two periods must show a similar degree of trending behavior.

Now let’s consider a second illustrative example. Here, not only are the two processes perfectly correlated, they also have an identical drift (14%).

Source: Wolfram Research

Despite their commonalities, it is again very unlikely that we would regard the behavior of the green process as being sufficiently “similar” to that of the red process. Here the problem is not the difference in drift, but in volatility, which is ten times higher for the green process than for the red process. Even though both processes are expected to appreciate at the same average rate, the high volatility of the green process means that it may appreciate at a much faster, or slower, rate than the mean. We say that the volatility “drowns out” the drift in the process, at least over the short term.

So, in conclusion, what we are looking for in our quest for “similarity” is a period in which the process has displayed, not only a similar trend, but also a comparable level of volatility.

The challenges don’t end there, however. Let’s say we find a period in history in which the overall appreciation and volatility of the process is “similar” to that of the period of interest, but the historical period is shorter, or longer, than the period we are evaluating. Does that matter? Well to some degree, it must: if during the historical period the stock appreciated by 50% in 3 days, vs the same percentage over a term of 3 months during the current period, it is again unlikely that we would regard the historical period as similar. On the other hand, if the two periods differed in length by, say, only 4-5 days out of a total of 30, would that be enough to make you discard the historical precedent? Probably not, other things being equal. So there has to be scope for some, but not unlimited, flexibility in making historical comparisons using periods of slightly differing durations.

Having presented the problem, let me now turn to the solution.

Dynamic Time Warping (DTW) is a machine learning algorithm originally developed for speech recognition that aims two align two sequences by warping the time axis iteratively until a match is found between the two sequences.

Consider two sequences

The two sequences can be arranged on the sides of a grid, with one on the top and the other up the left hand side. Both sequences start on the bottom left of the grid.

Inside each cell a distance measure is used to compare the corresponding elements of the two sequences. To find the best match or alignment between these two sequences one needs to find a path through the grid which minimizes the total distance between them. The procedure for computing this overall distance involves finding all possible routes through the grid and computing the overall distance for each one. The overall distance is the minimum of the sum of the distances between the individual elements on the path divided by the sum of the weighting function.

A great many measures of “distance” (the inverse of “similarity”) have been developed over the years, in fields as disparate as image analysis, signal processing and machine learning. In what follows we shall be using Euclidean Distance as our preferred distance metric, but there are a large number of alternative options:

The DTW algorithm optimizes the distance, constrained by limits on acceptable paths through the grid:

: the path will not turn back on itself, both the__Monotonic condition__*i*and*j*indexes either stay the same or increase, they never decrease.: the path advances one step at a time. Both__Continuity condition__*i*and*j*can only increase by at most 1 on each step along the path.: the path starts at the bottom left and ends at the top right.__Boundary condition__: a good path is unlikely to wander very far from the diagonal. The distance that the path is allowed to wander is the window width.__Warping window condition__: The path should not be too steep or too shallow. This prevents short sequences matching too long ones. The condition is expressed as a ratio__Slope constraint condition__*p/q*where*p*is the number of steps allowed in the same (horizontal or vertical) direction. After*p*steps in the same direction is not allowed to step further in the same direction before stepping at least*q*time in the diagonal direction.

These constraints allow to restrict the moves that can be made from any point in the path and so limit the number of paths that need to be considered. The power of the DTW algorithm lies in that instead finding all possible routes through the grid which satisfy the above conditions, it works by keeping track of the cost of the best path to each point in the grid. During the calculation process of the DTW grid it is not known which path is minimum overall distance path, but this can be traced back when the end point is reached.

Let’s apply the DTW approach to find a similar period in the history of the SPDR S&P 500 ETF (NYSEArca:SPY).

We download data for the SPY ETF and look for periods in the history of the series from 1995 that are similar to how the process has evolved in Q3 2016. Note that we are using log-returns rather than raw prices. This is because what we care about is that the rate at which the ETF has appreciated relative to its initial starting value should be comparable for both periods.

The algorithm picks out a 21-day period from 3/6/1995 to 4/7/1995 as being the most similar to the way the SPY ETF has progressed in Q3 2016. Let’s take a look at the two periods and assess their similarity. Note that we set the initial starting price at 100 in each case.

Source: Author

It is clear that the SPY appreciates by approximately the same percentage over the two periods, but the way in which it evolves looks rather different in each case. That’s because the earlier period covers only 25 business days, compared to the 51 days from the start of Q3 2016 period. If you can imagine dilating the shorter time span of the orange line in the above chart to cover the same span as the blue line, the patterns look very similar indeed.

We are going to use the historical period to forecast how the ETF is likely to evolve of the next 35 days to the end of Q3. We do this by mapping the evolution of the ETF over the 35 days following the end of the historical period on 4/7/95 and projecting that pattern onto the end of the current period, from 8/12/16. The result is as follows.

Source: Author

In other words, if the closely parallel history of the series from 1995 were to repeat itself in 2016, we might expect a further gain of perhaps 3.5% to 4% in the SPY ETF between now and the end of September.

Whether you hold this approach as valid or not depends largely on how credible you regard the thesis that history is inclined to repeat itself. There are significant differences in fundamental factors that are likely to affect the outcome over the next several months, compared to the similar period in 1995.

The early part of 1995 marked the beginning of a powerful market surge driven by an economic boom that lasted until late 1999. By contrast, in the more recent period, the economy has been emerging very slowly from the long-running credit crisis commencing in 2008. Likewise, in contrast to 1995 when the Fed was at the early stage of reducing interest rates, the Fed is now more likely to start raising rates at some point before the end of 2016.

While these and other aspects of the economic landscape are fundamentally different, there are parallels. The recent slowdown in productivity, for example, often cited as a cause for concern, mirrors a similar slowdown in 1995. As Bloomberg points out, such pauses are not uncommon and in fact the plateaus in 1993-1996 and 2003-2005 occurred at a time when the economy was pretty healthy. Conversely, several of the biggest productivity jumps have happened during recessions, in 2001 and 2009.

Source: Bloomberg

Doom-mongering is at record levels these days, as the market continues to make new highs. The Wall Street Journal, for example, recently ran a piece entitled “This Tech Bubble Is Bursting”, drawing parallels with the dot-com bust of 2000.

They could be right.

On the other hand, if history repeats itself we could see the market 4% higher going into the final quarter of 2016.

Sakoe,H. and Chiba, S. Dynamic programming algorithm optimization for spoken word recognition. IEEE Trans. on Acoust., Speech, and Signal Process., ASSP 26, 43-49 (1978).

The post Dynamic Time Warping appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>