The post The Systematic Volatility Strategy appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>

The post The Systematic Volatility Strategy appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>The post The Systematic Strategies Quantitative Equity Strategy appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>In 2012 we created a research program into quantitative equity strategies using machine learning techniques, earlier versions of which were successfully employed in Proteom Capital, a hedge fund that pioneered the approach in the early 2000’s. Proteom managed capital for Paul Tudor Jones’s Tudor Capital and several other major institutional investors until 2007.

Systematic Strategies began trading its Quantitative Equity Strategy, the result of its research program, in 2013. Having built a four year live track record, the firm is opening the strategy to external investors in 2017. The investment program will be offered in managed account format and, for the first time, as a hedge fund product. For more details about the hedge fund offering, please see this post about the Systematic Strategies Fund.

Designed to achieve returns comparable to the benchmark S&P 500 Index, but with much lower risk, the Quantitative Equity Strategy has produced annual returns at a compound rate of 14.74% (net of fees) since 2013, with an annual volatility of 4.46% and realized Sharpe Ratio of 3.31. By contrast, the benchmark S&P 500 Index yielded a compound annual rate of return of 11.93%, with annual volatility of 10.40% and Sharpe Ratio of 1.15. In other words, the strategy has outperformed the benchmark S&P 500 Index by a significant margin, but with much reduced volatility and downside risk.

For more details about the hedge fund offering, please see this post about the Systematic Strategies Fund. If you are an Accredited Investor and wish to receive a copy of the fund offering documents, please write to us at info@systematic-stratgies.com.

The post The Systematic Strategies Quantitative Equity Strategy appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>The post Strategy Portfolio Construction appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>The essence of the strategy portfolio approach lies in the understanding that it is much easier to create a diversified portfolio of equity strategies than a diversified portfolio of the underlying assets. In principle, it is quite feasible to create a basket of strategies for highly correlated stocks that are uncorrelated, or even negatively correlated with one another. For example, one could combine a mean-reversion strategy on a stock like Merck & Co. (MRK) with a momentum strategy on a correlated stock such as Pfizer (PFE), that will typically counteract one another rather than moving in lockstep as the underlying equities tend to do.

In fact this approach is widely employed by hedge fund investors as well as proprietary trading firms and family offices, who seek to diversify their investments across a broad range of underlying strategies in many different asset classes, rather than by investing in the underlying assets directly.

What is to be gained from such an approach to portfolio construction, compared to the typical methodology? The answer is straightforward: lower risk, which results from the lower correlation between strategies, compared to the correlation between the assets themselves. That in turn produces a more diversified portfolio, reducing strategy beta, volatility and tail risk. We see all of these characteristics in the Systematic Strategies Quantitative Equity Strategy, described below, which has an average beta of only 0.2 and a maximum realized drawdown of -2.08%.

Risk reduction is only part of the story, however. The Quantitative Equity Strategy has also yielded a combined alpha in excess of 4% per annum, outperforming the benchmark by a total of 16.35% (net of fees) since 2013. In other words, the individual equity strategies produce alphas that are conserved when combined together to create the overall portfolio. But even if the strategies produced little or no alpha, the risk-reduction benefits of the approach would likely still apply.

What are the drawbacks to this approach to portfolio construction? One major challenge is that it is much harder to create strategy portfolios than asset portfolios. The analyst is obliged to create (at least) one individual strategy for each asset in the universe, rather than a single strategy for the portfolio as a whole. This constrains the rate at which the investment universe can grow (it takes time to research and develop good strategies!), limiting the rate of growth of assets under management. So it is not an approach that I would necessarily recommend if your goal is to deploy multiple billions of dollars; but AUM up to around a billion dollars is certainly a feasible target.

From a risk perspective, the chief limitation is that we lack the ability to control the makeup of the resulting portfolio as closely as we can with traditional approaches to portfolio construction. In a typical equity long/short strategy the portfolio is constrained to have a specified maximum beta, and overall net exposure. With the strategy portfolio approach we are unable to guarantee a specific limit on the net exposure, or the beta of the portfolio. We may be able to quantify that, historically, the portfolio has an average net long exposure, of, say, 25% and an average beta of 0.2; but there is nothing to guarantee that a situation may not arise in future in which all of the strategies align to produce a 100% net long or net short exposure and a beta of +1/-1, or greater. This is extremely unlikely, of course, and may never happen even in geological timescales, as can be demonstrated by Monte Carlo simulation. What is likely, however, is that there will be periods in which the beta and net exposure of the portfolio may fluctuate widely.

Is this a problem? Yes and no. The point about constraining the portfolio beta and net exposure in a typical long/short strategy is to manage portfolio risk. Such constraints decrease the likelihood of substantial losses – but they cannot guarantee that outcome, as has been demonstrated during prior periods of severe market stress such as 2000/01 and 2008/09. Asset correlations tend to increase significantly when markets suffer major declines, often undermining the assumptions on which the portfolio and its associated risk limits were originally based.

Similarly, the way in which we construct strategy portfolios takes a statistical approach to risk control, using stochastic process control. Just as with the traditional approach to portfolio construction, statistical analysis cannot guarantee that market conditions may not arise that give rise to substantial losses, however unlikely such circumstances may be.

*Read more about stochastic process control:*

*More on genetic programming:*

Is the risk of as yet unknown future market conditions greater for strategy portfolios greater than for traditional equity long/short portfolios? I would say not. In fact, during periods of market stress I would prefer to be in a portfolio of strategies, some of which at least are likely to perform well, rather than in a portfolio of equities which are now all likely to be highly correlated to the benchmark index.

The benefit of being able to check the boxes for risk controls such as limits on portfolio beta and net exposure is, I would argue, somewhat illusory. They might be characterized as just a placebo for those who are, literally, unable (or, more likely, not paid to) to think outside the box.

But if this argument is not sufficiently persuasive, it is perfectly feasible to overlay risk controls on a portfolio comprising several underlying strategies. For example one can:

- Hedge out portfolio beta and net exposure above a specified level, using index ETFs, futures or options
- Reduce the capital allocations during periods of market stress, or when portfolio beta or market exposure exceeds a threshold level
- Turn off strategies that under-performing in current market conditions

All of these measures will impact strategy performance. By and large, our preference is to let the strategy play out as it may, trusting in the robustness of the strategy portfolio to produce an acceptable outcome. We leave it to the investor to decide what additional risk controls he wishes to implement, either independently, or within a separately managed account.

Systematic Strategies started out in 2009 as a proprietary trading firm engaged in high frequency trading. In 2012 the firm expanded into low frequency systematic trading strategies with the launch of our VIX ETF strategy, which was superseded in 2015 by the Systematic Volatility Strategy. The firm began managing external capital in its managed account platform in 2015.

In 2012 we created a research program into quantitative equity strategies using machine learning techniques, earlier versions of which were successfully employed in Proteom Capital, a hedge fund that pioneered the approach in the early 2000’s. Proteom managed capital for Paul Tudor Jones’s Tudor Capital and several other major institutional investors until 2007.

Systematic Strategies began trading its Quantitative Equity Strategy, the result of its research program, in 2013. Having built a four year live track record, the firm is opening the strategy to external investors in 2017. The investment program will be offered in managed account format and, for the first time, as a hedge fund product. For more details about the hedge fund offering, please see this post about the Systematic Strategies Fund.

Designed to achieve returns comparable to the benchmark S&P 500 Index, but with much lower risk, the Quantitative Equity Strategy has produced annual returns at a compound rate of 14.74% (net of fees) since 2013, with an annual volatility of 4.46% and realized Sharpe Ratio of 3.31. By contrast, the benchmark S&P 500 Index yielded a compound annual rate of return of 11.93%, with annual volatility of 10.40% and Sharpe Ratio of 1.15. In other words, the strategy has outperformed the benchmark S&P 500 Index by a significant margin, but with much reduced volatility and downside risk.

For more details about the hedge fund offering, please see this post about the Systematic Strategies Fund. If you are an Accredited Investor and wish to receive a copy of the fund offering documents, please write to us at info@systematic-stratgies.com.

The post Strategy Portfolio Construction appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>The post HFT VIX Scalper Leads on Collective2 appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>For more background on HFT scalping strategies see the following post:

The post HFT VIX Scalper Leads on Collective2 appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>The post Systematic Strategies Fund appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>In 2012 the firm began R&D of a new Quantitative Equity strategy, which was opened to investors at the beginning of 2017. Over the four years since inception in 2013, the Quantitative Equity strategy has achieved a compound annual rate of return of 14.74%, with a realized Sharpe Ratio of 3.31. More information about the strategy can be found in this post:

Due to increasing demand for the firm’s investment products, at the beginning of 2017 we have launched a new hedge fund vehicle, the Systematic Strategies Fund LLC, that will be open to accredited investors wishing to invest in any of our existing or future investment programs. These currently include the Systematic Volatility Strategy (Series A) and the Quantitative Equity Strategy (Series B). Additional investment programs will be added to the fund offering in due course. The new fund will give smaller investors access to investment programs that are available in SMA form only for larger managed accounts.

If you are an accredited investor and wish to receive copies of the fund offering documents, please contact us on info@systematic-strategies.com

The post Systematic Strategies Fund appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>The post The Algorithm appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>`teststring = "ItellyoumadamthecatisnotacivicanimalalthoughtisdeifiedinEgypt";`

`nlargest = 5;`

`TakeLargestBy[Cases[StringCases[teststring, __, Overlaps -> All], _?PalindromeQ], StringLength, nlargest] // Flatten`

which produces the n largest palindromes, in this case:

`{"deified", "acivica", "madam", "civic", "eifie"}`

My offering was fairly quickly superseded by a faster and more elegant solution from another Mathematica aficionado:

```
MaximalBy[StringCases[teststring, x : Repeated[_, {2, \[Infinity]}]
/; PalindromeQ[x], Overlaps -> True], StringLength, nlargest]
```

I have written previously about some of the new developments in programming languages, for example:

The joy of high-level, functional programming languages like Mathematica is that they often allow complex tasks to be accomplished easily, or at least in very few lines of code, compared to procedural languages like C, C++, Java or Python. The latter would probably require at least a dozen lines of code to achieve what Mathematica is able to accomplish in just one: see, for example, the table below, which compares the relative verbosity of various programming languages, including Mathematica.

Source: Wolfram Research

We can compare “tokens,” or any string of letters and numbers that are not interrupted by a number or punctuation. This lets us classify length in terms of “units of syntax,” which, while it isn’t perfect, gives us a clearer picture of the number of different elements required to build a function or program. Below, the Wolfram Language appears to, on average, increase in token count at a slower rate than Python. Using `FindFit`, we can estimate that a typical Python program that requires *x* tokens can be written in the Wolfram Language with 3.48 tokens, meaning a Python program that requires 1,000 tokens would require just 110 tokens in the Wolfram Language.

Source: Wolfram Research

So what is the message here?

For now, let me skate over the question of execution speed, typically the foremost objection that arises at this stage of the endless argument about interpreted vs. compiled programming languages. I promise to return to that topic very shortly. Instead let me first point out an often underestimated obstacle one faces in programming in Mathematica: the idiom. It can sometimes take as long to figure out the correct programming method in a single line of Mathematica code as it might take to code an algorithm more prosaically in a relatively verbose language like Java. And even after you have successfully achieved the goal, you are left with the challenge of understanding your own code when you return to it, perhaps months (or years) later. Even worse, you may have to disentangle the complex prose of another Mathematica programmer.

On the plus side, one of the great advantages of languages like Mathematica is that programs are largely a compilation of functions. So one assembles programs piece by piece, testing the functionality of each component function before adding another layer of complexity. Reverse engineering a Mathematica program – no matter how complex – follows roughly the same process, in reverse: in the above code you would first try to understand what the function MaximalBy does, then StringCases, then PalindromeQ (that one is fairly obvious!). The nature of functional programming languages makes it straightforward to assemble complex pieces of code from relatively simple sub-components, and also to reverse the process to dis-assemble them. The interpretative nature of the language is helpful here, too, since one can (unit) test each component on the fly, something that is much more tedious to do in a compiled language.

In everyday usage we often conflate the terms “algorithm” and “code”. The code given above is an implementation of an algorithm, but it is not the algorithm itself. An algorithm is a recipe or sequence of steps that will produce a specified result. It can be written down as a series of instructions, in English or any other language, just as easily as a computer language and can be executed manually, if necessary. During WWII, before the advent of modern computers, large teams of women were employed to do just that:

In the case of the present example the algorithm runs something like this:

- Examine every substring of the string
- Test each substring to see if it is a palindrome
- Add any that are palindromes to the list
- Sort the list of substrings by string length
- Pick the n longest substrings

In principle, there is nothing to prevent us executing the algorithm by hand, except possibly time and boredom. Computer code is just a very much more convenient and efficient means of execution. But this usefulness can give rise to a kind of mental laziness, in which we focus on the problem of producing the most elegant computer code, rather than the most efficient algorithm. We have, in essence, got the problem turned around: we should first think about the algorithm and only then begin to think about how we might implement it. Not proceeding in that logical fashion risks producing solutions that are distinctly sub-optimal, whatever the efficiency and elegance of the implemented code.

In this case, we have used a “brute force” algorithm that examines every substring for possible palindromes. This is highly inefficient, scaling with the square of the number of characters n in the string, i.e. O(n^2). To see this, consider the 4-character string “abcd”. There are a total of 10 substrings:

- 4 single character substrings: {a, b, c, d}
- 3 two-character substrings: {ab, bc, cd}
- 2 three-character substrings {abc, bcd}
- 1 four-character string {abcd}

In general, for a string of length n, there are:

- n single-character substrings,
- (n-1) 2-character substrings,

. . . . . . . . . . . . ,

- 2 substrings length n-1
- and a single string length n.

So the total number of substrings is:

Let’s demonstrate the inefficiency of the algorithm using Virgil’s Aeneid as our source of palindromes – perhaps not the smartest choice, given the nature of poetry!

`Aeneid = ExampleData[{"Text", "AeneidEnglish"}]; StringLength[Aeneid] 606071 teststring = StringTake[Aeneid, 1000]; nlargest = 20; StringTake[teststring, 100] I Arms, and the man I sing, who, forc'd by fate, And haughty Juno's unrelenting hate, Expell'd`

`MaximalBy[StringCases[teststring, x : Repeated[_, {2, \[Infinity]}] /; PalindromeQ[x], Overlaps -> True], StringLength, nlargest] // Timing`

{12.1068, {" I ", " I ", "ele", "t t", "a a", "t t", "ivi", "g g", " O ", "ses", "ovo", " a ", "s s", "h h", "ese", "exe", "t t", "awa", "t t", "s s"}

It takes a full 12 seconds to find the 20 largest palindromes in only the first 1,000 characters of the 600,00 character text.

If we double the length of the test string, the execution time increases by a factor of over 4x:

`teststring = StringTake[Aeneid, 2000];`

`MaximalBy[StringCases[teststring, x : Repeated[_, {2, \[Infinity]}] /; PalindromeQ[x], Overlaps -> True], StringLength, nlargest] // Timing`

{56.4254, {" I ", " I ", "ele", "t t", "a a", "t t", "ivi", "g g"," O ", "ses", "ovo", " a ", "s s", "h h", "ese", "exe", "t t", "awa", "t t", "s s"}

It is clear that even if we succeed in speeding up the code, the inefficiency of the algorithm will render it useless for all but the shortest strings – scanning the entire Aeneid for palindromes would take the algorithm at least 100 hours!

Fortunately, in 1975 Glenn Manacher came up with a much smarter algorithm that scales linearly with the length of the string, which you can read about here. In essence his approach exploits the symmetry in palindromes to reduce the search time.

An implementation of Manacher’s algorithm in Mathematica is given below. The code executes extremely quickly, taking only around a quarter of a second to scan the entire 600,000 character text of the Aeneid to find the longest palindrome (“man nam”).

AbsoluteTiming[findLongestPalindrome[Aeneid]] (* {0.236135, "man nam"} *)

A large part of the speed enhancement is due to the much greater efficiency of Manacher’s algorithm. Notice, too, however, that further significant speed gains are made by compiling the Mathematica code via C, which Mathematica is able to do as standard.

Doc, note: I dissent. A fast never prevents a fatness. I diet on cod

– Mathematician Peter Hilton

The lessons here are, firstly, to focus on the problem of the algorithm rather than the code, clearly distinguishing between the two.

Secondly, the complex idiom of functional programming languages like Mathematica can complicate the task of programming an algorithm, or understanding code written by others. On the other hand, the brevity of the language makes it easier to get an overview of the functionality of a program, while the functional structure makes the process of assembling – or disassembling – complex programs at least logically coherent.

Finally, it is important to understand that modern interpreted functional languages like Mathematica are a great deal more viable as production systems than they ever used to be, thanks to the introduction of capabilities to produce compiled code.

```
findLongestPalindrome[""] = "";
findLongestPalindrome[s_String] :=
FromCharacterCode @ findLongestPalindromeList[ToCharacterCode @ s];
findLongestPalindromeList = Compile[{{s, _Integer, 1}},
Module[{s2, p, c, r, n, m, i2, len, cc},
s2 = Riffle[s, -1, {1, -1, 2}];
p = ConstantArray[0, Length[s2]];
c = 1; r = 1; m = 1; n = 1; len = 0; cc = 1;
Do[
If[i > r,
p[[i]] = 0; m = i - 1; n = i + 1,
i2 = 2 c - i;
If[p[[i2]] < (r - i),
p[[i]] = p[[i2]]; m = 0,
p[[i]] = r - i; n = r + 1; m = 2 i - n
]
];
If[OddQ[m],p[[i]]++; m--; n++];
While[m > 0 && n <= Length[s2] && s2[[m]] == s2[[n]],
p[[i]] += 2; m -= 2; n += 2;
];
If[(i + p[[i]]) > r,
c = i; r = i + p[[i]];
];
If[len < p[[i]],
len = p[[i]]; cc = i;
],
{i,2,Length[s2]}
];
s[[Quotient[cc - len + 1, 2] ;; Quotient[cc + len - 1, 2]]]
]
]
```

The post The Algorithm appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>The post Trading Market Sentiment appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>In the early days of the developing field of market sentiment analysis, the supply of machine readable content was limited to mainstream providers of financial news such as Reuters or Bloomberg. Over time this has changed with the entry of new competitors in the provision of machine readable news, including, for example, Ravenpack or more recent arrivals like Accern. Providers often seek to sell not only the raw news feed service, but also their own proprietary sentiment indicators that are claimed to provide additional insight into how individual stocks, market sectors, or the overall market are likely to react to news. There is now what appears to be a cottage industry producing white papers seeking to demonstrate the value of these services, often accompanied by some impressive pro-forma performance statistics for the accompanying strategies, which include long-only, long/short, market neutral and statistical arbitrage.

For the purpose of demonstration I intend to forego the blandishments of these services, although many are no doubt are excellent, since the reader is perhaps unlikely to have access to them. Instead, in what follows I will focus on a single news source, albeit a highly regarded one: the Wall Street Journal. This is, of course, a simplification intended for illustrative purposes only – in practice one would need to use a wide variety of news sources and perhaps subscribe to a machine readable news feed service. But similar principles and techniques can be applied to any number of news feeds or online sites.

We are going to access the Journal’s online archive, which presents daily news items in a convenient summary format, an example of which is shown below. The archive runs from the beginning of 2012 through to the current day, providing ample data for analysis. In what follows, I am going to make two important assumptions, neither of which is likely to be 100% accurate – but which will not detract too much from the validity of the research, I hope. The first assumption is that the news items shown in each daily archive were reported prior to the market open at 9:30 AM. This is likely to be true for the great majority of the stories, but there are no doubt important exceptions. Since we intend to treat the news content of each archive as antecedent to the market action during the corresponding trading session, exceptions are likely to introduce an element of look-ahead bias. The second assumption is that the archive for each day is shown in the form in which it would have appeared on the day in question. In reality, there are likely to have been revisions to some of the stories made subsequent to their initial publication. So, here too, we must allow for the possibility of look-ahead bias in the ensuing analysis.

With those caveats out of the way, let’s proceed. We are going to be using broad market data for the S&P 500 index in the analysis to follow, so the first step is to download daily price series for the index. Note that we begin with daily opening prices, since we intend to illustrate the application of news sentiment analysis with a theoretical day-trading strategy that takes positions at the start of each trading session, exiting at market close.

From there we calculate the intraday return in the index, from market open to close, as follows:

Next we turn to the task of reading the news archive and categorizing its content. Mathematica makes the importation of html pages very straightforward, and we can easily crop the raw text string to exclude page headers and footers. The approach I am going to take is to derive a sentiment indicator based on an analysis of the sentiment of each word in the daily archive. Before we can do that we must first convert the text into individuals words, stripping out standard stop-words such as “the” and “in” and converting all the text to lower case. Naturally one can take this pre-processing a great deal further, by identifying and separating out proper nouns, for example. Once the text processing stage is complete we can quickly summarize the content, for example by looking at the most common words, or by representing the entire archive in the form of a word cloud. Given that we are using the archive for the first business day of 2012, it is perhaps unsurprising that we find that “2012”, “new” and “year” feature so prominently!

The subject of sentiment analysis is a complex one and I only touch on it here. For those interested in the subject I can recommend *The Text Mining Handbook*, by Feldman and Sanger, which is a standard work on the topic. Here I am going to employ a machine learning classifier provided with Mathematica 11. It is not terribly sophisticated (or, at least, has not been developed with financial applications especially in mind), but will serve for the purposes of this article. For those unfamiliar with the functionality, the operation of the sentiment classification algorithm is straightforward enough. For instance:

We apply the algorithm to classify each word in the daily news archive and arrive at a sentiment indicator based on the proportion of words that are classified as “positive”. The sentiment reading for the archive for Jan-3, 2012, for example, turns out to be 67.4%:

We can automate the process of classifying the entire WSJ archive with just a few lines of code, producing a time series for the daily sentiment indicator, which has an average daily value of 68.5% – the WSJ crowd tends to be bullish, clearly! Note how the 60-day moving average of the indicator rises steadily over the period from 2012 through Q1 2015, then abruptly reverses direction, declining steadily thereafter – even somewhat precipitously towards the end of 2016.

As with most data series in investment research, we are less interested in the level of a variable, such as a stock price, than we are in the changes in level. So the next step is to calculate the daily percentage change in the sentiment indicator and examine the correlation with the corresponding intraday return in the S&P 500 Index. At first glance our sentiment indicator appears to have very little predictive power – the correlation between indicator changes and market returns is negligibly small overall – but we shall later see that this is not the last word.

Thus far the results appear discouraging; but as is often the case with this type of analysis we need to look more closely at the conditional distribution of returns. Specifically, we will examine the conditional distribution of S&P 500 Index returns when changes in the sentiment index are in the upper and lower quantiles of the distribution. This will enable us to isolate the impact of changes in market sentiment at times when the swings in sentiment are strongest. In the analysis below, we begin by examining the upper and lower third of the distribution of changes in sentiment:

The analysis makes clear that the distribution of S&P 500 Index returns is very different on days when the change in market sentiment is large and positive vs. large and negative. The difference is not just limited to the first moment of the conditional distribution, where the difference in the mean return is large and statistically significant, but also in the third moment. The much larger, negative skewness means that there is a greater likelihood of a large decline in the market on days in which there is a sizable drop in market sentiment, than on days in which sentiment significantly improves. In other words, the influence of market sentiment changes is manifest chiefly through the mean and skewness of the conditional distributions of market returns.

We can capitalize on these effects using a simple trading strategy in which we increase the capital allocated to a long-SPX position on days when market sentiment improves, while reducing exposure on days when market sentiment falls. We increase the allocation by a factor – designated the leverage factor – on days when the change in the sentiment indicator is in the upper 1/3 of the distribution, while reducing the allocation by 1/leveragefactor on days when the change in the sentiment indicator falls in lower 1/3 of the distribution. The allocation on other days is 100%. The analysis runs as follows:

It turns out that, using a leverage factor of 2.0, we can increase the CAGR from 10% to 21% over the period from 2012-2016 using the conditional distribution approach. This performance enhancement comes at a cost, since the annual volatility of the news sentiment strategy is 17% compared to only 12% for the long-only strategy. However, the overall net result is positive, since the risk-adjusted rate of return increases from 0.82 to 1.28.

We can explore the robustness of the result, comparing different quantile selections and leverage factors using Mathematica’s interactive Manipulate function:

We have seen that a simple market sentiment indicator can be created quite easily from publicly available news archives, using a standard machine learning sentiment classification algorithm. A market sentiment algorithm constructed using methods as straightforward as this appears to provide the capability to differentiate the conditional distribution of market returns on days when changes in market sentiment are significantly positive or negative. The differences in the higher moments of the conditional distribution appears to be as significant as the differences in the mean. In principle, we can use the insight provided by the sentiment indicator to enhance a long-only day-trading strategy, increasing leverage and allocation on days when changes to market sentiment are positive and reducing them on days when sentiment declines. The performance enhancements resulting from this approach appear to be significant.

Several caveats apply. The S&P 500 index is not tradable, of course, and it is not uncommon to find trading strategies that produce interesting theoretical results. In practice one would be obliged to implement the strategy using a tradable market proxy, such as a broad market ETF or futures contract. The strategy described here, which enters and exits positions daily, would incur substantial trading costs, that would be further exacerbated by the use of leverage.

Of course there are many other uses one can make of news data, in particular with firm-specific news and sentiment analytics, that fall outside the scope of this article. Hopefully, however, the methodology described here will provide a sign-post towards further, more practically useful research.

The post Trading Market Sentiment appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>The post Financial Data in Mathematica appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>The post Selling Volatility appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>The post High Frequency Scalping Strategies appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>- The strategy is highly profitable, with a Sharpe Ratio in excess of 9 (net of transaction costs of $14 prt)
- Performance is consistent and reliable, being based on a large number of trades (10-20 per day)
- The strategy has low, or negative correlation to the underlying equity and volatility indices
- There is no overnight risk

The attractiveness of such strategies is undeniable. So how does one go about developing them?

It is important for the reader to familiarize himself with some of the background to high frequency trading in general and scalping strategies in particular. Specifically, I would recommend reading the following blog posts:

The key to understanding HFT strategies is that execution is everything. With low frequency strategies a great deal of work goes into researching sources of alpha, often using highly sophisticated mathematical and statistical techniques to identify and separate the alpha signal from the background noise. Strategy alpha accounts for perhaps as much as 80% of the total return in a low frequency strategy, with execution making up the remaining 20%. It is not that execution is unimportant, but there are only so many basis points one can earn (or save) in a strategy with monthly turnover. By contrast, a high frequency strategy is highly dependent on trade execution, which may account for 80% or more of the total return. The algorithms that generate the strategy alpha are often very simple and may provide only the smallest of edges. However, that very small edge, scaled up over thousands of trades, is sufficient to produce a significant return. And since the risk is spread over a large number of very small time increments, the rate of return can become eye-wateringly high on a risk-adjusted basis: Sharpe Ratios of 10, or more, are commonly achieved with HFT strategies.

In many cases an HFT algorithm seeks to estimate the conditional probability of an uptick or downtick in the underlying, leaning on the bid or offer price accordingly. Provided orders can be positioned towards the front of the queue to ensure an adequate fill rate, the laws of probability will do the rest. So, in the HFT context, much effort is expended on mitigating latency and on developing techniques for establishing and maintaining priority in the limit order book. Another major concern is to monitor order book dynamics for signs that book pressure may be moving against any open orders, so that they can be cancelled in good time, avoiding adverse selection by informed traders, or a buildup of unwanted inventory.

In a high frequency scalping strategy one is typically looking to capture an average of between 1/2 to 1 tick per trade. For example, the VIX scalping strategy illustrated here averages around $23 per contract per trade, i.e. just under 1/2 a tick in the futures contract. Trade entry and exit is effected using limit orders, since there is no room to accommodate slippage in a trading system that generates less than a single tick per trade, on average. As with most HFT strategies the alpha algorithms are only moderately sophisticated, and the strategy is highly dependent on achieving an acceptable fill rate (the proportion of limit orders that are executed). The importance of achieving a high enough fill rate is clearly illustrated in the first of the two posts referenced above. So what is an acceptable fill rate for a HFT strategy?

I’m going to address the issue of fill rates by focusing on a critical subset of the problem: fills that occur at the extreme of the bar, also known as “extreme hits”. These are limit orders whose prices coincide with the highest (in the case of a sell order) or lowest (in the case of a buy order) trade price in any bar of the price series. Limit orders at prices within the interior of the bar are necessarily filled and are therefore uncontroversial. But limit orders at the extremities of the bar may or may not be filled and it is therefore these orders that are the focus of attention.

By default, most retail platform backtest simulators assume that all limit orders, including extreme hits, are filled if the underlying trades there. In other words, these systems typically assume a 100% fill rate on extreme hits. This is highly unrealistic: in many cases the high or low of a bar forms a turning point that the price series visits only fleetingly before reversing its recent trend, and does not revisit for a considerable time. The first few orders at the front of the queue will be filled, but many, perhaps the majority of, orders further down the priority order will be disappointed. If the trader is using a retail trading system rather than a HFT platform to execute his trades, his limit orders are almost always guaranteed to rest towards the back of the queue, due to the relatively high latency of his system. As a result, a great many of his limit orders – in particular, the extreme hits – will not be filled.

The consequences of missing a large number of trades due to unfilled limit orders are likely to be catastrophic for any HFT strategy. A simple test that is readily available in most backtest systems is to change the underlying assumption with regard to the fill rate on extreme hits – instead of assuming that 100% of such orders are filled, the system is able to test the outcome if limit orders are filled only if the price series subsequently exceeds the limit price. The outcome produced under this alternative scenario is typically extremely adverse, as illustrated in first blog post referenced previously.

In reality, of course, neither assumption is reasonable: it is unlikely that either 100% or 0% of a strategy’s extreme hits will be filled – the actual fill rate will likely lie somewhere between these two outcomes. And this is the critical issue: at some level of fill rate the strategy will move from profitability into unprofitability. The key to implementing a HFT scalping strategy successfully is to ensure that the execution falls on the right side of that dividing line.

One solution to the fill rate problem is to spend millions of dollars building HFT infrastructure. But for the purposes of this post let’s assume that the trader is confined to using a retail trading platform like Tradestation or Interactive Brokers. Are HFT scalping systems still feasible in such an environment? The answer, surprisingly, is a qualified yes – by using a technique that took me many years to discover.

To illustrate the method I will use the following HFT scalping system in the E-Mini S&P500 futures contract. The system trades the E-Mini futures on 3 minute bars, with an average hold time of 15 minutes. The average trade is very low – around $6, net of commissions of $8 prt. But the strategy appears to be highly profitable ,due to the large number of trades – around 50 to 60 per day, on average.

So far so good. But the critical issue is the very large number of extreme hits produced by the strategy. Take the trading activity on 10/18 as an example (see below). Of 53 trades that day, 25 (47%) were extreme hits, occurring at the high or low price of the 3-minute bar in which the trade took place.

Overall, the strategy extreme hit rate runs at 34%, which is extremely high. In reality, perhaps only 1/4 or 1/3 of these orders will actually execute – meaning that remainder, amounting to around 20% of the total number of orders, will fail. A HFT scalping strategy cannot hope to survive such an outcome. Strategy profitability will be decimated by a combination of missed, profitable trades and losses on trades that escalate after an exit order fails to execute.

So what can be done in such a situation?

One approach that will not work is to assume naively that some kind of manual oversight will be sufficient to correct the problem. Let’s say the trader runs two versions of the system side by side, one in simulation and the other in production. When a limit order executes on the simulation system, but fails to execute in production, the trader might step in, manually override the system and execute the trade by crossing the spread. In so doing the trader might prevent losses that would have occurred had the trade not been executed, or force the entry into a trade that later turns out to be profitable. Equally, however, the trader might force the exit of a trade that later turns around and moves from loss into profit, or enter a trade that turns out to be a loser. There is no way for the trader to know, ex-ante, which of those scenarios might play out. And the trader will have to face the same decision perhaps as many as twenty times a day. If the trader is really that good at picking winners and cutting losers he should scrap his trading system and trade manually!

An alternative approach would be to have the trading system handle the problem, For example, one could program the system to convert limit orders to market orders if a trade occurs at the limit price (MIT), or after x seconds after the limit price is touched. Again, however, there is no way to know in advance whether such action will produce a positive outcome, or an even worse outcome compared to leaving the limit order in place.

In reality, intervention, whether manual or automated, is unlikely to improve the trading performance of the system. What is certain, however, is that by forcing the entry and exit of trades that occur around the extreme of a price bar, the trader will incur additional costs by crossing the spread. Incurring that cost for perhaps as many as 1/3 of all trades, in a system that is producing, on average less than half a tick per trade, is certain to destroy its profitability.

For many years I assumed that the only solution to the fill rate problem was to implement scalping strategies on HFT infrastructure. One day, I found myself asking the question: what would happen if we slowed the strategy down? Specifically, suppose we took the 3-minute E-Mini strategy and ran it on 5-minute bars?

My first realization was that the relative simplicity of alpha-generation algorithms in HFT strategies is an advantage here. In a low frequency context, the complexity of the alpha extraction process mitigates its ability to generalize to other assets or time-frames. But HFT algorithms are, by and large, simple and generic: what works on 3-minute bars for the E-Mini futures might work on 5-minute bars in E-Minis, or even in SPY. For instance, if the essence of the algorithm is something as simple as: “buy when the price falls by more than x% below its y-bar moving average”, that approach might work on 3-minute, 5-minute, 60-minute, or even daily bars.

So what happens if we run the E-mini scalping system on 5-minute bars instead of 3-minute bars?

Obviously the overall profitability of the strategy is reduced, in line with the lower number of trades on this slower time-scale. But note that average trade has increased and the strategy remains very profitable overall.

More importantly, the average extreme hit rate has fallen from 34% to 22%.

Hence, not only do we get fewer, slightly more profitable trades, but a much lower proportion of them occur at the extreme of the 5-minute bars. Consequently the fill-rate issue is less critical on this time frame.

Of course, one can continue this process. What about 10-minute bars, or 30-minute bars? What one tends to find from such experiments is that there is a time frame that optimizes the trade-off between strategy profitability and fill rate dependency.

However, there is another important factor we need to elucidate. If you examine the trading record from the system you will see substantial variation in the extreme hit rate from day to day (for example, it is as high as 46% on 10/18, compared to the overall average of 22%). In fact, there are significant variations in the extreme hit rate during the course of each trading day, with rates rising during slower market intervals such as from 12 to 2pm. The important realization that eventually occurred to me is that, of course, what matters is not clock time (or “wall time” in HFT parlance) but trade time: i.e. the rate at which trades occur.

What we need to do is reconfigure our chart to show bars comprising a specified number of trades, rather than a specific number of minutes. In this scheme, we do not care whether the elapsed time in a given bar is 3-minutes, 5-minutes or any other time interval: all we require is that the bar comprises the same amount of trading activity as any other bar. During high volume periods, such as around market open or close, trade time bars will be shorter, comprising perhaps just a few seconds. During slower periods in the middle of the day, it will take much longer for the same number of trades to execute. But each bar represents the same level of trading activity, regardless of how long a period it may encompass.

How do you decide how may trades per bar you want in the chart?

As a rule of thumb, a strategy will tolerate an extreme hit rate of between 15% and 25%, depending on the daily trade rate. Suppose that in its original implementation the strategy has an unacceptably high hit rate of 50%. And let’s say for illustrative purposes that each time-bar produces an average of 1, 000 contracts. Since volatility scales approximately with the square root of time, if we want to reduce the extreme hit rate by a factor of 2, i.e. from 50% to 25%, we need to increase the average number of trades per bar by a factor of 2^2, i.e. 4. So in this illustration we would need volume bars comprising 4,000 contracts per bar. Of course, this is just a rule of thumb – in practice one would want to implement the strategy of a variety of volume bar sizes in a range from perhaps 3,000 to 6,000 contracts per bar, and evaluate the trade-off between performance and fill rate in each case.

Using this approach, we arrive at a volume bar configuration for the E-Mini scalping strategy of 20,000 contracts per bar. On this “time”-frame, trading activity is reduced to around 20-25 trades per day, but with higher win rate and average trade size. More importantly, the extreme hit rate runs at a much lower average of 22%, which means that the trader has to worry about maybe only 4 or 5 trades per day that occur at the extreme of the volume bar. In this scenario manual intervention is likely to have a much less deleterious effect on trading performance and the strategy is probably viable, even on a retail trading platform.

(Note: the results below summarize the strategy performance only over the last six months, the time period for which volume bars are available).

We have seen that is it feasible in principle to implement a HFT scalping strategy on a retail platform by slowing it down, i.e. by implementing the strategy on bars of lower frequency. The simplicity of many HFT alpha generation algorithms often makes them robust to generalization across time frames (and sometimes even across assets). An even better approach is to use volume bars, or trade-time, to implement the strategy. You can estimate the appropriate bar size using the square root of time rule to adjust the bar volume to produce the requisite fill rate. An extreme hit rate if up to 25% may be acceptable, depending on the daily trade rate, although a hit rate in the range of 10% to 15% would typically be ideal.

Finally, a word about data. While necessary compromises can be made with regard to the trading platform and connectivity, the same is not true for market data, which must be of the highest quality, both in terms of timeliness and completeness. The reason is self evident, especially if one is attempting to implement a strategy in trade time, where the integrity and latency of market data is crucial. In this context, using the data feed from, say, Interactive Brokers, for example, simply will not do – data delivered in 500ms packets in entirely unsuited to the task. The trader must seek to use the highest available market data feed that he can reasonably afford.

That caveat aside, one can conclude that it is certainly feasible to implement high volume scalping strategies, even on a retail trading platform, providing sufficient care is taken with the modeling and implementation of the system.

The post High Frequency Scalping Strategies appeared first on QUANTITATIVE RESEARCH AND TRADING.

]]>