Signal Processing Archives - QUANTITATIVE RESEARCH AND TRADING

February 2, 2021February 2, 2021

Alpha Spectral Analysis

One of the questions of interest is the optimal sampling frequency to use for extracting the alpha signal from an alpha generation function. We can use Fourier transforms to help identify the cyclical behavior of the strategy alpha and hence determine the best time-frames for sampling and trading. Typically, these spectral analysis techniques will highlight several different cycle lengths where the alpha signal is strongest.

The spectral density of the combined alpha signals across twelve pairs of stocks is shown in Fig. 1 below. It is clear that the strongest signals occur in the shorter frequencies with cycles of up to several hundred seconds. Focusing on the density within
this time frame, we can identify in Fig. 2 several frequency cycles where the alpha signal appears strongest. These are around 50, 80, 160, 190, and 230 seconds. The cycle with the strongest signal appears to be around 228 secs, as illustrated in Fig. 3. The signals at cycles of 54 & 80 (Fig. 4), and 158 & 185/195 (Fig. 5) secs appear to be of approximately equal strength.
There is some variation in the individual pattern for of the power spectra for each pair, but the findings are broadly comparable, and indicate that strategies should be designed for sampling frequencies at around these time intervals.

Fig. 1 Alpha Power Spectrum

Fig.2

Fig. 3

Fig. 4

Fig. 5

PRINCIPAL COMPONENTS ANALYSIS OF ALPHA POWER SPECTRUM
If we look at the correlation surface of the power spectra of the twelve pairs some clear patterns emerge (see Fig 6):

Fig. 6

Focusing on the off-diagonal elements, it is clear that the power spectrum of each pair is perfectly correlated with the power spectrum of its conjugate. So, for instance the power spectrum of the Stock1-Stock3 pair is exactly correlated with the spectrum for its converse, Stock3-Stock1.

But it is also clear that there are many other significant correlations between non-conjugate pairs. For example, the correlation between the power spectra for Stock1-Stock2 vs Stock2-Stock3 is 0.72, while the correlation of the power spectra of Stock1-Stock2 and Stock2-Stock4 is 0.69.

We can further analyze the alpha power spectrum using PCA to expose the underlying factor structure. As shown in Fig. 7, the first two principal components account for around 87% of the variance in the alpha power spectrum, and the first four components account for over 98% of the total variation.

Fig. 7

Stock3 dominates PC-1 with loadings of 0.52 for Stock3-Stock4, 0.64 for Stock3-Stock2, 0.29 for Stock1-Stock3 and 0.26 for Stock4-Stock3. Stock3 is also highly influential in PC-2 with loadings of -0.64 for Stock3-Stock4 and 0.67 for Stock3-Stock2 and again in PC-3 with a loading of -0.60 for Stock3-Stock1. Stock4 plays a major role in the makeup of PC-3, with the highest loading of 0.74 for Stock4-Stock2.

Fig. 8 PCA Analysis of Power Spectra

April 2, 2019

Alpha Extraction and Trading Under Different Market Regimes

Market Noise and Alpha Signals

One of the perennial problems in designing trading systems is noise in the data, which can often drown out an alpha signal. This is turn creates difficulties for a trading system that relies on reading the signal, resulting in greater uncertainty about the trading outcome (i.e. greater volatility in system performance). According to academic research, a great deal of market noise is caused by trading itself. There is apparently not much that can be done about that problem: sure, you can trade after hours or overnight, but the benefit of lower signal contamination from noise traders is offset by the disadvantage of poor liquidity. Hence the thrust of most of the analysis in this area lies in the direction of trying to amplify the signal, often using techniques borrowed from signal processing and related engineering disciplines.

There is, however, one trick that I wanted to share with readers that is worth considering. It allows you to trade during normal market hours, when liquidity is greatest, but at the same time limits the impact of market noise.

Quantifying Market Noise

How do you measure market noise? One simple approach is to start by measuring market volatility, making the not-unreasonable assumption that higher levels of volatility are associated with greater amounts of random movement (i.e noise). Conversely, when markets are relatively calm, a greater proportion of the variation is caused by alpha factors. During the latter periods, there is a greater information content in market data – the signal:noise ratio is larger and hence the alpha signal can be quantified and captured more accurately.

For a market like the E-Mini futures, the variation in daily volatility is considerable, as illustrated in the chart below. The median daily volatility is 1.2%, while the maximum value (in 2008) was 14.7%!

The extremely long tail of the distribution stands out clearly in the following histogram plot.

Obviously there are times when the noise in the process is going to drown out almost any alpha signal. What if we could avoid such periods?

Noise Reduction and Model Fitting

Let’s divide our data into two subsets of equal size, comprising days on which volatility was lower, or higher, than the median value. Then let’s go ahead and use our alpha signal(s) to fit a trading model, using only data drawn from the lower volatility segment.

This is actually a little tricky to achieve in practice: most software packages for time series analysis or charting are geared towards data occurring at equally spaced points in time. One useful trick here is to replace the actual date and time values of the observations with sequential date and time values, in order to fool the software into accepting the data, since there are no longer any gaps in the timestamps. Of course, the dates on our time series plot or chart will be incorrect. But that doesn’t matter: as long as we know what the correct timestamps are.

An example of such a system is illustrated below. The model was fitted to 3-Min bar data in EMini futures, but only on days with market volatility below the median value, in the period from 2004 to 2015. The strategy equity curve is exceptionally smooth, as might be expected, and the performance characteristics of the strategy are highly attractive, with a 27% annual rate of return, profit factor of 1.58 and Sharpe Ratio approaching double-digits.

Dealing with the Noisy Trading Days

Let’s say you have developed a trading system that works well on quiet days. What next? There are a couple of ways to go:

(i) Deploy the model only on quiet trading days; stay out of the market on volatile days; or

(ii) Develop a separate trading system to handle volatile market conditions.

Which approach is better? It is likely that the system you develop for trading quiet days will outperform any system you manage to develop for volatile market conditions. So, arguably, you should simply trade your best model when volatility is muted and avoid trading at other times. Any other solution may reduce the overall risk-adjusted return. But that isn’t guaranteed to be the case – and, in fact, I will give an example of systems that, when combined, will in practice yield a higher information ratio than any of the component systems.

Deploying the Trading Systems

The astute reader is likely to have noticed that I have “cheated” by using forward information in the model development process. In building a trading system based only on data drawn from low-volatility days, I have assumed that I can somehow know in advance whether the market is going to be volatile or not, on any given day. Of course, I don’t know for sure whether the upcoming session is going to be volatile and hence whether to deploy my trading system, or stand aside. So is this just a purely theoretical exercise? No, it’s not, for the following reasons.

The first reason is that, unlike the underlying asset market, the market volatility process is, by comparison, highly predictable. This is due to a phenomenon known as “long memory”, i.e. very slow decay in the serial autocorrelations of the volatility process. What that means is that the history of the volatility process contains useful information about its likely future behavior. [There are several posts on this topic in this blog – just search for “long memory”]. So, in principle, one can develop an effective system to forecast market volatility in advance and hence make an informed decision about whether or not to deploy a specific model.

But let’s say you are unpersuaded by this argument and take the view that market volatility is intrinsically unpredictable. Does that make this approach impractical? Not at all. You have a couple of options:

You can test the model built for quiet days on all the market data, including volatile days. It may perform acceptably well across both market regimes.

For example, here are the results of a backtest of the model described above on all the market data, including volatile and quiet periods, from 2004-2015. While the performance characteristics are not quite as good, overall the strategy remains very attractive.

Another approach is to develop a second model for volatile days and deploy both low- and high-volatility regime models simultaneously. The trading systems will interact (if you allow them to) in a highly nonlinear and unpredictable way. It might turn out badly – but on the other hand, it might not! Here, for instance, is the result of combining low- and high-volatility models simultaneously for the Emini futures and running them in parallel. The result is an improvement (relative to the low volatility model alone), not only in the annual rate of return (21% vs 17.8%), but also in the risk-adjusted performance, profit factor and average trade.

CONCLUSION

Separating the data into multiple subsets representing different market regimes allows the system developer to amplify the signal:noise ratio, increasing the effectiveness of his alpha factors. Potentially, this allows important features of the underlying market dynamics to be captured in the model more easily, which can lead to improved trading performance.

Models developed for different market regimes can be tested across all market conditions and deployed on an everyday basis if shown to be sufficiently robust. Alternatively, a meta-strategy can be developed to forecast the market regime and select the appropriate trading system accordingly.

Finally, it is possible to achieve acceptable, or even very good results, by deploying several different models simultaneously and allowing them to interact, as the market moves from regime to regime.

January 15, 2019

Signal Processing and Sample Frequency

The Importance of Sample Frequency

Too often we apply a default time horizon for our trading, whether it below (daily, weekly) or higher (hourly, 5 minute) frequency. Sometimes the choice is dictated by practical considerations, such as a desire to avoid overnight risk, or the (lack of0 availability of low-latency execution platform.

But there is an alternative approach to the trade frequency decision that often yields superior results in terms of trading performance. The methodology derives from signal processing and the idea essentially is to use Fourier transforms to help identify the cyclical behavior of the strategy alpha and hence determine the best time-frames for sampling and trading. I wrote about this is a previous blog post, in which I described how to use principal components analysis to investigate the factors driving the returns in various pairs trading strategies. Here I want to take a simpler approach, in which we use Fourier analysis to select suitable sample frequencies. The idea is simply to select sample frequencies where the signal strength appears strongest, in the hope that it will lead to superior performance characteristics in what strategy we are trying to develop.

Signal Decomposition for S&P500 eMini Futures

Let’s take as an example the S&P 500 emini futures contract. The chart below shows the continuous ES futures contract plotted at 1-minute intervals from 1998. At the bottom of the chart I have represented the signal analysis as a bar chart (in blue), with each bar representing the amplitude at each frequency. The white dots on the chart identify frequencies that are spaced 10 minutes apart. It is immediately evident that local maxima in the spectrum occur around 40 mins, 60 mins and 120 mins. So a starting point for our strategy research might be to look at emini data sampled at these frequencies. Incidentally, it is worth pointing out that I have restricted the session times to 7AM – 4PM EST, which is where the bulk of the daily volume and liquidity tend to occur. You may get different results if you include data from the Globex session.

This is all very intuitive and unsurprising: the clearest signals occur at frequencies that most traders typically tend to trade, using hourly data, for example. Any strategy developer is already quite likely to consider these and other common frequencies as part of their regular research process. There are many instances of successful trading strategies built on emini data sampled at 60 minute intervals.

Signal Decomposition for US Bond Futures

Let’s look at a rather more interesting example: US (30 year) Bond futures. Unlike the emini contract, the spectral analysis of the US futures contract indicates that the strongest signal by far occurs at a frequency of around 47 minutes. This is decidedly an unintuitive outcome – I can’t think of any reason why such a strong signal should appear at this cycle length, but, statistically it does.

Does it work? Readers can judge for themselves: below is an example of an equity curve for a strategy on US futures sampled at 47 minute frequency over the period from 2002. The strategy has performed very consistently, producing around $25,000 per contract per year, after commissions and slippage.

Conclusion

While I have had similar success with products as diverse as Corn and VIX futures, the frequency domain approach is by no means a panacea: there are plenty of examples where I have been unable to construct profitable strategies for data sampled at the frequencies with very strong signals. Conversely, I have developed successful strategies using data at frequencies that hardly registered at all on the spectrum, but which I selected for other reasons. Nonetheless, spectral analysis (and signal processing in general) can be recommended as a useful tool in the arsenal of any quantitative analyst.