Systematic Trading | Using Autoencoder for Momentum Trading

In a previous post, we discussed the basic nature of various technical indicators and noted some observations. One of the ideas was: at a basic level, most indicators captures the concept of momentum vs mean-reversion. Most do so in the price returns space, but some in a non-linear transformation of the returns space, like signed returns or time since new high/ low. We presented the idea of a PCA approach to extract the momentum signals embedded in these indicators. From there to a trading model, the steps will be to collate this momentum signal (1st PCA component or higher if required) along with other input variables (like returns volatility and/ or other fundamental indicators) to train a separate regression/ classification model (like a random forest or a deep NN).

One of the issues with using simple PCA is that it is linear and hence may not be appropriate to summarize different measures captured across all these indicators. Here we discuss the next logical improvement - a nonlinear dimensional reduction approach using autoencoder.

As discussed here, the new Keras R interface has now made it very easy to develop deep learning models in R using the TensorFlow framework. Here we use this interface to train an autoencoder to fit the same set of technical indicators on NSE Nifty 50 Index as before. The steps involved are relatively straight-forward. First we generate and standardize the inputs (technical indicators levels). Then we build the computation graph.

To do so, first we define the encoding layers (2 hidden layers, the latent coded unit size is 3, to match the first 3 components of the PCA we use for comparison), and two different decoding layers. The two different decoding layers are to  enable us to train the auto-encoder as well as compute only decoding independently.

Next we combine these layers to create the computational graph. One for the encoder only, another for the decoder, and a third one for the end-to-end autoencoder, that we will actually train.

The rest of it is standard. We define a loss function to map the input to the output, measuring mean squared losses, and train the model. The training is done on data till 2013, and test set is since 2014 till present. Once the training is done, we can use the encoder and decoder separately to generate a dimensionality reduction of the input space and vice-versa.

The output of the dimensionality reduction is compared with the PCA. As it appears from the correlations, the PCAs are almost one-to-one mapped to the three latent dimensions in the hidden layer generating the encoding. So the encoded layers are orthogonal in our case, although this need not be true always.

	V1	V2	V3
PC1	1	-0.3	0.2
PC2	0.1	-0.2	0.8
PC3	-0.2	-0.9	0.5

The scatter plot below captures the same, but also highlights the some non-linearity, especially the first component of PCA vs the first latent dimension from the autoencoder.

From here the next step is obvious, replace the PCA factors inputs in the momentum trading model in the first paragraph with these latent dimensions from the autoencoder and re-evaluate. This will capture a richer set of inputs that can handle non-linearity and hopefully performs better than linear PCA. Here are some results what other reported (opens PDF). Here are some more (opens PDF) on the using autoencoder for cross-sectional momentum trading. The entire code is available here.

Off Topic| Wide and Deep Learning in R

R is an excellent environment for quick and dirty data science. I am a R user and obviously a bit biased, but between Python and R, R has always had the edge for data visualization and quick hypothesis testing. If you do not find the latest and greatest methods of bleeding edge analytics somewhere available already within the strong R package ecosystem, it is more or less safe to assume it does not exist anywhere else either. And forget Python, the IDE from R Studio is perhaps the best IDE across any development platform (although the Visual Studio perhaps has a better debugging interface). But one area where R has been weak is in machine learning, especially in the deep learning area. And with the explosion of interest (and fad?) in deep learning, this has become quite a glaring gap.

But hopefully not anymore. We already have Google's TensorFlow available in R for a while. But to be honest, it did not have much feel of R in it and looked like a deprecated version of the Python release. However, very recently the R Studio folks released a R support for the excellent Keras high level API, with back-end of TensorFlow. This feels like R (with some quirk like in memory modification of objects) and works like a charm. Although it runs on top of a local python platform, the package exposes pretty much all the functionalities Keras support.

You will already find a list of examples in their site here. Here is to add a basic example of how to set up a wide and deep learning network. This involves creating two separate learning network. The wide one has only one layer (effectively a logistic regression of sort). The deep network is created separately. The output of two is combined (concatenated) in a final decision layer (this is slightly different architecture than the TensorFlow example). This is run on the usual Census Income data-set. The error rate for this set up is around 16 - comparable to other methods officially reported.