Using the Dynamic Mode Decomposition (DMD) to Rotate Long-Short Exposure Between Stock Market Sectors

Co-Author: Eric Kammers

Part 1 – Theoretical Background

The Dynamic Mode Decomposition (DMD) was originally developed for its application in fluid dynamics where it could decompose complex flows into simpler low-rank spatio-temporal features. The power of this method lies in the fact that it does not depend on any principle equations of the dynamic system it is analyzing and is thus equation-free [1]. Also, unlike other low-rank reconstruction algorithms like the Singular Value Decomposition (SVD), the DMD can be used to make short-term future state predictions.

The algorithm is implemented as follows [2].

1. We begin with a {N}x{M} matrix, {\mathbf{X}}, containing data collected from {N} sources over {M} evenly spaced time periods, from the system of interest.

2. From this matrix two sub-matrices are constructed, {\mathbf{X}_1^{M-1}} and {\mathbf{X}_2^M}, which are defined below.

\displaystyle \mathbf{X}_j^k = \begin{bmatrix} \mathbf{x}(t_j) & \mathbf{x}(t_{j+1}) & ... & \mathbf{x}(t_k) \end{bmatrix}
\displaystyle \mathbf{X}_1^{M-1} = \begin{bmatrix} \mathbf{x}_1 & \mathbf{x}_2 & ... & \mathbf{x}_{M-1} \end{bmatrix}
\displaystyle \mathbf{X}_2^{M} = \begin{bmatrix} \mathbf{x}_2 & \mathbf{x}_3 & ... & \mathbf{x}_{M} \end{bmatrix}

We can consider a Koopman operator {\mathbf{A}} such that {\mathbf{x}_{j+1} = \mathbf{Ax}_j} and rewrite {\mathbf{X}_1^{M-1}} as

\displaystyle \mathbf{X}_1^{M-1} = \begin{bmatrix} \mathbf{x}_1 & \mathbf{A}\mathbf{x}_1 & ... & \mathbf{A}^{M-2}\mathbf{x}_1 \end{bmatrix}

whose columns now are elements in a Krylov space.

3. The SVD decomposition of {\mathbf{X}_1^{M-1}} is computed.

\displaystyle \mathbf{X}_1^{M-1} = \mathbf{U \Sigma V^*}

Then based on the variance captured by the singular values and the application of the algorithm, the number of desired reconstructions ranks can be chosen.

4. The matrix {\mathbf{A}} is constructed such that it is the best mapping between the two sub-matrices.

\displaystyle \mathbf{A}\mathbf{X}_1^{M-1} \approx \mathbf{X}_2^M

{\mathbf{A}} can be approximated with {\mathbf{\tilde{A}}} from evaluating the expression

\displaystyle \mathbf{\tilde{A}} = \mathbf{U^*X_2^M V\Sigma^{-1}}

where {\mathbf{U}}, {\mathbf{\Sigma}}, and {\mathbf{V}} are the truncated matrices from the SVD reduction of {\mathbf{X}_1^{M-1}}. The eigenvalue problem associated with {\mathbf{\tilde{A}}} is

\displaystyle \mathbf{\tilde{A}}\mathbf{y}_k = \mu_k \mathbf{y}_k \qquad k = 1,2,..,K

where {K} is the rank of approximation that was chosen previously. The eigenvalues {\mu_k} contain information on the time dynamics of our system and the eigenvectors can be used to construct the DMD modes.

\displaystyle \mathbf{\psi}_k = \mathbf{Uy}_k

5. The approximated solution for all future times, {\mathbf{x}_{DMD}(t)}, can now be written as

\displaystyle \mathbf{x}_{DMD}(t) = \sum_{k=1}^K b_k(0) \mathbf{\psi}_k(\mathbf{x}) \exp{(\omega_k t)} = \mathbf{\Psi} \text{diag}(\exp{(\omega t)}) \mathbf{b}

where {\omega_k = \ln(\mu_k)/\Delta t}, {b_k(0)} is the initial amplitude of each mode, {\mathbf{\Psi}} is the matrix whose columns are eigenvectors {\mathbf{\psi}_k}, and {\mathbf{b}} is the vector of coefficients. Finally, all that needs to be computed is the initial coefficient values {b_k(0)} which can be found by looking at time zero and solving for {\mathbf{b}} via a pseudo-inverse in the equation

\displaystyle \mathbf{x_0} = \mathbf{\Psi b}

To summarize the algorithm, we will “train” a matrix {\mathbf{\tilde{A}}} on a subset of the data whose eigenvalues and eigenvectors contain necessary information to make future state predictions for a given time horizon.

Part 2 – Basic Demonstration

We begin with a basic example to demonstrate how to use the pyDMD package. First, we construct a matrix \mathbf{X} where each row is a snapshot in time and each column can be thought of as a different location in our system being sampled.

\displaystyle \mathbf{X} = \begin{bmatrix} -2 & 6 & 1 & 1 & -1 \\ -1 & 5 & 1 & 2 & -1 \\ 0 & 4 & 2 & 1 & -1 \\ 1 & 3 & 2 & 2 & -1 \\ 2 & 2 & 3 & 1 & -1 \\ 3 & 1 & 3 & 2 & -1 \\ \end{bmatrix}

Now we will attempt to predict the predict the 6th row using a future state prediction from the DMD fitted on the first 5 rows.

import numpy as np
from pydmd import DMD
df = np.array([[-2,6,1,1,-1],
               [-1,5,1,2,-1],
               [0,4,2,1,-1],
               [1,3,2,2,-1],
               [2,2,3,1,-1],
               [3,1,3,2,-1]])

dmd = DMD(svd_rank = 2) # Specify desired truncation
train = df[:5,:]
dmd.fit(train.T) # Fit the model on the first 5 rows
dmd.dmd_time['tend'] *= (1+1/6) # Predict one additional time step
recon = dmd.reconstructed_data.real.T # Make prediction

print('Actual :',df[5,:])
print('Predicted :',recon[5,:])

Two SVD ranks were used for the reconstruction and the result is pleasantly accurate for how easily it was implemented.

\displaystyle \mathbf{x_{True}}(6) = \begin{bmatrix} 3 & 1 & 3 & 2 & -1 \end{bmatrix}

\displaystyle \mathbf{x_{DMD}}(6) = \begin{bmatrix} 2.8 & 0.8 & 2.6 & 2.0 & -0.9 \end{bmatrix}

Part 3 – Sector Rotation Strategy

We will now attempt to model the stock market as a dynamic system broken down by sectors and use the DMD to predict which sectors to be long and short in over time. This is commonly known as a sector rotation strategy. To ensure that we have adequate historical data we will use 9 sector ETFs: XLY, XLP, XLE, XLF, XLV, XLI, XLB, XLK, and XLU from 2000-2019 and rebalance monthly. The strategy is implemented as follows:

  1. Fit a DMD model using the last N months of monthly returns. The SVD rank reconstruction number can be chosen as desired.
  2. Use the DMD model to predict the next month’s snapshot which are the returns of each ETF.
  3. Construct the portfolio by taking long positions in the top 5 ETFs and short positions in the bottom 4 ETFs. Thus, we are remaining very close to market neutral.
  4. Continue this over time by refitting the model monthly and making a new prediction for the next month.

Though the results are quite sensitive to changes in the model parameters, some of the best parameters achieve Sharpe ratios superior to the long only portfolio while remaining roughly market neutral which is very encouraging and warrants further exploration with a proper, robust backtest procedure.

Figure_1-1

The code and functions used to produce this plot can found here. There are also many additional features of the pyDMD package that we did not explore that could potentially improve the results. If you have any questions, feel free to reach out by email at coltonsmith321@gmail.com

References

[1] N. Kutz, S. Brunton, B. Brunton, and J. Proctor, Dynamic Mode Decomposition: Data-Driven Modeling of Complex Systems. 2016.

[2] Mann, Jordan & Nathan Kutz, J. Dynamic Mode Decomposition for Financial Trading Strategies. Quantitative Finance. 16. 10.1080/14697688.2016.1170194. 2015.

3 thoughts on “Using the Dynamic Mode Decomposition (DMD) to Rotate Long-Short Exposure Between Stock Market Sectors

    1. Hey Tom, the plot in the post is using a 14 month rolling lookback and no SVD rank truncation. The sweet spot for the lookback seems to be 12-24 months when rebalancing monthly.

      Like

Leave a comment