The idea is to trade spreads on commodities, which exhibit at first glance:

1. Trending behavior (Trend)

2. Consistent drifts over time (Carry)

3. Rapid mean reversion when the the spread deviates substantially. (Mean Reversion)

4. Steepening of the contango curve as front contract gets closer to expiration, in commodities which are expensive to keep stock on (Carry)

5. Seasonal effects (Seasonality)

6. Rollover effects. Institutions rolling positions around first notice day (for deliverable commodities), or expiration (for financially settled commodities), which move prices.

Calendar spread strategies allow traders with smaller accounts to afford large contracts such as gasoline, corresponding to 42,000 gallons. We are going long and short different expiration contracts of the same market. There are hundreds of calendar spreads which can be accessed in the futures markets. When trading spreads you de-couple from the movement of the underlying market, hence building a portfolio of spreads would benefit diversification.

One great resource to learn spreads is this futures spreads course from CME.

There are also some great tools around for commodity spreads, such as SeasonAlgo. In fact, I got obsessed with the idea of trading spreads since I saw SeasonAlgo a few years ago but wasn’t able to develop strategies before because my coding skills were not good enough. Few years later… it’s time to take another look!

As usual, the rabbit hole goes very deep in commodities trading, so in this article we’ll focus on #2: Finding spreads which exhibit consistent drift throughout the year (not only a few weeks), and trading it. I tried #1 (trend following) but so far no joy because the behavior of the spreads in commodities is dominated by carry drift. Probably it’s just better in most cases to simply hold some of the spreads instead of trading them based on momentum or historical seasonal performance. Picking a given spread to trade over a given date range within the year compounds the risk of over-fitting, so I’ll try to keep it simple to avoid fooling myself.

Calendar Spread Sources of Return - Research

One thing that was frustrating before was that my code to research on the spreads ran very slowly and consumed all my computer memory. But recently I discovered a technology called DuckDB that runs fast, uses all the processors and manages memory gracefully. I just had to translate the logic to SQL and we’re good to go. ChatGPT and Claude tools make this easy now.

The data arrives from CSI Data (my futures data provider) in 1 CSV file for each individual futures contract (>10K historical contracts in total). An ETL code in R reads the CSVs, does some quality checks, and writes the data into the duckdb database. Thereafter we run 2 SQL scripts in duckdb. One to process the individual futures contracts into back-adjusted time series (one time series for each market), and the other one to compute all calendar spreads, from contiguous contracts to contracts 1 year apart, within the same market. This is how it looks more or less:

Research Stack

The database has 5 tables. One for individual contract data, one for backadjusted data, one for calendar spread data, one for metadata and one with the list of spreads to trade, based on research.

Database ER diagram

Here is a section of the SQL code which generates the calendar spreads:

create or replace table fut_calendar as 
with calendar AS (
  select 
      f.symbol,
      f.date,
      substr(f.delivery,1,4) as f_year,
      datediff('day', f.date, f.expirydate) as f_dte,
      f.delivery as f_delivery,f.expirydate as f_expirydate,f.open as f_open,f.close as f_close,
      f.oi as f_oi,f.volume as f_volume,
        b.delivery as b_delivery,b.expirydate as b_expirydate,b.open as b_open,b.close as b_close,
        b.oi as b_oi, b.volume as b_volume,
        substr(f.delivery,5,1)  || '_' || substr(b.delivery,5,1) as nm 
    from fut f
        inner join(
            select symbol,date,delivery,expirydate,open,close,oi,volume
            from fut
        ) b on f.symbol=b.symbol and f.date=b.date 
          and datediff('day',f.expirydate,b.expirydate)>20
          and datediff('day',f.expirydate,b.expirydate)<370 -- Max 1 year btw
    where f.symbol in (<put your list of markets here>)
)

select * from calendar;

In my actual implementation (not shown) I further stitch together, “backadjust” these calendar spreads, to get a single time series for each contract and month code combination. Thereafter the SQL code also does a simplified PnL calculation of holding the spread when the volume of the contracts of both legs is beyond a threshold of, say, 1000 contracts.

Let’s pick Corn as an example, so that you can cross check against SeasonAlgo (Analytics for Corn is available in the free subscription tier).

This is a matrix of the PnL of holding different corn spreads from 1994 to 2014, where volume is acceptable (>1000 contracts) on both legs. Let’s reserve the last 10 years for validation.

The month codes are listed here: CME contract month codes. Here is the table for easier reference:

Notice the consistent drift in some spreads? Probably we can simply open a spread position when the volume is sufficient, and hold it until first notice day, or roll to next year’s contracts (if sufficient volume available).

Let’s find an explanation for the observed behavior, based on literature and intuition. Here is the crop calendar for USA, which we can use to define the spreads for different agricultural commodities:

Another nice resource is RJO Futures website.

Notice the U_Z, U_H, U_K, U_N spreads (September (front) vs December, March, May, July) have a clear downward drift.

According to the agricultural calendar, September is the start of the harvest season for corn. April-May are next year’s planting season. I will pick the U_K spread, to capture somewhat the “old crop vs new crop” trade.

Searching the literature for common Corn spreads, we find that U_K is not such a commonly traded spread. But I believe it makes sense to trade it: it captures the price differences between the old crop and the new crop, due to storage costs, seasonal demand, and weather uncertainties. My theory: as we move forward towards expiration of the front contract, these uncertainties, which are prices in the spread, reduce, causing a part of the negative carry drift we observe.

Let’s also pick the Z_N spread, which is a more popular spread trade, also to capture differences in demand and supply expectations for old and new crop seasons.

This is the performance of holding 1 spread contract per 100K USD portfolio size, for U_K and Z_N, shorting the spreads then volume in both legs is >1000 contracts. Assuming 3 ticks slippage on each leg to be conservative.

The spread drift remains roughly consistent through the last 10 years (our out of sample period). We are onto something here….

Let’s confirm the observations using SeasonAlgo free tier:

Yep… looks good.

Risk Assessment

Now consider the risks:

• You see in courses, articles and blogs that spreads are less risky than outright futures. I do not believe that to be the case: when you normalize by dollar volatility, the spreads seem more risky (higher tail risk).

• Higher kurtosis (fatter tails on the return distribution). Everything looks nice and pretty, then one day ….kaboom… a huge sudden move. Need to reduce risk by trading many spreads with small positions.

• Underlying mechanics of the market (characteristics of the convenience yield, etc.) can evolve over time so the chosen spread list needs to be reviewed periodically.

• Spreads around season changes, like the infamous Nat Gas “Widow Maker” March vs April (H_J) spread are more unstable. I would prefer to pick spreads across middle of seasons like middle of summer vs middle of winter (July vs December, for example).

• Contract prices closer to expiration get more sensitive to current events. We may reduce risk by picking the front contract with >X days to expiration (liquidity fizzles away though).

• Low volume markets can get poor executions with market orders. Consider using limit orders or making an execution algo. Unfortunately adaptive algo orders are not available for futures spreads in Interactive Brokers.

A nasty example: Look at Wheat spreads at the onset of the Ukraine war (a war between two major wheat producers):

KA-BOOM. The spread blows up in your face. It recovers super-fast thereafter however (hopefully you didn’t panic close the position when bleeding).

As mentioned above, we would expect deeper drawdown in the spreads closer to expiration. N_Z (July vs December) should get a bigger drawdown than U_H (September vs March) based on our risk assumptions. Is this the case?

Yes that’s the case. BUT the spread closer to expiration (N_Z) also recovers faster. Interesting, innit?

Portfolio Performance

I repeated the same spread selection exercise using 1994 to 2014 data for all liquid commodities (US and International) and built a register of 0-2 spreads exhibiting drift for each market, with an explanation attempt for each spread. If I have to choose between 2 equally good spreads, I pick the one more commonly used by large producers and speculators, as per the literature.

Now, how to size the positions? In this example, I will size by ATR of the back-adjusted series for the outright futures market:

We can risk, for example, 1% of the account if the spread value moves 1 full ATR of the outright futures price. That would correspond to a leverage factor of 0.01 in the above formula.

This is the out of sample portfolio performance, including costs:

This simulation has no stop losses; no seasonality / month / week preferences; no trend validation. Simple carry so far. Definitely deserves deeper research.

Due to the fast mean reverting nature of the spreads, probably using stop losses would make the performance worse. I leave it to the reader to experiment with stop losses. For intraday stops, you would need intraday data to test it properly.

Diversification Analysis

The calendar spread portfolio performance should be uncorrelated to a trend following strategy. Let’s check:

Rolling correlations are around 0. This strategy should be a powerful diversifier. Let’s make a “Naive” 1/N allocation to both Trend Following and Calendar Spreads portfolio (50/50, with both strategies scaled to 15% annualized volatility). This is how it looks:

Combining the strategies yields a significant performance improvement. So yes, Trend + Carry is a winning combination.

Mean reversion strategy?

Given the nature of the spreads to suffer large spikes with subsequent fast recoveries, it would be great to test a potential mean reversion strategy using limit orders. This would need a different tech stack as higher frequency data is required to determine the spread’s daily high and low values. With daily data we can know the spread open and close price since both legs have the same opening and closing times, but we cannot know the daily spread high and low. With daily data we can evaluate a mean reversion on open and close data only. This research is in my to-do list.

Execution Considerations

Interactive Brokers does not provide adaptive algo functionality for futures calendar spread orders. It also does not provide calendar spread orders yet for the newly added Palm Oil (FCPO.MY) market.

Exchanges list dedicated spread contracts, which enters both legs simultaneously. The spread contract tick size is smaller than the outright futures contract tick size in some markets. Margin requirements for spreads are also lower than outright futures. See below example for Corn:

This makes it possible to trade a portfolio of spreads without consuming all the margin. But beware!…. Gotta trade small.

Conclusions

Commodity futures calendar spreads are a very interesting play.

• Commodity calendar spreads can exhibit a drift.

• The spread behavior is uncorrelated to the outright futures market.

• We can assemble a portfolio of spreads and trade them infrequently. Since correlation is small between spreads, risk adjusted returns would be decent.

• Trading a portfolio with many spreads at small size helps reduce the tail risk, like the wheat example shown.

• Calendar Spreads where front leg is closer to expiry are more exposed to event risks (news, weather issues, etc.).

• Calendar Spreads can experience large sudden losses. High kurtosis / tail risk.

• There is potential in mean reversion strategies on a portfolio of calendar spreads. To be researched further.

• Margin requirements for calendar spreads is lower than for outright futures, which allows assembling such portfolio without consuming all the account margin.

• Tech stack - wise, duckdb enables the fast computation required to research calendar spreads and keeps things tidy in database.

Disclaimer

This post is documentation of research by the author and does not constitute financial advice. Consider it for educational purposes only. Do your own homework.

Trend following is recognized as one of the most robust trading strategies. Meaning you know it will make money in the long run. But with robustness comes pain and long drawdowns. One of the reasons cited for why this strategy works is precisely because it’s painful to trade.

Institutional allocators typically struggle to justify any meaningful allocation to this strategy beyond 5%, even though it is well known that >30% allocation against a 60/40 portfolio is optimal.

The solution to the pain is to trade more uncorrelated strategies. The more the merrier. There’s a catch though… how much capital you need to do this (?) We’ll explore this aspect.

There’s a few schools of thought in the trend following arena, which can be grouped around 2 main philosophies:

• Classic: Entry and trailing stop, or channels. Long/short. Same rules long & short. Take every trade. Size position at entry, with no adjustment thereafter.

• Modern: Dynamically adjust position sizes for changing volatility and correlation of each market; and portfolio. Scale positions based on momentum scores. Volatility target portfolio.

The classic one is a more divergent type strategy with positive skew. The other one is more neutral skew as it dynamically reduces sizes in big winning positions.

In addition, some traders advocate to trade every market they can get their hands on. Others prefer to trade a fixed subset of markets. There are successful traders in all camps, which shows that there are lots of different ways to go about trend following.

It has been mentioned by several traders historically that you need >1MM USD to trade a diversified trend strategy with futures. We need to explore this again in 2024, as there is now a wider selection of smaller futures contracts that retail traders can access (CME micros / minis, and a few markets in European and Asian exchanges). The challenge for traders with smaller accounts is that they need to round position sizes, so they can’t get the fine control on position risk that institutions can afford. For example, you want to hold 1.7 contracts for each 100K USD account. If you have 1 MM$ you would trade 17 contracts. With a 100K$ account you would round it to 2 contracts. Now… how much difference does this make? Let’s explore.

In my view, traders with limited capital looking to trade multiple strategies would have to hold less & larger positions, as opposed to more & smaller positions, to be able to trend follow with futures. It’s a downside for the small trader. On the flip side, the small trader can trade less liquid markets which the large institutions can’t.

So our goal is to maximize risk adjusted returns, while (for the smaller trader) constraining the number of open positions in order to allocate more risk per position and be able to afford more markets.

Some ways to reduce the number of positions:

• Set position count caps:

  • Rank new setups in ascending order by ATR*Point Value, or by a momentum score. Take entries until all the available position slots are filled.

  • Kick out poorly performing positions to make room for new entries. This increases the turnover significantly, but may pay off..

• Long biases, or removing sectors.

In the next sections we will explore different ways to go at trend following, starting from data, strategy design, then assessment of performance for different account sizes.

Data

Let’s examine a universe of ~100 futures markets, out of which 10 are micros and minis, so 90 markets effectively. I use CSI Data as the sole data source for this study.

This is the list of markets:

Note some markets are currently delisted. Some markets are recently listed. Some are liquid and some illiquid today.

Futures contracts expire, so to facilitate the simulations, we stitch together the contracts into back-adjusted time series. I do 2 different back-adjusting of the data:

• Price-based adjustment; “Panama-Canal”.

• Returns-based adjustment.

Contracts are switched when the open interest of the back contract is higher than the front contract.

This is an example of continuous futures time series for a given market:

The “Panama Canal” method preserves the actual price changes of the individual contracts. The proportional method preserves the returns of the individual contracts. Note how the price-based adjusted (Panama Canal) time series can go negative in markets with historically positive carry (backwardation).

My simulation engine takes unadjusted, price adjusted and return adjusted series as input, in an R dataframe with columns such as:

symbol, date, delivery, nextdelivery, open, high, low, close, volume, oi, open.p, high.p, low.p, close.p, open.r, high.r, low.r, close.r

Where the suffix ‘.p’ is the price adjusted data, and the ‘.r’ is the returns adjusted data.

When the currently held contract open interest is lower than the next contract, the nextdelivery column will show the next contract, and the simulator will add commission and slippage for the corresponding roll-overs.

Contract selection criteria:

• For Fixed Income, Equities and Currencies, we hold the front or second contract; whichever has the highest open interest.

• For commodities and volatility, we hold the 2nd contract or later, whichever has the highest open interest. This is to remove some idiosyncrasies around commodity delivery, not have to worry about first notice day, and lessen the effect of carry on the trend signals (since I trade carry in a separate strategy). Front contracts in, say, natural gas have a stronger negative carry than the contracts farther away. See my post on commodities for more details.

• For deeply liquid interest rates, we hold the 8th contract onwards (2 years ahead in the future). As the front contracts have a low volatility and high tail risk.

Simulation

Classic Trend Following

Here’s a Classic “Loose Pants” Trend Following strategy:

• Rules:

  • Enter on 6-month high (for longs) or 6-month low (for shorts)

  • Exit based on 5-ATR trailing ‘chandelier’ stop and Donchian stop (rolling 2 month low)

• Fixed fractional position sizing. Each position risks a fixed % of principal.

• Size positions at entry and keep fixed.

• Trade every liquid market. Every market we can get our hands on.

• Long + Short the same way in every market. Same rules.

• Take all trades (can end up holding lots of positions).

OK, let’s explore the strategy at high level, assuming we could afford every liquid market.

Note the number of open positions goes up as new markets become available for trading within our universe. Therefore we shouldn’t model taking the same position risk we use today for the past, because there were fewer markets before. The risk per position needs to be scaled down as the number of available liquid markets increase.

Let’s scale down the position risk at entry with the liquid market count.

OK this makes more sense. We start risking 1% of principal, and move progressively to 0.6% risk as more markets become available and more positions can be opened.

This is the equity curve:

The performance is in line with the TTU TF index, BTOP50 and SG Trend Index.

Dynamic Position Sizing

Using fixed position sizes can leave us with pretty large positions sometimes which would drive the volatility of the portfolio. This feature helps us make more money over the long run, but it’s hard to stomach, as it exposes us to quick and large losses when trends end.

adjust position sizes if target size is >50% off from the target size:

The performance looks visually similar.

The dynamic sizing logic will impact the outlier trades, which are the main driver of returns of the strategy.

Let’s check the impact of dynamic position size adjustment for the recent London Cocoa trade:

The size.c variable is the calculated size for the current volatility (ATR). The size variable is the actual position size we hold. The charts illustrates that fixed sizing ’Classic” system captures a larger profit in outlier trades. But can also incur greater losses. In the Cocoa example you can see the huge run and the large loss at the end for the fixed sizing, and a smoother (but less profitable overall) PnL curve for the dynamic position sizing. Fixed sizing is hard to stomach in outlier trades as the position volatility drives the volatility of the entire portfolio.

Limiting Entries

We can try to simply not enter a new position if we already fill a predefined position count limit. This would limit the number of trend opportunities we can catch early on. But big outlier trades would be caught on eventually, a bit later, as they keep making new highs (for longs), or lows (for shorts). This is what we’ll do:

• Rank all potential entries, “setups” by dollar volatility: PointValue * ATR. To focus on more ‘affordable’ liquid markets (like CME micros, for example).

• Setup a number of slots to fill. Let’s try 10,15,20,30 positions and compare results.

• First check for exits, to free the available slots. Then fill the available slots with entries.

Here is the analysis for multiple position counts, with leverage scaled to have similar median volatility, allowing for apple-to-apple comparison:

Surprise surprise. When normalized to the same leverage, we see the position caps of 10 and 15 positions for our futures universe actually improve performance. I think this is a pretty good insight for the smaller trader who can afford less positions than the institutions.

Assuming we could afford any market we wanted, what if we do the entry setup ranking using a momentum score instead? Intuition says that entering markets with more consistent trends (by whichever method used to quantify this) would improve performance.

Let’s try this one:

momscore=max(−2,min(2,0.42x0.456xEWMAC(16,64)+0.16x0.312xEWMAC(32,128)+0.42x0.212xEWMAC(64,256)))

Source: Derived from Robert Carver’s books and blog.

Interesting… There is a small improvement in ranking entries by momentum score vs. by affordability. Traders with limited capital can do the momentum ranking on the affordable markets only. More on this later.

Good. We are getting somewhere.

Kicking out poorly performing positions

Let’s try a variation of the entry limiting. How about we get rid of some of the worst performing positions to make room for new entries?

• Rank open positions by #ATRs drawdown from position high in descending order (highest drawdown on top).

• Kick out the worst positions, beyond certain drawdown score.

• Fill the available slots.

  AvailableSlots=PositionLimit−Exits−KickOutsAvailableSlots=PositionLimit−Exits−KickOuts

This increases the turnover of the strategy, but hopefully will improve the performance.

Observing the cumulative probability line, we can try kick-out thresholds for drawdown scores of 0.5 (kicks out bottom 25% of positions), 0.75 (kick outs ~10% of positions), 0.9 (kicks out <5% of positions). And let’s also simulate kicking out every position required to enter every new setup. The lower the drawdown score threshold, the larger the trade count and costs, so I don’t have an intuition of where this will take us…

Very interesting! Kicking out the worst performing positions (thresholds 0.5, 0.75, 0.9) reduces performance. Why this may be?

I think because it exits early some positions which would otherwise continue to be profitable, then you have to wait until hitting a new high (or low) to enter again.

However, if we make room to take every setup (threshold 0), the strategy becomes more profitable since 2008. This is surprising, because the increased trading costs would kill the performance in more expensive markets (larger spreads per volatility unit) such as interest rates, and in more mean reverting markets such as equities. That means that cheaper, higher volatility markets improved performance significantly. Using rankings for entries and exits (rotation strategy) is a different way to do trend following, which we will explore later in this post.

Long Bias

OK we’ve found that skipping entries improves risk adjusted returns and keeps position sizes in check, but kicking out poor performers to make room for more entries does not.

Our baseline strategy is now:

• Fixed fractional position sizing based on ATR.

• Adjust position sizes if target<0.5 x size, or target > 2 x size. (To sleep well at night).

• Trade every liquid market.

• Long + Short the same way in every market.

• Skip entries to control open position count.

Now it’s time now to analyze the performance attribution by sector and by trade direction (long vs short). These are the sector results for the “Loose Pants” strategy:

Alright. There’s a lot to unpack here:

• Around half the trades are shorts (see position charts in previous section).

• Shorts contribute negatively or little in Equities, Fixed Income, Grains, Softs, Metals.

• Shorts were profitable in interest rates after interest rates started to increase in 2020.

• Limiting entries changes the performance attribution, but the overall behavior is similar.

• Equity indices are less profitable than other sectors with these rules.

• Volatility and natural gas shorts are profitable, most likely due to strong negative carry, steep contango.

Let’s see market by market performance in our Loose Pants reference strategy to get additional insights.

Observations:

• Each individual market has a poor performance (crappy equity curves), but when stacked together in the strategy, the resulting performance curve is much smoother. Diversification at play.

• Some markets are unprofitable through their entire history, then the big profits arrive, such as Cocoa. I would not discard historically unprofitable markets, except for most shorts as they are consistently unprofitable over many uncorrelated markets.

• Several markets in universe are relatively illiquid.

Let’s check the impact of outliers in longs vs. shorts, for different strategy configurations:

It looks like limiting entries with momentum ranking improves profitability of shorts, but there are not enough big outliers in shorts to match the performance of longs.

OK now I will make a decision to get rid of shorts in Fixed Income, Equities, Volatility, Grains, Softs, Metals . The rationale, based on my context is:

• I already short volatility in other 2 strategies: VIX Basis and Curve Carry. Trend following would be shorting volatility most of the time, which would add too much exposure to volatility in my overall portfolio of strategies.

• Other strategies provide better profile for shorts. Better to invest my principal in those.

• I short interest rates already in the carry strategy.

• Equity indexes like S&P 500 are actually a slow momentum strategy (picks top N stocks by market capitalization, the ones which have outperformed, rebalancing quarterly), so shorting it is like betting against a momentum strategy. It wouldn’t make sense to me for a momentum strategy to short another momentum strategy.

• Fixed income shorts are consistently unprofitable for all the markets.

I will also get rid of volatility sector altogether, as I manage long volatility trades in the carry strategies.

The above decisions are debatable, but also note that I trade shorts in other strategies: Carry, Curve Carry, Mean Reversion, single ticker strategies, etc..

Alright. Let’s check the performance of the long biased, position-controlled strategies now:

Performance looks better overall, with similar equity curves for 15 and 10 positions. This means we can capture the essence of the trend following returns with a smaller number of positions, which is great news for retail traders. Check out the performance curve.

Not bad for this “simple” system.

It told you it was going to be a looong article. Time for a stretch….

Continuous Trading System

A different way to do trend following is by continuous trading, without stop losses. Here’s what we will try:

• Compute several momentum scores.

• Scale momentum scores by their standard deviation. Cap to +/- 2 standard deviations to remove extreme values (which are possibly mean reverting).

• Simulate a strategy for each indicator, to enter long or short when |momentum score|>1 (a meaningful trend) and exit when momentum score changes sign.

• Run the strategy with the combined momentum scores using different weighing schemes. Decide way forward.

• Use momentum scores as scaling factor for position sizes.

• (Hopefully) The performance should be better and more robust with this added complexity.

Alright, let’s get our hands dirty. Here is the code to compute the momentum scores and scale them for 2 categories of momentum measures: returns over moving window, and exponential moving average crossover (similar to ROC and MACD respectively).

Now for the scaling…

The indicator distributions look interesting. The ewmac indicator looks similar to a normal distribution, and the lr looks like a triangular distribution. Now let’s simulate a strategy for each momentum score definition.

And the analysis performance of each momentum score, normalized to the same volatility:

Very interesting. Notice the dispersion of the performance with different momentum scores.

Now the ‘trick’ is to figure out how to combine the individual scores into a single one. Let’s keep it simple and assign a “U-shape” allocation to the 3 lookback of each of the scaled momentum scores:

This should be a good rule of thumb, as correlation of strategy returns between the short and long formation period is lower than the correlation of strategy returns between mid vs short and long formation periods. Note the other 3 indicators are variations of a single concept of calculated returns, so we can pick the best one. I like the “lr” (log returns) because the distribution looks more symmetric, but we have to check the performance to verify.

Let’s check 2 things:

• Performance of sizing positions based only on volatility.

• Performance of sizing positions based on volatility and the combined momentum scores. (Higher turnover but hopefully higher risk-adjusted returns).

Here’s the performance of the combined momentum scores:

The way this strategy is configured, there is no additional edge in sizing positions dynamically using the momentum score.

Also it looks like the ‘cr’ (compound returns) score is a bit better than the others.

I would prefer to keep it simple and stick to the dynamic position sizing on volatility only. But for this article let’s go with sizing for vola and momentum, to demonstrate a “continuous trading” system.

Our resulting strategy, combining the ‘cr’ and the ‘ewmac’ scores 50/50, and dynamically sizing on vola and momentum is:

There’s no additional gain from the combining the two momentum scores into one. But we will still keep them combined. Here is the equity curve:

Looks nice. But it will be hard to afford for people with small accounts due to the large number of positions. Let’s limit the positions to 10:

The performance dropped.

One last attempt. Let’s use just the EWMAC score alone. I like it because anyways because it uses more data, vs. return based indicator which uses just a few points.

More research required into the interactions between the strategy moving parts which is causing such divergence in performance across momentum scores when capping position count.

Fascinating. With the EWMAC scores with 10-position limits, the character of the strategy has changed, and drawdowns are shorter. It doesn’t beat our (simpler) breakout with trailing stop strategy though.

Summary of rules:

• Compute momentum score with 3 lookback sets of EMA crossovers (ewmac).

• Scale momentum scores to a range of +/- 2.

• Enter when |mom score| >=1 (a meaningful trend). Exit when mom score changes sign.

• Dynamic position sizing on volatility and momentum, with 50% threshold (halve the size to reduce risk, double size to increase risk)

• Long bias: Skip Fixed Income, Equities, Metals, Grains, Softs sectors.

• No volatility sector

A Ranking System

Let’s re-use the same combined momentum score and structure the system in a different way:

• Every day, rank all markets by momentum score.

• If momentum score within top 7, enter.

• If momentum score outside of top 14, exit.

This system would hold up to 14 positions. Let’s see how it looks:

Very nice. It works also. A lot of additional research can be done on this type of system. I will leave it for other posts.

Combination of strategy types

What would happen if we trade the 3 trend following approaches together?:

1. Breakout with trailing stop

2. Continuous Trading System

3. Ranking System

Let’s check the correlations:

Pretty high correlations. So we don’t expect much gain from combining trend systems (which increases capital requirements, unless you trade fractional CFD contracts). Better to focus on creating other uncorrelated strategies (carry, mean reversion, etc.).

Anyways, here’s the combined performance with Naïve 1/N allocation across the 3 strategies:

As expected, some improvement (worth it still under some contexts), but not hitting the ball out of the park.

Summary of Strategy Design

• Explored 3 trend following approaches: Breakout with trailing stops, Continuous Trading, Ranking System.

• Checked performance attribution and decided to exclude shorts in some sectors.

• Compared performance of uncapped vs capped position count and decided to cap position count.

• Tested trading the 3 strategies simultaneously. Performance improved, but not game changing.

All the approaches are profitable. Given my personal context, up to this point (where we assumed we can afford everything), I’ll go with the simplest one; breakout with trailing stops. Summary of rules:

• Limit positions to 10

• Enter on 126 day breakout.

• Exit on 5 x ATR(20) stop or rolling 42 day low.

• Rank entry setups by momentum score. Note…it’s just a bit better than ranking by affordability. No major difference.

• Long bias: Skip Fixed Income, Equities, Metals, Grains, Softs sectors.

• No volatility sector

• Dynamically adjust position sizes when target size < 0.5 x current size or target size > 2 x current size.

Implementing the Strategy in Smaller Portfolios

OK finally we got to the main subject of the article. How much capital do we need to trade this stuff?.

CFDs

OK we got our reference strategy. Now let’s check the performance of trading it with expiring CFDs. CFDs are very heterogeneous. But let’s assume we can trade 1/5th of the futures contract for each CFD. Examining the IG Markets website, we can see that the spreads on expiring CFDs are much wider than in the futures. Let’s assume we lose 4 ticks per trade (entries, exits, rolls), for simplicity.

Note for trading CFDs, it is important to analyze properly the costs in detail with actual historical data. That requires another study in its own. A great resource overall to dig deeper into trend following, including implementation with CFDs is ATS Trading Solutions

The wider spreads heavily impacts the strategy, but it is still profitable.

Let’s check the performance using CFDs, for different account sizes. Assuming we cannot hold fractional CFDs (although in some brokers we can!). Due to differences in number of positions held, leverage factor is adjusted to achieve roughly 15% volatility.

We see that you can start trading a trend following strategy with CFDs with 25K USD, which is in line with capital requirements of systematic equity strategies. This is good because it allows sophisticated retail traders with access to CFDs to trade this strategy.

Futures

Now let’s check the performance using futures, for different account sizes. Due to differences in number of positions held, leverage factor is adjusted to achieve roughly 15% volatility.

Fascinating. We see at100K USD you can start trading a trend following strategy with futures effectively. This is much less than what is cited in different sources.

These are the position count charts for 100K and 1000K USD account for 10-position limit:

Interesting. Frequently we would hold less than 10 positions in the 100K USD case because of the smaller selection of available markets hence entry opportunities. Whereas for 1 MM$ we can access a lot more opportunities, so we hold 10 positions most of the time. But this does not make a noticeable difference in performance anyways.

One way to increase the number of opportunities is to relax the liquidity filter a bit. But this also means we would get markets with wider spreads which impact the performance.

Another approach is to ditch the position count control altogether and go back to taking every affordable trade. Let’s test this:

We can reach a similar conclusion to the previous strategy. A minimum of 100K USD is required. Let’s check the position counts now:

For smaller accounts, the position count is kept under control because of the reduced number of affordable markets. The performance looks good as well.

Check out the list of positions for 29th July 2024, for the strategy which takes all affordable trades, by account size:

Pretty cool heh?

Execution Considerations

Execution is challenging because as you saw in the previous table, we are trading in exchanges all over the world, in different currencies, and in markets which have different opening and closing times, and different times in which there is a decent trading volume (liquid hours).

The data fetch runs at midnight New York time, while most US, Canada and European markets are closed. Strategy algo runs thereafter, and outputs a CSV file with the desired positions and sizes normalized to 100K USD.

An automated execution algorithm runs every 5 minutes for Interactive Brokers in an infinite loop, and does the following:

• Get the current net liquidation value (NLV…account size). Compare against a manually input reference NLV.

  • If deviation >2% between the reference NLV and actual NLV, the trader is informed via Telegram message.

  • If deviation>5% between reference NLV and actual NLV. The system does not place any order and the trader is informed.

• Current positions and orders in broker are compared against the expected positions and orders for the system. Sizes are computed based on the reference NLV.

• Difference between Broker and System orders & positions are summarized into a table of orders:

  • When we need to close one position and open a new one in a different delivery for a given market (i.e. rolls, etc.), a calendar spread order is submitted as a market order at the start of the liquid trading hours of the day.

  • Outright futures orders (not calendar spreads) are submitted using the Interactive Brokers patient adaptive algo at the liquid hours of the day, to try to get a better execution.

• Once every 2 hours, a Telegram message is sent to the trader informing that the execution engine is running (live check), and provides the current NLV.

I regularly check the execution engine is working as expected, and keep a record of the trades. The algo runs Python ib_insync library, wrapped in R using reticulate package. Integration between both languages works well.

Conclusions

This was a long article! I hope it sparks several ideas and discussions. Let me know if you find errors!.

Here are some conclusions:

• Yes, sophisticated retail traders can run a diversified trend following system with futures. But need >100K USD in capital to do so effectively with futures. Good news is that this is much less than what is often touted.

• Shorts are in general less profitable than longs except in high contango markets and in currencies (where longs and shorts are equivalent). I think it’s OK to skip the trend following shorts in some sectors to release capital other strategies which have better edges on shorts.

• “Loose Pants” vs. Other styles and variations of trend following: All profitable and valid.

• Trading all markets we can get our hands on vs. a limited portfolio: Both profitable and valid.

• Controlling position counts by limiting entries, vs. taking all the trades: Both profitable and valid.

• Combining different trend following strategies: Good idea, if capital available.

• Non-US residents with small accounts can use CFDs instead of futures. But need to study the costs, spreads and swaps (for cash CFDs) very well before pulling the trigger. Explore using mature CFD/Forex tools such as MT4/MT5 for easier execution. No escape from having to write a lot of code.

• Need an automated execution engine to trade this system plus other strategies effectively over multiple exchanges across multiple time zones. Unless you have time and dedication to place trade manually or semi-automatically.

Disclaimer

This article is documentation of research done by the author adapted to his own context. It does not constitute financial advice. Do your own homework.

Ray Dalio says the holy grail of investing is in trading multiple uncorrelated strategies. Diversification can be achieved by trading systems with different underlying principles, time frames and asset classes, over a wide range of instruments.

Following an ideas-first approach to trading; not trying to uncover patterns in the data but working with base principles. As per AQR’s definitions of style premia:

• Trend: Tendency of asset recent relative performance to continue in the future.

• Carry: Tendency of higher yielding assets to outperform lower yielding assets.

• Value: Tendency of relatively cheap assets to outperform relatively expensive ones.

• Defensive: Tendency for lower-risk and higher-quality assets to generate higher risk-adjusted returns.

I’m exploiting these base principles (so far) using several daily timeframe strategies:

1. [Trend] Trend Following: Long/Short momentum on equities, currencies, commodities, fixed income, rates.

2. [Carry] Carry: Long markets in backwardation, short markets in contango. Confirm with short term momentum.

3. [Carry] Curve Carry: Calendar spreads on commodities, exploiting the change in curvature of the term structure for each contract as contracts move towards expiration.

4. [Carry] Futures Contract Expiration: Long/Short cash-settled contracts when close to expiration, as the futures price converge to spot.

5. [Carry] VIX Basis: Exploit the VIX term structure. Short VIX futures (sell volatility) when contango. Long VIX futures when backwardation.

6. [Value] Relative Value: Exploit price relationships between commodities and stocks (producers, processors/refiners).

7. Asset Allocation: Vola risk parity long-only portfolio of stock index, bond and gold.

Strategies Performance

All the above strategies are implemented with Futures, except the relative value one, which trades stocks. The above strategies don’t deliver a great risk/reward individually but the combination does.

Let’s see how the individual strategy performance looks like:

It’s hard to compare individual strategy performance properly, since each one has a different leverage. Let’s normalize the strategies to 12% volatility for comparison, for apples-to-apples comparison.

Here we multiply the position allocations by these factors to get the same leverage in each strategy:

Overall positive, with each strategy doing its own thing, with some sizeable drawdowns every so often. The asset allocation “risk parity” strategy got hammered badly twice since 2020, as the equity and fixed income correlation character changed. Now let’s evaluate the historical correlations of the strategies. We expect low correlations since we are attacking the markets from different angles, timeframes and instruments:

Yearly rolling correlations look decent overall. Except for asset allocation and trend following, which oscillate around +/- 0.5, since trend following portfolio periodically gets concentrated with long bonds and equities.

Let’s check what happened during the 2007-2009 global financial crisis, and during the Covid crisis of 2020.

Correlations remained low, or changed gradually during the GFC which is interesting. During Covid crisis a sharp change of correlations is seen across several strategies. Notably the VIX Basis strategy switches from short volatility to long volatility in March 2020, which offsets the losses during the crisis in the asset allocation strategy.

Combining the Strategies into a Portfolio

How much to allocate to strategies at any given time depends on the correlations, volatility, expected returns, and an assessment on the robustness of the strategy.

Let’s try out different methods: Naive 1/N allocation and risk parity: volatility, correlation, and volatility + correlation.

Naive 1/N portfolio allocation

Let’s allocate 1/7th of the portfolio principal to each strategy at any given time, meaning each when a new signal comes, the strategy would size the orders using 1/7th of the principal.

Note I’m talking about the “dry powder”, which is your account net liquidation value minus open profits/losses. This is to avoid pyramiding.

Net Liquidation Value = Principal + Open PnL

OK, here we go:

The chart shows the portfolio performance in thick black line, and the contributions for each strategy to the portfolio performance.

That’s a CAGR of 16.7% with historical drawdowns of <6.2%. Looking pretty good.

Note how during drawdowns, some strategies can lose most or even more than the money they made historically up to that point. This is because we progressively increase the cash allocated to each strategy as our portfolio value increases.

Historical drawdowns are manageable. Let’s leverage the strategy 2X and see how it looks:

A CAGR of 33% and max historical drawdown of ~11.2%. Here we assumed 1 tick slippage for each trade (outright or calendar spread leg). Rolls are assumed as 2 trades, but in practice I use calendar spreads for rolls, to reduce slippage. Let’s double the slippage assumption and see how it looks:

CAGR decreased to 30% and drawdown increased to 12%. This will be our reference case from this point onwards.

Volatility Risk Parity (VRP)

Let’s now try adjusting the position sizes based on the volatility of the equity curve of the individual strategies. Daily volatility is estimated as the standard deviation of log returns over 30 days, and annualized by multiplying by squared root of time. 1 year has ~256 trading days, so we annualize the daily volatility by multiplying by 16.

annualized_vola=roll_sd(ln(vami/lag(vami)),width=30)*16*100

Note how the volatilities go all over the place for the “Future Contract Expiry” strategy. This is because we only hold positions for short period of times, and trades are clustered around mid-month or end-of-month. So we’ll just use a fixed number of 12% for volatility of this strategy and use the actual estimated volatility for the rest.

OK, now we have to convert the volatility of each strategy to a scaling factor, such that the sum of the scaling factors is equal to 1. For the previous naive 1/N allocation we had a fixed allocation of 14.29% for every strategy (1/7), but in this case the allocation of each strategy will change over time.

Important note: no vola adjustment for open position for trading strategies, except for the asset allocation strategy during rebalances. Let’s see how the volatility risk parity allocation impacts or base portfolio:

The average drawdown is reduced to 9.7%, and the CAGR is reduced to 25%. There is also a wider dispersion in the strategy contributions. Looks encouraging.

Correlation Risk Parity

OK, now let’s see how it looks if we scale the entry order sizes based on the pairwise correlations. We do it like this:

• Get log returns time series for each strategy.

• Arrange log returns for each strategy pair.

• Compute 90-day rolling correlations for each strategy pair.

• Compute the scaling factor using Gaussian scaling, for each strategy and date:

(each strategy and date would have 6 rows. Corresponding to pairwise correlation between the given strategy and the other 6; 42 rows per date)

Where:

• 𝑍 is the Z-score.

• 𝑋 is the pairwise correlation

• μ is the mean of the correlation matrix

• σ is the standard deviation of the correlation matrix.

Now, for the performance, scaling entry sizes by correlation only:

We get a CAGR of 27% and max drawdown of 12.6.%, with an overall wider dispersion on the individual strategy contributions.

Volatility + Correlation Risk Parity

OK now let’s try to adjust by both volatility and correlation. Let’s compute a combined scaling factor which weighs 50% to inverse volatility and 50% to inverse correlation factor, then scale it so it sums to 1 for the portfolio.

Interesting… it looks very similar to the Naive 1/N base case.

Let’s check the performance of the combination:

OK, CAGR 26% with max drawdown 10.7%. The added complexity doesn’t hit the ball out of the park!.

Other portfolio management ideas to try, later on

• Equity Curve Trading: Enter/Exit the strategy based a criteria. For example 200-day moving average or more advanced regime filters.

• Martingale: If we assumed every strategy is robust and will recover from every drawdown, we can allocate to strategies following the “Martingale” method, where we put more capital during drawdowns (up to certain level), and less capital during equity highs.

• Other advanced asset allocation methods.

Portfolio comparisons

Let’s summarize our observations into a chart and a table:

It looks like the volatility risk parity improves performance a bit, and removes some outliers on positive and negative side.

But there is a key downside of volatility risk parity (in my particular context): reducing the leverage in strategies in a relatively small account can prevent from opening positions because it would require to buy less than 1 contract sometimes. This can happen in the Naive 1/N allocation also, but less often. Careful simulations are required to evaluate the impact of missing some trades and rounding position sizes.

Overall there is a huge positive impact in trading multiple strategies. Note how the Sharpe ratio of the portfolio is much larger than the Sharpe ratio of the individual strategies. So we are moving in the right direction.

Conclusions

• Keep-It-Simple. Use 1/N strategy allocation, or a fixed allocation setting based on the conviction on the robustness of each individual strategy. Unless you manage a large pool of capital, where volatility targeting the porfolio makes sense.

• Focus on developing more uncorrelated strategies with different:

  • Asset classes (equities, commodities, fixed income, rates, crypto)

  • Instruments: Futures, Equities, Crypto (spot & futures)

  • Styles: Trend following, mean reversion, carry, relative value

  • Timeframes: Long Term, Short Term

  • Directions: Long + Short

What’s next?

To research other strategy ideas for futures:

• Add synthetic instruments, such as popular calendar spreads (with economic sense) to existing (or new) strategies.

• Short term mean reversion.

• Additional relative value trades.

• Feature engineering (COT report, term structure, etc.) and machine learning model to predict next week’s market direction.

And away from futures:

• Mean Reversion in stocks & crypto.

• Trend following in stocks & crypto.

• Stock Pairs trading (easy to execute but very hard to find the best pairs to trade).

A few days ago, I came across this piece by Sofien Kaabar on the “Internal Bar Strength” indicator, on his All About Trading! Substack.

Very simple concept using this indicator:

𝐼𝐵𝑆=(𝑐𝑙𝑜𝑠𝑒−𝑙𝑜𝑤)/(ℎ𝑖𝑔ℎ−𝑙𝑜𝑤)

With these simple trading rules: Long entry when IBS<0.2 and Exit when IBS>0.8

Results look pretty interesting in the TradingView platform. It looks like an interesting strategy to trade with US equity index futures, to avoid the “wash sales tax” when trading ETFs (for US citizens/residents, due to the higher trade frequency), and to reduce trading costs.

Analysis

Let’s expand Sofien’s analysis to a lot of markets, beyond S&P500, and check how it performs. Here’s a market universe of CSI Data symbols with categories by yours truly.

symbol.frame=data.frame(
  symbol=c('MES','MNQ','M2K' ,'MYM' ,'FNG',
           'A50','SIN','JTM',
           'FXP','FDX',
           'C2','W2','S2','SM2','BO2',
           'EMA','COM','BL2',
           'KC2','SB2','CT2','LB','LBR','CC2',
           'LCC','LRC','LSU',
           'GC2','SI2','PA2','PL2','HG2',
           'LC','LH', 'FC',
           'CL2','RB2','NG2','HO2',
           'VX','FVS',
           'EBL','TY'
           ),
category=c('US.EQ','US.EQ','US.EQ','US.EQ','US.EQ',
           'ASIA.EQ','ASIA.EQ','ASIA.EQ',
           'EUR.EQ', 'EUR.EQ',
           'US.GRAIN','US.GRAIN','US.GRAIN','US.GRAIN','US.GRAIN',
           'EUR.GRAIN','EUR.GRAIN','EUR.GRAIN',
           'US.SOFT' ,'US.SOFT', 'US.SOFT','US.SOFT','US.SOFT','US.SOFT',
           'EUR.SOFT','EUR.SOFT','EUR.SOFT',
           'US.METAL','US.METAL','US.METAL','US.METAL','US.METAL',
           'US.MEAT' ,'US.MEAT','US.MEAT',
           'US.ENERG','US.ENERG', 'US.ENERG','US.ENERG',
           'VOLA','VOLA',
           '10YBOND','10YBOND')
)

Now let’s compute performance for each market without compounding, and sizing by inverse volatility for apples-to-apples comparison; since each future has a different contract size and natural volatility. To see if we can find some patterns…

BA=fGetFuturesBA(symbol.frame$symbol)%>%
  inner_join(symbol.frame,by='symbol')%>%
  group_by(symbol)%>%
    arrange(date)%>%
    #Indicators
    mutate(
      annvola=roll_sd(log(close.r/lag(close.r)),30)*16*100,
      ibs=(close.p-low.p)/(high.p-low.p)
    )%>%
    #Signal
    mutate(signal=case_when(
      ibs<0.2 & lag(ibs)>=0.2 ~ 1,
      ibs>0.8 ~ 0,
      T ~ as.numeric(NA)
    ))%>%fill(signal)%>%mutate(signal=replace_na(signal,0))%>%
    #Sizing  
    mutate(
           #Position sizing such that you lose 5% of account
           #when 7 standard deviation move against position
           size.c=0.05*fx*1e5/(fullpointvalue*7*(annvola/16/100)*close),
           size=case_when(
             signal & !lag(signal) ~ size.c,
             T ~ as.numeric(NA))
    )%>%fill(size)%>%mutate(size=replace_na(size,0))%>%
  ungroup()%>%
  mutate(
      strategy='Internal Bar Strength',
      allocation=1,
      ordertype='MKT'
  )

This is the IBS indicator performance with no slippage:

#No slippage
BA%>%
  fComputePnL(slippageTicks = 0)%>%
  mutate(symbol.cumpnl=symbolcumret-symbolcumcost)%>%
  ggplot(aes(x=date,y=symbol.cumpnl,color=symbol))+
    geom_step()+
    theme_bw() + 
    ylab('Cumulative PnL, USD')+
    guides(color = guide_legend(override.aes = list(linewidth = 2))) + 
    facet_wrap(~category,scales='free_x',ncol=4)+
    ggtitle('IBS indicator performance',subtitle = 'No Slippage')

This is the IBS indicator performance with 1 tick slippage:

This is the performance with 3 ticks slippage:

Interesting... Here are some observations:

• The strategy is highly sensitive to slippage and liquidity, as expected since it is short term in nature.

• There is edge in some other markets other than US equity indexes but it goes away when slippage is introduced.

It looks like this strategy would only work effectively in the highly liquid US index futures. Let’s do a backtest using a naïve 1/N position sizing (which performs better usually) to gauge the strategy performance:

Universe=symbol.frame%>%
  filter(category %in% c('US.EQ'))


BT=fGetFuturesBA(Universe$symbol)%>%
  inner_join(Universe,by='symbol')%>%
  filter(date>='2000-01-01')%>%
  #Get available market count
  fPadData()%>%
  group_by(date)%>%
    mutate(N=n())%>%
    filter(!gapfilled)%>%
  group_by(symbol)%>%
    arrange(date)%>%
    #Indicators
    mutate(
      annvola=roll_sd(log(close.r/lag(close.r)),30)*16*100,
      ibs=(close.p-low.p)/(high.p-low.p)
    )%>%
    #Signal
    mutate(signal=case_when(
      ibs<0.2 & lag(ibs)>=0.2 ~ 1,
      ibs>0.8 ~ 0,
      T ~ as.numeric(NA)
    ))%>%fill(signal)%>%mutate(signal=replace_na(signal,0))%>%
    #Sizing  
    mutate(
           size.c=1e5/N/close/fullpointvalue,
           size=case_when(
             signal & !lag(signal) ~ size.c,
             T ~ as.numeric(NA))
    )%>%fill(size)%>%mutate(size=replace_na(size,0))%>%
  ungroup()%>%
  mutate(
      strategy='Internal Bar Strength',
      allocation=1,
      ordertype='MKT'
  )

Here’s the performance with 1 tick slippage:

#Compute Backtest
BT%>%
  mutate(slippageTicks=1)%>%
  fComputePnL()%>%
  fCompoundPositions()%>%
  fComputePnL()%>%
  fPlotInteractivePerformanceChart()

And with 2 tick slippage, to check the magnitude of the edge:

Still positive.

Now let’s try a different IBD range, with 1 tick slippage. Enter Long when IBD<0.15 and Exit when IBD>0.85.

It holds up. Overall not bad performance for such a simple strategy.

Now let’s test a similar strategy in RealTest, which trades SPY, QQQ, IWM and DIA ETFs, also with 1/N sizing and IBD>0.2 entry, to double check:

Looks OK. Now…. note there is a relatively high correlation to S&P500

What’s next? To do similar analysis into different technical analysis indicators (RSI, etc..) and rules; and come up with a collection of technical analysis trading strategies.

These strategies which work in a few markets but fail in the rest, without a fundamental economic reason for doing so, are less robust and more prone to be random. The solution? To stack together a lot of these weak edges and watch closely over time. Trade with paper trading account for a few months before going live, to remove the crappy ones, then kill the ones that turn out to be random after a few months of live trading. And allocate less capital to these strategies than more robust strategies such as trend and carry.

This particular one goes to my strategy “incubator” in the paper trading account.

Lemme know what you think!

The Idea

Got an interesting idea from @PyQuantNews tweet and a recent podcast from Better System Trader. on crude oil crack spreads, where you can do a long/short trade on the crack spread vs. a refiner stock such as Valero VLO. The main premise is that when the crack spread increases in value, the refiners profit margins increase hence the stock price should increase. There is a lag on the change of the stock price when the crack spread changes which we can profit from.

Here’s the tweet which got my attention.

I’ve traded the risk premia harvesting strategy before, and it works. I assume the “manager rebalancing in treasury ETFs” refers to the end of month seasonal effect on treasuries and stocks. That one also works well, although it has struggled since 2022.

First time hearing about the other 3: Crack spread vs refiner stocks, factor-based rebalancing against volatility, near expiration option straddles before earnings. Let’s test out the Crack Spread idea….

The crack spread is typically described as 3:2:1 or 5:3:2 ratios of CL, RB, HO markets (crude oil, gasoline, heating oil), short the crude front contract and long the distillates back contract. The appropriate crack spread ratios to use on a given refinery depends on the properties of the oil feed and the refinery configuration. This CME article gives some useful insights on crack spreads and its use for hedging.

Computing the Crack Spread

The code below gets the stock data from Norgate and Futures data from CSI Data. This is from my personal toolkit in R. Stock data is total return series, which includes splits, corporate actions and dividends in the time series. Futures data is for individual contracts.

STK=fGetStockData(symbols=c('VLO', #Valero
                            'MPC', #Marathon
                            'PSX'  #Phillips 66
                            ))%>%
  fAdjustStockDataFrame() #To make time series look like futures back adjusted data (for personal backtest engine)


FUT=fGetFutures(c('CL2', #Crude Oil
                  'RB2', #Gasoline
                  'HO2'  #Heating Oil
                ))

Now that we got the futures contracts data, we can compute the crack spread. There are a few challenges:

• Heating Oil HO and Gasoline RB contracts on same delivery expire the same day, but Crude Oil CL contract expires 1 week before.

• Hence there is ~ 1 week of the month where there is no front futures contract for crude to compute the spread.

• Crude price went negative on 20 April 2020 upon contract expiration.

• There is 1 crack spread for each expiration month. Which one to use, and how?

Let’s use the crude spot price when no crude front contract is available. Negative crude price would show as ultra-profitable crack spread spike for refiners for that day. That’s fine for now.

Let’s also compute a synthetic crack spread with a fixed expiration dates, such as 45, 60 and 90 days, using linear interpolation.

library(tidyverse)
library(lubridate)

CrackSpread=FUT%>%
  mutate(delivery=if_else(delivery=='Cash',' Cash',delivery))%>%
select(symbol,date,delivery,expirydate,close,volume,oi)%>%
pivot_wider(names_from=symbol,values_from=c(close,expirydate,volume,oi))%>%
  mutate(expirydate=expirydate_RB2)%>%
  #Compute days to expirateion
  mutate(dte=as.numeric(expirydate-date))%>%
  group_by(date)%>%
    arrange(delivery)%>%
    fill(close_CL2)%>% #When front crude contract not available, use spot
    mutate(
      crack.spread.321=-1*(3*lag(close_CL2)
                        -2*42*close_RB2
                        -1*42*close_HO2)/3,
      crack.spread.532=-1*(5*lag(close_CL2)
                        -3*42*close_RB2
                        -2*42*close_HO2)/5
    )%>%
  ungroup()%>%
  filter(delivery!=' Cash')%>%
  group_by(date)%>%
    arrange(delivery)%>%
    mutate(
           #Compute the synthetic fixed expiration crack spreads
           cs321.90D=approx(x=dte,y=crack.spread.321,xout=90)$y,
           cs321.60D=approx(x=dte,y=crack.spread.321,xout=60)$y,
           cs321.45D=approx(x=dte,y=crack.spread.321,xout=45)$y,
           cs532.90D=approx(x=dte,y=crack.spread.532,xout=90)$y,
           cs532.60D=approx(x=dte,y=crack.spread.532,xout=60)$y,
           cs532.45D=approx(x=dte,y=crack.spread.532,xout=45)$y
           )%>%
  ungroup()


#Write to a CSV file, to use in RealTest
CrackSpread%>%
  distinct(date,.keep_all=T)%>%
  select(date,starts_with('cs321'),starts_with('cs532'))%>%
  pivot_longer(cols=-date,names_to='symbol',values_to='close')%>%
  write.csv('D:/RESEARCH/Crack.csv',row.names = F)

Code blocks look pretty ugly in Substack, without syntax highlighting. Sorry.

Testing strategy ideas

We got the crack spread time series. Now let’s test a very simple long-only idea:

Go long the refiner company stock when the crack spread is increasing. Exit when it’s decreasing. Trend defined by ROC (rate of change) indicator. Short term ROCs would be required since this effect doesn’t take too long to be arbitraged away.

This should be enough for you to experiment. This is how it looks in my simulation tool for a given set of parameters:

Very nice. Now let’s do the same test with RealTest software, to verify:

These are the stats:

Note the max exposure is 100% more than 99% of the time. There are 2 blips when PSX and MPC stocks gets listed in the exchange. Will figure out how to handle it better in the code.. but this does not affect the result.

Conclusion

Overall a decent strategy to add to the portfolio. If you haven’t yet, please listen to Better System Trader podcast. Big thanks to the podcast guests for sharing profitable ideas.

On the next posts I’ll make an update on the portfolio of trading strategies, and how I’m implementing multiple Futures and Stocks/ETF strategies together.

Commodities can be accessed by retail traders through linear instruments such as:

• Futures

• CFD, Cash-based

• CFD, Expiring

• Spread Bets (*UK only)

• ETF, Physical Holding

• ETN (exchange traded notes) (unsecured debt instruments)

• ETC (exchange traded commodities). UK regulated.

• ETF of commodity producers

• Stocks of commodity producing companies

The payoff of linear instruments such as futures and stocks have a direct relationship with market movements. Their value changes linearly with underlying asset price. Non-linear instruments, such as options, have asymmetrical payoff. I’m not trading options (yet).

ETP, ETF, ETN, ETC differences

When digging for ETF information online, you can find several different names thrown around, which can be confusing. Here’s some quick clarification:

• ETP: A broad term that includes any investment product that trades on a stock exchange and tracks an underlying asset or index.

• ETF: A marketable security that tracks an index, a commodity, bonds, or a basket of assets like an index fund but trades like a stock on an exchange. (Source: TradingView)

• ETN: A type of unsecured debt security that tracks an underlying index of securities and trades on a major exchange like a stock. (Source: Investopedia)

• ETC: Similar to ETN, but collateralized by holding of physical commodity. Traded in London stock exchange (Source: Investopedia)

ETFs have lower counter-party risk, especially the ones which have physical holdings. ETNs and ETCs have higher counter-party risk because the entire value of is dependent on a single counterparty’s ability to make good on its promises.

There’s a nice explanation on counter-party risks from ETF.com, for more details on risks: ETP counter-party risks explanation

CFD Cash-based & Expiring

CFDs offer access to commodities for (non-US) traders with small accounts. The CFD providers with widest commodity CFD coverage I’ve found after lots of digging are IG and SaxoBank.

Let’s pick an example from IG, for Lumber. Note how IG offers 2 types of CFDs: Undated (cash-based) and Futures (expiring). In some other markets, there is also a selection of contract sizes to choose from as well.

Check the contract size for Futures (expiring) CFDs and Undated ($ cash-based) CFD of $0.275, which matches the point value of the future contract (Lumber Futures Contract Specs - CME Group , CSI Data Commodity Factsheet). But what’s special is that you can trade fractional contracts in the expiring CFD (!). See CSI commodity factsheet tool screenshot below for point value parameter. The IG CFD for the Lumber market has the same point value as the future contract.

So if you bought 1 future contract LBR in Interactive Brokers for example, you get an exposure to Lumber on 23 Apr 2024 of ~14000 USD. If you buy 0.1 expiring CFD in IG you get an exposure of ~1400 USD. Big difference.

I used to trade with IG but had to drop it because I struggled to automate trading with the API due to my sub-par programming skills. But you may be smarter than me. Check out the trading-ig Python library. Output requires a lot of processing to convert to a simple data frame and to match the IG tickers to the market names from your historical data provider.

Now, why would you trade expiring CFDs, if you can simply trade undated CFDs? It boils down to costs:

• Undated (cash-based) CFDs carry an overnight funding cost, which you pay daily.

• Futures (expiring) CFDs don’t have funding costs. Costs are passed by widening the spread.

In short, for longer term trading, it’s cheaper to trade expiring CFDs. For short term/intraday, it’s cheaper to trade undated CFDs. For long term, the funding costs kill you. Overall it’s easier to make money trading longer term systems (position duration>1 week), so on my personal context I would stick to the expiring CFDs.

From IG: Why is overnight funding charged and how is it calculated? | IG AE

A great resource to learn about these instruments is the Leveraged Trading book by Robert Carver. One well known trader who trades CFDs is Richard Brennan. He shares a wealth of knowledge on trading. Worth listening to his podcasts and YouTube videos to get ideas.

Spread Bets

These instruments are similar to CFDs, but have different tax treatment under some jurisdictions, such as the UK. In the UK, spread bets offer a tax advantage because profits are not subject to capital gains tax nor stamp duty. This is because spread betting is considered as gambling whereas CFDs are considered as financial investments. As far as I understand, broker accounts in countries other than the UK don’t offer spread bets.

Futures

The cheapest and most effective way to trade commodities is through Futures markets. However, it has been historically challenging for retail traders to trade futures with small accounts, as contract sizes for some markets can be huge. However, futures trading is now becoming accessible with the CME (Chicago Mercantile Exchange) micro contracts, and other relatively small contracts available in international exchanges.

Futures are contracts between buyer and seller (such as commodity producer and consumer) to buy or sell a commodity at a given date for a given price. Most commodities are traded this way, as a means for producers and consumers to hedge their exposure to market and environment risks, such as weather impact in crops, etc.

Futures contracts for each market are standardized to a given size (Example 1 US crude oil contract CL gives you exposure to 1000 barrels of oil, and 1 US micro oil contract MCL gives you exposure to 100 barrels of oil), and trade for different expiration months depending on the market. They can trade in different currencies and different denominations (dollars or cents). Some are physically settled, and others are cash-settled, etc. Quite a lot of heterogeneity.

Most brokers don’t allow physical settlement and liquidate the positions upon “first notice date”. Specification of the first notice date is different in each market.

The commodity trader thus needs to build a database of futures market metadata to account for differences in specifications across markets, and to map the ticker from the data source, the exchange and the broker. The below R code shows how to get the list of futures markets available in Interactive Brokers, with their current margin requirements, which is basically web scraping this link. If you use another language, just paste this code to ChatGPT and ask it to translate.

#######################################
#### Fetch futures margins from IB ####
#######################################
#Load required libraries
library(rvest)
library(tidyverse)

# URL of the Interactive Brokers' margin page
url='https://www.interactivebrokers.com/en/index.php?f=26662'

# Read the webpage
webpage=read_html(url)

# Scrape the tables from the webpage
tables=webpage %>% html_table(fill = TRUE)

# Get the list of futures available in IB
tbl=bind_rows(
  lapply(tables,function(x){
    if('symbol' %in% tolower(names(x))){
      NULL
    }else{
      names(x)[1]=strsplit(names(x)[1],' ')[[1]][1]
      names(x)=tolower( gsub("\r|\n|\\s|\\d|,", "", names(x)))
      x%>%
        mutate(across(everything(), as.character))
    }
  })
)%>%
  arrange(exchange,underlying)

If you have an account size of ~100K USD, as a rule of thumb, you can access contracts of <=2000 USD maintenance margin. I’ve found that margin requirements correlate with fractional position sizing: ~ 5 ATR (average true range) loss on 1 contract (very) roughly approximates the margin required by Interactive Brokers site. So you can use this logic to search for which contracts you can trade based on your account size. The chart below shows the rough correlation for several markets with margin <5000 USD.

For strategy simulation I use CSI data. You can get the contract metadata from CSI Data website. Next, you need to build a dictionary of the ticker/symbol of each market from CSI (or your data provider of choice), vs. Interactive Brokers (or your broker of choice).

Examples:

| Market | Int. Brokers | CSI Data | Exchange Symbol | Exchange | FullPointValue |…

|Lean Hogs | HE | LH | HE | CME | 400 |…

|Iron Ore | SCI | SEF | FEF | SGX | 500 |…

Examples:

Live Cattle

Let’s say we expect the cattle price will increase for whatever reason and we want to buy. Checking the futures contract LE from Chicago Mercantile Exchange, we find that 1 contract corresponds to 40,000 pounds of live cattle, which corresponds to ~30 steers and heifers (castrated males and females which have not borne calf). So you’re entering into a deliverable (physically settled) contract to buy the equivalent of ~1 truck of cattle (!). That’s a pretty sizeable contract.

Of course, as a retail trader you can’t take delivery of cattle. The exchange would liquidate your position at first notice date. See Interactive Brokers contract description showing first notice date.

Usually the open interest (number of contracts held by traders in active positions) of the front (earliest delivery) would dry out by first notice date, and the open interest of the back contract (second earliest delivery) would increase, as traders “roll over” their contracts to avoid delivery and seek liquidity. You can watch this at play in another great resource: BarChart website.

There are no ETF/ETNs which track live cattle price. There is a live cattle IG CFD with point value 2$ (1/2 of futures size), and minimum contracts allowed of 0.25. So you can have 1/8th of the exposure of 1 future contract. (As of today, 1 future contract gives exposure ~70K USD, so 0.25 IG CFD(2$) gives you an exposure of 8800$). Pretty nice if you have access to CFDs, and you either figure out how to automate trading with IG, or are happy with submitting orders manually.

OK… enough with CFDs vs. Futures for now.. now let’s dig into other aspects of commodity markets.

Gold

Gold is not a perishable commodity. It can be stored indefinitely in vaults.

This presents interesting characteristics, since some financial instruments can be backed by a physical holding in a vault (hence the instrument price moves in lockstep with spot gold prices, minus some long-term drift due to management fees, storage and insurance costs), and others can be based on futures contracts:

• Future: GC (full size, 100 troy ounces), MGC (micro size, 10 troy ounces)

• Physically-Backed ETF: GLD, IAU

• ETN: UGL

• Gold Mining ETF: GDX, GDXJ

• Gold Mining Stocks: Barrick Gold (GOLD)

Plotting the gold prices, normalized by the last close, we can see some interesting differences.

The physically backed ETF tracks the gold price very closely, and has a tiny drift over time, attributed to small fees, storage and insurance costs. The micro and full futures contracts also match almost exactly in price, showing a very efficient arbitrage is taking place.

Note how the proportionally adjusted futures time series are historically above the spot. This is because the gold market is usually in Contango, which is jargon for having the future price higher than the current price. Contango is caused by:

• Storage costs: warehousing, insurance, maintenance, etc.

• Carry costs: costs of financing the purchase of physical commodity to store until the time is sold.

• Interest rates: opportunity cost of holding gold vs. returns earned from other investments (such as US treasuries).

• Market expectations (supply/demand/speculation/sentiment/inflation)

In the case of gold most of the contango seem to be caused by the interest rates, as we see from the ETFs that the storage costs contribute to a smaller portion of the carry.

Natural Gas

The “carry” (contango and backwardation) is an important aspect to trading commodities. One of the clearest examples is Natural Gas. Gas is expensive to store, and its demand is seasonal, hence we expect to see large differences in prices for different futures contract expirations.

Natural gas consumers, mainly electric utilities companies, hedge their exposure to swings in natural gas prices by purchasing natural gas at different future deliveries, going even years into the future.

The chart below shows natural gas Term Structure (prices at a given date for different available futures contracts with different expiry dates). Notice any pattern? Let’s see…. There’s a hump around winter months, and a kink from the spot to the front (nearest) futures contract. There’s a high volatility (prices swing around 2 to 8$/MMbtu). Also, there’s steeper price differences around Feb-April expiries, and a change of slope at the April expiry. The March-April spread is called the “widow-maker” due to the sometimes-extreme volatility. Some traders profit from these patterns through a set of strategies which can be bundled as “carry” using outright futures or calendar spreads.

You can check today’s term structure in BarChart.

There is a mini contract available for Natural Gas, QG, which is 1/4 of the size of the full contract NG, albeit with lower liquidity. Good enough for retail traders. As explained in previous sections, futures contracts expire so to simulate some trading strategies which require rolling contracts (such as trend following) we have to stitch the time series together, through “back-adjusting” process. See below the proportionally back-adjusted time series for natural gas for full and mini contracts, rolled by open interest (holding the most liquid contract at any given time - usually the front contract). The first contract close (unadjusted) is also plotted for comparison. Note the long-term negative trend, characteristic of a steep contango.

Note also how historical performance of NG (full) and QG (mini) don’t track each other as closely as gold’s GC (full) and MGC (micro). Digging into the contract specifications, I found two differences: 1) Contract expiration date is 1 day earlier for mini contract. 2) Mini contract is cash settled, unlike full contract, which is deliverable. This explains the differences. Remember the market is highly efficient and differences in otherwise identical contracts are arbitraged away by a myriad of market participants.

For a proper performance analysis, it’s better to use log charts. Just be careful to use the correct back-adjusted time series (don’t plot “Panama-canal” back-adjusted series with log charts). Proportionally back-adjusted series can be represented in log charts, as below.

Yes, there is a steep contango observed (futures front contract prices decline as they approach expiry; that “kink” at the front of the term structure, which we observed), so let’s spin off a very simple strategy to go short 1 natural gas NG futures contract for each 100K account size, compounding at each contract roll, holding the front contract until 1 week before expiry when the front contract is higher than the spot price. Let’s see how it looks:

A bit ugly, but profitable. Let’s tweak it a bit by adding a threshold, such as contract close is 4% higher than spot price:

Much nicer. Notice the nasty events in 2004,2005,2006. These “left tail” events are typical in carry strategies. I wouldn’t dare to trade this strategy standalone but would be happy to trade it as part of a portfolio of strategies, with a small allocation (assuming enough capital is available. Big challenge for us small traders).

Now let’s try with a more affordable market, QG, holding 4 contracts, for apples-to-apples comparison with NG (QG contract size is 1/4th of the full NG contract), and shorting the contract only if volume is >500 contracts (volume filter due to lower liquidity):

Not as nice as the full contract, but also profitable. Correlation between the QG and NQ strategies increased a lot after 2010. Note you have to be aware of Interactive Brokers Futures Cut-Out Rules when trading QG and other contracts.

A similar strategy can be implemented in the form of “Curve Carry”, whereas you go long a calendar spread of the front contract vs. a farther out contract. (A long calendar spread shorts the front contract and buys the back contract). Sounds good, but you would have to figure out how to handle seasonality impacts, to avoid getting into a “widow maker” situation.

This is the curve carry strategy shorting the front (first) contract and buying the 7th contract. Example, today (May 2024) you would short May “K” NG contract and buy November “X” NG contract, and so on.

Very nice. More affordable strategy with similar return profile. Notice the tail events, which may be caused in part because we are crossing seasons (summer vs. winter for example). Let’s see if we can trade contracts 1 year apart, to trade the same seasons.

From the BarChart screenshot, we see that liquidity for farther out months dry out, so big players can’t trade natural gas calendar spreads of contracts which are 1 year apart. But small traders can!. Let’s check out the curve carry strategy trading contracts 1 year apart, such as NG May “K2024” vs NG “K2025”, with a 500 contract volume filter.

Looks very nice. Note the few trades in the 90’s due to low liquidity, but the increase in liquidity after 2000. An attractive strategy to add to the portfolio. I don’t know why the strategy was mostly flat throughout the 2010’s, but the natural gas market has gone through some huge changes in fundamentals over the past 2 decades in the USA.

I got this strategy idea from the Top Traders Unplugged podcast. Check out the CCRV ETF as well.

Conclusion:

The beauty of commodities is that it makes it easier to construct uncorrelated trading strategies. You can trade all the markets in the same way, as with a Trend Following approach, and you can also profit from the nuances in each market, with strategies such as the ones above.

I have given you one of my strategies for free, to capture your attention. In future posts I may put these insights behind a paywall, but for now…. let’s see how it goes. Lemme know what you think!

Disclaimer: This content is for educational purposes only and is not investment advice. Do your own homework.

I am currently researching and deploying trading strategies on the Futures, Equities and cryptocurrency space, with the Futures strategies as the core of my portfolio.

The strategies (both deployed and under research) as of March 2024 are divided into:

• Futures:

  • [Active] Trend Following

  • [Development] Carry

  • [Active] Seasonality

  • [Active] Single-Market intraday strategies

• Equities:

  • [Development] Momentum

  • [Development] Short Term Mean Reversion Long/Short

  • [Active] Volatility

• CryptoCurrency:

  • [Early Development] Trend Following

All using linear instruments (futures, equities, spot crypto) and profiting from well-known return factors.

I use these data providers for End-Of-Day data:

• Futures end-of-day: CSI Data. Very reliable, high data quality and extensive coverage futures markets across the globe.

• Equities end-of-day: Norgate Data. Best in class data provider for equities, including delisted stocks. Tightly integrated with RealTest portfolio simulation software.

• Futures intra-day: Tradestation. I code my single market intra-day strategies in TradeStation, as it provides an all-inclusive package (historical data, strategy research, execution and broker all in one), and removes the complexity of having to integrate multiple tools and data sources.

My core broker is Interactive Brokers because it’s reliable, low-cost, high reputation, and has the widest market coverage. Market coverage is very important for me since I’m targeting more exotic, less liquid markets which the bigger players don’t trade, to get the “small trader advantage” in my favor.

The second broker I like is Tradestation, which as I explain above, removes most of the complexity involved in developing and executing intra-day strategies. And it provides long history of intraday futures data which can be used for simulations within the platform using its scripting language “Easy Language”.

For end-of day strategies I’m using R as my core language for strategy development, because the TidyVerse package makes it much easier for number crunching than other languages… and there’s quite a lot of number crunching required. Easier to code and read, and more concise than Python’s Pandas and similar (prove me wrong). But for interfacing with other software (Telegram for trading notifications, and Interactive Brokers API for execution) I use Python. Namely the python-telegram-bot and ib-insync libraries. But, to make life simpler and have a common look & feel, I wrap the Python within my R code using R’s reticulate package.

Equity strategy simulation has a lot of nuances and requires significantly more data due to the huge universe of tradeable stocks. I use the RealTest software for equity simulations, because it integrates seamlessly with Norgate Data, is easy to use, simulates portfolios of strategies and; most of all, because it has a great community forum. I read that lots of long-term Amibroker users are migrating to RealTest, so had to give it a shot, and got impressed.

For end-of-day strategies, I made a simple telegram bot which informs me anytime my strategies run or send orders to the broker. And I also configured the brokers to send me SMS when orders are executed.

Note I’m not endorsing or getting commissions from any of these trading technology providers (if/when I do I’ll tell you). I use them heavily today hence are the only ones I can talk about confidently today.

The CSI Data, Norgate Data, RealTest and Tradestation softwares run on Windows, so they need a PC to be constantly online. I use a cloud Virtual Private Server (VPS) with Windows to run these tools continuously.

There are lots of different technology stack combinations possible. For Equities, probably Norgate + RealTest offers the shortest learning curve. For portfolio simulations in futures, building everything from scratch with R or python like I did would take lots of time & effort, but nowadays there is a huge amount of code and libraries in GitHub to get started. Plus there’s ChatGPT… a game changer.

Looks like there’s pleeenty to talk about in other posts. Let’s talk about trading strategy development process on the next ones.

Tell me… how does your tech stack look like? Why do you set it up that way? I answer to comments ;-).

Hi, I’m Carlos Mata. I am an engineer with a busy schedule, but with a passion for markets and trading. This publication is a journal of the progress of my trading strategy development and implementation. Each post offers insights into strategy development thought process, and shares ideas that you may find useful for your own trading journey.

I’m also sharing the portfolio performance and positions, to build a public track record. Who knows… It may be useful in the future!. At least it’ll help to stay on track through the ups & downs.

This is the performance since the last strategy update. Let’s take this as a starting point and see where it gets to in 3,5, 10 years and beyond!.

Subscribe now