Testing Popular Ideas · Lesson 9
The higher-timeframe filter
Trade short bars only when a longer bar's trend agrees: which bar, which trend, and the leak in a bar that hasn't closed.
Know the basics? Read the lesson notesBars of different lengths
The Opening-range breakout lesson traded on five-minute bars: candles, each holding the open, high, low and close of five minutes. Each day here is one session, 09:30 to 16:00. A bar can cover any length of time. A longer bar is built from the shorter ones inside it: the open of the first, the highest high, the lowest low, the close of the last. Traders call the longer bars the higher timeframe.
The prices are simulated: made up by a computer, not a real market. Simulated markets can't show whether an idea works in a real market; here they show what a rule does when there is nothing to find (markets built so past prices say nothing about which way the next move goes). Look at the same market on all four bar lengths.
One simulated market, four bar lengths
Each bar: a thin line from its low to its high, a box from its open to its close (empty if it closed higher). Every view ends at 16:00 on 2 Jan 2017; the weekly one at the last full week.
A daily bar holds the day's 78 five-minute bars; a weekly bar, five days of them, 390. The longer the bar, the fewer bars and the slower they change.
What counts as a trend?
The higher-timeframe filter: take a trade on short bars only in the direction a longer bar's trend points. "Trend" needs an exact definition. Three common ones, each read from the last longer bar that has closed:
- colour: up if that bar closed above its own open, down if below;
- above average: up if its close is above the average of the last 20 closes (a simple moving average, as in the Moving-average crosses lesson);
- averages: up if the average of the last 10 closes is above the average of the last 30.
Three bar lengths times three readings: 3 × 3 = 9 ways to read "the trend". Each is a version: the same idea with different settings (the How we test an idea lesson). On the day shown, at 10:00, they split.
Nine readings of the trend at 10:00 on 2 Jan 2017 (simulated)
Each read from the last bar that had closed by 10:00: the 30-minute bar from 09:30, the daily bar of 30 Dec, the weekly bar of the week before.
From 2016 to 2024, on how many simulated days did the three readings of one bar length all agree?
So "the trend" is not one thing. Which bar and which reading are choices, and each choice is another version to count.
The filter on one day
The rule comes from the Opening-range breakout lesson, with a filter added. The opening range is the high and low of 09:30 to 10:00. At 10:00 the filter reads the trend once. Up: an order waits at the range's high and buys when a bar reaches that price (the Orders lesson). Down: an order waits at the low and sells short; selling short means selling first and buying back later, so the trade gains when the price falls. A stop (an order that closes the trade at a set price) at the other side of the range ends a losing trade; otherwise the trade ends at 16:00.
Unfiltered, both orders wait and whichever price is reached first trades. Try no filter, then filters until you have seen one read up and one read down.
The rule on 2 Jan 2017, five-minute bars (simulated)
Shaded: the opening range. Dashed: the price the waiting order needs. Dot: the entry, ring: the exit, when the rule traded.
Unfiltered, the rule bought at 106.42 in the 10:15 bar. A filter that read down waited at the low, 105.99, which the price never reached: no trade. One waiting order is reached less often than two, so across 1,000 simulated markets the nine filters traded on 190 to 222 days a year, the unfiltered rule on 260.
At 10:00, today's bar isn't finished
Take the daily colour. A file of daily bars (the stored prices a test reads) has one finished row per date. A backtest (running a rule on past prices) that looks up "today's daily bar" at 10:00 gets the row with today's date, and that row holds today's 16:00 close: a price six hours in the future. A filter that reads today's unfinished bar partly reads how the day will end, and then trades in that direction. Move the clock and watch today's daily bar build.
Today's daily bar, so far and in the file, beside the finished bar of 30 Dec (simulated)
Left: five-minute bars up to the clock; grey: what hasn't happened yet. Right: 30 Dec's finished daily bar, what the honest reading uses; today's bar as it stands at the clock; and today's bar as the file holds it.
At 10:00, 6 of the session's 6.5 hours were still to come. The file's bar already knew the close, 106.43, so it read up; the finished bar, 30 Dec's, read down.
Reading a longer bar before it has closed is a kind of look-ahead bias (the Look-ahead bias lesson): a backtest using information that wasn't available at the moment of the decision. Call it the unfinished bar; the honest reading uses the last finished bar.
The leak, counted
Run the daily colour filter both ways on 1,000 simulated markets, from 2016 to 2021 (the first year, 2015, only builds the averages). Finished: yesterday's bar. Unfinished: today's, from the file.
On the finished reading, 40.6 trades in 100 gained, before costs (as if trading were free). Guess the unfinished reading's figure.
Trades that gained before costs, daily colour filter, 2016–2021
Every trade on 1,000 simulated markets combined. Dashed: your guess.
Unfinished, 61.6 trades in 100 gained, and the average trade (its gain or loss in % of the price) was +0.23% before costs, against −0.02% for the finished reading. The difference comes from today's bar, six hours of which had not happened at 10:00. With the finished reading fewer than half gained, yet the average stayed near zero: a gaining trade averaged +0.60%, a losing one −0.44%.
Before costs it even beat just holding (buying on the first day and never selling) in all 1,000 markets.
Every version, finished and unfinished
Now all nine versions, 2016 to 2021. The years 2022 to 2024 stay held out: set aside before any version ran, to be used once at the end (the Held-out test lesson). Each bar is a version's average trade before costs, over every trade on the 1,000 simulated markets. Look at both readings.
Average trade before costs, nine versions, 2016–2021 (simulated)
One bar per version, grouped by bar length; in each group C: colour, A: above average, Av: averages. Filled: the reading picked with the buttons; outline: the other one. Every trade on 1,000 simulated markets combined. Orange: 0.05% or more.
Finished, every version averaged between −0.020% and −0.017% a trade before costs. Unfinished, some leaked far more than others.
The leak is largest where the unknown part of the bar is the trade itself: today's bar ends at 16:00, with the trade. A weekly bar also holds later days, a 30-minute bar only the trade's first half hour, and in an average today's close is one of 10, 20 or 30 closes.
The honest nine against just holding
Back to the finished readings. Each buy and each sale costs 0.1% of the price (the Costs lesson); holding pays that once. A version's lead is its growth in % a year minus holding's, in points (a point is 1% a year). After costs, in how many of the 1,000 simulated markets did at least one of the nine beat holding?
Average lead over holding after costs, 2016–2021, 1,000 simulated markets
Grey: the nine filters, grouped by bar length. Black: the unfiltered rule.
In none. The filters averaged −42.4 to −47.1 points a year, the unfiltered rule −52.0. Each trade pays 0.2%, so each trading day keeps 0.998 of the account: 0.998 multiplied 190 times is 0.68, so costs alone take about a third of it a year.
The filters lost less because they traded on fewer days, so paid less in costs. Their average trade before costs was the same as the unfiltered rule's, −0.02%.
These markets were built so that past prices say nothing about which way the next move goes; they only rise slowly on average. So these counts show what the rules do when there is nothing to find, not whether a higher-timeframe filter works in any real market.
Does the best stay best?
On each market, take the best of the nine finished-bar versions after costs on 2016 to 2021 and call it the pick; it then runs once on the held-out years. A blind draw, a version picked at random from the nine, would be best there in about 111 of 1,000 (1,000 ÷ 9 = 111.1). Guess how often the pick stayed best.
The pick's place on the held-out years, 1,000 simulated markets, after costs
One bar per place, 1st on the left. Dashed: a blind draw.
The pick stayed best in 130 of 1,000 simulated markets, against a blind draw's 111, and beat holding there in none.
The two 30-minute versions trade most, so pay most in costs, and were rarely best, so the contest was mostly among seven: 130 is close to the 143 a seven-way draw gives (1,000 ÷ 7). That is not skill.
Read a longer bar only after it closes
A rule may use a higher-timeframe bar only once that bar has closed. Using it earlier is look-ahead bias; traders say such a reading repaints: a backtest shows it reading finished bars, while live it read the bar so far. In code, count the short bars from 0 where the long bars start.
- i
- the short bar that has just closed, counted from 0 at the start of the first long bar
- m
- how many short bars make one long bar
- j
- the last finished long bar, counted from 0, when every long bar holds exactly m short bars
- ⌊ ⌋
- round down to a whole number
Why it works: i + 1 short bars have closed; ÷ m gives the whole long bars closed; − 1 turns that count into a number counted from 0. j = −1 means none has finished yet.
Five-minute bars into 30-minute bars from 09:30: m = 6. At 10:00 the bar just closed is the 09:55 one, i = 5. j = ⌊6 ÷ 6⌋ − 1 = 0: the 09:30 to 10:00 bar. The leak reads ⌊6 ÷ 6⌋ = 1, the 10:00 to 10:30 bar, which hasn't started.
Into daily bars, m = 78. At 10:00 on day d (counted from 0), i = 78d + 5, so j = ⌊(78d + 6) ÷ 78⌋ − 1 = d − 1: yesterday.
- every long bar holds exactly m short bars;
- both counts start at the same moment;
- the rule decides at the close of short bar i.
Bars are uneven: 60-minute bars from 09:30 leave a 30-minute bar at 15:30, and a week with a holiday has four days. Then use end times: the last finished bar is the latest one that ended at or before the decision.
Five-minute bars go into 15-minute bars from 09:30. A rule decides at the close of five-minute bar i = 10 (10:20 to 10:25). Which 15-minute bar is the last finished, counted from 0?
What this means for your test
- Write the exact rule before you run it.
Which longer bar, which reading of trend, the moment the rule reads it, which bar counts as finished, and the costs.
- Read only finished bars, then check the average trade before costs.
On markets with nothing to find, the finished readings averaged −0.02% a trade; today's unfinished daily bar gave +0.23% and won 61.6 trades in 100 instead of 40.6.
- Count the versions, compare with holding after costs, hold out years.
9 here. After costs none beat holding in any of 1,000 simulated markets, and the best of 9 stayed best on held-out years in 130, against about 111 for a blind draw.
A filter that wins
What is the likeliest reason for the jump?
Which daily bar?
Which daily bar, counted from 0, is the last finished one?
Hint: m = 78. Work out ⌊(i + 1) ÷ m⌋, then take 1 away.
An honest reading
Which reading is honest at that moment?
Lesson done
A higher-timeframe filter takes a trade on short bars only in the direction a longer bar's trend points. Which bar (30-minute, daily or weekly) and which reading of trend made 9 versions here, and the three readings of one bar length agreed on fewer than half the simulated days. A longer bar may be read only after it closes: on markets with nothing to find, reading today's unfinished daily bar at 10:00 lifted the average trade before costs from −0.02% to +0.23%. These markets were built so that past prices say nothing about which way the next move goes, so the counts show what these rules produce when there is nothing to find: with finished bars and after costs, no version beat just holding in any of 1,000 simulated markets, and the best of 9 stayed best on held-out years in 130 (a blind draw: about 111). The counts cannot show whether a higher-timeframe filter works in any real market.
Key ideas
- Higher timeframe
- longer bars built from the shorter bars inside them: the open of the first, the highest high, the lowest low, the close of the last
- Higher-timeframe filter
- a rule on short bars that trades only in the direction a longer bar's trend points
- Trend reading
- an exact definition of "trend", such as a bar's colour, its close against an average, or a short average against a long one; each is a separate version
- Finished bar
- a bar that has closed; the only kind of longer bar a rule may read
- Unfinished bar
- a longer bar read before it closes; a backtest sees its final close, a kind of look-ahead bias (traders say the reading repaints)
- Version
- the same idea with different settings; count every one
Formulas
j = ⌊(i + 1) ÷ m⌋ − 1: the last finished long bar at the close of short bar i, when every long bar holds exactly m short bars and both counts start at the same moment