What does technical analysis measure? A view through conditional distributions

The previous note showed that returns have almost no linear autocorrelation, so that for the direction of prices the random walk hypothesis holds up well as a first approximation. Yet technical analysis — reading what comes next from past movements in price and volume — has been used in financial markets for over a century (1). This note does not ask whether technical analysis “works”. It asks what, restated in statistical terms, technical analysis is actually measuring, and answers from the classic studies.

Two vocabularies, one claim

In their textbook on financial econometrics, John Campbell, Andrew Lo and Craig MacKinlay set two statements side by side (2). One is in the language of technical analysis: clearly identified support and resistance levels, together with a one-third retracement when prices lie between them, point to strong buying and selling opportunities in the near term. The other is in the language of econometrics: the size and decay pattern of the first twelve autocorrelations, and the significance of the Box–Pierce Q-statistic, point to a short-horizon predictable component in stock returns. Lo, Harry Mamaysky and Jiang Wang cite the pair to observe that both say the same thing — that past prices contain information for predicting future returns — and yet most readers find one plausible and the other puzzling (3).

Strip away the vocabulary, and for the claims of technical analysis to be true, at least the following must hold. Let S be some quantity computed from prices (and volume) observable up to time t. When S takes a particular value, the distribution of the subsequent return r differs from its distribution when S is unknown. In symbols,

p(r | S) ≠ p(r)

A moving-average crossover and a chart pattern such as head-and-shoulders are simply different ways of constructing S. Each of the studies below tests some part of this inequality.

The inequality is a necessary condition, not the claim itself. It says only that the distribution differs — not that trading on it makes money. The distinction is taken up again at the end.

Linear prediction, and what lies outside it

Forecasting the future from past values has an established theory. For weakly stationary processes, Wiener–Kolmogorov prediction theory gives the linear combination of past values that minimises the mean squared error. In a 1991 paper, Salih Neftci asked why market participants prefer informal technical rules to this optimal linear forecast. He then examined whether technical rules can be written down as formal algorithms, and under what conditions they might capture properties of prices that linear models leave unexploited (4).

Framing the question this way makes one thing clear. If returns form a weakly stationary Gaussian process — one whose joint distribution across time points, not merely at each point, is normal — then the conditional expectation given the past is a linear combination of past values (or its limit, when the infinite past is used), and coincides with the optimal linear forecast. The whole conditional distribution is then normal, centred on the linear forecast with constant variance. No non-linear function of past prices, however elaborate, improves either the forecast of the mean or the forecast of the spread. So if a technical rule adds anything to the optimal linear forecast, it can only be because the linear forecast errors, while uncorrelated with one another, are not independent of the past — that is, because there is non-linear dependence. Departure from the normal distribution alone is not enough: if the linear forecast errors themselves are drawn independently of the past from the same distribution, the conditional expectation given the past still coincides with the linear forecast even if that distribution is not normal. Time variation in the size of moves is a typical form this non-linear dependence takes.

As the previous note showed, actual returns are “almost uncorrelated, but not independent”. In particular, large moves cluster together, and the autocorrelation of absolute returns is positive and decays only slowly (5). Since dependence that does not show up in linear autocorrelation does exist, there is room in principle for non-linear rules to pick something up. The questions are whether tests can see it, and, if they can, what exactly they are seeing.

Testing moving averages: Brock, Lakonishok and LeBaron (1992)

William Brock, Josef Lakonishok and Blake LeBaron applied two of the simplest and most widely used families of rules — moving-average rules and trading-range breaks (breaks above or below the recent high or low) — to daily data on the Dow Jones Industrial Average from 1897 to 1986 (6). Together these made 26 rules (1).

The key to the method is the bootstrap. Brock and his co-authors fitted four models to the data as null hypotheses — a random walk, an AR(1), a GARCH-M and an EGARCH — and generated many artificial price series by resampling the returns themselves in the random-walk case and the residuals in the other three, then applied the same rules to them (6, 1). Significance is judged by where the result on the actual data falls within the distribution of results on the artificial series. The strength of the design is that, by swapping the null model, one can ask whether what a rule captures is structure the model already explains. GARCH-M, for instance, builds in the idea that the size of moves varies over time and feeds back into average returns.

They reported that returns after buy signals and after sell signals differed both in mean and in volatility, that the periods after buy signals were less volatile, and that none of the four null models could reproduce these differences (6). Put another way, the differences in the mean and volatility of returns following the signals were not reproduced even by models that explicitly incorporate time-varying volatility.

The authors themselves acknowledged the study’s weak point. Moving-average rules can be designed in endless variety, and some will, without doubt, “work”; the dangers of data snooping are great. And they could not compute a single test across all their rules that took account of the dependence between the rules’ results (1).

The test begins when the rules are chosen

Data snooping occurs when the same set of data is used more than once for inference or model selection (1). In 1970, Michael Jensen and George Benington put the problem this way: given enough computer time, one can surely find a mechanical trading rule that “works” on a table of random numbers — provided one may test the rule on the same table that was used to discover it (7).

The bias need not come from any single researcher. As Ryan Sullivan, Allan Timmermann and Halbert White point out, over many years investors and researchers try countless rules; those that happen to have done well in the past attract attention and are passed on. Testing only the surviving rules mistakes luck for merit (1).

To correct for this, White proposed a procedure known as the “Reality Check” (8). Rather than testing the best surviving rule on its own, it places that rule within the full universe of rules from which it was drawn, and tests the null hypothesis that even the best of them is no better than the benchmark, by bootstrapping the distribution of the maximum statistic.

Sullivan and his co-authors applied the procedure to a universe of nearly 8,000 rules that included the 26 used by Brock and his co-authors. They used the same Dow Jones data, extended to cover 1897 to 1996 (1). The results split in two.

• For the period Brock and his co-authors studied (1897–1986), the null hypothesis that even the best rule in the universe was no better than the benchmark (holding cash) was rejected, even after correcting for data snooping. The Reality Check decides whether at least one rule in the universe outperforms the benchmark; it does not identify which. The evaluation used both mean returns and the Sharpe ratio, which adjusts for total risk.
• For the following ten years (1987–1996), the null could not be rejected at conventional significance levels, even for the best rule. To limit the influence of transaction costs and short-sale constraints, Sullivan and his co-authors also tested the rules on S&P 500 index futures (over the 13 years from 1984 to 1996), and found no evidence there that the rules beat the benchmark.

What this shows is that the differences seen in the 1897–1986 sample could not be fully explained by the choice of rules, but were not confirmed outside that sample. Whether the differences faded over time or were peculiar to that sample from the start, these results alone cannot tell.

Testing patterns: Lo, Mamaysky and Wang (2000)

A moving average can be written as a formula; a chart pattern such as head-and-shoulders looks different to different observers. Lo, Mamaysky and Wang identify this subjectivity as one of the main reasons technical analysis has received so little academic scrutiny (3).

They tried to resolve it with non-parametric kernel regression. Prices are treated as the sum of a smooth non-linear function and noise; kernel regression averages out the noise to estimate the smooth curve, and patterns are then defined mechanically by the sequence of local maxima and minima on that curve. A head-and-shoulders, for example, becomes a run of five extrema: maximum, minimum, a higher maximum, minimum, maximum. In this way they defined ten patterns — five types (head-and-shoulders, broadening, triangle, rectangle and double), each with a top and a bottom form (for head-and-shoulders, the inverse form) — and detected them in 38-trading-day windows rolled forward one day at a time (3).

The design of the test differs from that of Brock and his co-authors. Lo and his co-authors noted that a test of trading profits cannot, without a fully specified equilibrium asset-pricing model, tell whether those profits are anything more than compensation for bearing risk. They posed a more basic question instead: does the distribution of returns after a pattern appears differ from the unconditional distribution of returns (3)? It is fair to call this a test of the opening inequality itself.

• Data: from the US CRSP database, NYSE/AMEX stocks and Nasdaq stocks taken separately, with 1962–1996 divided into seven five-year periods. In each period, 10 stocks were drawn at random from each of five market-capitalisation quintiles.
• What was compared: the distribution of the one-day return starting three days after a pattern is completed, and the unconditional distribution of one-day returns. Returns were normalised for each stock by subtracting the mean and dividing by the standard deviation.
• Tests: a goodness-of-fit (chi-squared) test based on deciles, and the Kolmogorov–Smirnov test.

Under the Kolmogorov–Smirnov test, the hypothesis that the two distributions are equal was rejected at the 5% level for five of the ten patterns among NYSE/AMEX stocks, and for all ten among Nasdaq stocks. From this the authors conclude that several patterns do provide incremental information, especially for Nasdaq stocks. They state at the same time that this does not necessarily imply that trading on the patterns is profitable, and that patterns best suited to detecting statistical anomalies need not be best suited to trading (3).

This study, too, has limits. The sampling distributions of both test statistics are derived under the assumption that returns are independent and identically distributed (i.i.d.). The authors themselves acknowledge that this assumption is not plausible for financial data, and that normalising each stock’s returns does not remove the dependence or heterogeneity (3). The clustering of large moves seen earlier is exactly one such form of dependence.

Dependence affects not only the test statistics but how the results are read. The Kolmogorov–Smirnov test responds not only to a difference in the location (mean) of the distributions but also to a difference in their spread (volatility). Patterns are defined by the shape of the preceding price movements, so in a market where large moves cluster, it would be no surprise if volatility after a pattern differed from volatility overall. In that case, a rejection would reflect information about the size of moves, not necessarily about their direction. The authors do check, by Monte Carlo resampling, how well the asymptotic distribution approximates the test statistic; but because that exercise redraws returns independently, it does not address the dependence problem (3). And while Lo and his co-authors did not choose the ten patterns by looking at this data, the patterns have been handed down through long practice, so they are not outside the “surviving rules” problem discussed in the previous section.

What is being measured

Set side by side, the studies look like this.

Study	Conditioned on	Compared with	What remains open
Brock et al. (1992)	26 moving-average and trading-range-break rules	Artificial series generated from four null models	No single test across all the rules
Sullivan et al. (1999)	Nearly 8,000 rules	The benchmark of holding cash (judged against the distribution of the best result across all rules: the Reality Check)	The correction covers only the rules included in the universe; no difference was seen outside the sample
Lo et al. (2000)	Ten patterns defined by kernel regression	The unconditional distribution of returns	The tests assume i.i.d. returns; rejections may reflect differences in volatility

On this basis, “what does technical analysis measure?” can be answered as follows.

1. What is measured is the distribution of subsequent returns, conditioned on a function of past prices. Volatility is in scope as well as the mean. In the results of Brock and his co-authors, the differences appeared in both.
2. Whether there is “a difference” depends on what it is compared with. A difference that is significant against a random walk need not be significant against a model that builds in time-varying volatility. That is why Brock and his co-authors lined up four null models, and why the i.i.d. assumption is a limit on the tests used by Lo and his co-authors.
3. The choice of rules is itself part of the test. A rule picked from thousands of candidates looks good simply because it was picked. Once that is corrected for, there is at least one case where the difference disappeared outside the sample.
4. A difference in distributions and profit from trading are separate questions. Lo and his co-authors say so themselves. And as the survey by Park and Irwin points out, many studies that tested trading profits are subject to problems in their testing procedures: data snooping, ex post selection of trading rules or search methods, and the difficulty of estimating risk and transaction costs (9).

Restated as a statistical question, the claim of technical analysis becomes one form of the question of whether there is dependence between past prices and future returns — both dependence that shows up as linear autocorrelation and dependence that does not. What was it conditioned on, what was it compared with, and how many candidates was it chosen from? These three checks apply not only to technical analysis but to any study claiming to have found regularity in past prices.

---

This series explains research in quantum finance and econophysics from the published literature. It offers no view on any particular market or asset.