The accuracy of trading performance back testing

Even beginner trader is well aware how important is testing of trading strategy on historical price data to estimate its efficiency. Everybody who tried to write own strategies in Metastock® or Trade Station® was able to watch performance charts of initial capital growth throughout the period of strategy execution and also everybody observed typical statements: “past performance is not a guarantee or a reliable indication of future results” which typically accompany performance charts. One can wonder why at all to pay so much attention to quantities without any exact reliability. The simple answer tells: because there is nothing better to measure trading prospects.

People with basic knowledge of mathematics would surely remember to mention probability theory associated with stock market modeling and high degree of error associated with market trend forecasting. This serves as a root to numerous strategies of risk management, which seek to achieve the balance between profits from aggressive trades and associated losses in case when the market direction estimate is incorrect. However, in many cases such estimates appear also wrong.

There is simple and evident proof for such a high volatility of market forecasts. Market itself is extremely complex system and to date there is no proven theory to explain it in any finite model allowing for easy trend calculation. The most risk estimates suppose that market behaves as some black box system with associated plain probability measures such as moving averages and variances approximating error limits. Nevertheless, there is quite generic evidence that these criteria appear inadequate in most market conditions.

This phenomenon is due to multiplication effect of market influences. Consider long pipeline transporting raw oil. Suppose every part of pipeline can suffer accident with some known probability. It may seem that the ultimate probability of pipeline fault as a whole will appear as a simple sum of associated probabilities for every fragment of a pipeline. However, it is not the case because the fault in every pipeline segment would cause the fault of the pipeline as a whole. Therefore, cumulative probability would appear not as a sum but as a product of associated probabilities. In mathematical terms, it will mean that there will be severely violated the conditions of so known Central Limit Theorem, which defines the statistical variance as a reliable measure of probability distribution associated with the value. It reveals that the most risk management strategies implemented worldwide are not reliable.

Only few mathematicians with good background in statistics can understand and rigidly prove this point. However, one should not be mathematician to observe its real world manifestation in abrupt huge losses of trading institutions and world market instabilities, which, by far, are not the results of errors of some evil minded individuals, as often told to wide public, but are due to fundamental stochastic laws which rule the modern market.

There exist special mathematical techniques devised to diminish the influences of cause multiplication factor on the accuracy of ultimate statistical estimates. Known as nonparametric statistics, they are well described in many statistical textbooks. The simplest of such measures is median point of distribution, which acts as a reliable nonparametric replacement of the average value. Another measures concern special statistical hypothesis testing techniques aimed to estimate reliability in cases where regular variance looses its sense as illustrated above.

Aura Forecast Engine (http://stockfusion.net) is the special forecasting platform, which makes use of nonparametric statistics and other nonlinear techniques such as Friedman coefficient, Hurst coefficient, fractal dimension, K-Sample and Kruskal-Wallis tests and Shannon probability to make automatic expert choice of best market forecasting strategy appropriate to any given set of historical data. It allows making back testing alternatives and measuring the risk in different terms against common trading performance standard. Even despite past profitability chart may look impressive in term of immediate strategy goal, it appears not reliable in terms of estimating future trading performance, which greatly dissolves common risk estimates based on it. Much less known nonparametric alternatives can serve as better reliability guides to investors seeking the better weighted revenue indicators.