掘金社区

关于回测中常见的三类错误Pinned highlighted

逍遥游 发表在掘金2 2015-08-18 02:33:27

掘金2
1099
2
0

> 回测是完善一个交易策略的重要工作,但通常回测中会有三类错误导致你的回测带给你一个错误的结论,分别是 “后见之明的错误,偷窥了数据”、“太多的变量”、“市场急剧的变化”。
下文引自外汇交易系统中的一个总结,供参考。

Many successful traders share one habit – they backtest their trading strategies. Backtesting your trading strategy will not alone guarantee that you will become profitable, but it is a giant step in the right direction. In this article we examine some potential biases that can creep into your backtesting, and we will look at how to minimize the impact of these biases. There are many problems that can occur when you backtest your trading system, but most problems fall into one of three categories:
postdictive errors, too many variables, or failing to anticipate drastic changes in the market . Each of these errors is explained, along with methods of avoiding errors.

  1. Postdictive Error
    The postdictive error is just a fancy way of saying that you have used information only available “after the fact” to test your system. Believe it or not, this is a very common error when testing trading systems.
    This error is easy to make. Some software will allow you to use today’s data in testing a trading system, which is always a postdictive error (we do not know if today’s data is useful yet for predicting the future, but we certainly do know if it is useful in predicting the past). Wouldn’t you love to be able to use the closing price of the GBP/USD to predict what the market will do today? Of course you would, I definitely would, but unfortunately, this information is not available to us until the day is over. For example, you may have a system that incorporates the closing price, then this obviously means that the trade cannot be initiated until the day is over, otherwise this is a postdictive error. Another example may help illustrate the postdictive error, if you have a rule in your trading system about highest prices, then you will have a postdictive error.
    This is because highest prices are often defined by data that comes later, in the future.
    The way to avoid the postdictive error is to make sure that when you backtest a system that only information that is available in the past at that point in time is used in backtesting. With manual backtesting or backtesting with forex tester you can accomplish this quite easily, but with automated
    backtesting the postdictive error can sneak into your trading system.

  2. Too Many Variables
    This is also known as the “Degrees of Freedom” bias. This simply means that you have too many variables, or trading indicators in your trading system. It is very possible to come up with a trading system that can explain past price behavior of a currency pair. In fact, the more indicators you add, the easier it often becomes. The problem arrives when you want to apply this system to the future.
    Often when a trading system has too many indicators it can predict the behavior of the market during a time period extremely well. But, that’s all the system is good for, because in the future the system
    falls apart.
    The above statement is often difficult for traders to come to grips with, but it is true. Consider what William Eckhardt, of the New Market Wizards has to say about trading systems, In general, the delicate tests that statisticians use to squeeze significance out of marginal data have no place in trading. We need blunt statistical instruments, robust techniques.
    Obviously, he is warning against the degrees of freedom error and suggesting that simple trading systems are more likely to stand test of time. This is absolutely true.
    Some of the most powerful trading systems available are extremely simple.
    Keep this in mind as you trade, and as you attempt to find a profitable trading system. Most traders will find that with experience, they become more likely to embrace the view that simpler trading is preferred over a complex approach.

  3. Drastic Changes in the Market
    Many traders forget to anticipate unforeseen events that will occur in the future. It doesn’t really matter that you don’t know what is going to happen in the future – because you do know this:
    there will be times in the future when the markets will behave erratically. When this happens, you should have designed your trading system to remain functioning during these times.
    Perhaps some examples may help with this: When Saddam Hussein was found (over the weekend), the currency markets reacted quite drastically on Monday’s opening. When the global financial crisis started unfolding in September 2008, most currency pairs traded with much more volatility than had been seen for years.
    The fact is that there will be unexpected events in the future, and these events will affect the markets, so the best thing you can do is to be prepared. How do you prepare for the unexpected? Consider these simple solutions:

    1. Exaggerate your expected losses. If your backtesting reveals a maximum loss of $5000, assume a maximum loss of $10,000. Will your trading systems still be profitable under these conditions?
    1. Decide on an appropriate level of risk for each trade. Remember that even this level of risk is likely to be exceeded. If you have decided to risk 1% on each trade, you should assume that sometime in the future, you may be in a trade and an unexpected event will occur, and your trade will not lose 1%, but instead 5% will be lost.
    1. You should have a contingency plan set up. That is, how will you exit a trade if something bad happens and you cannot access your account? For instance, what happens if your trading platform is
      inaccessible and you desperately want out of a trade? Most brokers offer a telephone line to traders for these instances. Do you have the phone number?
    1. Do you have a maximum risk level set? This would be applicable if you have several trades open simultaneously. If you decide to risk 1% per trade and you have 7 trades open simultaneously, does this mean that you will be risking 7% of your account? Or have you decided on a maximum risk level of say, 3%? Keeping in mind that the unexpected will occur, you should probably have a maximum risk level for those times when you have several open trades.
    1. What is the maximum drawdown (amount of money your trading system loses over an extended period of time) you are willing to tolerate?
      Keeping in mind that you (and you are not alone) are more likely to overestimate the severity of drawdowns that you can withstand, it is important to be realistic. If you lose 30% of your account will you stop trading? What about if you lose 50%? Or if you see 70% of your account disappear? Again, the best way to plan for drawdowns is to do extensive backtesting to find out what sort of historical drawdowns your trading system experiences and then plan for even worse drawdowns in the future.
      Anticipating drastic changes in the markets is the single best way to preserve the equity in your account.
      So, you know that successful traders share this habit – they backtest their trading strategies. You know that backtesting separates the wealthy traders from those who lose money. You also know several ways of incorporating backtesting into your trading regimen. And you know of the pitfalls – what to look out for – when you are backtesting, so that you can get the most out of the process. But, what exactly, will you get out of backtesting your trading system? In the next article I will explore the side effects of backtesting.

>Walter Peters, PhD is a professional forex trader and money manager for a private forex fund. In addition, Walter is the co-founder of Fxjake.com, a resource for forex traders. Walter loves to hear from other traders, he can be reached by email at walter@fxjake.com.

评论: 2
  • 掘金产品中回测要注意的问题:

    1. 不要在数据回调函数 on_tick, on_bar中使用print, 或logger的打印, 会严重减慢回测的速度
    2. 在回测过程,以及实盘策略中,不要用查询历史数据,查询持仓,查询资金这类操作,掘金提供了专门的数据推送机制,方便你通过在策略初始化时,先做好查询,然后根据推送的数据更新这些信息。
      回测中,首要的目的是验证交易信号的准确度,其次才是通过仓位管理来检验收益数据,这两个部分建议最好是分开进行,以方便你进行策略的绩效归因工作。
    3. 回测前建议先明确回测的目的,因为历史数据是你有目的性地选择的一个时间段,在这个时间段中,行情走势有个大概的特征是你事先了解的,那么你选择这段数据来回测的目的是什么? 是为了测试模型的稳定性? 还是希望策略在这么一个特殊的行情中产生“完美”的收益曲线? 抑或是其他的什么目的,你需要预先分析一下,这样有助于你接下来的策略调优和分析改进。

    以上纯属个人观点,供参考。

    2015-08-18 02:50:41
  • > 关于回测的定义,最重要的一句话是: “过去的绩效并不能保证未来的收益”

    DEFINITION of 'Backtesting'

    The process of testing a trading strategy on prior time periods. Instead of applying a strategy for the time period forward, which could take years, a trader can do a simulation of his or her trading strategy on relevant past data in order to gauge the its effectiveness.
    Most technical-analysis strategies are tested with this approach.

    INVESTOPEDIA EXPLAINS 'Backtesting'

    When you backtest a theory, the results achieved are highly dependent on the movements of the tested period. Backtesting a theory assumes that what happens in the past will happen in the future, and this assumption can cause potential risks for the strategy.
    For example, say you want to test a strategy based on the notion that Internet IPOs outperform the overall market. If you were to test this strategy during the dotcom boom years in the late 90s, the strategy would outperform the market significantly. However, trying the same strategy after the bubble burst would result in dismal returns. As you'll frequently hear: "past performance does not necessarily guarantee future returns".

    Read more: http://www.investopedia.com/terms/b/backtesting.asp#ixzz3j8JKIgJh

    2015-08-18 03:22:23

Looks like your connection to 掘金量化社区 - 量化交易者的策略交流学习社区 was lost, please wait while we try to reconnect.