Example of decoupling strategy and backtesting

[systats]

Hi guys,

very nice package! I am planning in comparing the most prominent open source backtesting packages and wonder whether it is possible to use only your backtest tool without strategies? Do you have an example how to input something like timestamps (minute), close (price) and action (buy/sell) and return a backtest dataframe?

Thanks for your help!

[jaymon0703]

Hi Simon, thanks. @braverock and the rest of the crew who made this code open source did the R and Trading community a massive service.

To your specific question, the package is mainly for backtesting and research (although the package can be integrated with a live trading system). What you backtest is a strategy, stored as a strategy object. Its possible to perform a backtest without quanstrat, using just blotter. See this demo for an example. There are multiple quantstrat demos which you can use to benchmark with.

In terms of viewing the results, there are several options. You can use perTradeStats(), DailyStats() or tradeStats(). There is pretty good documentation for all functions.

The below example uses the maCross demo, which uses AAPL daily bar data. If you have bars with 1-min frequencies, you could use that instead.

require(quantstrat)

demo("maCross")

perTradeStats('macross')
dailyStats('macross')
tradeStats('macross')

Not sure if this helps or not.

Regards
Jasen

[braverock]

Do you have an example how to input something like timestamps (minute), close (price) and action (buy/sell) and return a backtest dataframe?

As @jaymon0703 already pointed out, quantstrat operates on a strategy object to run the backtest. You can just generate treades in blotter and get the P&L of a series of transactions, but I think you wanted the extra backtesting scaffolding.

Here is a minimal example of some random 1-minute 'Close' data and random entry and exit signals added as columns to the data. You would of course replace these with your own signals and close data.

The strategy in this case just consists of two rules, one for entries and one for exits, to tell quantstrat what to do when you get a signal.

require(quantstrat)

strat.st <- "FAKESTRAT"
nsamples <- 30
len<- 600

rm.strat(strat.st)

# generate random 1-min returns from a Student's t distribution, and add drift
rnd.rets<-xts(((rt(len,6)/200)+.00024),order.by=as.POSIXct(x=60*1:len, origin='2021-07-01'))
# turn that into a price series
rnd.price<-100*exp(cumsum(rnd.rets))
colnames(rnd.price)<-'Close'
FAKE<-rnd.price

# now generate some random signal columns

FAKE$long<-0
FAKE$long[sample(1:nrow(FAKE),size = nsamples, replace=FALSE)]<-1

FAKE$short<-0
FAKE$short[sample(1:nrow(FAKE),size = nsamples, replace=FALSE)]<-1

currency("USD")
stock("FAKE",currency = "USD")

initPortf(strat.st, symbols="FAKE")
initEq<-100000
initAcct(strat.st, portfolios=strat.st, initEq=initEq)
initOrders(portfolio=strat.st)

strategy(name=strat.st,store=TRUE)

add.rule(strat.st,"ruleSignal", 
         arguments=list(sigcol="long",
                        sigval=TRUE,
                        orderqty=100,
                        ordertype='market', 
                        orderside='long'),
         type="enter",
         label="enter"
        )

add.rule(strat.st,"ruleSignal", 
         arguments=list(sigcol="short",
                        sigval=TRUE,
                        orderqty="all",
                        ordertype='market', 
                        orderside='long'),
         type="exit",
         label="exit"
)

out<-applyStrategy(strat.st , portfolios=strat.st, verbose=TRUE)

updatePortf(strat.st)
chart.Posn(strat.st)

[systats]

Wonderful! I would not have had the time to get into that depth. But this will help a lot when comparing frameworks. I will let you know about the results as soon there is something to report. Cheers guys!

[jaymon0703]

Looking forward to the report.

[systats]

Hi - just a small question: Is it possible to get statistics in percent instead of absolute value? For example, if i want to know what is the equity and max_drawdown in percent without compounding? Compound is of course very important but also misleading due to increased leveraging (higher weight per trade over time). Any ideas how to compare to a fixed size investment amount per trade?

$ portfolio          <fct> bt
$ symbol             <fct> DATA
$ num_txns           <dbl> 305
$ num_trades         <int> 152
$ net_trading_pl     <dbl> 3096991
$ avg_trade_pl       <dbl> 21041.8
$ med_trade_pl       <dbl> -4689.5
$ largest_winner     <dbl> 495243
$ largest_loser      <dbl> -303736
$ gross_profits      <dbl> 6181005
$ gross_losses       <dbl> -2982651
$ std_dev_trade_pl   <dbl> 104897.4
$ std_err_trade_pl   <dbl> 8508.302
$ percent_positive   <dbl> 42.76316
$ percent_negative   <dbl> 57.23684
$ profit_factor      <dbl> 2.072319
$ avg_win_trade      <dbl> 95092.38
$ med_win_trade      <dbl> 48186
$ avg_losing_trade   <dbl> -34283.34
$ med_losing_trade   <dbl> -17164
$ avg_daily_pl       <dbl> 21041.8
$ med_daily_pl       <dbl> -4689.5
$ std_dev_daily_pl   <dbl> 104897.4
$ std_err_daily_pl   <dbl> 8508.302
$ ann_sharpe         <dbl> 3.184333
$ max_drawdown       <dbl> -1027792
$ profit_to_max_draw <dbl> 3.013247
$ avg_win_loss_ratio <dbl> 2.77372
$ med_win_loss_ratio <dbl> 2.807388
$ max_equity         <dbl> 3619611
$ min_equity         <dbl> -9747
$ end_equity         <dbl> 3096991

I really like your event driven backtest and thanks for your help!

[jaymon0703]

The short answer is no. You can of course divide the End.Equity and Max.Drawdown statistics by initEq.

If you have a custom order sizing function which commits a fixed number of dollars to a trade, then it would be possible to back out the stats in percent terms. If not, then it would be more work.

Re your last point, technically quantstrat is a vectorized backtest. I think you are referring to the path-dependent nature of quantstrat...which is essential as that is the real-world in production.

[systats]

Hi guys,

just some more information. Now I got 4 packages and we can sparsely compare the metrics. First of all I very happy that there is already a lot of constituency. Some questions remain though:

1.) The biggest difference so far is drawdown. Any ideas why this metric is all over the place?
2.) Its seems like quantstrat has 1 trade less than the others (could also be a bug from the data input)
3.) Is there any chance that you can expand the metrics to include for example time_exposure, drawdown_mean, win_sd or loss_sd? I will do the same for backer.

        var         backer     quantstrat  vectorbt      PA
1  equity_last        307.08     297.04        -       -
2   equity_cum       1231.55          -  1231.55 1231.55
3   equity_max        310.81     311.32        -       -
4    return_sd          8.21       8.27        -    8.21
5  return_mean          2.01       2.07        -    2.01
6     time_exp          0.41          -     0.41       -
7    down_mean         -5.28          -    -1.05   -8.84
8     down_max        -19.76     -43.08    -30.2  -18.52
9    win_ratio          2.20       2.97        -       -
10    win_rate          0.44       0.43     0.44       -
11    win_mean          8.41       8.81     2.01       -
12     win_max         43.56      44.22    43.61       -
13      win_sd          8.68          -        -    8.68
14     win_sum        563.62     572.37        -       -
15   loss_mean         -2.98      -2.96        -       -
16    loss_sum       -256.54    -257.94        -       -
17     loss_sd          2.09          -        -    2.09
18    loss_max         -9.74      -9.87    -9.75       -
19     loss_05         -6.32          -        -   -5.67
20     sortino          0.96          -        -    0.74
21      calmar          0.38        6.9        -    7.62
22    n_trades        153.00        152      153       -

For equity_cum I would have to run quantstrat once with and without fixed position sizing. But the number is correct. Which is not the case for calmar and some few others. So I have also to fix some stuff. The following metrics are already in the dictionary in case you spot an issue.

equity_last = end_equity
equity_max = max_equity
equity_min = min_equity
return_mean = avg_trade_pl
return_med = med_trade_pl
return_sd = std_dev_daily_pl
win_max = largest_winner
win_sum = gross_profits
win_rate = percent_positive/100
win_ratio = profit_factor
win_mean = avg_win_trade
win_med = med_win_trade
win_ratio = avg_win_loss_ratio
loss_max = largest_loser
loss_sum = gross_losses
loss_mean = avg_losing_trade
loss_med = med_losing_trade
down_max = max_drawdown
calmar = profit_to_max_draw
n_trades = num_trades

Do you have an idea which framework to port next, how to achieve higher constituency or wehere to expand performance metrics?

[jaymon0703]

If you are looking to audit the statistics, you can step through the code in debug mode of course, or use a combination of perTradeStats and getTxns. To audit the signal process you can view the mktdata object. Re your specific questions:

1. We have got nothing to compare our drawdown metric with, nor even the code used to produce the quantstrat drawdown metric. The function tradeStats is from the blotter package. The code is here - https://github.com/braverock/blotter/blob/9914d676651b1289e2e4ac882eabd391015f53ea/R/tradeStats.R#L105 and the specific line for maxDD is here - https://github.com/braverock/blotter/blob/9914d676651b1289e2e4ac882eabd391015f53ea/R/tradeStats.R#L253.
2. Cannot comment without a minimum reproducible example. Just note that we make a distinction between a round turn trade (which is comprised of an opening transaction and a closing transaction or set of closing transactions) and a transaction.
3. That would be a feature request. Feel free to create an issue for it with some more info.

Not sure i understand your last question...are you asking if there is another backtesting package in R which you can compare with? If so, then no, unfortunately i do not.

[systats]

Okay perfect. I will look into blotter for the drawdown issue, the n_trade difference and if needed provide an repex. The feature request should be also for blotter and not for quantstrat?

[jaymon0703]

Yes, it would be for blotter. Thanks!