NOTE: Advanced topic. Make sure to read previous parts of the tutorial first. In order to interpret properly Monte Carlo simulation results you need to read this section of the manual. Non-trivial settings and non-obvious details are explained below. Please don't skip it.
Introduction
Generally speaking "Monte Carlo" methods represent broad class of computer algorithms that use repeated random sampling to obtain statistical properties of given process. It was invented by Polish mathematican Stanislaw Ulam working on nuclear weapons projects at the Los Alamos lab. As he was unable to analyse complex physical processes using conventional mathematical methods, he thought that he could set up a series of random experiments, observe the outcomes and use them to derive statistical properties of the process.
More on Monte Carlo methods in general can be found here: https://en.wikipedia.org/wiki/Monte_Carlo_method
In trading system development, Monte Carlo simulation refers to process of using randomized simulated trade sequences to evaluate statistical properties of a trading system.
There are many ways to perform actual computations that differ when it comes to implementation details, but probably the most straightforward and reliable is bootstraping method that performs random sampling with replacement of actual trade list generated by the back-test.
See https://en.wikipedia.org/wiki/Bootstrapping_(statistics) for detailed discussion of bootstrapping method.
Various Monte Carlo simulation methods allow to verify robustness of the trading system, find out probability of ruin and many other statistical properties of the trading system.
How does it work in AmiBroker?
In order to perform Monte Carlo simulation (or bootstrap test) of your trading system, AmiBroker performs the following:
A. Creating input set
A.1 Perform back-testing of your trading system to produce original set of N trades
B. Repeatedly (1000+ times)
B.1 pick randomly trades from the original trade list to produce new, random set of N trades (called 'realization')
This random set contains the same number of trades, they are ordered randomly and some original trades may be skipped and some used more than once (permutation with repetition, or random sampling with replacement).
Since number of unique realizations is N^N (so with just 100 input trades we have 100100 unique realizations), with sufficient number of trades (>100) the probability of picking identical sequence as original is virtually zero.
B.2 sequentially perform gain/loss calculation for each randomly picked trade, using position sizing defined by the user to produce system equity
B.3 record system equity in the distribution
C. Post-process
C.1 Process data obtained in B to generate distribution statistics and charts
All of the above happens when you press Backtest button in the New Analysis window. AmiBroker's Monte Carlo simulator is so fast that it usually costs just a fraction of second on top of normal backtest procedure.
It should be well noted that simulated trades during bootstrap are performed sequentially. If your original trading system traded multiple positions at once (so some or all of the trades are overlapped) it may result in smaller system drawdowns being reported by bootstrap test, because drawdowns from individual trades would occur sequentially (not in parallel as with overlapping trades).
Settings
The way how Monte Carlo simulator works can be controlled from the Analysis Settings page, "Monte Carlo" tab:
Enable Monte Carlo simulation
this check box controls whenever MC simulation is performed automatically as a part of backtest (right after backtest generates trade list)
Number of runs
defines the number of MC simulations to run (should be 1000 or more)
Simulate using portfolio equity changes
this option causes that MC simulation uses bar-by-bar portfolio equity percent changes instead of individual trades. Those individual equity changes are randomly picked and permutted to create simulation run. In this mode bar-by-bar equity changes are computed as ratio (so 10% increase is represented as 1.1), selected randomly and multiplied cumulatively. This setting allows to handle situations when you have multiple overlapping trades in your system and does not require any special setting for position sizing.
Simulate using trade list
this option causes that MC simulation uses individual trades from the original backtest to create simulation run. To perform simulation in this mode MC simulator randomly picks original trades and applies new position sizing as defined below. This mode is useful in cases when you don't have overlapping trades.
Position sizing
defines position sizing method used by MC simulator in "trade list" mode:
Don't change - uses original position size as used during backtest. Keep in mind that it always uses original dollar value of the trade (or whatever currency you use), even if your formula is using percent of portfolio equity.
Fixed size - uses fixed number of shares/contracts per trade
Constant value - uses fixed dollar amount for opening any trade
Percent of equity - uses defined percent of current simulated equity value. Be careful when using this setting - it causes that position size of one trade depends on profits on previous trades (compounding profits) and creates serial dependence. It may also lead to extra compounding effect when you have overlapping trades in your original backtest as bootstrap performs trades sequentially (so they don't overlap). For this reason its use is limited to cases when no overlapping trades occur.
Enable MC equity curves (Min/Max/Avg)
turns on MC equity charts (including highest, lowest and average equity plots
plus straw broom equity charts). Note that green and red lines (min/max equity)
are not really single "best" and "worst" equities.
They are bar-by-bar highest (max) and lowest( min) points of ALL equities generated
during MC.
So they are actually best points from all equities and worst points from all
equities. And blue line (avg) is the average from all equity lines (all
runs).
Show absolute values in linear scale - displays equities in absolute dollar values using linear scale chart
Show absolute values in logarithmic scale - displays equities in absolute dollar values using semi-log chart
Show Percent change - displays equities as "rate of change" since the beginning
Straw broom chart plots - defines how many individual test equites should be plotted as 'straw broom chart' (large number may slow down processing/drawing)
Use logarithmic scale for Final Equity
Displays final equity CDF chart using semi-log scale instead of linear
Use logarithmic scale for $Drawdown
Displays dollar drawdown CDF chart using semi-log scale instead of linear
Use negative numbers for Drawdown (reverse Drawdown CDF)
When this option is turned on, both dollar and percent drawdown are reported as negative numbers. This has also effect on CDF distribution. It reverses the ordering of "drawdown" column in the MC table and reverses the meaning (i.e. 10% percentile value means that there is 10% chance of drawdowns being equal or worse (more negative) than presented amount. With this option turned off (as in old versions), drawdowns are reported as numbers greater than zero (positive) and 10% percentile value means 10% chance of drawdowns being equal or better (smaller) than presented amount.
Best practices
To remove risks of serial correlation affecting the results of Monte Carlo simulation it is highly encouraged to use fixed position sizing (either fixed dollar value of trades or fixed number of shares/contracts), so the order in which given trade occurs in original sequence does not affect its profit/loss due to compounding.
Also depending on whenever your system opens multiple overlapping positions, choose the simulation method as follows
The results of Monte Carlo simulation are displayed in the "Monte Carlo" page of Backtest report.
At the top of the page we can see a table that gives values of few key statistics derived from the cumulative distribution charts (CDFs) of Monte Carlo simulation results.
Here are sample results (highlights are added manually for the purpose of illustration). Starting equity was 10000 in this example. Test was done over 7 years (EOD data).
| Percentile | Final Equity | Annual Return | Max. Drawdown $ | Max. Drawdown % | Lowest Eq. |
|---|---|---|---|---|---|
| 1% | 5706 | -7.37% | 1302 | 7.23% | 3618 |
| 5% | 7987 | -3.02% | 1549 | 9.76% | 5853 |
| 10% | 9706 | -0.41% | 1726 | 11.32% | 6690 |
| 25% | 12851 | 3.48% | 2136 | 14.38% | 8107 |
| 50% | 16174 | 6.78% | 2747 | 19.77% | 9135 |
| 75% | 19632 | 9.64% | 3563 | 27.63% | 9640 |
| 90% | 23258 | 12.21% | 4626 | 38.48% | 9922 |
| 95% | 25269 | 13.48% | 5292 | 45.47% | 10000 |
| 99% | 29139 | 15.71% | 7685 | 63.82% | 10000 |
First column shows percentile level (the value below which a given percentage of test observations (realizations) fall). So say 10th percentile tells us that 10% of time observed value is below shown amount. For example, the annual return value at 10th pecentile (in this case -0.41%) means that 10% of tests (realizations) had annual profit less or equal than shown (-0.41%). So we can say that there is about 10% chance that our system would not make any money (would not breakeven). A max. drawdown figure at 90th percentile (38.48%) means that in 90% of cases drawdown will be less than 38.48%. So in other words, we can say that there is 10% of chance that it will be higher than that. If we look further in the table we can also notice that in 99% of cases drawdown will be less than 63.82%. It is important to note that the table above is generated with "Use negative numbers for Drawdown" turned OFF.
If we turn ON "Use negative numbers for Drawdown" option, all drawdown numbers will become negative, and the order would be reversed and the meaning of drawdown column would be reversed too, like in the table below:
| Percentile | Final Equity
|
Annual Return
|
Max. Drawdown $
|
Max. Drawdown %
|
Lowest Eq.
|
|---|---|---|---|---|---|
| 1% | 5706
|
-7.37%
|
-7685
|
-63.82%
|
3618
|
| 5% | 7987
|
-3.02%
|
-5292
|
-45.47%
|
5853
|
| 10% | 9706
|
-0.41%
|
-4626
|
-38.48%
|
6690
|
| 25% | 12851
|
3.48%
|
-3563
|
-27.63%
|
8107
|
| 50% | 16174
|
6.78%
|
-2747
|
-19.77%
|
9135
|
| 75% | 19632
|
9.64%
|
-2136
|
-14.38%
|
9640
|
| 90% | 23258
|
12.21%
|
-1726
|
-11.32%
|
9922
|
| 95% | 25269
|
13.48%
|
-1549
|
-9.76%
|
10000
|
| 99% | 29139
|
15.71%
|
-1302
|
-7.23%
|
10000
|
This time the meaning of drawdown column is reverse - it tells you that the drawdowns would be worse (more negative) than specified amount, 99% percentile value of -7.23% means that in 99% of cases you will see drawdowns worse (more negative) than -7.23%. 1% percentile valuue of -63.82% tells you that in 1% of cases you would experience drawdowns equal or worse (more negative) than -63.82%
This way the table can be read "row-wise" and the top of the table (small percentiles) refer to "pessimistic" scenarios.
Below the table we can find min/avg/max + straw broom chart of simulated equities:
Note that green and red lines (min/max equity) are not really single "best" and "worst" equities. They are bar-by-bar highest (max) and lowest( min) points of ALL equities generated during MC. So they are actually best points from all equities and worst points from all equities. And blue line (avg) is the average from all equity lines (all runs). The 'cloud' of gray lines represents individual test equities - as we can see the same trading system may generate different outcomes when market conditions change and MC simulation attempts to simulate various outcomes and provide you some statistical information on how bad/good it may be.
After straw broom chart you can find cumulative distribution function (CDF) charts of final equity, CAR, drawdowns and lowest equity (again green and red annotation lines were added manually):
Cumulative distribution charts presents the same information that was included in the table at the top of "Monte Carlo" page but in the graphical form. Again, when we take a look at annual profit % (CAR) distribution chart we can see that in approximately 10% of cases our system would not break even (produces negative CAR). We can also see that in approximately 35% of cases our CAR would be below 5%. Profits above 10% per year only occur in top 20% of tests.
All other charts in the MC page are constructed the same and you can read them using the same methodology.
Final equity chart shows the cumulative distribution function of final value of the equity (at the end of test period)
Annual return chart shows the cumulative distribution function of compound annual percentage return of the test
Max. Drawdown $ and Max. Drawdown % charts show the cumulative distribution function of drawdowns (maximum peak to valey dollar/percent distances) experienced during the test
Lowest Equity chart shows the cumulative distribution function of lowest equity ever experienced during the test
How to control it from the formula level?
In addition to using Settings dialog, you can control Monte Carlo simulator using SetOption() function. You can also retrieve those values using GetOption function.
SetOption("MCEnable", 0 ); // value == 0 disables MC simulation
SetOption("MCEnable", 1 ); // value == 1 enables MC only in portfolio
backtests (default)
SetOption( "MCEnable", 2 ); // value == 2 forces MC to be enabled
everywhere (in every mode including optimization - SLOW !)
Note that enabling MC in optimization is highly discouraged unless you actually
use MC metrics as optimization target via custom backtester
or otherwise use MC distributions in the optimization process.
Monte Carlo process is computationally costly and while few hundred milliseconds
added to one backtest don't matter much,
in case of optimizations when these are multipled by number of steps you can
easily increase optimization time by orders of magnitude.
So unless you REALLY need MC distribution as custom metric and optimization target,
do NOT enable MC in optimization.
SetOption("MCRuns", 1000 ); // define number of MC simulation runs (realizations)
Other MC parameters that can be set using SetOption and retrived using GetOption:
How to add custom metric based on MC test distribution(s) to the backtest report ?
In addition to built-in MC report, you can add your own custom metrics to the report using GetMonteCarloSim() method of the Backtester object and MonteCarloSim object that this function returns. If you are new to custom metrics, please consult "How to add custom metrics to backtester report" part of this manual first.
MonteCarloSim object has one function GetValue( "field", percentile ) that allows to access CDF values. Available "field" values are:
Now here is the sample code that presents how to add 30th percentile FinalEquity and CAR to the report:
SetOption( "MCEnable", True );
SetOption( "MCRuns", 1000 );
SetCustomBacktestProc( "" );
if( Status( "action" )
== actionPortfolio )
{
bo = GetBacktesterObject();
bo.Backtest(); // run default backtest procedure
// get access to Monte Carlo results
// note 1: it may be NULL
if MC is NOT enabled
// note 2: MC results are
available after Backtest() or PostProcess
// as MC simulation is done
in final phase of post processing
mc = bo.GetMonteCarloSim();
if( mc )
{
// get 30-th percentile of final equity and CAR distribution
bo.AddCustomMetric( "FinalEq30",
mc.GetValue( "FinalEquity", 30 )
);
bo.AddCustomMetric( "CAR30", mc.GetValue( "CAR", 30 )
);
// you can also combine MC stats with normal
stats
st = bo.GetPerformanceStats(0);
bo.AddCustomMetric( "CAR30/MDD", mc.GetValue( "CAR", 30 )
/ st.GetValue( "MaxSystemDrawdownPercent" )
);
}
}
Once custom metrics is added, it can be used as Optimization target (don't forget to change MCEnable to 2) and used in Walk Forward test process as objective function. To select custom metric as optimization target, you would need to type its name exactly as it appears in the AddCustomMetric call into "Optimization Target" field in the Settings dialog, Walk Forward page. This way you can run optimization / walk forward test that is directed by values of MC simulation distribution. So for example instead of using CAR/MDD you can use CAR30/MDD (30th percentile MC CAR divided by max. system drawdown).
How about Monte Carlo randomization instead of bootstrap test?
The Monte Carlo randomization is different than bootstrap test because it does not use actual (realized) trade list from the backtest but it attempts to use "all individual returns whenever they are realized or hyphotetical". For example when trading system is generating way more signals than we can actually trade due to limited buying power, then we have to choose which trades we would take and which we would skip. Normally this selection is a part of trading system and in AmiBroker PositionScore variable tells the backtester which positions are preferred and should be traded. In randomization test, instead of using some analytic/deterministic PositionScore, you use random one. If there are more signals to open positions than we could take, this process would lead to randomized trade picks. Now using Optimize() function and random PositionScore we can run thousands of such random picks to produce Monte Carlo randomization test:
step = Optimize( "step", 1, 1, 1000, 1 ); //
1000 backtests
// with random trade picks from the broad universe (make sure you run it on
large watch lists)
PositionScore = mtRandom();
Randomization test has one big disadvantage: can not be used in many cases.
When
system does not produce enough signals each bar there is not much (if any)
to choose
from.
Also, more importantly, MC randomization makes false assumption that all "trading
opportunities" (signals) are equal. In many cases they are not. Pretty often
our trading system has specific, deterministic
way to pick trades from many oppotunities by some sort of ranking/scoring.
When system is using a
score
(rank)
as a
core
component
of the system (rotational
systems
do that) -
if you replace analytic score of with random number you are just testing white
noise not the system.