A strategy presentation mixes three kinds of numbers: simulated (backtested) results, results from accounts independent of the product, and the product's own live track record. Only the third is the instrument's performance; the first two are evidence about the approach, and evaluating them requires knowing exactly how each can mislead.
Every quantitative strategy arrives with an impressive chart. The honest question is never "how good is the chart?" but "which of these numbers would have been available to me, net of costs, if I had been invested?" This guide gives you the vocabulary and the checklist to answer it.
What a backtest is, and what it is not
A backtest applies a set of trading rules to historical data and reports what would have happened. Done rigorously, it is a legitimate research tool: it shows how a model behaves across regimes, drawdowns and crises that no live track record is long enough to contain.
General rule What it is not is a promise. Backtested figures are simulated: they do not represent actual trading. Where a firm shows simulated past performance in a marketing communication, EU rules require it to be based on actual past data for the instrument or index concerned, to cover a full period of at least five years or the instrument's whole life, and to carry a prominent warning that the figures refer to simulated past performance and are not a reliable indicator of future performanceEUR-Lex. Four failure modes account for most of the gap between simulation and reality: Four failure modes account for most of the gap between simulation and reality:
- Overfitting. Rules tuned until they fit historical noise perform beautifully on the past and poorly on the future. The more parameters a strategy has, and the more times it was revised to improve the backtest, the higher the risk.
- Survivorship bias. Testing on today's index members quietly excludes the companies that failed, inflating returns.
- Look-ahead bias. Using information (prices, fundamentals, revisions) that was not actually available at the simulated decision time.
- Frictionless execution. Simulations often assume perfect fills. Live trading pays spreads, slippage and fees on every trade.
The scale of the problem has been measured rather than asserted. A study of 888 algorithmic strategies developed on the Quantopian platform, each with at least six months of out-of-sample performance, found that the backtest Sharpe ratio explained less than 2.5 per cent of the variation in out-of-sample Sharpe, and that the more a strategy had been backtested, the wider the gap between its simulated and its subsequent resultsSSRN. Two qualifications belong with that number, and are usually dropped when it is quoted: the cohort was crowdsourced retail-developed strategies on a single platform, not institutional managers, and the same study found that other backtest features — volatility, maximum drawdown, whether the portfolio was hedged — did carry predictive signal. The finding is not that backtesting is worthless. It is that the headline risk-adjusted number is the least informative thing in a backtest, and the testing process behind it is what deserves the interrogation.
Four levels of performance evidence
Before comparing two numbers, establish that they are the same kind of number. These four tiers are ordered by how hard the evidence is to manufacture, strongest first:
| Tier | What it is | What makes it checkable |
|---|---|---|
| Audited live record | The instrument's own realised performance, examined by an auditor | Auditor named, period stated, net of all fees; the strongest evidence available |
| Observable live record | Prices an independent third party published as they happened — an exchange, an administrator, a data vendor | Reproducible by anyone with the ISIN; no reliance on the manager's own reporting |
| Personal-account record | The developer's own trading, on their own account | Real money, but different size, costs, constraints and risk settings; not the instrument's performance |
| Simulation / backtest | How rules would have behaved on historical data | Weakest tier; only as good as the disclosed methodology, and exposed to the four biases above |
Note The tiers are an editorial framework, not a regulatory classification. The labelling obligations that attach to simulated figures are set out in Commission Delegated Regulation (EU) 2017/565, Article 44EUR-Lex.
Six things a performance figure must tell you
Whatever tier it sits in, a figure you cannot place is a figure you cannot use. For every number in a presentation, establish:
- The observation window — exact start and end dates, not "since inception" or "last 5 years".
- Live or simulated — and, if the series switches partway, the exact date it switches.
- Gross or net — and net of which fees: management, performance, custody, issuance, all of them?
- Trading-cost assumptions — what spread, slippage and commission a simulation charged itself.
- Whose account — the instrument, a model portfolio, or a person's own trading.
- Who published the underlying data — an independent source, or the manager. If no independent source exists, that should be disclosed rather than left to inference.
Three kinds of track record, three levels of evidence
| Number | What it shows | What to remember |
|---|---|---|
| Backtest | How the rules would have behaved on history | Simulated; exposed to the four biases above; check the methodology |
| Personal account | The developer's own trading, on their own account | Real money, but different size, costs, constraints and risk settings; independent of the instrument and not indicative of its performance |
| Instrument track record | The product's own priced history, net of fees | The only number that is actually the instrument's performance; past results still do not guarantee future ones |
Serious product documentation keeps these categories separate and labels them. Framont example On Framont Access, HERAKLIT's long-horizon figures are explicitly marked as backtested simulations rather than the instrument's realised performance. For the instrument's own history, the issuer publishes the product record, including ISIN and currency, on its own siteiMaps, which is the independently observable series rather than a manager-reported one. Neither figure is audited, and this page does not claim otherwise. When a presentation blurs these lines, treat it as a finding in itself.
Fee mechanics: reading the cost of the wrapper
Strategy products typically charge a management fee (accrued daily in the instrument's price) plus a performance fee on gains. Two details determine whether a performance fee is fair:
The high-water mark. A fee "with high-water mark" is charged only on net new highs. Example: the instrument rises from 100 to 110 and pays 20% on the 10 points of gain. It then falls to 95 and recovers to 110. No second fee is due on the recovery, because 110 was already paid for; the next fee applies only above 110. Without a high-water mark, investors can pay repeatedly for the same recovered ground.
The money-terms total. The PRIIPs KID must show total costs in euros over the recommended holding period, combining management, performance and transaction costs. That single figure, not the headline percentage, is the number to compare across products.
Governance: who actually holds the wheel
A strategy is only as trustworthy as the structure executing it. For securitised products such as ETIs and AMCs, map the roles: the strategy developer supplies the model; the portfolio manager, a regulated entity (for Framont Access instruments, Framont & Partners Management Ltd, an MFSA-authorised AIFM), executes it inside the wrapper; the issuer maintains the segregated portfolio; an independent security trustee holds the collateral pledge. Separation of these roles, with regulated entities in the money-handling seats, is what turns an interesting model into an investable instrument. Our guides to ETIs and AMCs cover the structures in detail.
The seven-question checklist
- Which numbers are simulated, and which are live? Ask for the split explicitly and for the backtest methodology (out-of-sample testing, costs assumed).
- Who is the regulated portfolio manager, and under which authority? Verify the licence on the regulator's public register.
- Is the structure collateralised, and who is the security trustee?
- What are total costs in money terms over the recommended holding period, per the KID?
- Is the performance fee subject to a high-water mark?
- How do I exit, at what price, and who quotes it? Check listing venue, market-making terms and spreads.
- What is the realistic worst case, per the KID's stress scenario, and does it fit your capacity for loss?
If a distributor cannot answer these seven questions in writing, the product is not ready for your capital, whatever the chart says.