M Alpha Core logoM·ALPHA·COREBuy a license

Home / Guides / Backtest on unseen data

A backtest on data the search never saw

Almost any strategy looks great on the data it was tuned on. The real question is how it does on data it has never seen. That is what M Alpha Core shows in every report.

Last updated:

Section “What this says about the future” in the M Alpha Scan gold report: the share of price history the search never saw, the mean per trade and how all trades are distributed
Section “What this says about the future” in the M Alpha Scan gold report: the share of price history the search never saw, the mean per trade and how all trades are distributed (Demo with the gold scan of November 12, 2026)

Example: gold, 1 hour chart

Search on the first65 % of the data
Test on the last35 %, never seen by the search
Trades in the test period153
Profit factor on searched data2.53
Profit factor on unseen data2.81
Nvidia: profit factor searched / unseen2.19 / 4.90
Nasdaq 100: profit factor on unseen data2.17
ETH example: worst move against a trade15.64 %
ETH example: leverage limit6.39x

If the profit factor on unseen data drops far below the searched part, the settings were probably fitted to noise. If it holds up, the settings found something that also worked outside the data they were chosen on.

Why normal backtests lie

A backtest runs a set of rules over past prices and shows what would have happened. That sounds objective, and for one fixed set of rules it is. The trouble starts when you search. As soon as you try many settings and keep the best one, the best one is partly the luckiest one. With thousands of settings, some will look perfect in hindsight just by chance, even on random prices.

This is called overfitting. The settings describe the noise of the past instead of a pattern that repeats. An overfitted strategy shows a beautiful curve on the screen and then fails once real money is involved. The more settings a search tries, the larger this risk. A tool that tests thousands of settings, as M Alpha Core does, must therefore check its own result on fresh data. Otherwise the search is a machine for producing pretty, useless curves.

The 65/35 split

M Alpha Core splits every test period in two parts. The first 65 % is the search part. Here the scan tests thousands of settings and picks the best ones. The last 35 % is the unseen part. The search never looks at it. Once the settings are fixed, they run on this part without a single change, and the report shows the result next to the search result.

The split is in time order. The unseen part always comes after the search part, just like the future comes after the past. Mixing the order, for example by testing on random months, would let information from the future leak into the search and make the result look better than it is.

M Alpha Scan settings: 65 percent of the candles for the search, 35 percent kept unseen for the check, 500 trials per starting point and at least 60 trades
M Alpha Scan settings: 65 percent of the candles for the search, 35 percent kept unseen for the check, 500 trials per starting point and at least 60 trades

How to read the two numbers

The key figure is the profit factor: dollars won for every dollar lost. Above 1 a strategy made money in that period, below 1 it lost. Compare the searched part with the unseen part.

Risk in plain numbers

A good profit factor alone can still hide a dangerous trade. The report therefore shows how far the price ever moved against you inside a single trade. In one ETH strategy that was 15.64 %. With leverage, a move of that size against you can wipe out the margin of the position. From the worst move the report calculates the highest leverage the trade would have survived: 6.39x in that ETH example, 18.2x for gold and 9.03x for Nvidia.

That limit is not a recommendation to go all the way. The next bad move can be larger. It is a hard boundary that tells you where the backtest itself would already have failed. You can test this relation for any move with the leverage and liquidation calculator.

Section “What these numbers do not say” in the M Alpha Scan gold report on an Android phone, naming the limits of a backtest
Section “What these numbers do not say” in the M Alpha Scan gold report on an Android phone, naming the limits of a backtest (Demo with the gold scan of November 12, 2026)

Every trade, also the losers

A number can hide a lot. That is why the report also shows every single trade of the test on the chart, the winners and the losers. You see where the strategy entered and left, how long trades lasted and what the bad phases looked like. If the unseen part only made money in one wild week, you see that at once.

From test to bot, step by step

  1. Install the app. M Alpha Core runs on Windows 10 and 11, on macOS 11 or newer (Apple Silicon and Intel) and on Android 10 or newer. Get it on the download page. One Standard license covers one computer and one phone, Pro covers three devices in any mix.
  2. Scan your market. Type the symbol of a coin, of gold or of a stock in the scan screen. Pick a time frame and a date range, then start the scan. It runs on our scan server, so you can close the app or put your phone away while it works.
  3. Read the report. Look at the number of trades, the profit factor on the searched part and on the unseen part, the result with and without leverage, the worst move against a single trade and the leverage that move would have allowed. Open the chart and look at every trade, the losers included. The methodology page explains how each figure is calculated.
  4. Rent a small server and start your bot. The app walks you through renting a small server for about $12 a month and sets it up for you. Your bot first runs as a dry run: it only writes down what it would do and sends every step to your Telegram.
  5. Connect your exchange with a trading only key. Create an API key on one of the ten supported exchanges. Create an API key that may trade but may not withdraw. The key is stored in the protected vault of your device and on your own server. Our scan server never sees it. The security page explains how keys, servers and the control panel are protected.
  6. Go live when you are ready. A practice bot stays a practice bot, so for real money you set the bot up once more without the dry run tick. From then on it trades on your own account, and you can set it to long, short, reverse or stop with one tap in the app or in the control panel in your browser.

The bot then runs with exactly the settings that passed the unseen test. Nothing is tuned again in between. If you later want new settings, you run a new scan with a fresh split.

By hand, own script or M Alpha Core

You can do an out of sample test by hand or with your own code. The question is how much work it is and whether you really do it every time.

What mattersBy handOwn scriptM Alpha Core
Testing many settingsSlow, a few settings per eveningPossible, if you write and maintain the codeThousands of settings per scan on our scan server
Test on unseen dataRarely doneOnly if you build the split yourselfIn every report: search on 65 %, test on the last 35 %
Leverage risk in numbersGuessworkOnly if you calculate itWorst move inside a trade and the leverage that would have survived it
Trading around the clockOnly while you watch the screenNeeds your own server and monitoringYour bot on your own server, checked every minute by a watchdog
Programming neededNoYesNo, you tap through the app
Who holds the exchange keyYouYouYou, in the vault of your device and on your own server
CostYour timeYour time plus a serverFrom $49 a month plus a small server for about $12 a month

If you are weighing M Alpha Core against other bot tools, our side by side pages cover M Alpha Core vs 3Commas, M Alpha Core vs Pionex, M Alpha Core vs Cryptohopper and M Alpha Core vs Gainium.

Limits of any backtest

The guides on out of sample testing and how to backtest a trading strategy go deeper into these points.

Frequently asked questions

What is out of sample testing?

It means testing a strategy on data that was not used to find it. M Alpha Core uses the last 35 % of the test period for this.

Why 65 % and 35 %?

The search needs enough data to find settings, and the unseen part needs enough trades to say something. A 65/35 split in time order keeps both parts meaningful.

Is a good result on unseen data a guarantee?

No. It is a much stronger sign than a normal backtest, but markets change and past results are no promise.

What is overfitting?

Settings that fit the noise of past prices instead of a pattern that repeats. They look great in the backtest and fail afterwards.

What if the unseen result is much worse?

Then the settings were most likely fitted to noise. Do not trade them, however good the searched part looks.

Why can the unseen profit factor be higher than the searched one?

The unseen part is shorter, so a few lucky trades weigh more. Treat a higher value as a pass, not as a forecast.

How many trades does a test need?

The more the better. Our examples range from 89 trades for Nvidia to 153 for gold in their test periods.

Is this the same as walk forward testing?

It is a single walk forward step: search on the past, test on the following period. Full walk forward testing repeats this over many windows.

Which markets can I test?

Crypto, gold, silver, platinum, copper, oil, natural gas, the S&P 500, the Nasdaq 100 and more than 200 stocks.

Does the bot change its settings later?

No. The bot trades exactly the settings that passed the test. New settings need a new scan.

Download for Windows, Mac and AndroidAll guides

Compare M Alpha Core

Side by side with other bot tools, with sources for every statement about them.

Exchanges in M Alpha Core: Binance · Bybit · Bitget · OKX · MEXC · Gate · BingX · KuCoin · Crypto.com · Hyperliquid

All figures come from backtests on past market data. No result is a promise of future profit. Trading, especially with leverage, involves high risk. M Alpha Core is a software tool, not investment advice.