v4.3.0: MT5 Strategy Tester HTTP API + per-terminal live/backtest mode
Lands PR #2 from algotradingspace/backtester (Marin). The headline feature
is a tester-mode terminal that runs alongside live trading terminals on the
same Docker / Windows VM deployment, behind the same /<broker>/<account>/...
nginx routing.
Why mode exists
---------------
MT5 is single-instance per portable data directory. A running terminal64.exe
holds an exclusive lock on its dir; any second terminal64.exe spawned against
the same dir exits silently with code 0. That makes it impossible to drive
the Strategy Tester through a terminal that's also backing the live SDK.
Mode declares intent at terminal-startup time:
- mode: live (default) -- terminal64.exe stays running, SDK is initialized,
all live trading + market data endpoints work as before.
- mode: backtest -- portable dir is prepared but terminal64.exe is NOT
launched and the MT5 SDK is NOT initialized; the data dir stays free for
the tester subprocess that POST /backtest spawns on demand.
A single install can run both modes side by side.
New endpoints
-------------
POST /backtest/build-ini -- stateless JSON -> tester.ini helper
POST /backtest -- multipart submit (ini + ex5 + set), 202
GET /backtest/<job_id> -- queued/running/completed/failed + summary
GET /backtest/<job_id>/report -- the MT5 HTML report
GET /backtest/<job_id>/log -- captured terminal log for the run
GET /ping -- now echoes {"status":"ok","mode":"..."}
Tester INI is re-encoded UTF-16-LE+BOM+CRLF before MT5 reads it (MT5
silently rejects [Tester] Login under UTF-8). [Common].Login/Password/Server
are always overwritten from the URL-selected account in config.yaml -- the
caller cannot inject credentials. EA/.set inputs accept either inline upload
or a host-managed name resolved against ./assets/{experts,sets}/ mounted
read-only into the VM (path traversal in *_name is rejected).
Concurrency: one tester per API process, serialized by an internal RUN_LOCK;
additional submissions queue. Jobs left in-flight when the API restarts are
marked failed by sweep_orphans() at next boot.
Configurable backtest timeout
-----------------------------
Default 6h. Override chain (highest priority first):
1. POST /backtest form field 'timeout' (e.g. "30m", "3h30m")
2. config.yaml backtest_timeout
3. BACKTEST_TIMEOUT env var
4. hardcoded DEFAULT_BACKTEST_TIMEOUT="6h"
Parsing reuses parse_duration_to_seconds (same grammar as utc_offset).
Other runtime hardening (also in PR #2)
---------------------------------------
- Per-terminal symbol_suffix: optional broker-specific suffix appended to
[Tester].Symbol when missing (e.g. ".r" for RoboForex, "p" for FTMO).
Lets the same EA/.set/.ini run against multiple terminals without
hand-editing the symbol per broker.
- reboot_interval=0 is now respected at startup so long backtest runs are
not interrupted by the scheduled auto-reboot.
- nginx generated config: client_max_body_size 25m + client_body_timeout
120s so EA + .set uploads up to ~25MB succeed.
- Parsed report summary now includes bars/ticks/symbols so empty-history
failures are visible in the JSON response without opening the HTML.
- psutil added to the base pip install (was already imported by
mt5api/mt5client.py but missing from the install list).
Compatibility
-------------
Fully additive. mode defaults to "live"; existing config files keep working
verbatim. Live terminal API surface is unchanged. Multi-terminal routing,
Docker/VM deployment, and the v4 single-file config model are unchanged.
Host-managed asset pool is opt-in (use inline upload to skip the mount).
Tests
-----
121 pytest cases pass, +28 new covering the backtest handler, INI builder,
and job lifecycle. End-to-end tester run validated against a real Darwinex
demo terminal (job completed in 114.5s, 276KB report, parsed summary
netProfit=129.89, profitFactor=1.34, totalTrades=123).
Also in this release
--------------------
CHANGELOG.md introduced as the readable digest of every annotated git tag
in the project's history (v1.0.0 -> v4.3.0). Tags remain the canonical
source of truth; CHANGELOG.md links each entry back to its tag.
Files (PR #2 already on origin/master; CHANGELOG.md added in this tag's commit):
CHANGELOG.md (new in this commit)
mt5api/backtest/__init__.py
mt5api/backtest/handler.py
mt5api/backtest/ini_builder.py
mt5api/backtest/jobs.py
mt5api/config.py
mt5api/handlers/terminal.py
mt5api/main.py
mt5api/server.py
scripts/api_runner.bat
scripts/config_helper.py
scripts/start.bat
run.sh
.dockerignore .gitignore
config/config.yaml.example docker-compose.yml.example
assets/experts/.gitkeep assets/sets/.gitkeep
README.md .agents/.skills/mt5-httpapi/{SKILL.md,references/setup.md}
tests/test_backtest_handler.py
tests/test_backtest_ini_builder.py
tests/test_backtest_jobs.py