1. The two-machine split
believe runs on two kinds of hosts. The broker tier is a Windows VM on Azure whose only job is to talk to IB Gateway, run Sierra Chart, and execute trades on CME. The research tier is a Linux cluster that ingests ticks and the MBP-10 book, trains and retrains every head of the ensemble, runs the backtester, and hosts the monitoring stack. The research tier never places an order; the broker tier never writes to the training store.
Broker Tier (Azure Win VM)
- IB GatewayreqMktData, reqMktDepth, placeOrder
- Sierra ChartT29 DLL tick capture
- Python execution bridgeclientId=251, dom_ib.jsonl writer
Signed Data Path
- Trade journal uploadfills.jsonl, orders.jsonl
- Tick + DOM capture uploadrsync over WireGuard
- Model card downloadbelieve-v<N>.tar.gz + hash
Research Tier (Linux)
- Training storeParquet shards, content-addressed
- Backtesterdeterministic replay on captured ticks
- Retrain + model card builderF2_dom, XGB 5m, Tick ML
- Monitoringfeature drift, fill-rate, parity
2. Broker tier
The Windows VM is sized for reliability, not speed. It runs Sierra Chart and IB Gateway
side by side, each with its own tick capture. The Python execution bridge is the only
piece of our own code that ever sends an order. It subscribes to the IB client with a
pinned clientId=251 and writes the full depth stream directly
to dom_ib.jsonl.
- Sierra Chart is the tick-capture primary: T29 DLL writes to disk synchronously, so restarting Sierra does not lose ticks silently.
- IB Gateway is the execution surface. Orders are placed as marketable limits, bracketed at submission time. We do not rely on client-side exits to close a position.
- The bridge is a Python process supervised by NSSM. A crash spawns a fresh one, and the watchdog alarms on the Gmail channel if the supervisor cannot restart it.
3. Research tier
The Linux cluster carries the weight. It holds the tick store, the MBP-10 store, the training code, the backtester, and the monitoring dashboard. It is the only tier that knows how to produce a new model.
- Training store. Parquet shards keyed by
{instrument}/{feed}/{date}/{shard}.parquet. Content-addressed; every hash referenced in a model card points to a byte-for-byte reproducible shard. - Backtester. Deterministic replay over the live tick capture. Same feature code path as live, same order-fill logic, same commission schedule. A single seed, a single set of outputs.
- Retrain pipeline. One job per head: F2_dom weekly, XGB 5m monthly, Tick ML monthly, each with purged walk-forward and embargo. Output is a model card plus a signed tarball.
- Monitoring. Feature-drift alarms, bracket fill-rate tracking, and the backtest-vs-live parity job.
4. Dashboard and logging
The public dashboard lives at log.bhf.capital — a Cloudflare Worker backed by Workers KV. It shows live state (bridge up, last tick, last fill, current session regime) and historical context. All of the dashboard’s inputs come from the signed data path; the Worker has no route to the broker and no credentials to trade.
Click telemetry from this site is posted to the same Worker (log.bhf.capital/click),
stored in KV, and reviewed during monthly marketing hygiene.
5. Watchdogs and alarms
- Tick heartbeat. If no tick has been seen for a configurable window during a known-open session, the watchdog pages via Gmail.
- CME maintenance-aware. The daily 17:00-18:00 ET window is masked. A clean shutdown of Sierra during maintenance is not an incident.
- Bridge supervisor. NSSM restarts the bridge on crash; an unrecoverable crash (three consecutive restart failures) escalates to email.
- Walk-forward drift. If the most-recent fold AUC on any head falls more than 2σ below its 30-day rolling mean, the retrain pipeline holds the next deploy and opens a review.
6. Ship path for a new model
The end-to-end deploy of a new head is mechanical:
- Retrain pipeline builds model + model card on research tier.
- Purged walk-forward metrics are inspected against the prior release; regression triggers a hold.
- Signed tarball is pushed across the data path.
- Broker tier unpacks and verifies the hash; mismatch refuses the load.
- Bridge hot-swaps the head at the next RTH open, logs the swap, and continues.
No ad hoc “let me scp this model quickly” paths exist. The ship path is the only path.
7. Stack summary
| Tier | Host | Key components |
|---|---|---|
| Broker | Azure Windows VM | Sierra Chart, IB Gateway, Python bridge (NSSM) |
| Research | Linux cluster (claw) | Parquet store, backtester, retrain pipeline, monitoring |
| Public | Cloudflare Pages + Worker | bhf.capital, log.bhf.capital, trade.bhf.capital |