Most "profitable" backtests are phantoms. Upload your ledger or backtest and we run the same overfit battery and phantom-PnL detectors our own trading desk uses to kill fake edges — then tell you, per market, whether the edge is real.
Sweep tools like vectorbt make it trivial to run 10,000 backtests before breakfast — and that ease is exactly how retail overfits: the best of 10,000 random strategies always looks brilliant. DSR and PBO exist to price that multiplicity, and this battery runs them on every ledger.
This quarter we found and eliminated nine backtest-phantom classes that were inflating our own paper edges by up to ~96%. Every one is now a detector you can run against your ledger.
Entries stamped at or after settlement — an edge that "knew" the outcome. Point-in-time entry-timestamp guard.
Gross PnL booked without the fees/funding/slippage that make it net-negative. Re-couples every leg to true net.
Schema/feed drift injecting synthetic 0.5/0.0 marks that fabricate fills. Signature-keyed contamination scan.
Fills booked before a tradable book existed — liquidity that wasn't there. Book-existence gate at entry epoch.
Re-materialized legs booked outside the payable band, inflating capital-velocity. Engine-rev + band audit.
Percent-return divided by a near-zero entry cost → explosive fake %/day. Cost-basis-floor + denominator guard.
Dead / delisted / resolved-away markets excluded from the roster before any statistic is computed. Roster-census guard.
Edges that vanish when every signal is lagged to what was truly knowable at entry. Point-in-time blind re-run.
Split/adjustment boundaries left unadjusted in the tape fabricate 20x "moves". Adjacent-bar split-boundary guard.
Not a truncated "DSR + PBO + CPCV" checklist — the complete quant + forecast-verification battery (72 instruments, ~24 tractable-at-scale), the same one that gates every strategy on our desk. Grouped by family:
DSR · PBO · CPCV · White Reality-Check · Hansen SPA · walk-forward · MC-permutation · n_eff deflation · PSR · MinTRL · model-confidence-set · Pesaran-Timmermann · Diebold-Mariano · BH-FDR · BY-FDR · Holm-Bonferroni · Romano-Wolf · variance-ratio · regime-break · online-FDR (SAFFRON) · anytime-valid CS
Info-ratio · Sortino · Omega · tail-ratio · Kelly-growth · capacity-adjusted · Calmar · Ulcer-PI · CDaR · CVaR · Rachev · K-ratio · √-law capacity · carry-to-risk · vol-surface no-arb
Brier · Brier-skill · log-loss · reliability/ECE · CLV · CRPS · resolution-decomp · calib slope/intercept · Spiegelhalter-z · ranked-prob-score · Mincer-Zarnowitz · conformal interval
realized-spread · adverse-selection · effective-spread · impl-shortfall · fill-ratio · inventory-risk · Kyle-λ · Amihud · VPIN · queue-value · Hasbrouck info-share · Roll-Corwin-Schultz
net-after-fee · settlement-truth · paired-leg-completeness · convergence-time/capital-lock · cointegration/OU-halflife
hit-rate decomp · greeks-PnL attribution · mtm-velocity · information-coefficient · carry-return attribution · variance-risk-premium · QLIKE
Full = 72 instruments. Every persona on our own fleet is judged by the family-correct subset of these — a directional edge by DSR + Kelly-growth, an options edge by greeks-attribution, a maker by realized-spread net of rebates, a prediction market by Brier + CLV.
One-stop integrity shop. The universal overfit battery + phantom detectors apply to everything; each market adds its own failure modes. Pick your packs.
Near-settle lookahead · feed field-drift · settlement-truth · paired-leg-completeness · the full calibration family (Brier/CLV/ECE/CRPS/RPS/reliability). Kalshi · Polymarket · Limitless · sports.
Buy pack · $19/mo →Greeks-PnL attribution · IV-lookahead · c0 cost-basis division · vol-surface no-arbitrage · variance-risk-premium · QLIKE. Tradier · IBKR · alpaca options.
Buy pack · $19/mo →Funding-lookahead · carry-to-risk · carry-return attribution (is it carry or disguised spot?) · convergence-time/capital-lock. Perp DEX/CEX.
Buy pack · $19/mo →Contango/roll artifacts · continuous-contract roll-adjustment · cointegration/OU-halflife · carry-to-risk. CME/commodity/gold futures.
Buy pack · $19/mo →Wash-trade & venue-outage gap detection · mtm-velocity · Amihud/Kyle-λ illiquidity · VPIN toxicity · Hasbrouck cross-venue info-share. Coinbase/Gemini/spot.
Buy pack · $19/mo →Survivorship · point-in-time (look-ahead) fundamentals · corp-action/split adjustment · information-coefficient · variance-ratio. Alpaca/IBKR/Robinhood equities.
Buy pack · $19/mo →Any taker or maker: realized & effective spread net of rebates · adverse-selection · implementation-shortfall · fill-ratio · inventory-risk · queue-value.
Buy pack · $19/mo →The 72-instrument selection/risk battery + all 9 phantom detectors + survivorship. Market-independent. The core of every plan.
| Check | Predict | Equities | Options | Perps | Futures | Spot/Crypto |
|---|---|---|---|---|---|---|
| Overfit battery (72) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Net-decoupled PnL | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Pre-liquidity fills | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Recycle / velocity | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Survivorship | ✓ | ✓ | ✓ | · | ✓ | ✓ |
| Near-settle lookahead | ✓ | · | ✓ | · | ✓ | · |
| Feed field-drift | ✓ | · | · | · | · | ✓ |
| c0 cost-basis div | ✓ | · | ✓ | · | · | · |
| Calibration family | ✓ | · | · | · | · | · |
| Greeks / IV lookahead | · | · | ✓ | · | · | · |
| Funding / carry | · | · | · | ✓ | ✓ | · |
| Roll / contango | · | · | · | · | ✓ | · |
| Look-ahead fundamentals | · | ✓ | · | · | · | · |
| Wash-trade / outage gap | · | · | · | · | · | ✓ |
À-la-carte: add any single market pack to Quant for $19/mo, or run a one-off single-ledger audit. Prediction-market-only traders: the dedicated PM Validator is $29.
Phantom Auditor is the trading-desk face of a shared integrity platform — three verification engines that all ship only verified findings, one account, one console. A quant fund that trades via code needs all three: a real edge in a buggy, insecure bot still loses.
Strategy & backtest integrity. Is the edge real, or a phantom / overfit / look-ahead artifact? The 72-instrument battery on your ledger.
Code integrity. AST bug-class detectors + CI auto-fix PRs on your execution code — the bot that runs the edge.
Security & posture integrity. A LIVE-vs-canonical parity oracle that ships only findings it has reproduced as exploitable — zero false positives, a proof for every finding. The keys & infra the bot runs on.
See Solvd →TradeHouse · Elite AI Empire. Phantom Auditor runs the same overfit battery and phantom-PnL detectors that gate our own trading fleet. Upload a ledger (CSV / fills / backtest), get a per-market integrity verdict. The battery is real: 72 instruments, 9 phantom classes incl. survivorship — enumerated, not marketing.