pbody. From RL Trading Bot to Research Program

Overview

pbody started as a learning project. I wanted to understand how a reinforcement-learning agent learns to trade CFDs on a simulated IG account, not to chase returns. Between August 7 and September 9, 2026 it grew into a research program with 38 numbered experiments and 23 pre-registrations. I led the project and used Claude to speed up the fine-tuning. The honest outcome is that not trading was the best strategy in forex, and that one deployable candidate remained on the US equity market. At the close every artifact was backed up in three places and restore-tested.

Implementation

Infrastructure

The simulation ran as a gRPC episode server in Go and modelled a CFD account under ESMA rules. The trainer and research scripts were written in Python with PyTorch and Stable-Baselines3. The Polygon flat files from 2003 to 2025 went into a binary cache of 18.3 billion records on 1.1 terabytes. Training took place on a home machine with an RTX 5090, where all 24 GPU runs ran as Docker containers, while a Debian worker carried live operation on systemd timers.

Method

Pass criteria were written down before the numbers existed, and a sealed period stayed untouched until the final test. A leak test that deliberately fed the next return into the observation fired in 49 of 50 evaluations. That proved the pipeline finds a signal when one exists, so a flat result meant a missing signal and not a bug.

Results

Forex

The RL agent stayed flat in every run. Supervised models did find a real signal, but it sat in the evening rollover hours, where spreads run about thirteen times the daytime level. Live across 158 decisions on nine currency pairs, the system rightly placed not a single trade.

US equities

Volatility-managed cross-sectional momentum reproduced published research results on 21 years of data, returned about 16 percent a year in the sealed period and survived six pre-registered corrections. What stands between it and live trading is tax classification and capital size, not missing evidence.

Stumbling blocks

Around two dozen bugs turned up, none of them in the RL algorithm itself. Most sat in data and calendars, such as two daylight-saving errors around the forex weekend break. In operations, systemd tied the runner to the live server via Requires=. When the worker rebooted four times, the runner opened a fresh episode each time, and the running live test was lost after 105 of 200 decisions.

Stack

Go, gRPC, Python, PyTorch, Stable-Baselines3, TensorBoard, Docker, Polygon.io, IG and Interactive Brokers, Debian with systemd.