Skip to content

About

Benchmark based on harbor tasks and surfpool that evaluate trading agents on Solana

Resources

Stars

9 stars

Watchers

0 watching

Forks

Repository files navigation

Machine Trading

Harbor dataset

Machine Trading is a workshop for frontier trading agents on Solana. It starts with a benchmark. Agents write trading bots, and the bots are graded on hidden market scenarios replayed against an offline copy of Solana mainnet. Real programs, real order books and real fees run in a sandbox that no bot can cheat.

Getting started

just install mtx      # build and install the mtx CLI into ~/.cargo/bin

A scenario is a scripted market: a sequence of changes (prices that crash or squeeze, funding, fees, a halt) applied at given slots to an offline copy of Solana mainnet, restored from a pinned snapshot. Bots are graded on hidden scenarios; mtx scenario play has a coding agent write one from a request, then plays it on a local surfnet and charts it live:

mtx scenario play "SPCX crashes 5% in three legs, then rebounds halfway"

mtx bench review browses a bench's results: each model's skill and costs, every run, and every scenario step by step. Give it a published release (bench/results/<version>) or a batch you ran:

mtx bench review 0.10.0

Write your own bot

You need Rust with the wasm32-wasip2 target, just and mtx (above); no Docker. A bot is a strategy that trades one session: the guide to writing a bot has it all. You write it on mtx-kit and build it into a WebAssembly component, and the grader does the rest: it reads the market before every step, calls your strategy, and sends its orders inside the mandate.

use mtx_kit::prelude::*;

#[derive(Default)]
struct FadeTheDip { peak: f64 }

impl Strategy for FadeTheDip {
    fn step(&mut self, market: &Market, orders: &mut Orders) -> BotResult {
        self.peak = self.peak.max(market.mark);
        if market.mark < self.peak * 0.97 {
            orders.target_position(market.cap);    // cut to the position cap for you
        } else if market.mark >= self.peak {
            orders.target_position(0.0);
        }
        Ok(())
    }
}

mtx_kit::export!(FadeTheDip);
mtx bot new ~/my-bot          # a strategy that watches the market and never trades
cd ~/my-bot
mtx bot run --debug           # watch it play one session, step by step
mtx bot practice              # build it, and grade it on a task's practice scenarios

The step-by-step guide builds a first strategy and watches it trade. Market is a plain value (the step, mark, oracle, index and best bid and ask, position, estimated PnL and drawdown, the mandate's room), and Orders records intentions (target_position, quote, buy, sell, bid, ask, cancel_all), so a strategy can be unit-tested without a chain; mtx_kit::rpc reads anything else from the chain. mtx bot practice --task <name> practises another task in bench/tasks, and scenarios mtx scenario generate draws can be graded too: mtx bot practice more/*.json. mtx bot check and mtx bot pack check a bot directory and package its source. The strategy is src/lib.rs, and just build builds it into bot.wasm, the one file the grader loads: no files, network or clock inside, and the same result on any machine.

The rest (the bench's own bots, grading, publishing and comparing releases) is in docs/development.md.

The bench

harbor run -d solana-foundation/machine-trading -a claude-code -m anthropic/claude-sonnet-5-5 -k 3
Task A good bot…
📉 flash-crash-spcx fades crashes and squeezes on the SPCX perp that revert, and survives the ones that don't
⚖️ market-making-spcx earns the SPCX spread from noise, and steps back before informed flow
💸 funding-carry-spcx earns the SPCX premium and its funding when positioning opened it, and follows when the index follows
🌙 weekend-gap-spcx reads which weekend flow on the SPCX perp knows where SPCX will open, and holds that gap into the reopen

Results are published at lgalabru.github.io/machine-trading/bench, the same view mtx bench review gives locally.

Planned tasks are tracked as issues labelled new task.

How bots are judged. The score is PnL: skill is a bot's total PnL over the hidden set, divided by a reference bot's on the same sessions. Risk is a set of hard limits that work like a trading desk's: one market, a position cap, and a loss limit. A bot that breaks one in any session scores 0. Statistics such as tail loss, drawdown and Sharpe are reported alongside the score. The details are in the dataset README.

Where this is going

📊 Bench Harbor tasks: each one is both an eval and an RL environment Shipping
🛠️ Harness An agent harness for trading on Solana, over mtx-kit Next
🧠 Models Trading models trained on the bench's rewards and trajectories Later

Repository

bench/      📊 tasks (the Harbor dataset), checks, images, published results
crates/     🦀 mtx (CLI), mtx-grader, mtx-kit (trading and the bot SDK), mtx-core (shared types and the mandate)
bots/       🤖 starter, reference and baseline bots
snapshots/  📸 pinned mainnet state
specs/      📜 the mandate: what an agent may trade, and its limits
skills/     🧠 agent skills, starting with how to design trading tasks
docs/       📐 development guide and decision records

To build, grade, review a bench or add a task, see docs/development.md.

About

Benchmark based on harbor tasks and surfpool that evaluate trading agents on Solana

Resources

Stars

9 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages