I built a bot that trades weather markets on Polymarket. The bot bets on which bucket a city’s daily high temperature will land in and this is a log of what I learnt building it.
The model is deliberately simple: a single Gaussian over the day’s high, its mean taken from the ECMWF forecast and nudged by the gap between that forecast and the live airport METAR reading, its width taken from the spread of the 51-member ensemble. The area under the curve between a bucket’s boundaries is the model’s probability, the difference from the market’s price is the edge, and positions are sized at quarter-Kelly with a hard per-trade cap. Every minute it scanned: pulled the latest METAR observation, recomputed the distribution, and compared each bucket’s probability to the live order book. Every signal that scan produced was logged - price, size, fill, and eventual outcome, whether it traded or passed. When it acted, it took one of two positions:
- A BET buys YES on a bucket priced below the model’s probability - usually a cheap longshot in the tail of the distribution.
- A FADE buys NO against a bucket priced above it, which lands on the deep, liquid favourite side of the book.
The bot has gone through 34 versions, some 52,000 logged signals, and 4 days of real money in the search for edge in every place it could plausibly live: a better probability model, a better distribution shape, the overlays on top of them, faster information, friendlier execution, the other side of the book, even the physical sensor each market settles on. Every lever is now closed, and most of them closed with the same signature. Market efficiency sounds like an end state; from the inside, it was something I discovered mechanism by mechanism. What follows is that search, lever by lever - the evaluation lessons it produced are a separate piece.
A better model
The first real lesson I learnt was that bugs can manufacture edge. Early versions multiplied the forecast uncertainty by a time-scaling factor, widening the distribution for distant horizons. The ensemble spread already encodes horizon uncertainty, so the multiplier double-counted it and produced phantom edges everywhere the widened tails crossed a market price. Removing it was the single improvement that turned the paper book profitable, making the model better by doing less.
The second was that the distribution shape is priced. The most instructive bug: with a day’s high already observed at 19°C and the remaining forecast maxing at 17.2°C, the model still gave 19°C a 42% probability, because a symmetric Gaussian centred on the forecast spreads mass below a bound the temperature had already set. About a third of the probability mass sat on physically impossible outcomes. Fixing that was necessary, but the deeper finding came from trying to keep improving the curve: a statistically correct sigma recalibration destroyed the strategy by shrinking exactly the longshot probabilities that carried it - the only net-negative version, at −5.6% ROI, a regression from its parent significant at p=0.007; a running-max model replayed null against the plain Gaussian; and skew and GEV shapes replayed null on top of both. Four results converged on one conclusion: over public forecast inputs, the market has already priced the shape of the daily-high distribution. A better curve does not add edge, because everyone’s curve is looking at the same forecasts.
The overlays fared no better. A station blacklist cut about 75 bets that were net winners. A sigma floor on fades removed exactly the profitable ones. Of two Kelly refinements, an ensemble-dispersion haircut was inert at best, and a price-fragility tilt gained 1.3 points on the version it was fitted to and inverted on both sibling versions out of sample - overfit, caught by walk-forward before adoption. Several versions existed only to carry these filters, and were retired as strictly dominated by the plain model underneath them. Every layer added above the model subtracted value; the lesson of getting better by doing less turned out to be the rule, not the exception.
What did work, for a while, was none of this. The one mechanism that ever produced real fills at what looked like real edge exploited stale quotes - being early to a market that had not yet repriced a fresh observation. Which pointed at speed, and at the analysis that would eventually explain those fills away.
Faster information
I rebuilt the data path so the bot reacts in 1-2 minutes from a new weather observation, down from roughly 35 minutes. Then I measured whether the race was winnable at all, on a base of 117 fair-value-moving observations - the same denominator for every figure that follows: 38% were repriced before the bot could see them, but 58% took over 3 minutes to reprice and 45% took over 10. On timing alone, the race is winnable in a majority of events.
It is also worthless, and the reason is the interesting part. The market is slow to reprice exactly when the new print has not yet pinned the outcome - when the temperature might still climb further and fair value is genuinely unresolved. When the print is decisive, repricing is effectively instant. Replayed at executable prices, every variation loses: buying each crossing and exiting at the bid costs 4.4 cents a share across 220 events, with a confidence interval entirely below zero, against a hindsight ceiling of +17.7 cents for an oracle that knows which prints continue. Harvesting the slow window therefore requires predicting continuation, so I tested whether it could be predicted: a logistic hazard model on physics features - forecast headroom, hours to peak, ensemble spread - pre-registered before running, fit on 371 crossings and evaluated once on 239 out-of-sample events, no iteration allowed. In sample it discriminated cleanly; out of sample no rule cleared the bar, and the failure mode was the real finding. Trading only where the model beat the ask produced a 9% win rate against 22% for trading every event, because a model-versus-price gate preferentially selects the model’s own errors. The market was not asleep; it was waiting for the same information I was.
The settlement-sensor thread ended the same way, twice. Polymarket settles on Wunderground readings while the bot reads aviation METARs, and the settling sensor’s readings genuinely lead the market - at the mid. By the first ask you can actually lift after a fresh print, the lead is gone; restricting to asks that have not yet moved recovers an edge of exactly zero. Then the premise itself collapsed: across 783 matched timestamps the two sources differ by 0.00°C, because they are the same instrument. The apparent gap had been an artefact of my own recording. What survived is a map of which sensor settles each market, kept for safety rather than profit.
Friendlier execution
The longshot tail was where the model showed real selection skill - its cheap picks won at 3-5 times the base rate for their price: 7.7% against a 1.5% base in the 2-5c band, 12.3% against 4.3% in 5-10c, 24.4% against 5.0% in 10-12c, with the deepest 0-2c band a confirmed loser at the ask. The order book is where that skill went to die. Below 5 cents, only 31% of intended dollars could fill at quoted depth, and the unfillable trades were precisely the winners. Resting bids instead of taking the ask makes it worse, not better: on the resolved winning longshots - 92 distinct events, once version-duplicated rows are collapsed - the price sat above my hypothetical bid for roughly 3/4 of its post-signal life, against about 2% for the losers. A resting bid in the tail fills when the longshot is losing and gets left behind when it wins. Adverse selection inverts the edge.
Widening the trading window does not rescue it either. The books are deeper earlier in a market’s life - my trading window sat at the liquidity trough - but where fills exist, the cheap basket resolves YES around 2.2% against a 4-5% ask. The picks that showed skill and the picks that fill are not the same picks - depth appears exactly where the market is content to sell, so the fillable subset is the adversely selected one. Even where the edge at the ask stays point-positive, every band’s cluster-bootstrapped confidence interval includes zero on 171 events. Liquidity was the bottleneck to filling, edge was the bottleneck to profiting, and easing one does not move the other.
The other side of the book
Half the book was never longshots at all. The FADE side had none of the execution problem: sitting on the deep favourite side, it filled over 90% of intended dollars. Across roughly 21,500 resolved fades it was profitable up to a market probability of 0.85, running +11% to +17% in the middle bands, and a trap beyond, where near-lock fades bleed at −34% and worse - the structural mirror of the longshot tail. For a while it looked like the answer, showing around +11.6% ROI on its best version. Two tests closed it. A benchmark that faded on price alone, ignoring the model entirely, matched the model’s selections within every price stratum; and at realistic fills the realised ROI’s confidence interval grazes zero, +6.7% spanning −0.1% to +13.2%. What looked like model edge was a structural favourite-side premium any rule could collect, and not demonstrably positive even then. The planned FADE-only live probe was stood down. This lever closed with a different signature - not the spread, but attribution.
The signature
Everywhere else, one pattern repeats. The model’s directional signal is real at the mid - worth 2-4 points across more than 6,000 resolved market-bucket events, measurable, stable, strongest on fresh quotes. At the price you can actually transact, capture is 1.1 points with a confidence interval from −1.9 to +4.3 - statistically indistinguishable from zero - because the execution premium costs almost exactly what the signal is worth. The sensor lead was real at the mid and dead at the ask. The repricing lag was real in time and dead at the fill. The market’s last line of defence is not better information; it is the spread.
The search is closed, and so, mostly, is the system. I wound the operation down to a signal logger - live trading off, the research lines shut, each lever keeping a written bar for what evidence would reopen it. Stopping was the last decision the evidence forced, and I trust the conclusion more for how it was reached. Efficiency, from the inside, is not the absence of signal. It is the market charging you exactly what your signal is worth.