Thompson Sampling for Outbound Copy Variant Selection

A/B testing email copy at fixed 50/50 splits wastes sends on losing variants. If variant A is clearly underperforming after 30 sends, the next 220 sends splitting evenly between A and B are partially discarded value. The regret is real: every send to a losing variant is a send not going to the best-known variant.

The Avo Engine outbound system needed to select among copy variants automatically, route sends to better-performing variants faster than static splits, and promote winners without manual review on every campaign. The secondary problem was data sparsity. Cold email campaigns generate thin feedback loops. A good campaign might hit 5 to 10% open rate, meaning 90 to 95 sends return no signal at all. The algorithm needed to function with low observation counts.

Thompson sampling over a Beta distribution was chosen over epsilon-greedy and UCB1 for three reasons. First, Beta-Bernoulli conjugacy made posterior updates exact and cheap: an open event updates the alpha parameter, a send without an open updates the beta parameter. Second, Thompson sampling performs well under low observation counts: UCB1 requires careful tuning of the exploration coefficient at low N, while Thompson sampling automatically reduces exploration as posteriors sharpen. Third, the system needed to support CTR as a secondary reward signal: a two-stage Beta model with independent posteriors allowed compositional scoring as open_prior times ctr_prior.

Prior initialization used Beta(1, 1) (uniform) for new variants. For variants with prior campaign history on similar audience segments, a weakly informative prior of Beta(2, 18) was used, encoding the background expectation of a roughly 10% open rate.

The bandit state was stored in a Postgres table with columns variant_id, campaign_id, alpha FLOAT, beta FLOAT, alpha_ctr FLOAT, beta_ctr FLOAT, sends INT, opens INT, clicks INT, updated_at TIMESTAMPTZ. No external ML library. Posterior parameters were maintained as running floats, updated in a single UPDATE statement on each event.

At send time, the send-selector drew one sample per active variant by computing a sample from beta_distribution_sample(alpha, beta) using the Rust statrs crate. The variant with the highest sample value won the send slot. Variants with wide posteriors win occasionally due to random sampling, providing natural exploration.

Auto-promotion fired when any variant's sends counter crossed the N=50 threshold and its sampled expected reward exceeded all other variants by a margin of 0.03 (3 percentage points). Below that margin, sampling continued. The margin prevented premature promotion when variants were within noise.

One postmortem issue: the first implementation recalculated posteriors from raw event counts on every draw rather than storing running parameters. This hit Postgres read contention under concurrent sends. Switching to stored alpha/beta floats updated incrementally eliminated the contention. A second issue: webhook delivery for open events was not exactly-once. A dedup table keyed on (message_id, event_type) with a unique index prevented double-counting.

Offline evaluation ran on 10,486 historical sends from the prior rule-based system. Thompson sampling reduced simulated cumulative regret by 31% relative to round-robin over the first 200 sends per campaign. By send 150, it had converged to the best variant in 89% of simulated campaigns. Round-robin had not converged by send 200 in any campaign.

Online results from the first three live campaigns: mean open rate on Thompson-selected variant allocations was 19.2% versus 13.7% when using the fixed 50/50 holdout. The auto-promotion path eliminated manual review on 100% of routine campaigns. Only 12% of campaigns were escalated for human review when the margin between top two variants stayed below 0.03 at N=200.