Why this is harder than it looks

After some tidying up (keeping sites that report reliably in both 2023 and 2024), we have 705 sites. I set aside a random third of them, 235 sites, as targets: places where we want to know the air quality but won’t put a monitor. The other 470 are candidates.

How many ways are there to pick 50 sites from 470? About 10^68. That’s a 1 followed by 68 zeros, more than the number of atoms that make up the Earth (a mere 10^50 or so). Even checking a billion networks a second, you’d still be going long after the Sun burns out.

So we need to be clever.

Step 1: Learn how the air behaves

Air pollution doesn’t happen site by site. A weather system parks over the Midwest, or wildfire smoke drifts in from Canada (as it did across the Northeast in June 2023), and whole regions go up and down together.

That’s useful. If you know how a few places are doing, you can make a good guess about their neighbours, a bit like guessing the score of a football match from the crowd noise outside the stadium.

Using the 2023 data, the model learns 20 regional patterns (weather systems, smoke plumes, the seasons) and how strongly each site follows each one. Those 20 patterns explain about 76% of the day-to-day ups and downs. The rest is local: a busy road, a neighbour’s barbecue, a wobbly sensor.

Once we have that model, we can score any network: how much of the regional pattern at the targets does it leave unexplained? Lower is better.

Step 2: Swap the light switch for a dimmer

Each candidate site is like a light switch: monitor on or off. All those on/off choices are what make the problem so huge.

So we cheat, temporarily. We replace every switch with a dimmer, so a site can have 0.4 of a monitor, or 0.93 of a monitor. The rule is that the dimmers must add up to 50.

That sounds silly, but it turns a bumpy, impossible search into a smooth convex problem. Picture a bowl: wherever you drop a marble, it rolls to the same lowest point. Jaddle’s convex solver found the bottom of this bowl in 36 steps, about 3 seconds.

Then we turn the dimmers back into switches. We keep the 50 brightest sites, then try swapping a chosen site for an unchosen one, keeping any swap that helps. After just 5 swaps nothing else improves.

Step 3: Get a receipt

This is my favourite part.

The dimmer version of the problem has more freedom than the real one: every real network is just a dimmer setting where each dial happens to be at 0 or 1. So the best dimmer answer is a floor. No real network of 50 monitors can ever do better than it.

Our network lands within 1.3% of that floor. So without checking all 10^68 options, we can say with confidence that no 50-monitor network beats ours by more than about 1.3% (as judged by the model).

Most methods hand you an answer. This one hands you an answer and a receipt.

A few things jump out of the map:

• The chosen sites aren’t evenly spaced. There are clusters along the West Coast and in the Northeast, where lots of people (and lots of target sites) live.
• Big empty areas still get a monitor or two. One well-placed sensor can “speak for” a whole region that moves together.
• Most dimmers ended up fully on or fully off. Only 47 of the 470 were in between, so turning the dimmers back into switches was easy.

Did it actually work? The 2024 exam

A model that’s only tested on the data it learned from is like a student marking their own homework. So the real test uses 2024, a year the model never saw.

For each network, we predicted the daily PM2.5 at all 235 target sites from that day’s readings at the 50 monitors, then measured the typical error:

The optimised network cuts the error by about 8% compared with random picks, on data it had never seen.

Two things surprised me:

1. Spreading monitors evenly does worse than random. It feels fair, but it puts sensors in big, empty, mostly clean areas. The largest errors come from smoggy days in crowded places, and an even spread under-covers those.
2. The simple “greedy” method is just as good here. I’ll be honest about that. Greedy is a strong, well-known approach. What it can’t give you is the receipt. Only the dimmer trick can tell you that nothing out there does much better.

How many monitors is enough?

Fifty was an arbitrary choice, so I asked Jaddle to solve the problem for 16 different budgets, from 5 to 150 monitors, in one batched call (using JAX’s vmap, which runs many copies of the solver side by side).

The headline:

• About 35 well-placed monitors predict as well as 150 randomly placed ones. Placement matters more than quantity.
• Returns shrink fast. The 150th monitor is worth roughly a fiftieth of the 16th. For a state deciding which monitors to retire, that curve shows where cutting starts to really hurt.
• The receipt gets better as the budget grows: within 5% of the floor from about 24 monitors, and within 0.1% at 150.

Bonus: which monitor deserves an upgrade?

The solver can also say how much better the best possible network would get if any single site had a less noisy sensor (for example, by installing a second instrument next to the first). One calculation covers all 705 sites at once.

The top of the list: Tacoma (South 36th St), Washington, followed by sites in California, Colorado, Idaho and Nevada. So the same tool that picks locations also points to where an equipment upgrade would help most.

The takeaways

• Many “choose the best few” problems are secretly huge. Picking 50 from 470 has more options than there are atoms in the Earth.
• Relaxing the rules helps you solve the strict version. Swapping switches for dimmers turns an impossible search into a 3-second solve.
• A good answer with a guarantee beats a good answer alone. The relaxation doesn’t just find a strong network; it proves there isn’t a much better one.
• Test on data the model hasn’t seen. The improvement held up in 2024, though it was smaller than the model predicted, because some pollution is just local and no network can see it coming.

Try it yourself

The full example, from downloading the EPA data to drawing the map, is examples/cp/monitor_network_design.py in the Jaddle repository. It runs in a few minutes and uses only public data.

If you work on sensor placement, experimental design or any other “where do I put my limited stuff?” problem, I’d love to hear how you approach it.

Keeping score

The test bed is SciFact, a set of scientific claims paired with the research abstracts that support or refute them. Every score in this post is nDCG@10 on its held-out test claims. The metric looks only at the top ten results. It gives more credit for a relevant abstract near the top than near the bottom, and a perfect ranking scores 1. A gain of 0.01 is one “point”.

The pipeline at a glance

flowchart LR
  Q[Claim] --> K[Keyword search<br/>BM25]
  Q --> D[Meaning search<br/>embeddings]
  K --> F[Rank fusion]
  D --> F
  F --> S[Top 100<br/>candidates]
  S --> R[Referee<br/>17-feature reranker]
  R --> T[Top 10<br/>to the model]

Layer 1: Match the words

The classic starting point is BM25. Search for “does coffee help memory?” and it scores each page by how often coffee, help and memory appear in it, with two refinements:

• Rare words count for more. Matching caffeine is far more telling than matching the.
• Long pages don’t win by bulk. A page can’t win just by containing a lot of words.

BM25 is fast, simple and hard to beat. It scores 0.681.

Its blind spot is vocabulary. If the claim says car and the abstract says automobile, BM25 can’t tell they’re related.

Layer 2: Learn which words matter

SciFact comes with training labels: for each training claim, the abstracts that are actually relevant. We can use them to tune BM25.

Give every word its own multiplier, starting at 1. Then adjust the multipliers so that, for each training claim, relevant abstracts outscore irrelevant ones by a margin. Words that point towards the right answer get boosted, and words that are mostly noise get turned down.

That lifts the score to 0.708. It’s better, but the helper still only sees words.

Layer 3: Learn what the words mean

A text embedding turns a passage into a list of numbers (384 of them here, from the compact bge-small model). Passages with similar meanings get similar lists, so car and automobile end up close together even though they share no letters.

To search, we embed the claim the same way and look for abstracts whose vectors point in the same direction. This is dense retrieval, and on its own it scores 0.720, already ahead of the tuned keywords.

Keyword search and dense retrieval make different mistakes, so it pays to combine them. Reciprocal rank fusion does this using only each page’s position in the two rankings. An abstract that ranks well in both rises to the top. Because it uses positions, it never has to reconcile two scores on different scales. The fused ranking scores 0.731. Its top 100 also holds 96.5% of the relevant abstracts, up from 92% for BM25’s top 100.

Layer 4: A referee for the shortlist

We don’t have to trust any single ranking. Take the fused top 100 as a shortlist, and bring in a referee to reorder it.

The referee looks at 17 features of each (claim, abstract) pair:

• Keyword signals: BM25 on the full text, on the title alone and on exact word forms, TF-IDF, and the learned term-weight score from Layer 2
• Meaning signals: embedding similarity, plus an older technique called LSA
• Coverage: how much of the claim the abstract mentions, including runs of consecutive words
• Context: the lengths of the claim and the abstract, and the fused rank itself

Every feature means the same thing for every claim. “High embedding similarity” is good news whatever the topic, so what the referee learns on training claims carries over to claims it has never seen.

The referee itself is simple: multiply each feature by a weight and add them up. The question is what the weights should be.

Where Jaddle comes in

One quadratic program is easy enough. But we don’t know the right in advance, so we solve the problem for 11 values of and keep whichever does best on a set of validation claims.

appears only in the objective’s linear term. Every problem on the path therefore has the same structure, and Jaddle solves all 11 side by side in a single batched call on the GPU. Each problem has about 3,600 variables, and the whole path takes about 3 seconds. As a check, the chosen problem is solved again with Clarabel, an interior-point solver, and the optimal values agree to about 1e-5.

Since the whole path is this cheap, there’s no need to guess . We can try every value and keep the best. The same trick fits the term weights in Layer 2.

Results

The full referee beats rank fusion by 4.0 points (95% interval +2.1 to +6.0, from a paired bootstrap over test claims). The validation set agrees: 0.817 against 0.791.

The embeddings do two jobs:

• They widen the net. Fusion’s shortlist contains relevant abstracts that BM25’s top 100 misses.
• They give the referee a signal keywords can’t. Without the three embedding features, the referee loses to plain fusion (0.724 against 0.731).

The learned weights show what the referee relies on. Embedding similarity carries the most weight. The learned term score and coverage come next, contributing the precision of exact keyword matches. LSA gets no weight at all, because the embeddings already provide everything it offers.

The takeaway

Good retrieval is built in layers:

1. Match the words. BM25 is a strong, cheap baseline.
2. Learn which words matter from labelled examples.
3. Add meaning with embeddings, so car and automobile finally get along.
4. Bring in a referee to reorder the shortlist, and let a fast batched solver choose its weights.

None of these layers is clever on its own. Stacked together, they pick the right pages more often, which gives the model a better chance of answering correctly.

The full example is examples/cp/rag_reranker.py in the Jaddle repository.

3. When the answer is needed now

A large language model performs billions of calculations for every word it writes. For a chatbot that is fine, since people read slowly. For a robot arm that must react within a few milliseconds, or a power grid balancing itself second by second, it is far too slow.

Convex problems used for control are often tiny. Picture a chain of four blocks joined by springs, with a wall at each end and three small motors pushing neighbouring blocks apart. Every half a second a controller looks at where the blocks are, plans the next few seconds of motor pushes, applies the first one, then plans again. This is called model predictive control. Each plan is a quadratic program with about a hundred variables. A good solver finishes it in a fraction of a millisecond.

Now make it a fleet of a thousand such systems: a thousand delivery drones, say, or a thousand batteries on a grid. Their problems all have the same shape and differ only in their current state. A batched convex solver on a GPU handles all thousand at once, in lockstep, and each plan starts from the previous half-second’s plan shifted forward, which is an excellent first guess. On one laptop GPU this works out to about one fortieth of the time per problem of a well-tuned CPU solver working through the fleet one problem at a time.

There is an honest caveat. For a single system on its own, the CPU solver wins, because a lone tiny problem leaves a GPU sitting idle. A GPU is a bus, not a sports car. It shines when there are lots of passengers going the same way.

Two more things make convex solvers a comfortable fit for deadlines:

• They keep their promises along the way. A solver can be built so that every intermediate answer already respects the hard limits, such as “the motor can’t push harder than this”. If the deadline arrives before the solver has fully converged, the plan you take is slightly less than perfect, but it is not illegal.
• The trained model is often tiny. A linear search ranker scores a document with seventeen multiplications and some additions. That is cheap enough to run on every candidate document without anyone noticing.

Friends, not rivals

That pattern repeats, and it suggests a sensible division of labour.

• Foundation models make features, convex models make decisions. A neural network turns messy text, images or sound into useful numbers. A convex model combines those numbers under rules you can state and audit.
• Convex layers act as guardrails. A neural network proposes, and a convex problem projects that proposal onto the set of things that are allowed: within budget, within physical limits, within the fairness rules.
• Language models can write the bowl. Describing a problem in a modelling language is something language models are increasingly good at. The convex solver then supplies what the language model cannot: a certified optimum and a set of shadow prices that explain it. The language model writes the essay and the solver checks the arithmetic.

There are also places where convexity is simply the wrong tool. Recognising a cat in a photo, translating a poem, holding a conversation: nobody knows how to write these down as bowls, and the mountain-range methods are the right answer. The trick is to notice when part of your problem is a bowl, and to treat that part accordingly.

Why general matters

For decades, convex optimisation came in specialised boxes: one solver for linear programs, another for quadratic programs, a third for this particular statistics problem. Each had its own input format and its own quirks. Using one meant first translating your problem into the box’s language, which was often the hardest part.

A general solver turns this around. You write the objective and the rules as ordinary code, and the solver works out the rest. Here is the whole definition of a monotone curve fit:

import jax.numpy as jnp
import jaddle.jaddle_convex as jc

y = jnp.linspace(-1.0, 1.0, 100) ** 3   # data to fit

cp = jc.JaddleCP(
    num_variables=100,
    objective=lambda p: jnp.sum((p - y) ** 2),   # stay close to the data
    constraints_eq=lambda p: jnp.zeros(0),       # no equality rules
    constraints_ineq=lambda p: p[:-1] - p[1:],   # each value <= the next
    lower_bounds=-jnp.ones(100),
    upper_bounds=jnp.ones(100),
)

result = jc.solve(cp)

The same machinery then handles a search ranker, a fair lender, a thousand-strong fleet of controllers, a log-optimal investment portfolio or a noisy-channel capacity calculation, on a laptop CPU or a GPU, without a new solver for each. Because it is built on the same tools as modern deep learning, it can be batched, differentiated and dropped into a larger model like any other layer.

When the cost of trying a convex formulation falls to a few lines of code, it stops being the thing you reach for once everything else has failed and becomes the first thing you try. That is the real argument for a general solver. Convexity was never about the mathematics being pretty (though it is). It is about making the guarantees, explanations, speed and cheap experiments easy enough to use every day.

In short

Foundation models are superb mountaineers, and the world is full of mountains. But it is also full of bowls: budgets, schedules, controllers, rankers, fair classifiers, portfolios. When you meet one, you are in luck. You can find the very bottom, prove that you have found it, explain why it is there, and do it again for a thousand slightly different bowls before lunch.

So the next time a problem lands on your desk, before reaching for the biggest model you can find, it is worth asking a small question first:

Is this a bowl?

The benchmark set

The test set is the 383 MIPLIB 2017 instances used to evaluate PDLP and cuPDLP, taken from the mip_relaxations_instance_list in FirstOrderLp.jl. Each instance has between 100,000 and 10 million nonzeros. Integrality is dropped, so every instance is solved as a plain LP.

The set is hard for first-order methods for reasons that have little to do with size:

• Wide coefficient ranges. Many instances mix coefficients of 1 with big-M terms of 1e6 or more. Diagonal scaling cannot fully remove this, and it slows Jaddle down a lot.
• Hidden objectives. Some models put the objective in a dense equality row, such as min z subject to z = Σ aⱼxⱼ. The real cost structure is then hidden in the constraints, and the primal iterates drift off feasibility.
• Degeneracy and long tails. Many relaxations have large optimal faces or near-parallel constraints. Iterates then circle the optimum for a long time before the duality gap closes.

The instances range from 150 to 1.45 million columns and from 36 to 2.95 million rows. One configuration has to handle all of them.

Experiment setup

Every instance ran through the same pipeline with the same solver settings. Nothing was tuned per instance.

An instance counts as converged when Jaddle returns a full optimality certificate on the presolved LP. All three residuals must be below the tolerance. Jaddle does not stop on a heuristic such as objective stalling.

The headline time is the iteration loop, including the one-off XLA compile on the first epoch. Wall time adds presolve, scaling, setup and postsolve. Unsolved instances count at the full 3,600 seconds in every average, as in the PDLP papers.

Presolve: HiGHS plus a Jaddle-specific reduction

Each instance goes through jl.solve_with_presolve with the defined-variable elimination switched on. It runs five steps:

1. HiGHS presolve removes redundant rows and columns, fixes variables and tightens bounds.
2. Defined-variable elimination, Jaddle’s own reduction, removes dense defining rows that HiGHS keeps.
3. Jaddle solves the reduced LP.
4. Elimination postsolve recovers the eliminated variables and the duals of their rows.
5. HiGHS postsolve maps the primal and dual solution back to the original model.

Presolve is cheap next to the solve. The median presolve time was 0.28 seconds. HiGHS kept a median of 95% of the columns. Presolve also solved one instance outright: neos-787933, with 236,376 columns, which counts as one of the 382.

Results

Jaddle certified 382 of 383 instances, and most of them in seconds. Half finished in under 3.2 seconds, 90% in under 25 seconds, and only four needed more than five minutes.

The table below gives shifted geometric means with a 10-second shift (SGM10), the PDLP papers’ summary statistic. hgms62 counts at 3,600 seconds.

The median instance needed only 8 epochs. The long tail is short. Apart from hgms62, the slowest instances were snp-02-004-104 at 761 seconds, hgms30 at 1,190 seconds and fhnw-binschedule1 at 1,550 seconds. All three finished well inside the limit.

The gap between the first two rows is compile time. Every new problem shape triggers an XLA compile, which costs one to two seconds. On small instances it is often most of the solve. Jaddle caches compiled programs on disk, so repeat solves of a same-shaped model skip it.

Case study: hgms62, the one that got away

hgms62 is the only instance Jaddle did not certify with default settings. It ran for the full hour, about 2.9 million iterations, and stopped close to the tolerance but not inside it.

import jaddle.jaddle_linear as jl

result = jl.solve_with_presolve(
    "hgms62.mps",
    eliminate_defined_vars=True,  # as in the benchmark
    primal_feasibility_tolerance=1e-4,
    dual_feasibility_tolerance=1e-4,
    dual_gap_tolerance=1e-4,
    diag_clamp=1e8,  # default 10.0
)

Caveats and reproducing the run

Three points matter when reading these numbers.

• The certificate is on the presolved LP. After HiGHS postsolve, 297 of the 383 points also pass the 1e-4 certificate on the original LP. Postsolve can enlarge a first-order solution’s small dual errors. Presolve can also rescale a problem a lot, which changes what a relative tolerance means. On leo1, presolve folds a row with 1e7 coefficients into the cost, so the objective is right but the original dual test fails. When the original certificate matters, polishing, iterative refinement or crossover can tighten the postsolved point.
• The time limit matters for the hardest few. With a 600-second limit, earlier runs certified 380 of 383. hgms30 and fhnw-binschedule1 need 20 to 26 minutes.
• GPU runs are not bit-for-bit repeatable. The order of scatter-add operations varies, so identical runs can differ by several epochs.

The MPS files are not shipped with Jaddle. Download the 383 instances from the MIPLIB 2017 site into data/, then run from the repository root:

python benchmarks/benchmark.py --tol 1e-4 --max-seconds 3600 --highs-solver none

--highs-solver none skips the HiGHS reference solve, which can take longer than Jaddle’s. Add --diag-clamp 1e8 to rerun hgms62 with the wider clamp. The harness and its options are documented in benchmarks/README.md.

What problem are we solving?

Jaddle is a program that solves optimisation problems. That means: pick some numbers so that something is as small (or as big) as possible, while following a set of rules.

Here is a tiny one. You are baking. Flour costs $2 a kilo, and your recipe needs at least 3 kilos. How much flour should you buy to spend as little as possible?

You already know the answer: buy exactly 3 kilos and spend $6. Easy.

Now imagine an airline planning thousands of flights, a power company deciding which power stations to switch on, or a delivery company routing hundreds of trucks. These problems have millions of numbers to choose and millions of rules to follow. Nobody can eyeball those. We need a recipe that a computer can follow, step by step, and that recipe is what this article is about.

Mathematicians write this kind of problem (a linear program) like this:

In words:

• is the list of numbers we get to choose (how much of each thing to buy).
• is the list of prices, and is the total cost.
• Each row of is one rule (“you need at least this much of that”).
• are simple limits on each number, like “you can’t buy a negative amount of flour”.

For the flour problem there is just one number, = kilos of flour. The cost is , the rule is (which we can write as ) and the limit is .

Jaddle solves these problems in five big ideas. Let’s go through them one at a time.

1. Turn rules into fines (dualising and the Lagrangian)

Rules are awkward for computers. They act like invisible walls, and walls are hard to deal with when there are millions of them.

So we play a trick: we replace every rule with a fine.

Imagine a referee who watches the shopper. For each rule, the referee chooses a fine, , to charge for every kilo you are short. If you buy kilos you are kilos short, so the fine is . Your total bill becomes

This bill has a famous name: the Lagrangian, after Joseph-Louis Lagrange, a mathematician from the 1700s. For any linear program it looks like this:

There is one fine for each rule, and the list of fines is called the dual variables. (The shopper’s numbers are the primal variables.) Swapping rules for fines like this is called dualising the constraints. Fines can’t be negative, because a referee doesn’t pay you for breaking a rule. If you have extra flour, is negative, and the referee’s best move is to set the fine to zero. (Rules of the form “exactly equal to” also work. Their fines are allowed to be negative.)

2. Take turns walking downhill and uphill (alternating gradient descent–ascent)

How do the shopper and the referee find the saddle point? They feel which way the ground slopes and take small steps.

The gradient is the slope: it says which direction is uphill and how steep it is. For the flour bill the slopes are

These say something sensible:

• If the fine is less than $2, flour costs more than the fine it saves you, so the shopper wants to buy less.
• If is less than 3, the shopper is short of flour, so the referee wants to make the fine bigger.

For any linear program the slopes are for the shopper and for the referee.

The shopper walks downhill (makes the bill smaller) and the referee walks uphill (makes the bill bigger). They take turns, and the referee always reacts to the shopper’s newest choice. That is why it is called alternating:

Here (tau) and (sigma) are step sizes: how far each player moves per turn. “clip” pushes back inside its limits, and stops fines going negative.

Each turn only needs two multiplications by the big table of rules: one by and one by its flipped version . That is cheap even when has millions of rows, and graphics cards (GPUs) are very good at it. Methods that only use slopes like this are called first-order methods.

function simulate(eta, n) {
  let x = 0, y = 0, sx = 0, sy = 0;
  const pts = [];
  for (let i = 1; i <= n; i++) {
    x = Math.max(0, x - eta * (2 - y));   // shopper walks downhill
    y = Math.max(0, y + eta * (3 - x));   // referee walks uphill
    sx += x; sy += y;
    pts.push({i, x, y, ax: sx / i, ay: sy / i});
  }
  return pts;
}

function saddlePlot(pts, showAvg) {
  const marks = [
    Plot.line(pts, {x: "x", y: "y", stroke: "#4c78a8", strokeOpacity: 0.45}),
    Plot.dot(pts, {x: "x", y: "y", r: 2.5, fill: "#4c78a8"})
  ];
  if (showAvg) {
    marks.push(Plot.line(pts, {x: "ax", y: "ay", stroke: "#f58518", strokeWidth: 2.5}));
    marks.push(Plot.dot(pts.slice(-1), {x: "ax", y: "ay", r: 6, fill: "#f58518"}));
  }
  marks.push(Plot.dot([{x: 3, y: 2}], {x: "x", y: "y", r: 7, fill: "#e45756", stroke: "white"}));
  marks.push(Plot.text([{x: 3, y: 2}], {x: "x", y: "y", text: d => "saddle point (3, 2)", dy: -14}));
  return Plot.plot({
    width: 520,
    height: 420,
    grid: true,
    x: {domain: [0, 6.5], label: "x = kilos of flour (shopper) →"},
    y: {domain: [0, 5.5], label: "↑ y = fine per missing kilo (referee)"},
    marks
  });
}

viewof eta1 = Inputs.range([0.05, 1.2], {value: 0.3, step: 0.05, label: "Step size"})
viewof n1 = Inputs.range([5, 300], {value: 60, step: 5, label: "Number of turns"})
saddlePlot(simulate(eta1, n1), false)

3. The average is smarter than the walker (averaging)

What is in the middle of a circle? The centre! And if you take lots of points spread evenly around a circle and average them, you land near the centre.

So Jaddle keeps a running average of everywhere the players have been. Write for both players’ positions together. After turns the average is

Each position is weighted by the step size used to reach it (more on in the next section), so bigger steps count for more. Jaddle doesn’t store every old position. It nudges the average a little towards each new point, which costs almost nothing.

Think of a dog on a lead running in circles around a post. Take a photo every second and average the dog’s position over all the photos: you get the post.

The orange line below is the average. Turn up the number of turns and watch it home in on the answer.

viewof eta2 = Inputs.range([0.05, 1.2], {value: 0.3, step: 0.05, label: "Step size"})
viewof n2 = Inputs.range([5, 300], {value: 150, step: 5, label: "Number of turns"})
saddlePlot(simulate(eta2, n2), true)

There is a catch. The average gets there slowly: to be 10 times more accurate you need about 10 times more turns. It also never forgets. The very first guesses, which were far from the answer, keep dragging the average away forever. We fix both problems in section 5.

4. Choosing step sizes: , , and

Steps that are too big overshoot and fly off. Steps that are too small take forever. Worse, different numbers in the same problem can need very different step sizes. One might be measured in millimetres and another in kilometres.

Jaddle builds every step size out of four pieces:

is the step for the shopper’s -th number, and is the step for the referee’s -th fine. Think of a group out on a hike:

Piece	Who it affects	Hiking version
(eta)	everyone	how fast the whole group walks
	shopper vs. referee	whether the shopper team or the referee team walks faster
	the shopper’s -th number	how long one shopper’s stride is
	the referee’s -th fine	how long one referee’s stride is

Before any of this starts, Jaddle rescales the problem so that the numbers in , and are all roughly the same size. It is a bit like converting every measurement to the same units. (The methods are called Ruiz and Pock–Chambolle scaling, if you want to look them up.) On top of that, and start at 1 and are then adapted as the solver learns more about the problem. Giving each number its own multiplier like this is called diagonal preconditioning: “diagonal” because if you wrote the multipliers into a table, they would only appear along its diagonal.

and : every number gets its own stride

and treat big groups of numbers the same. But inside a problem with a million numbers, some are marching steadily towards their answer while others are jittering back and forth. and give each number its own stride.

At each restart, Jaddle looks back over the last stretch at each number and works out two things:

• its net distance : how far it ended up from where it started;
• its total wiggle : the size of every step it took, squared and added up.

Then it divides one by the other:

Here is what (rho) looks like for three numbers that each took 16 steps of size 1, so for all of them:

Number	What it did	Net distance		Meaning
A	16 steps forward	16	4	marching steadily
B	10 forward, 6 back	4	1	wandering a bit
C	forward, back, forward, back…	0	0	jittering on the spot

A big means the number knows where it is going, so it should take bigger steps. A small means it keeps overshooting and coming back, so it should take smaller steps. Jaddle updates each number’s stride with

where is the geometric mean of all the ’s. If only A and B were in the problem, :

• A’s stride is multiplied by , so it gets 41% longer.
• B’s stride is multiplied by , so it gets 29% shorter.

Dividing by the typical value keeps the group’s overall pace the same, so and stay in charge of that. Each stride is also kept within 10 times bigger or smaller than where it started, and numbers that didn’t move at all (for example ones stuck at a limit) give no information, so they keep their stride. The referee’s fines get exactly the same treatment to update .

This idea borrows from machine learning methods called AdaGrad and DoG (“distance over gradients”). On a standard collection of hard test problems (the MIPLIB 2017 linear programs) it cut Jaddle’s solve times by about 18% on average.

5. Restarts: when in doubt, start fresh

Remember the two problems with averaging? It is slow, and it never forgets the bad early guesses.

The fix is the restart. Once the average has got a lot closer to the answer, Jaddle jumps both players to the average, throws away all the history, and starts a fresh average from there.

Why does that help so much? The size of the circle the players run around depends on how far from the answer they start. Start closer and the next circle is smaller, so the next average is closer still, and so on. Each restart shrinks the error by a fixed fraction. “Ten times more turns for ten times more accuracy” turns into “a few more restarts for each extra digit of accuracy”.

restartData = {
  const merit = (x, y) => Math.max(
    Math.max(0, 3 - x) / 4,
    Math.max(0, y - 2) / 3,
    Math.abs(2 * x - 3 * y) / (1 + Math.abs(2 * x) + Math.abs(3 * y))
  );
  const step = (x, y, eta) => {
    x = Math.max(0, x - eta * (2 - y));
    y = Math.max(0, y + eta * (3 - x));
    return [x, y];
  };
  const eta = 0.4, N = 300, floor = 1e-16, rows = [];

  // No restarts: score the current point and the average.
  let x = 0, y = 0, sx = 0, sy = 0;
  for (let i = 1; i <= N; i++) {
    [x, y] = step(x, y, eta);
    sx += x; sy += y;
    rows.push({i, score: Math.max(floor, merit(x, y)), method: "Current point only"});
    rows.push({i, score: Math.max(floor, merit(sx / i, sy / i)), method: "Average"});
  }

  // With restarts.
  x = 0; y = 0; sx = 0; sy = 0;
  let n = 0, mLast = merit(0, 0), prev = Infinity;
  for (let i = 1; i <= N; i++) {
    [x, y] = step(x, y, eta);
    n++; sx += x; sy += y;
    const ax = sx / n, ay = sy / n;
    const ma = merit(ax, ay), mi = merit(x, y), m = Math.min(ma, mi);
    rows.push({i, score: Math.max(floor, m), method: "Average + restarts"});
    const good = m <= 0.2 * mLast;
    const stalling = m <= 0.8 * mLast && m > prev;
    if (n >= 10 && (good || stalling)) {
      if (ma <= mi) { x = ax; y = ay; }
      sx = 0; sy = 0; n = 0; mLast = m; prev = Infinity;
    } else {
      prev = m;
    }
  }
  return rows;
}

Plot.plot({
  width: 620,
  height: 380,
  grid: true,
  x: {label: "turns →"},
  y: {type: "log", label: "↑ wrongness score (log scale)", tickFormat: ".0e"},
  color: {
    legend: true,
    domain: ["Current point only", "Average", "Average + restarts"],
    range: ["#4c78a8", "#f58518", "#54a24b"]
  },
  marks: [
    Plot.line(restartData, {x: "i", y: "score", stroke: "method", strokeWidth: 2})
  ]
})

Putting it all together

Here is the whole recipe in one place:

rescale the problem so its numbers are all about the same size
start with x = 0 (inside its limits), y = 0, η = 1, k = a guess, all t and s = 1

repeat:
    take a trial alternating step with τ = (η / k)·t and σ = η·k·s
    if the step was too big:  shrink η and try again
    otherwise:                keep it, update the average, let η grow a little

    every so often:
        work out the wrongness score
        if all three checks are tiny:  stop, we have the answer!
        if a restart rule fires:
            jump to the better of the current point and the average
            update k, t and s
            start a fresh average

Every part of this is either a multiplication by or , or simple arithmetic on each number separately. There are no big equation-solving steps, which is why it works on problems with millions of numbers and runs so well on graphics cards. Jaddle is written in JAX, a Python library that turns the whole loop into one fast compiled program. The same code runs on a laptop CPU, a GPU or a TPU.

Jaddle also has a second solver for curvier problems (convex programs), where the cost and rules can be more complicated than straight lines. It uses the same five ideas. The only difference is that it can’t write the slopes down as , so JAX works them out automatically.

Cheat sheet

Symbol	Name	What it means
	primal variables	the shopper’s choices
	dual variables	the referee’s fines, one per rule
	Lagrangian	the total bill: cost plus fines
	step sizes	how far the shopper / referee move per turn
	average	the weighted average of everywhere they’ve been
	base step	the whole group’s pace, learned as it goes
	safe step	the largest step that won’t make them overreact
	primal weight	shares the pace between shopper and referee
	diagonal	each number’s personal stride
	drift ratio	net distance ÷ total wiggle
	wrongness score	the largest of: broken rules, unfair fines, gap

The rules of the game

We need to decide what fraction of our money goes into each of the 500 assets. The rules are:

• The fractions must add up to 100%. We invest all the money, no more and no less.
• No fraction can be negative. In finance-speak, the portfolio is long-only: we can’t bet against an asset or borrow to invest.
• The portfolio’s volatility (how much it’s expected to wobble) must stay below a limit we choose.

Within those rules, we want the highest expected return.

Two kinds of wobble

To measure wobble, we use what’s called a factor model. It says each asset’s ups and downs come from two sources.

Shared wobble. Some forces push lots of assets around at once. Think of an ice-cream shop and a sunscreen company: a heatwave is good for both, and a rainy summer is bad for both. In our example there are 50 of these shared forces, or factors, and each asset reacts to them in its own way.

Personal wobble. Every asset also has its own private surprises. A particular shop might get a rave review or a broken freezer. These are unrelated to everything else.

This distinction matters because the two kinds of wobble behave differently when you combine assets:

• Personal wobbles tend to cancel out. If you own lots of different things, one asset’s bad day is often balanced by another’s good day. This is the magic of diversification.
• Shared wobbles don’t cancel. If all your assets love sunny weather, owning more of them doesn’t protect you from rain.

Where the ice-cream cone comes in

To work out a portfolio’s total wobble, you collect all its exposures, to each shared factor and to each asset’s personal risk, and measure their combined size, the way you’d measure the length of an arrow from its parts.

The rule “this combined size must be below the limit” has a surprising shape. If you drew all the portfolios that satisfy it (and let the limit itself vary), you’d get a shape like a cone. It even has a nickname among optimisation folk: the ice-cream cone. Its formal name is the second-order cone.

Why does that matter? Because some optimisation solvers can only handle flat-sided shapes, built from straight-line rules like “this plus that must be at most ten”. A cone has a curved, rounded surface, so those solvers can’t take it directly. Jaddle can: at each step it takes a stride towards a better answer, then gently pushes the result back inside the cone if it’s wandered out.

Tracing the efficient frontier

Rather than picking one wobble limit, we try five, ranging from cautious (6% volatility) to adventurous (20%). For each one, we find the best possible portfolio.

If you plot the results, with risk along the bottom and expected return up the side, you get a curve that finance people call the efficient frontier. Every point on it is a portfolio where you can’t get more expected return without accepting more risk. Any portfolio below the curve is leaving money on the table.

Two things make tracing this curve quicker:

• Warm starts. Each new solve begins from the previous answer instead of from scratch. Moving the limit from 6% to 8% doesn’t change the best portfolio completely, so starting nearby saves a lot of work.
• Reusing the setup. The problem is described once, and only the wobble limit changes between solves.

The slope tells a story

Every time the solver obeys a rule, it can also report how much that rule is costing. For the wobble limit, this number answers the question: “If I allowed a little more wobble, how much extra return would I get?”

That’s precisely the slope of the efficient frontier at that point, and it shrinks as the limit grows. When you’re being very cautious, accepting a bit more risk buys you a healthy boost in return. When you’re already being adventurous, each extra bit of risk buys less and less. Like adding sugar to tea: the first spoonful makes a big difference; the fifth, not so much.

What the portfolios look like

At the cautious end, the best portfolio is spread across many assets, leaning hard on diversification to cancel out personal wobbles.

As the limit loosens, the portfolio concentrates on fewer assets with higher expected returns. Freed from the need to stay calm, it stops spreading its bets and starts backing its favourites.

Checking our work

To make sure Jaddle’s answers are right, we solve every problem a second time with Clarabel, a well-respected solver that ships with CVXPY. The returns agree to within a tiny fraction of a percent, and so do the portfolios themselves.

The takeaway

Investing well isn’t just about picking winners. It’s about understanding how your choices wobble together. By describing risk as the size of an arrow, and using a solver that’s comfortable with the curved, cone-shaped rules that come with it, we can find, for any level of nerve, the best portfolio for that amount of risk. And we get a bonus: a clear reading of exactly how much extra return each extra bit of courage would buy.

Two questions hiding inside one

It turns out “how big a battery should I buy?” contains a second question tucked inside it: “how would I actually use it?”

You can’t judge a battery without knowing how it would be run. A big battery used badly might save less than a small battery used cleverly. So, to size the battery, we need to know how to operate it well on every kind of day it might face.

That gives us two layers:

1. The operating problem (every day): given a battery of a certain size, when should it charge and when should it discharge to make the electricity bill as small as possible?
2. The sizing problem (once): given that we’ll always run the battery as well as we can, which size gives the lowest total cost, counting both the battery’s price and the bills?

One day at a time

We split each day into 24 hours. Each hour has:

• an electricity price, which peaks in the morning and, even more, in the evening;
• a demand, the amount of power the building needs, which follows a similar pattern;
• some solar output, which peaks around lunchtime and depends on how cloudy the day is.

The battery has two numbers that describe its size, and it’s worth knowing the difference:

• Energy capacity (in kilowatt-hours) is how much it can store. Think of it as the size of a water tank.
• Power rating (in kilowatts) is how fast it can charge or discharge. Think of it as the width of the tap.

A huge tank with a tiny tap can’t empty fast enough to cover the evening rush. A wide tap on a tiny tank runs dry in minutes. You need a sensible match.

Planning one day’s battery use, given the prices, the demand, the sunshine and the battery’s size, is a classic linear program: a puzzle where you choose a bunch of numbers (how much to charge, discharge and buy each hour) to minimise a cost, while obeying straightforward rules like “you can’t store more than the tank holds” and “the lights must stay on”. The battery also has to end the day as full as it started, so it can’t cheat by borrowing from tomorrow.

Thirty-two days of weather

One day isn’t enough to make a decision. Some days are sunny, some are gloomy, and prices bounce around. So we generate 32 sample days, each with its own mix of prices, demand and sunshine, and judge each battery size by its average performance across all of them.

That means every time we want to test a battery size, we have to solve 32 operating puzzles. That could get slow.

The bulk-discount twist

There’s one more wrinkle, and it’s the reason this example exists. Batteries enjoy economies of scale: a bigger battery costs more, but not proportionally more. In our model, doubling the battery’s size makes it about 1.74 times as expensive, not twice as expensive.

That sounds like good news for your wallet, and it is. But it’s bad news for the maths. If cost were perfectly proportional to size, the whole thing (sizing plus all 32 days) could be bundled into one giant linear program and solved directly, very quickly. The bulk discount bends the cost curve, and that bend breaks the neat linear structure. We need a different approach.

Feeling for the slope

Here is the trick. Imagine you’re standing on a hillside in thick fog, and you want to reach the bottom of the valley. You can’t see the bottom, but you can feel which way the ground slopes under your feet. So you take a step downhill, feel again, take another step, and so on. This is called gradient descent, and it’s one of the most widely used ideas in all of computing.

To use it, we need the slope: if we made the battery a tiny bit bigger, would the total cost go up or down, and by how much?

The lovely part is that the operating puzzles hand us this information for free. When a linear program finishes, its solution comes with extra numbers that say how much the best possible cost would change if each limit were loosened a little. One of those limits is “the battery can’t hold more than its capacity”, so the solver already knows how much a slightly bigger battery would have saved on that day. We don’t have to do any extra work to get the slope.

Jaddle solves all 32 daily puzzles at once, in a single batch, collects the slopes, and combines them with the slope of the battery’s price. Then we take a step downhill and repeat. Forty steps later, we’ve arrived.

The answer

Across our 32 sample days:

• With no battery, the site spends about $6.02 per day on electricity.
• With the battery that gradient descent found, the total daily cost (battery and bills) drops to about $3.83 per day.

The best battery stores roughly 14 to 15 kilowatt-hours.

Did we find the real bottom?

Because of that bulk-discount bend, the landscape isn’t a perfectly smooth bowl. Gradient descent always walks downhill, but it can get stuck in a small dip that isn’t the deepest point. And there’s no exact solver for this problem we can check against.

So we did the next best thing: we started again from a completely different place. The first run began with a tiny battery and grew it. The second began with an absurdly oversized battery and shrank it. Both hikers walked down through the fog and met in the same valley. Their costs agreed to within 0.03%.

Their battery sizes agreed slightly less closely (about 14.3 versus 14.6 kilowatt-hours). That’s not a problem; it tells us the valley floor is wide and flat. Near the best size, a little bigger or a little smaller makes almost no difference to the cost. That’s handy in real life, where batteries only come in certain sizes anyway.

What the battery actually does

If you look at the plan for the sunniest day, the battery behaves just as you’d hope. It fills up around midday when the solar panels are producing more than the building needs, then drains during the expensive evening peak, cutting the amount of power bought from the grid when prices are highest.

The takeaway

Big decisions often depend on lots of small ones. Here, a one-off purchase depends on thousands of hour-by-hour choices across many different days. By solving all the small problems together, and using the slopes they give away for free, we can steer the big decision step by step until it’s just right. Not too big, not too small.

Two sources of information

You have two things to go on:

1. The measurements, your friend’s shouted positions. Most are close; some are wild.
2. The physics. Boats don’t teleport. They glide along, speeding up and slowing down smoothly, nudged now and then by gusts of wind you can’t predict.

The trick is to combine them. You’re looking for a path that obeys the physics and stays reasonably close to the measurements. If a measurement says the boat jumped five metres to the left in an instant, the physics says “that’s not possible”, and the path should mostly ignore it.

Finding the best such path, after the fact, using all the measurements at once, is called smoothing.

The classic recipe: the Kalman smoother

In the 1960s, Rudolf Kalman developed a beautifully efficient method for exactly this kind of problem. Versions of his ideas helped steer the Apollo spacecraft to the Moon, and today they run inside phones, cars, drones and countless other gadgets.

The classic approach chooses the path that keeps two things small at once:

• the wind gusts needed to explain the path (smaller gusts mean a more natural path), and
• the misses, the gaps between the path and each measurement.

To combine all the misses into one number, it squares each one and adds them up. Squaring makes sense when errors are small and well-behaved: it treats misses in either direction equally and is very convenient mathematically.

The problem with squaring

Squaring has a weakness. A miss that’s twice as big counts four times as much. A miss that’s twenty times as big counts four hundred times as much.

In our example, the occasional wild measurement is about twenty times noisier than a normal one. So when the classic smoother tries to keep the squared misses small, those few wild measurements dominate everything. They act like bullies, dragging the estimated path towards themselves, because ignoring them would be “too expensive”.

It’s a bit like a teacher averaging test scores where one student’s answer of “a million” for a question with answer “seven” drags the class average into the stratosphere.

A fairer referee: the Huber penalty

The fix is to change how misses are scored. Instead of squaring every miss, we use the Huber penalty, named after the statistician Peter Huber:

• For small misses, it behaves just like squaring. Normal, everyday noise is treated exactly as before.
• For big misses, it switches to growing in a straight line. A miss twice as big costs only about twice as much, not four times.

This version is called the robust Kalman smoother. “Robust”, in statistics, means “doesn’t fall apart when some of the data is bad”.

The experiment

We simulate an object moving around a flat plane for 50 seconds, measured 1,000 times. It has a position and a speed, a little friction slowing it down, and random pushes knocking it around. Each measurement has a one-in-five chance of being an outlier.

Then we estimate the path twice: once with the classic smoother and once with the robust one.

The results are striking. The classic smoother’s path is visibly dragged around by the outliers, wandering off towards the wild measurements. The robust smoother’s path sticks closely to the true trajectory, as if it had simply noticed which shouts to take with a pinch of salt.

What Jaddle does differently

Both versions of the problem are written in CVXPY, a popular language for describing optimisation problems. In fact, this is one of CVXPY’s own showcase examples.

The classic smoother is a relatively simple kind of problem (a quadratic program), and Jaddle solves it directly.

The robust version is more interesting. Normally CVXPY would translate the Huber penalty into a bundle of cone-shaped rules, which works but makes the problem bigger and more complicated. Jaddle takes a different route: it keeps the Huber penalty as the smooth, gently curving function it really is, and solves the problem in that form. It also notices that the “miss” for each measurement is completely determined by the path, so it can drop those as separate unknowns and work with the path alone. Fewer moving parts, same answer.

We check both of Jaddle’s answers against Clarabel, a well-established solver that comes with CVXPY, and they agree.

The takeaway

Real-world data is rarely perfectly behaved. Sensors glitch, signals bounce, people mishear. A method that assumes every measurement is roughly right can be thrown badly off course by a handful that aren’t.

By changing one small thing, how we score big misses, the robust Kalman smoother learns to listen to the crowd without being bullied by the loudest fibbers. It’s a good lesson beyond boats and ponds: when you’re combining lots of evidence, make sure no single piece gets to shout over all the rest.

The jiggling-blocks system

This setup comes from a well-known 2010 research paper by Yang Wang and Stephen Boyd, and it’s become a standard test for control methods. The rules are:

• Each motor has a strength limit. It can only push or pull so hard.
• Each block should stay within a safe zone around its resting position.
• We want the blocks to settle down without the motors working harder than necessary.

Thinking ahead: model predictive control

How do you decide what the motors should do? One of the most powerful approaches is called model predictive control, or MPC.

MPC works a lot like a good chess player:

1. Look at the board. Measure where every block is and how fast it’s moving.
2. Think several moves ahead. Using a model of how springs and blocks behave, plan the best sequence of motor pushes for the next 15 steps (about seven and a half seconds).
3. Make only the first move. Apply the first push in the plan.
4. Look again and re-plan. The world never goes exactly as predicted (those random shoves!), so throw the old plan away and make a fresh one from the new position.

This loop of plan, act, re-plan makes MPC both clever and adaptable. It’s used in self-driving cars, chemical plants, robots, drones and even some heating systems.

Each plan is a small optimisation puzzle called a quadratic program, with 105 numbers to choose and 120 rules to obey.

Soft walls instead of hard ones

There’s a subtle problem. Sometimes a random shove knocks a block so hard that no plan, however clever, can keep it inside its safe zone. If the rules said “the block must stay inside, no exceptions”, the puzzle would have no solution and the controller would be stuck.

So the safe zones are made soft. A block is allowed to stray outside, but the plan pays a hefty fine for every bit it goes over. The fine is large enough that whenever staying inside is possible, the plan does exactly that. It’s only when there’s no other choice that a block is allowed to go a little over, and even then the plan does its best to bring it back.

The motor limits, on the other hand, stay hard. A motor physically can’t push harder than its maximum, so the plan is never allowed to ask it to.

One at a time versus all together

A single one of these puzzles is tiny. A good solver running on an ordinary computer processor (CPU) can crack one in well under a thousandth of a second. That’s very hard to beat.

But we don’t have one system. We have a thousand, each starting from its own random position and getting its own random shoves. At every step, we need a thousand fresh plans.

Here’s where the two approaches differ:

• The one-at-a-time approach solves the thousand puzzles one after another, using a well-known solver called SCS. It’s a bit like a single brilliant runner delivering a thousand letters, one trip at a time.
• The all-together approach uses Jaddle on a graphics card (GPU). All thousand chains have the same springs, motors and rules; only their positions differ. So Jaddle solves all thousand puzzles side by side, in lockstep, like a marching band where every musician takes the same step at the same time.

Both approaches use a warm start: each new plan begins from the previous step’s plan, shifted forward by one step. Since the world hasn’t changed much in half a second, the old plan is usually a great starting guess.

And the winner is…

It depends on how many systems you have, which is the most interesting part of the whole example.

• One system: the one-at-a-time solver wins. A single tiny puzzle leaves a GPU almost entirely idle; it spends most of its time just getting started.
• Tens of systems: the batched approach catches up.
• A thousand systems: the batched approach on a laptop GPU spends about one-fortieth as long per puzzle as the one-at-a-time solver, and its plans are slightly more accurate too. Even running the batched approach on an ordinary CPU, it’s about three times faster than looping.
• Ten thousand systems: something new happens…

The slowest member of the band

Remember the marching band? The band only finishes when its slowest member finishes. In a fleet of ten thousand, there are always a few chains in a tricky spot, perhaps just shoved past the edge of their safe zone, and their puzzles take longer to solve. Everyone else ends up waiting for them.

The fix is something every real controller needs anyway: a deadline. We give the solver a fixed amount of effort per step. Any system that hasn’t fully finished by then simply uses the best plan it has so far. Because Jaddle always keeps its answers within the motor limits as it works, even an unfinished plan never asks a motor to do the impossible.

With a deadline in place, the time per puzzle for ten thousand systems drops back down near its best.

Do they agree?

Speed is no use if the answers are wrong. So we check both approaches against a reference answer solved to extremely high accuracy. Both are close, and over the whole run, the two fleets end up with nearly identical total costs. The blocks stay calm, the motors stay within their limits, and the chains settle down in both cases.

The takeaway

This example isn’t saying “always use a GPU”. If you’re controlling one robot and need an answer within a strict fraction of a millisecond, a traditional CPU solver is still the right tool.

But a surprising number of real problems involve lots of similar decisions at once: a fleet of delivery robots, a wind farm full of turbines, or testing one controller against thousands of possible futures to see how it copes. In those cases, it’s worth asking the marching-band question: could all of these take the same step together? When the answer is yes, solving them side by side can be dramatically faster than solving them one at a time.

A world with a thumb on the scale

In this example we build a small make-believe world so we can see exactly what is going on. There are applicants from two groups, which we’ll call Group A and Group B. About three in ten applicants belong to Group B.

Every applicant has a few “skill” scores, things like income stability or repayment history. Here is the important part: the skills are spread out in exactly the same way in both groups. Nobody in Group B is any less capable, on average, than anyone in Group A.

In a fair world, whether you get a loan depends only on those skills. But the historical decisions the bank kept came from a world with a thumb on the scale. Every Group B applicant had a fixed amount knocked off their chances, just for being in Group B.

So the pile of old decisions, the one we’re asking the computer to learn from, quietly contains a bias.

“Just don’t tell it the group!”

The first idea most people have is simple: don’t show the computer which group each applicant is in. If it can’t see the group, surely it can’t discriminate?

Sadly, it isn’t that easy. In our world, each applicant also has two extra details, a bit like a postcode, that happen to differ a little between the groups. They say nothing about whether someone will repay a loan. They are what statisticians call proxies: clues that let you guess the group without being told.

A computer trained to copy the old decisions is very good at spotting patterns. It notices that applicants with a certain kind of “postcode” were turned down more often, and it learns to lean on that clue. Sure enough, when we train a plain model, it puts a negative weight on those postcode-like features and approves Group B applicants far less often than Group A.

How the robot banker thinks

The model we use is called logistic regression, and it is one of the oldest and most trusted tools in machine learning. You can think of it as a points system:

• Each feature (each skill score, each postcode detail) earns or loses the applicant some points.
• The points are added up into a single score.
• If the score is above zero, the answer is “yes”. Below zero, “no”.

“Learning” just means adjusting how many points each feature is worth until the model’s answers match the training data as closely as possible. Finding the best point values is an optimisation problem: a search for the settings that make some measure of error as small as it can be.

Adding a fairness rule

To push back against the bias, we give the optimiser an extra rule to obey while it learns. The rule we want is demographic parity: both groups should be approved at the same rate.

There is a small snag. Approval rates jump in steps: one applicant flips from “no” to “yes” and the rate suddenly hops. Optimisers prefer smooth hills to staircases. So, following a well-known approach by Zafar and colleagues (2017), we use a smoother stand-in for the rule:

The model’s average score shouldn’t depend on which group you’re in.

We also add a dial, which we’ll call the fairness bound. Turn the dial all the way up and there’s no rule at all: you get the plain, biased model. Turn it all the way down to zero and the two groups must have exactly the same average score. Anywhere in between is a compromise.

What we found

We trained two models: one with no fairness rule, and one with the dial at zero.

The fair model closes most of the gap in approval rates between the two groups. And something interesting happens to its accuracy:

• Measured against the biased historical decisions, the fair model looks worse. Of course it does: it is refusing to copy the bias.
• Measured against the fair decisions (the ones that depend only on skill), the fair model looks better.

In real life we would never get to see those “fair decisions”. We only know them here because we built the world ourselves. But it shows something important: if the bias lives in the labels rather than in the people, then “accuracy against the past” can be a misleading report card.

The whole trade-off in one go

One fair model and one unfair model only tell part of the story. What about all the settings in between?

We picked 40 different positions for the fairness dial and asked Jaddle, the optimisation library this example is built with, to train all 40 models at the same time, as one batch. That gives us a full curve showing how accuracy changes as the model becomes fairer, rather than a couple of dots.

Reading the curve from the unfair end to the fair end, accuracy against the biased labels slides down while accuracy against the fair labels climbs up.

The price of fairness

Here is my favourite part. When an optimiser obeys a rule, it can also tell you how much that rule is costing. Mathematicians call this a dual variable or shadow price. Think of it as a receipt that comes with each answer, saying: “if you tightened this rule by one more notch, the training error would go up by this much.”

For our fairness rule, that receipt is a price of fairness. We found that:

• When the dial is loose enough that the rule isn’t actually stopping anything, the price is zero. The rule is just along for the ride.
• As the dial tightens, the price climbs steadily. Each extra step towards equal treatment costs a little more than the one before.

That’s useful information for real decision-makers. Rather than arguing in the abstract about whether fairness is “worth it”, you can see the actual exchange rate and talk about it openly.

A few footnotes

Fairness is a deep and much-debated subject, and this example is a small, tidy illustration rather than a final answer.

• Demographic parity is only one definition of fairness. Others ask for equal error rates, or equal treatment of similar individuals. They can disagree with each other, and choosing between them is a question about values, not just maths.
• Our world is made up. Real data is messier, and real bias is rarely a single fixed penalty.
• The rule we used is a stand-in. Making average scores equal tends to bring approval rates together, but it doesn’t guarantee they match exactly.

The takeaway

A model trained to imitate the past will happily imitate the past’s mistakes, even when you hide the obvious clues. Adding a fairness rule to the optimisation is a direct, transparent way to push back. And because the optimiser hands you a price for every rule it follows, you can see exactly what that fairness is costing, and decide with your eyes open.

Drawing the problem

Mathematicians picture this as a graph: a set of dots (the people) joined by lines (the bickering pairs). Splitting the people into two teams is like drawing a line across the page. Every connection that crosses your line counts as “cut”, and you want to cut as many as possible.

In our example there are 30 people, and each pair has a 30% chance of being a bickering pair. That gives a graph with 119 connections.

Why it’s hard

With 30 people, each person can go on one of two teams, so there are 2 × 2 × 2 × … (thirty times) possible splits. That’s over a billion options. Add one more person and the number doubles. Add ten more and it’s a thousand times bigger.

Checking every option is hopeless for big graphs, and as far as anyone knows, there’s no clever shortcut that always finds the perfect answer quickly. Max-Cut belongs to a family of problems called NP-hard, which is computer-science speak for “nobody knows how to solve these fast, and many people suspect nobody ever will.”

So we settle for the next best thing: an answer that’s provably close to perfect.

The trick: pretend it’s easier

In 1995, Michel Goemans and David Williamson came up with a beautiful idea.

In the real problem, each person has to pick a team. You could picture each person holding a compass needle that must point either straight north (Team North) or straight south (Team South). Bickering pairs want their needles pointing in opposite directions.

Now let’s loosen the rules. Instead of only north or south, each person’s needle can point in any direction at all, not just on a flat map but in a space with many dimensions. Bickering pairs still want their needles to point as far apart as possible, but now there’s a whole sphere of in-between options.

This loosened version is called a relaxation, and it has two wonderful properties:

1. It’s much easier to solve. The relaxed problem is a type of puzzle called a semidefinite program, which, unlike the original, can be solved efficiently and exactly.
2. Its answer is a ceiling. Since every real split is also allowed in the relaxed version (north and south are still valid directions), the relaxed problem can only do better than or equal to the real one. So its score tells us: no split can possibly cut more than this.

Back to reality: the random slice

Of course, people can’t actually join “Team Slightly North-East-Up”. We need to turn the needle directions back into two teams.

Goemans and Williamson’s answer is delightfully simple: slice the space with a random flat sheet. Everyone whose needle points to one side of the sheet goes on one team; everyone on the other side goes on the other.

Why does this work? Bickering pairs ended up with needles pointing far apart. The further apart two needles point, the more likely a random sheet will pass between them, putting them on opposite teams. Pairs who got along fine ended up with needles close together, so they’ll usually stay on the same side.

That was a landmark result. For decades, nobody has found an efficient method with a better guarantee, and there are good theoretical reasons to believe nobody can.

In our example, we try 100 random slices and keep the best one. It comfortably clears the 87.8% bar.

A certificate of quality

There’s one more neat feature. When the solver finishes the relaxed problem, it also produces a set of numbers, one per person, called dual values. Add them all up and you get exactly the same ceiling, approached from the opposite direction.

This acts like a certificate. Anyone can take those numbers and check, without re-solving anything, that no split could ever beat the ceiling. You don’t have to trust the solver; you can verify its homework.

How Jaddle solves it

The tricky bit of the relaxed problem is a rule saying the needle directions must “fit together” consistently. (In technical terms, a certain table of numbers must be positive semidefinite.) Jaddle handles this by taking small steps towards a better answer and, after each step, nudging the result back into the set of answers that fit together.

The example writes the problem in CVXPY, a popular language for describing optimisation problems, and hands it to Jaddle to solve. We double-check Jaddle’s ceiling against SCS, another well-established solver, and they agree.

The takeaway

When a problem is too hard to solve exactly, you don’t have to give up. You can:

1. Relax it into an easier problem that gives you a ceiling;
2. Solve the easy version;
3. Round its answer back into a real solution;
4. Compare that solution to the ceiling to know exactly how good it is.

This “relax and round” recipe shows up all over computer science, from scheduling to circuit design to clustering data. And it all started with a clever question: what if the needles could point anywhere?

Step 1: Turn the rules into a fight

Suppose you want to minimise a cost, but you must obey a rule. A classic trick is to add a penalty for breaking the rule and let a second player decide how big that penalty should be.

Call the first player X, who is trying to make the cost as small as possible. Call the second player Y, who is trying to make the penalty as large as possible whenever the rule is broken. Y is a bit of a troublemaker. If X breaks a rule, Y pounces and charges X for it.

Mathematicians call this dualising the constraints. A problem that said, “minimise the cost, subject to the rules” becomes “X minimises, while Y maximises, a combined score.” The problem has gone from a minimisation to a min-max.

Step 2: Meet the two-player game

What we’ve built is a two-player zero-sum game. One player’s gain is exactly the other’s loss. X wants the score low, Y wants it high, and neither can simply walk away.

Picture a tug-of-war on a seesaw. Every move X makes pushes the score up or down, and every move Y makes pushes it back. Neither player is trying to be nice.

Step 3: The sweet spot is a Nash equilibrium

In game theory, a Nash equilibrium is a situation where neither player can improve their outcome by changing their move alone. Each player is doing the best they can, given what the other is doing.

In our min-max game, this equilibrium has a lovely geometric name: a saddle point. Think of a horse’s saddle. Walk along it from front to back and you’re climbing up; walk side to side and you’re going down. The saddle point is the spot that is the lowest in one direction and the highest in the other at the same time.

That is exactly the answer to our original problem. The saddle point gives us the best design that also obeys the rules.

Step 4: Let the players learn from each other

How do we actually find the saddle point? One approach is to let both players learn as they play. Each player watches what the other has been doing and adjusts their strategy accordingly.

The simplest version is called gradient descent-ascent. X takes a small step downhill on the score, Y takes a small step uphill, and both repeat. It is like two hikers feeling their way across a foggy landscape, one trying to get lower and the other trying to get higher, each reacting to where the other has just stepped.

Step 5: The twist (and the magic)

Here is the honest bit. In general, these learning dynamics don’t converge. In some games, the players chase each other in circles, or their steps grow larger and the game spirals away from the answer. Two stubborn players reacting instantly to each other are a recipe for wobbling.

But then something rather magical happens. If you stop looking at each individual move and instead track the long-running average of what each player has done, the averages settle down. The wobbles cancel out, and the averaged strategies drift steadily towards the saddle point, the Nash equilibrium, the answer we were looking for.

Why it matters

Saddle point problems appear everywhere: training generative adversarial networks, robust control under uncertainty, fair resource allocation, and constrained learning in general. The core lesson is delightfully simple. When two opposing learners keep playing, don’t trust any single moment. Trust the average over time.

From Idea to Engine: Jaddle

Everything above is a story about ideas: turn the rules into a fight, find the calm point where neither player wants to move, let both learners adapt, and trust the long-run average. Turning those ideas into code that actually runs fast is what Jaddle is for.

Jaddle (short for JAX Saddle Point Solver) is a library built on JAX, Google’s numerical computing library. It takes the two-player picture seriously. You describe the game, one player minimising and one maximising, and Jaddle runs the learning dynamics for you.

Here’s how the story maps onto the library:

• Dualising the constraints becomes a function you write: the cost, plus the penalised constraints, as one combined score.
• The two-player game is the object Jaddle solves. Each player’s update is computed from the gradient of that shared score, so you never hand-derive the learning rules.
• The Nash equilibrium is the output you care about: the saddle point, the pair of strategies where neither player gains by moving alone.
• Learning from the other player is the iterative update loop, with both players stepping at once and reacting to each other.
• The long-running average is where the magic from the last section lives in practice.

JAX makes this practical. Automatic differentiation supplies the gradients, jit compiles the update step so it runs at speed, and vmap lets many problems run side by side. The same code runs on CPU, GPU, or TPU without rewriting.

So if the earlier sections gave you the intuition, Jaddle is the workshop where you put it to use. Try it on a small game first, watch the iterates wobble, then watch the averages settle. That moment is worth seeing for yourself.

Two ways to climb a hill

Finding the best answer to an optimisation problem is a bit like climbing a hill in thick fog. You can’t see the top, but you can feel which way is uphill, and you keep moving that way.

Some solvers, such as Clarabel, take careful steps. Each step is expensive: the solver works out a lot about the shape of the hill around it, which means solving a sizeable calculation. Because each step is so well-informed, you need only a few of them to reach the top.

Jaddle takes a different approach. Its steps are cheap. Each one is a handful of quick sums, the sort of arithmetic a computer can repeat at great speed. You need many more steps to reach the top, but each one costs very little.

Why many cheap steps can win

It sounds backwards. Surely a few smart steps should beat lots of dumb ones? Not always. Here’s the catch: the cost of one careful step grows quickly as the problem gets bigger. Double the size of the problem and a careful step can cost several times as much. A cheap step grows much more gently. Eventually the careful solver spends so long on each step that the cheap solver, taking far more steps, still arrives first.

There’s a second benefit too. Cheap steps are easy to run in parallel. A graphics card, for example, is built to do thousands of simple sums at the same time. Lots of small, similar steps fit that kind of hardware beautifully.

The evidence

I’ve tried this on several problems in the series, and the picture is consistent.

Small problems: the careful solver wins. When I ran a robust Kalman filter, which tracks a moving object through noisy and sometimes wrong measurements, Clarabel finished in a fraction of a second. Jaddle took a few seconds. On a 30-node version of the network-splitting puzzle, the same story held: SCS was far quicker. The problems were small enough that the cheap steps didn’t have enough work to add up.

Big problems: the cheap solver wins. On a network with 1,000 nodes and about 150,000 links, which means a grid of a million numbers to arrange, SCS took about 126 seconds. Jaddle finished in about 18 seconds, roughly seven times faster, and both reached the same answer.

Many problems at once: the cheap solver wins. When I controlled a fleet of 1,000 small machines, each one needed its own plan every moment. Solving them one at a time with SCS took about 420 microseconds per plan. Jaddle, solving all the plans together in one batch, took about 10 microseconds each, around forty times faster on a laptop graphics card. Even on an ordinary processor, the batched version was about three times faster than the one-at-a-time approach.

The honest bit

This is not a magic trick, and the crossover is real. On a single small problem, the careful solver is usually the right choice, and I’d rather tell you that than sell you something you don’t need. Jaddle’s advantage shows up when problems are large, or when you have many of them to solve together. If you’re solving one tiny problem a week, use the specialist tool.

A quick decision guide

• One small problem, needed occasionally: use Clarabel.
• One large problem: try Jaddle, and compare it against Clarabel on your own data.
• Many similar problems at once: Jaddle, especially batched on a GPU.
• You need a high-accuracy answer fast on a tiny problem: Clarabel. Its careful steps pay off when precision matters more than throughput.

The takeaway

When you choose a solver, don’t ask only which one is cleverest. Ask how big your problem is, how many you need to solve, and whether a crowd of cheap steps can beat a small team of expensive ones. For small problems, use Clarabel. For big ones, lots of cheap steps often win, and Jaddle is built around exactly that idea.