Imagine you’re trying to follow a remote-controlled boat drifting around a pond, but you can’t see it directly. Instead, a friend on the bank shouts out its position every few seconds. Most of the time they’re pretty accurate, give or take a little. But your friend is easily distracted, and about one time in five they shout out something completely off, as if they’d glanced at a duck instead.
Can you still work out where the boat actually went?
Two sources of information
You have two things to go on:
- The measurements, your friend’s shouted positions. Most are close; some are wild.
- The physics. Boats don’t teleport. They glide along, speeding up and slowing down smoothly, nudged now and then by gusts of wind you can’t predict.
The trick is to combine them. You’re looking for a path that obeys the physics and stays reasonably close to the measurements. If a measurement says the boat jumped five metres to the left in an instant, the physics says “that’s not possible”, and the path should mostly ignore it.
Finding the best such path, after the fact, using all the measurements at once, is called smoothing.
The classic recipe: the Kalman smoother
In the 1960s, Rudolf Kalman developed a beautifully efficient method for exactly this kind of problem. Versions of his ideas helped steer the Apollo spacecraft to the Moon, and today they run inside phones, cars, drones and countless other gadgets.
The classic approach chooses the path that keeps two things small at once:
- the wind gusts needed to explain the path (smaller gusts mean a more natural path), and
- the misses, the gaps between the path and each measurement.
To combine all the misses into one number, it squares each one and adds them up. Squaring makes sense when errors are small and well-behaved: it treats misses in either direction equally and is very convenient mathematically.
The problem with squaring
Squaring has a weakness. A miss that’s twice as big counts four times as much. A miss that’s twenty times as big counts four hundred times as much.
In our example, the occasional wild measurement is about twenty times noisier than a normal one. So when the classic smoother tries to keep the squared misses small, those few wild measurements dominate everything. They act like bullies, dragging the estimated path towards themselves, because ignoring them would be “too expensive”.
It’s a bit like a teacher averaging test scores where one student’s answer of “a million” for a question with answer “seven” drags the class average into the stratosphere.
A fairer referee: the Huber penalty
The fix is to change how misses are scored. Instead of squaring every miss, we use the Huber penalty, named after the statistician Peter Huber:
- For small misses, it behaves just like squaring. Normal, everyday noise is treated exactly as before.
- For big misses, it switches to growing in a straight line. A miss twice as big costs only about twice as much, not four times.
The squared penalty is like a rubber band that pulls harder and harder the further you stretch it. The Huber penalty is a rubber band that, beyond a certain point, stops getting stronger and just pulls with a steady force. Wild measurements still tug on the path, but they can no longer yank it off course.
This version is called the robust Kalman smoother. “Robust”, in statistics, means “doesn’t fall apart when some of the data is bad”.
The experiment
We simulate an object moving around a flat plane for 50 seconds, measured 1,000 times. It has a position and a speed, a little friction slowing it down, and random pushes knocking it around. Each measurement has a one-in-five chance of being an outlier.
Then we estimate the path twice: once with the classic smoother and once with the robust one.
The results are striking. The classic smoother’s path is visibly dragged around by the outliers, wandering off towards the wild measurements. The robust smoother’s path sticks closely to the true trajectory, as if it had simply noticed which shouts to take with a pinch of salt.
What Jaddle does differently
Both versions of the problem are written in CVXPY, a popular language for describing optimisation problems. In fact, this is one of CVXPY’s own showcase examples.
The classic smoother is a relatively simple kind of problem (a quadratic program), and Jaddle solves it directly.
The robust version is more interesting. Normally CVXPY would translate the Huber penalty into a bundle of cone-shaped rules, which works but makes the problem bigger and more complicated. Jaddle takes a different route: it keeps the Huber penalty as the smooth, gently curving function it really is, and solves the problem in that form. It also notices that the “miss” for each measurement is completely determined by the path, so it can drop those as separate unknowns and work with the path alone. Fewer moving parts, same answer.
We check both of Jaddle’s answers against Clarabel, a well-established solver that comes with CVXPY, and they agree.
The takeaway
Real-world data is rarely perfectly behaved. Sensors glitch, signals bounce, people mishear. A method that assumes every measurement is roughly right can be thrown badly off course by a handful that aren’t.
By changing one small thing, how we score big misses, the robust Kalman smoother learns to listen to the crowd without being bullied by the loudest fibbers. It’s a good lesson beyond boats and ponds: when you’re combining lots of evidence, make sure no single piece gets to shout over all the rest.