Prediction Intervals: Reading Forecast Uncertainty

The band around the forecast is a claim you can test

A forecasting model hands you two things at once. There’s the number everyone reads, say a projected close of 182.40 for tomorrow, and there’s the quieter part printed beside it: a band running from 178.90 to 185.90. Most people glance at that band, take it as a rough sense of scale, and move on. It’s doing more work than that. The band is a prediction interval, a testable statement about where a future value will land, and it can be right or wrong in a way you can count over time.

That object deserves a careful definition, because every clause earns its place. A prediction interval is a range built to contain a future realised observation under a set of stated modelling assumptions. It’s about a future observation, not a hidden parameter. It holds only while the assumptions hold, and when they break, the band breaks with them. A trader who reads the interval as a guarantee has misread it before the first tick prints.

What the interval covers, and three things it does not

Start with the point forecast, because the interval is built around it. The point forecast is the single best guess: 182.40. On its own it says nothing about how sure the model is. Ask it for tomorrow’s close and it answers in the same confident tone whether the market has been dead calm or whipping four percent a day. The interval is where that missing information lives.

Three neighbours get confused with it, and keeping them apart is most of the battle.

• A confidence interval describes an estimated parameter, like the true mean daily return, and says nothing about a single future observation. It narrows as you gather more data, because the parameter sits still while your estimate of it sharpens. A prediction interval doesn’t collapse that way. Even with infinite history, tomorrow’s close still carries genuine randomness, so the band keeps a floor width.
• A probability attached to a discrete event answers a yes-or-no question, such as the chance the close finishes above 185. That’s a single number for one outcome. An interval spans the whole range of where the price could land.
• A point forecast with no band is a guess wearing a suit. It looks precise and tells you nothing about the spread of what could happen.

Here’s the misread I see most often. A 95 percent interval doesn’t mean the model is 95 percent sure the price hits 182.40. It means that if the assumptions hold, about 95 out of every 100 future closes should fall inside the stated range. The centre is the guess. The band is the honesty.

How a model turns one number into a range

Every interval comes from the same two ingredients: a central estimate and an estimate of its own uncertainty. The interesting part is how the second ingredient becomes a range, and there are a few standard routes.

The simplest assumes the model’s errors follow a fixed bell shape. You take the standard deviation of the historical residuals, the gaps between what the model predicted and what actually happened, and you scale it. For a 95 percent band under a normal assumption, that scale factor is about 1.96. If the one-day residuals have a standard deviation of 1.8 percent around a 182.40 forecast, the band runs roughly 182.40 give or take 3.5 percent, so about 176 to 189. This is where a working grasp of standard deviation as a spread of outcomes stops being academic and starts setting the width of the band you’d trade around.

Other routes drop the bell-shape assumption. Quantile methods estimate the 2.5th and 97.5th percentiles of the outcome directly, which lets the band sit off-centre when downside and upside aren’t symmetric. Simulation methods generate thousands of possible paths and read the interval straight off the spread of endpoints. The goal is the same every time: turn a fuzzy sense of uncertainty into two hard numbers.

Why the band breathes

A fixed-width interval is a red flag, because real uncertainty isn’t constant. Three forces push the band wider, and knowing them tells you when to trust a tight band and when to distrust it.

Volatility is the first. When the market’s day-to-day range expands, the residuals expand with it, and any honest interval has to widen in step. That’s why a band built on last quarter’s calm can be badly too narrow the week a stock starts moving. It pays to read a band next to a measure of historical volatility: if volatility has doubled and the band hasn’t moved, the model is asleep.

Horizon is the second. Forecasting tomorrow is a different task from forecasting a month out. Under a simple random-walk assumption, uncertainty grows with the square root of time, so a two-week band runs closer to three or four times the one-day width, not the double a quick guess expects. The first time I widened a one-day residual by hand for a ten-day horizon, scaling 1.8 percent by the square root of ten, it grew to about 5.7 percent, and the honest band was suddenly far looser than the tidy daily version had suggested.

The third force is the quiet one. A model meets conditions unlike anything in its training sample, a new rate regime, a market structure that didn’t exist a year ago, and its uncertainty estimate is now guessing about territory it has never seen. The band it prints looks as confident as ever. That confidence is the problem.

Coverage and sharpness: the two questions that decide everything

Once you have a band, two questions tell you whether it’s any good. They pull against each other, which is what makes them useful.

Coverage asks how often the future actually lands inside the stated range. A 95 percent interval makes a promise: over a long run of closes, roughly 19 in 20 should fall inside it. So you count. On a 95 percent band checked across 250 trading days, I expect somewhere near 12 or 13 breaches. When I count closer to 40, the band is running near 84 percent coverage, well short of the 95 it advertises, and every position sized off that band was sized on a fiction. Coverage is the first thing I measure and the last thing I take on trust.

Sharpness asks the opposite question: is the band informative, or so wide it says nothing. I can hand you an interval with perfect coverage by quoting a range from zero to infinity. It will never be breached and it will never once help you. A useful interval is the tightest band that still keeps its coverage promise. That tension is the whole game. Squeeze for sharpness and you risk breaking coverage; pad for coverage and you drown the signal.

Both questions have to be asked on data the model never touched during fitting. A band that looks perfectly calibrated on its own training sample tells you almost nothing, the same trap that walk-forward analysis exists to close. Coverage measured in-sample is a mirror. Coverage measured out-of-sample is a test.

Where the numbers quietly fail

The failure that hurts is quiet. A band that’s obviously wrong gets discarded on sight; the one that damages you looks right on average and comes apart exactly when you need it. Average coverage across a calm year can read a clean 95 percent while hiding a stretch where the market broke and the band was breached on a third of days. The mean smooths over the moment that actually costs money.

This is why calm-period and stressed-period coverage deserve separate scoring. A model can earn its 95 percent badge across a quiet sample and still come apart through a structural break, the point where the relationship the model learned simply stops holding. Nominal coverage, the number the model claims, and realised coverage, the number the market delivers, drift furthest apart precisely when conditions turn unusual.

The practical answer is recalibration. After a regime shift, an interval fitted to the old world needs refitting to the new one, or it keeps quoting yesterday’s confidence into today’s disorder. A trader using these bands might treat a run of breaches clustered in a short window as a signal that the model needs refitting, the same way a cluster of stops tells you the character of the tape has changed.

Conformal prediction, and why it isn’t a cure

One newer family deserves a mention, because it shows up in model reports more each year. Conformal prediction builds intervals with a light touch on distributional assumptions. The split-conformal version is easy to picture: hold out a slice of data the model never trained on, measure the model’s errors on that slice, and take the appropriate quantile of those errors as your band. Done right, you get a coverage guarantee that holds without assuming the errors follow any particular shape.

That’s genuinely useful, and it’s no cure. The guarantee leans on an assumption called exchangeability, loosely, that the future looks statistically like the calibration data in no particular order. Markets violate that assumption for a living. A regime change, a volatility spike, a structural break, and the neat guarantee is quoting a world that no longer exists. Conformal methods widen the toolkit. They don’t repeal the fact that a model can only calibrate against a past it has already seen.

A checklist for reading a prediction interval

When a report puts a band in front of you, a short interrogation separates an honest interval from a decorative one.

• Name the target and the horizon. What exact variable is being bounded, tomorrow’s close or next week’s return, and over what window. A band with no stated horizon isn’t finished.
• Ask where coverage was measured. Untouched, out-of-sample data is the only fair test, and in-sample calibration flatters every model.
• Split calm from stressed. Ask for coverage in both, separately, and treat a band that only survives quiet periods as untested where it counts.
• Check for recalibration. Was the interval refitted after the last regime change, or is it still quoting confidence from an older market.
• Match sharpness to the decision. A band too wide to inform a stop or a size is technically honest and practically useless.

Run that list and you’ll throw out more intervals than you keep. That’s the point. Most bands look authoritative and fold under the first real question.

Treat the band as a hypothesis, not a fence

The deepest error is to read a prediction interval as a boundary on what the market can do. It is a statement about the model’s own uncertainty under a chosen procedure, and the market never agreed to stay inside it. That lesson runs straight through Nassim Taleb’s lessons on fat tails: the rare move that shreds a 99 percent band is exactly the move a smooth model is worst at imagining. Financial data can shift faster than any interval adapts, and nominal coverage fails hardest at the moment it matters most.

So the interval earns its keep as a working hypothesis. You keep testing it, and you never lean your whole weight on it. Count the breaches. Score calm against stress. Refit after the world turns. Held that way, a band tells you something true about how much your model actually knows. Read as a promise, it just tells you a comforting story until the day it doesn’t. Learn the pattern. Ride the trend. Keep the gains.

Educational content only. Not investment advice. Trading involves risk. You are responsible for your decisions.