Gaplad › Free tools & guides › Which gap-filling method, when
Guide · with a method pickerHow to fill gaps in sensor data, honestly.
The right way to fill a gap depends on how long it is, what else was recording at the time, and what the sensor was doing. Describe your gap and get a method, with the reason and the catch.
Which method for my gap?
The short version
| Gap | Best honest option | Why |
|---|---|---|
| Up to ~6 hours | Linear interpolation | Hard to beat at this length, nothing to tune, never overshoots. |
| 6 hours to weeks, another sensor recorded | Model on the other sensor(s) | The other sensor carries the real ups and downs; validate against linear per gap length. |
| Up to a week, daily cycle, nothing else recorded | Daily profile, joined at both ends, flagged | Keeps the cycle; cannot know the weather of the missing days. |
| Long, and every sensor died together | Leave it unfilled, say why | Any fill is a guess. Report the span as needing reference data. |
| Across a re-installation | Split the record first | Two baselines bridged by a fill make a ramp that never happened. |
| Nights in a daytime-only sensor | Nothing: not a gap | Filling them invents data and skews daily statistics. |
Step 1: find and classify the gaps
Before choosing anything, list the gaps and their lengths. A record that is "92 % complete" can be four hundred short dropouts (easy) or one month-long outage (possibly unfillable). The missing timestamp finder lists every gap by length class and tells you which ones hit all sensors at once.
Step 2: clean before you fill
A model trained on spikes and error codes learns spikes and error codes. Remove -9999 placeholders, despike, check for frozen sensors and look for re-zeros first. The QC check does all of it; the Hampel despiking tool lets you tune the spike detector.
Step 3: short gaps get linear interpolation
For gaps of a few readings, a straight line is as accurate as anything else and impossible to argue with. Smoother curves (cubic splines) rarely win at this length and can overshoot next to a sudden change. See the four classic methods race on the same gap.
Step 4: longer gaps need information from somewhere else
Interpolation only knows the readings at the edges of the gap. Anything that happened in the middle (a warm afternoon, a rain event, a pump test) must come from a sensor that was still recording: another depth, a neighbouring station, a co-located variable. Regression on that sensor is the classic approach; gradient-boosted trees with lagged and rolling features handle non-linear and delayed relationships better. Either way, the model must beat linear interpolation on held-out data before it is allowed to fill.
Step 5: validate on synthetic gaps
Take stretches where you have data, hide them, fill them blind and compare. Do it per gap-length class, because a method that wins on 2-hour gaps can lose on 3-day gaps. Report RMSE and, for hydrology, NSE and KGE. Use the residuals to put an honest uncertainty band on each fill.
Step 6: flag everything and report what you could not fill
Every filled value gets a flag saying it was filled and how. Spans where the predictors left the range they were trained on are flagged as extrapolation. Gaps that could not be filled are listed with the reason. Never compute extremes or compliance exceedances inside filled spans.
Questions
What is the best way to fill missing values in sensor time series?
There is no single best method; it depends on the gap. Linear interpolation for short gaps (a few hours), a regression or machine-learning model on a correlated sensor for longer gaps, a daily-profile fill for cyclic signals when nothing else is available, and nothing at all when the gap is long and no other sensor saw it. Whatever you choose, validate it on synthetic gaps and flag the filled values.
How long a gap can I interpolate?
As a working rule, up to about 6 hours for hourly environmental data, and never longer than the shortest important cycle in the signal (a day for temperature, a tide for coastal levels). Beyond that, interpolation is guessing. Test it on your own record: carve gaps of that length from observed data and see how big the errors are.
Should I fill gaps before computing statistics?
Only if the statistic needs a continuous record (a flow-duration curve, a daily total) and only with flagged, validated fills. Never compute extremes or exceedances inside filled spans: a fill cannot know about a peak it never saw.
How do I report filled data?
Keep a flag column marking every filled value and how it was filled, state the method and its validation error per gap length, and list the spans you could not fill. Gaplad writes all three: a flagged CSV, a methods paragraph and a trust report.
What about gaps in a sensor that only logs during the day?
Nightly blanks in a daytime-only sensor (PV power, irradiance, some flux sensors) are not gaps. Do not fill them; they distort every daily statistic.
Steps 1 to 6, in one tool
Gaplad does all of this in your browser: gap map, QC, validation per gap class against linear interpolation, filling with the winner, refusal where filling is not honest, a flagged CSV and a trust report.