Gaplad › Free tools & guides › Sensor data QC check

Free tool · runs in your browser

Quality-check your sensor data in one drop.

Find the readings that should not be trusted before they reach a model, a report or a gap fill: error codes, spikes, frozen sensors, impossible values and sensors that were re-installed at a different level.

Drop a CSV or Excel file here

Read in this browser tab. Nothing is uploaded. One timestamp column (or Year/Month/Day columns) and any number of value columns.

Paste data instead

The five checks, and the thresholds behind them

1. Error codes

Loggers write placeholder numbers when a sensor fails: -9999, -999, 6999, 32767 and friends. A value counts as an error code only when it repeats at least three times and sits far outside the spread of everything else (more than five interquartile ranges beyond the quartiles), so a real reading of -999 m below datum would not be thrown away.

2. Spikes

A Hampel filter: each reading is compared with the median of its 12 neighbouring observations, and flagged when it is more than 5 robust standard deviations (1.4826 × the median absolute deviation) away. It is deliberately conservative: a real flood rise is wide, a spike is one or two readings.

3. Flatlines

24 or more identical consecutive readings with no gaps between them. The first reading of the run is kept; the repeats are the suspects.

4. Plausible range

A robust automatic range (first and third quartile ± 6 interquartile ranges) catches the absurd. For real QC you should set physical limits for your sensor (relative humidity 0–100 %, a stage that cannot be below the gauge zero); the main tool lets you.

5. Level steps (re-zeros)

A jump far larger than the normal reading-to-reading change (30 × the median short-range difference), that persists rather than bouncing back, and that is large compared with the series' spread. Jumps after a long outage are ignored, since a level can drift legitimately over months.

Questions

What does the QC check look for?

Five things: logger error codes (-9999 and similar values repeated far outside the data), spikes (Hampel filter), flatlines (24 or more identical readings in a row), values outside a robust plausible range, and level steps where a sensor was re-installed or re-zeroed.

Does it delete anything?

No. Every flag is a suggestion. The page lists flagged points and lets you download them; deciding what to exclude is yours. In the main tool you switch each class of flag on or off before filling.

Why are flatlines not always errors?

A rain gauge reads 0 for days, and a quantised sensor can repeat a value honestly. That is why flatlines are listed separately and are off by default when Gaplad cleans a record: only you know whether a frozen value is plausible for your sensor.

What is a re-zero, and why does it matter?

When a pressure transducer is pulled and re-hung, or a sensor is recalibrated, its readings can jump to a new baseline. Interpolating or training a model across that jump blends two baselines into a ramp that never happened, so Gaplad splits the record there and never fills across it.

My solar or PV sensor is empty every night. Is that flagged?

Nightly blanks in a daytime-only sensor are not defects, and nothing here turns them into errors. Leave them empty; do not fill them.

Clean, then fill, then prove it

Gaplad runs these same checks, lets you choose which flags to exclude, splits the record at re-zeros, and then fills the gaps with a method validated on your own data. You get a flagged CSV and a trust report you can attach to a deliverable.

Open Gaplad, free How to choose a method