GeoWeb Guru

Putting a map on a screen, and everything that goes wrong on the way

Browse all thirty entries
A vernier caliper resting on the metal bed of a lathe machinePlate 01

08 — Survey & Measurement

Accuracy against precision

Six decimal places does not mean six decimal places of truth.

A vernier reports to a tenth of a millimetre. Whether it is right about the object is a different question.

Photo: FFD Restorations / Pexels

The gap nobody logs

A GPS receiver displaying 51.501476°N, 0.124572°W looks authoritative. Six decimal places of a degree imply a ground resolution of roughly ten centimetres. But that number describes precision — the fineness of the reported value — not accuracy, which is how close that value is to the thing it claims to measure. A clock that has stopped is perfectly precise; it is right twice a day and wrong the rest of the time.

The confusion runs deep enough to matter in map production. Precision is a property of the instrument and its output format. Accuracy is a relationship between the output and reality. The two are independent. A consumer GNSS unit on a clear day might quote coordinates to six decimal places while sitting two to five metres from the true position; the precision is decimetric, the accuracy is not. A carefully controlled survey mark, established by classical triangulation and carrying a formal uncertainty statement, may be recorded to only five decimal places of a degree — about a metre on the ground — and be accurate to within a metre.

When a dataset arrives carrying many decimal places, the natural instinct is to trust it. That instinct is the trap. Decimal places cost nothing to store; they tell you about the number format, not the measurement. What actually matters is the provenance of the measurement — the method, the equipment, the conditions, and whether anyone published an uncertainty estimate alongside the coordinate. A coordinate without an uncertainty figure is an assertion, not a measurement.

This has direct consequences for how data should be handled and combined. Conflating a survey-grade dataset accurate to half a metre with an open dataset accurate to five metres and then publishing the result at six decimal places produces something that looks more reliable than either source. The precision of the output format flatters the accuracy of the inputs. The coordinates that underpin every position were always measured from something, under conditions that introduced error; recording more decimal places cannot subtract that error.

Close detail of printed contour lines and spot heightsPlate 2

The same ground published twice, with contours and without. What a sheet leaves out is a decision about its purpose.

Photo: Topographic and planimetric sheets, Fort Bragg · Wikimedia Commons

The practical correction is simple in principle and difficult in habit: match the precision of your output to the genuine accuracy of your weakest input, state uncertainty where you can, and treat inherited decimal places with suspicion rather than respect. A value stored to three decimal places that is accurate to three decimal places is worth more than six decimal places wrapped around a guess.

Hands laying a transparent overlay over a printed grid
Every drawn grid is an agreement about where things sit — on tracing paper as much as on a screen.Photo: Ksenia Chernaya / Pexels

Precision is what the instrument reports. Accuracy is what the world contains. The map reader sees neither distinction — which is exactly why the mapmaker has to hold it clearly.

Also in Survey & Measurement

Next section — 01 Projections Read on