GPS, Statistics, and Philosophy

Here’s a scenario which might already be familiar to some of you. You enter a rugged trail run event with your buddies and everyone is GPS’ed up to their eyeballs (speaking of which when is someone going to sort a heads up display for me). When you all finish and compare distances you find a fair bit of variation and none of you record the advertised distance.

So which distance is the true one?

  1. The event advertised distance
  2. The distance indicated by a topo map
  3. The longest of any of the GPS recorded distances
  4. An average of a bunch of GPS recorded distances
  5. None of the above

If you selected 5. None of the above, you are likely to be the winner.

Being a trail run, the event organiser is unlikely to have measured the course three times by wheel, as they do in official road races, or by any other survey method for that matter. Which means either they calculated it via a map or from previous GPS tracks themselves.

Topo maps are horrible for measuring true trail distance due to the 1:50k scale being hopelessly small. The mapped tracks smooth-out countless kilometres of twisting trail. Though at least you know the measured distance will be short.

So what about GPS? In testing the accuracy of a number of current and past running GPS watches in NZ trail conditions, you are forced to make a choice as to which results represent the best outcome. No GPS is 100% accurate and all have a distribution of variability with respect to distance. The question is what should this distribution look like? Some models play it conservative in tough conditions and make sure they never exceed the true distance traveled, and others have bit more freedom and measure short some days and long others.

Everyone can agree that the less variability the better (put another way, the more predictable the better). But if a watch had more variability, recording say between 90-110% of the true distance but coming out with a long term average of 100%. Is that better than a watch that averages 99% accurate with a very tight range of 97% – 99.9%?

Turns out it looks like the best performing current high end trail running GPS tend to aim towards the latter – to try and make every run as (predictably) accurate as they can even if it means they will never get to 100% over the long term. But you know with a bit more certainty what the distance was on that day.That’s certainly my preference, even if the wide varying models look better by some simple statistical measures (eg. long term average).

So where’s that leave the original scenario? Well if your sample of GPS models are all current high performers, averaging isn’t going to help much as they are all short. But the longest might be close. On the other hand if you are all sporting old school models and GPS conditions weren’t too bad, and the constellations were aligned, the average could feasibly be close, but don’t count on any one of them. If your comparison group includes both types, give up. The truth is it was a great day running and who really cares.

The results of our actual GPS trail tests are rather interesting. We were certainly surprised by a few of the results and what to steer clear of… Will be progressively posting them in the coming weeks.

Ps. same patterns seen in total ascent/descent data as distance, only the margins are somewhat greater.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s