Yes, it’s been a while. The sabbatical in Italy put a dent in testing, sure. Then came a disc bulge which took any decent running load out of the picture. On the upside I took the time out to brush up on my geospatial-data-dorking skills in R and PostGIS and rebuilt the process entirely in R script. Yes, it’s nerd. The result is a complete re-analysis of collected data to give better insight into the capabilities of the GPS models tested.
In short, the updated methodology is leaning more towards quantified GPS performance bench-marking rather than a subjective review. Plus the future collection and analysis workflow is super easy which equal less typing and more running. Or beer, or whatever.
We can make pretty pictures like this track point cloud that shows GPS positional accuracy over a large number of runs (hint – this is a recent model GPS).
Having powered up the analysis we can now see the impact of tree-cover, bendy paths, running speed, sampling rate, satellite availability, and even how much the satellite data is actually filtered out from the recorded distance (ie. does the watch work like a glorified activity tracker). The results give some great insights on which watches work best in actual trail conditions. With a couple of statistical tests we can formally identify which comes out as the better, or worse, performers. And the data can be modeled to show which factors effect watch accuracy.
As a teaser, the chart below shows the accuracy of all models across easy, mixed, and difficult conditions as recorded by the watch (ie. as you see it) in purple and as recorded by the raw GPX data (ie. buried underneath what you see) in yellow. The brown is the overlap of the two. The dashed lines are means, and grey is the true distance. We can see here the accuracy deteriorating between the conditions nicely. At the individual watch level there are huge difference in performance between these categories. Given we normally spend a lot of time running in the mixed to difficult conditions (when we are not testing) this analysis gives a great view on which watch will give us the good trail running numbers.
Plus, we’ve sold a kidney and promised a first born child to get hold of a Suunto Spartan Ultra for testing. The first round of formal surveyed trail testing is done, though sadly the +20hr endurance run test is off the cards for now (back prognosis points to a Spring running recovery). At the same time we tested a Sony Xperia Z5c using the SportTracker app. The results are interesting… our advice would be hold off any purchase till you read the review.
We’ll be writing up the Sunnto Spartan Ultra and Sony Xperia Z5c and updating existing reviews over the coming weeks. And data collection is in progress for the TomTom Adventurer.