Trail/Ultra Running GPS Review Roundup (June 2016 update)

The 2017 update will be available once we’ve completed the Xperia Z5c/SportsTracker app write up, and testing of the TomTom Adventurer and Garmin fēnix 5X. It’ll be real interesting to see if the 5X improves on the sub-average accuracy of the fēnix 3. The initial Spartan Ultra review is up now.

We’ve also upgraded our testing analysis to give accuracy scores on distance and position by GPS trail difficulty so you can better identify which model might suit you best. This will be applied retrospectively to past models over time and will written in to the next roundup.

Our Reviews and (what they mean to) You

Look, there is really no such thing as ‘the best trail GPS watch’ for everyone all the time. What we are trying to assess is how well each model we test does against a bunch of parameters. We are explicit and open about this so you can figure out which model might suit your needs. Best for me probably isn’t best for you, hell, best for me depends on what kind of event I’m doing in what conditions and what socks I’ve got on.

To bypass the intro and head straight to the review round-up click here

We are pretty selective in what comes up for review. Mostly because we actually have to get hold of the watch in the first place (and convince the trail-running owner to swap it out for the formal testing period – ie. a couple of months minimum). Basically to make the cut, the marketing material has to suggest the model has the chops to trail run for 14hrs then someone we know takes a punt a buys one. The required endurance and specifications means the watches reviewed are all going to be towards higher end of things ($$$), so they should perform accordingly.

Our reviews have a specific trail focus and we try to be as objective as we can (see our test methodologies). We don’t go in to the full set of features each watch has. The reviews don’t reflect a watch’s ability in pure road running, let alone cycling, swimming, or triathlons. There are other review sites on the web that give a more general overview – check them out if you need more. Check them out anyway, it’s never a good idea to rely on one source to form an opinion.

We are also not reviewing watches to boost sales. We are reviewing past models as well as current in order to give runners a sense of whether an update is indeed an upgrade for them (if you like your watch just be happy, ignore reviews and go for a run). Given the kind of testing required to come to a conclusion on a model, this is likely 6 months or a year behind the release date. That’s not going to get a buzz on for you early adopters. There’s just no getting around the fact that the time required to come to a conclusion that is robust and generalizable to most trail runners is significant.

What’s worse is that our results may well be out of date by the time you read them. Frequent firmware and app updates are par for the course now and these can change things. What doesn’t change though is the fact that you should never by a GPS watch on a promise of future improvements. Statements like “Yeah, it’s a bit buggy now but a firmware update will fix that”, “or that feature is coming in Q4 of 2025” are simply alternate realities as of today.

Current State of GPS Trail Play

So far our review pool includes Garmin’s fēnix 2, fēnix 3, FR310XT/FR910XT, Suunto’s Ambit3 Peak and Ambit2, and Polar’s V800. The Sony Xperia Z5c is currently being tested to see how it fares against these watches.

As is discussed in the testing write up and each review, positional accuracy is only indirectly related to accuracy in distance and pace. Watch based GPS filtering can improve distance and pace performance considerably, but filtering can only do so much with rubbish data. With that in mind, positional accuracy remains a pretty good summary of potential GPS running performance (altitude excluded).

Updated June 2016 –  So Suunto’s Ambits still come in first and second in terms of positional accuracy under the limited GPS conditions. In terms of distance the Ambit3 Peak hits +/-1% of true distance 80% of the time with the vintage Garmin FR310/910XT coming in second at 68% of the time. Regarding elevation accuracy it’s the Ambit3 Peak again at 94% followed by the Ambit2. So a clean sweep for Suunto’s Ambit3 Peak then.

Notes: the FRx10XT is the combined 310 and 910 data (they proved virtually the same) the offset axis is also truncated as the FRx10XT outliers go all the way up to 77 metres. The cycling GPS Garmin Edge705 and 810 units are included for comparison.

Position Error 2016 June UpdateQuick boxplot interpretation tutorial

A boxplot (the chart above) is a nifty way of showing distribution, here the distribution is the distance in metres trackpoints are away from the surveyed run course (in GPS limited conditions). The dark line in the centre of the box is the median, which is the halfway point of the distribution, 50% of points are greater than this and 50% of points are less. The ends of the box are the 1st and 3rd quartiles, these are like the median but at 25% and 75% respectively. The whiskers are the lines out the end of the box which are the 1st and 3rd quartiles plus one and a half of the box length (or zero). Beyond this are the individual ‘outliers’ – nasty little circles indicating the GPS was having a tough time that day.

Distance and Altitude Accuracy

Distance accuracy as plotted here is how often each GPS gets within 1% of the true surveyed distance. The Ambit3 Peak again stands out at 80%, and in a surprising second place at 68% are the FR310/910XT combo. The fēnix 2/3 and Ambit2 all sit around 65%.

Altitude accuracy is how close the model is at capturing total climb. Ambit3 Peak leads again at 94% followed by the Ambit2 at just under 90%, with the fēnix 3 and V800 a little behind at 84%. Note only the FR910XT elevation data is used in the FRx10XT set.

Dist Alt Accuracy

 Trackpoint Density Clouds

Darker colours indicate higher trackpoint density, a perfect trackpoint cloud would be a dark red line overlaying the black survey line. The width/location of the cloud indicates ‘how wrong’, whereas the colour ‘how often’. The trackpoint clouds depicted here are from all runs over a subset of the surveyed route with varying conditions. The bottom left tracks in each frame have limited (though not terrible) sky GPS conditions, the rest are relatively open sky.

Despite the FR310XT’s obvious few ‘where am I again?’ days it actually follows the course very well for the bulk of trackpoints. The fēnix 3 (+3.60FW), is a little wider on the dark strips but without too many light coloured outliers . Conversely the Ambit’s pretty much just truck along regardless (though note the spotty outliers on the Ambit2 reduced GPS sample rate runs).

GPS Trackpoint CloudsSuunto Ambit3 Peak (click me)

Good for serious trail and ultra-runners. Top of the class in terms of consistent accuracy, reliability, and endurance. Hits the +/-1% distance accuracy 80% of the time. It’s design focus appears clear, trim any feature that impedes accuracy or endurance. Vibrate still doesn’t make the feature list, and screen design means trail legibility is limited compared with other brands, buttons also a bit spongy and suffer a bit of lag (no good for 400m splits). At the end of the 15.5hr trail test it came out on 1st with 19% battery remaining. No real waypoint management so no good for you OSM trail mappers. And you’ll need to swap out your ANT+ sensors for BLE if you are upgrading from the Ambit2 or Garmin. The Android companion app is still causing pain for many (as of June 2016).

Garmin fēnix 3 (click me)

The fēnix 3 has certainly come along way since our August 2015 pre-3.60 “avoid this model” review. If you want a watch to find your phone, skip your music list, and notify you of appointment, txts, etc, and can also do some trail GPS then this is the one for you. You will have to give a bit ground to most other models as the current fēnix 3 firmware version is credible though not stellar in accuracy or reliability. Hits the +/-1% distance accuracy 65% of the time. Ultra runners will also want to take note of the 12:30hr real world battery life and occasional UltraTrac reliability issues we’ve noted. Production quality also appears a bit hit and miss as two of three we’ve had ended in a warranty return within a month as it was recording complete nonsense. However, get a good one and the fēnix 3’s design and feature set is excellent Garmin are commendable on their openness of data exchange and fully featured Android app (even if the firmware/app upgrades break connectivity from time to time).

Suunto Ambit2 (click me)

Overall pretty similar to the Ambit3 Peak, reliable plus in all-conditions. Accuracy is practically the same, though the sampling rate is down and recorded distance is a bit less. Hit’s the +/-1% distance accuracy rather less at 65% of the time. At just under 14hrs, the Ambit2 also has less endurance (battery and memory) than the Ambit3 Peak, it lacks activity tracking stuff, and no mobile upload facility. Though as a bonus your anybrand ANT+ sensors will work with it.

Polar V800 (click me)

Unfortunately is not quite a complete trail running watch. Though apparently very accurate in easy GPS conditions distance accuracy varies quite a bit in difficult conditions. Hits the +/-1% distance accuracy a mere 35% of the time on our mixed condition trail routes. Endurance proved solid as it went a full 13hrs in real world trail testing. While the V800 has some excellent running features, sadly is still a bit locked up in proprietary Polar land, doesn’t fully support sensor standards and data needs to be wrestled to get into other apps. Android app is also finally reaching maturity after initially lagging the iOS version. However, currently the mobile Flow app refuses to sync runs most of the time.

Garmin FR310XT/FR910XT Combined (click me)

These are combined as are very similar in performance aside from elevation. Will do the business for trail running (most of the time). The granddaddies of trail run GPS watches, on a good day they are still capable of showing the youngin’s a thing or two about accuracy. Hits the +/-1% distance accuracy 68% of the time. Third party developers also come to rescue with Android mobile upload capability. Unfortunately is also prone to overestimating things, and on a bad day can get a bit confused about where it actually is (ie. is completely off-track). The FR310XT Without a barometer, elevation and climbing data is also a bit of a lottery.

Garmin fēnix 2 (click me)

A lemon for trail runners, don’t even think about it. Prone to random reboots, eats your data before it’ll upload it. Battery life is very poor and UltraTrac endurance mode doesn’t work in trail conditions, and even if it did wouldn’t record your heart rate anyway. Altitude accuracy is horrible, positional accuracy highly variable. Suprisingly overall distance accuracy isn’t too bad (if a little long), but doesn’t save it from it’s ‘avoid at all costs’ trail rating. Not including it in the summary table because the lack of respect is mutual.

 Summary Feature Table

General Trail Running Ambit2 Ambit3 fēnix3 FR310/910XT V800
GPS accuracy under canopy Excellent Excellent Good Great Good
Consistent GPS performance Yes Yes Yes(ish) No Yes (though not distance)
Rapid GPS Acquisition Yes Yes Yes No Yes(ish)
HRM Yes Yes Yes Yes Yes
Cadence pod option Yes Yes Yes Yes Yes
Battery 8hr with HRM and high accuracy setting Yes Yes Yes Yes Yes
Barometer Yes Yes Yes FR910XT Yes
Breadcrumb with waypoint navigation Yes Yes No No No
Vibration alerts No No Yes Yes Yes
Trail readable display Poor Poor Great Good Great
Open data access Limited Limited Yes Yes No
Fully ANT+ or BLE Compliant Yes Yes Yes Yes No
Ultra Feature Set Ambit2 Ambit3 fēnix3 FR310/910XT V800
Battery 14hr+ with HRM and high GPS accuracy Yes Yes No Yes No
Electronic Compass Yes Yes Yes No Yes(ish)
Nice to Have Features Ambit2 Ambit3 fēnix3 FR310/910XT V800
Mobile uploads No Yes[1] Yes Android only Yes(ish)
Cadence (without footpod) Yes Yes Yes No Yes
HRV (R-R) recording with recovery feature Yes Yes Yes No Yes
Footpod GPS override Yes Yes No No No[2]
Basic interval workout ability Yes Yes Yes Yes Yes
Pacing function RT/CP RT/CP CP/HP CP/HP RT/CP
Position/waypoint autolapping No No No Yes Yes[3]
Custom timer reminders Yes Yes Yes No Yes
Everyday watch Yes Yes Yes No Yes
Activity Tracking No Yes Yes No Yes
Mobile Notifications Yes Yes[1] Yes No Yes

NYT: not yet tested

Pacing function: RT race time, CP constant pace, HP historic pace

[1] At time of writing Suunto Mobile app (1.2.26) worked well with Android 5.1.1, can’t speak for earlier handsets. However versions 1.2.27 and 1.2.28 have seen connectivity issues return for some handsets (frequent repairing required)

[2] Feature only works with proprietary Polar footpod

[3] Only autolaps at the point at start of recording or previously recorded waypoint