Garmin fēnix 3 Trail Test

Original Review date: August 2015. Tested with Garmin Run HR Strap and 3.20-5.10 firmware (updated Aug 2017)

While data collection finished in May 2016, we’ve re-analysed the data using our current enhanced methodology. The comparison watch group has also been updated to include the Spartan Ultra.

FYI data collection is in progress for the fēnix 5X.

Summary

A nicely designed GPS watch with many fantastic features, great interface and companion software. Pre-3.60 firmware GPS and distance accuracy were dismal but Garmin has pulled off some impressive updates to improve accuracy considerably. The fēnix 3 still won’t win any accuracy awards as it still sits below the group average but has proven itself to be adequate on NZ trails – with a some caveats. Ultra runners take note – real world trail battery life came out at 12.5hrs, and UltraTrac isn’t always a reliable option. You’ll be wanting to pack that portable charger for anything longer.

A current model round-up can be seen here, and the testing methodology is here

What kind of trail runner is it good for?

The post 3.60 firmware fēnix 3 has shown itself to be adequate (depending on your standards) in most conditions but offers a huge set of trail running/activity tracking/smart watch type features. The fēnix 3 would suit a trail type who is happy to concede accuracy, some reliability, and endurance to get a watch to do nearly everything else. Be warned, you’ll want to be a fast ultra runner (12.5hr battery on most accurate setting), and 100 miler types are taking a bit of gamble on the UltraTrac side of things so running with a charger is your best option.

What should you expect as a long term running companion?

It appears there’s still a bit of chance at play in becoming a happy fēnix 3 owner. Of the three fēnix 3’s in the MEC running pool two were returned on warranty as they were freezing or recording rubbish data. Early updates were a bit risky as they tended to break stuff though it seems sorted now.  Perhaps more worryingly, there’s still a chance that occasionally the recorded trail run will throw a bit of a wobbly in anything more that moderate GPS conditions. UltraTrac in particular has seen some really off recording days. Perhaps this sounds worse than it its, there’s probably a good chance you’ll get along just fine (the real question here is – do you want to leave your precious event data recording to chance?).

Good Stuff

Great set of useable features for trail/ultra-running, excellent tactile buttons, and a great display. Plays nicely with other brand ANT+ sensors, data is directly accessible via the widely supported FIT format. Garmin is pretty open with active third party development interfacing web services and watch. Mobile app works well with both Apple and Android (in most firmware versions). UltraTrac mode is capable (but not certain) of working very well at recording distance with the Garmin Run HR strap or a footpod, comparable in fact to smart recording on the trail. Charging clip is good enough to power it on the run which is a good thing given limited battery performance.

Bad Stuff

For a high end trail GPS watch the fēnix 3 is below the group average in accuracy, and is well down on the venerable FR910XT. We still see the very occasional day where moderate to difficult GPS conditions result in a sawtooth type track dominated by random scatter. While these are far and few between that’s definitely a mark down on absolute reliability. Course navigation function is limited with no waypoint type proximity alerts on the breadcrumb trail, and no positional auto-lapping. Also no means of prioritising footpod pace/distance over GPS on normal GPS setting. Battery life on most accurate setting and HR during real world trail test was 12:24hrs which was dead last in the test pool. You’ll want a battery I.V. hookup for your really big days.

Plea to Garmin

Thanks very much for improving the accuracy since our first review Garmin! Most appreciated. Just the odd wobbly day and UltraTrac performance reliability to sort now. A bit more honesty on real world battery life (or at least stating the conditions that will reduce it). Putting some fit-for-purpose navigation features back in it would be also nice. And finally quality control, seeing some units just don’t work and firmware updates that break things aren’t a good look.

A feature request too, an easy way to optimise battery life when setting up a sport/app (ie. turn on smart recording, turn off bluetooth, etc).

Formal Testing

Accuracy

So far fēnix 3 testing has covered in excess of 200km on the surveyed courses over 30 separate days with a total of 73 recorded laps. Outing conditions included rain and shine, night and day. Our testing is designed to provide a statistically robust measures in real world conditions – check our methodology here.

The fēnix 3 was tested with GLONASS on and off, 1 second recording and smart recording under firmware ranging from 3.20 to 4.90. While testing under the various setup permutations may have been too limited to pick up an effect, neither would any improvement be substantial. We couldn’t pick any difference between GPS only and GPS+GLONASS accuracy though we didn’t try too hard to find any.

The GPS pool against which the fēnix 3 was tested included the following models: Suunto’s Ambit2, Ambit3 Peak, and Spartan Ultra, Polar’s V800, Garmin’s FR910XT and fēnix 2, and a Sony Xperia Z5c running the SportsTracker app.

During testing multiple GPS devices are carried as a means of validation, if all units have a bad GPS day then a tin foil hat is donned. Though some days with poor GDoP potential saw all units a little degraded, no testing days saw GPS chaos across all models.

Note: Aside from the initial overall figure, which tells us if will run long or short on long term averages, we report distance accuracy using absolute percentages. In other words 5% wrong is 5% wrong, it doesn’t matter if it’s long or short, they don’t cancel each other out during your run.

Distance

Thankfully the distance accuracy of the fēnix 3, tracked nicely with the improvements in positional accuracy. On the few direct and open-sky road outings, or runs along 1km markers distances were within 1% of the measured distance which is excellent (note: limited sample size here).

That said the fēnix 3 is consistently below the group average across all conditions tested. Overall  it registered 98.0% of the total distance, meaning it’d short you around 2% in your long-term training log if you run in conditions similar to ours.

In easy GPS conditions its performance is marginally down on the group average of 98.6% at 98.3%. However in both mixed and difficult conditions the fēnix 3 dropped to last place recording 97.6% and 94.5% respectively (the group average was 98.5% and 96.1%). By comparison the best performer came in at 99.2% (easy) 98.8% (mixed), and 96.6% (difficult).

Overall the fēnix 3 just manages to escape last place in distance accuracy with an adjusted score of 97.6%. The worst performer came in 97.5% and best at 98.5%. Yes margins appear tight here, though the next set of tests opens things up.

fenix3.abs.mean.accuracy

At this point you might be thinking those margins are pretty small, practically the same even. But there’s another way to look at accuracy where we call each recorded distance within 1% of the truth as a success. Think of it this way – you’ve just run a 10km PB and you want your glory on Strava or the online service of your choice. But your watch shows 9.9km at the finish, so no digital PB, and no bragging rights for you. So even a 1% error can be a bummer, let alone 4%.

Our ‘within 1% success test’ is much more sensitive in identifying how reliable each  recorded distance run is. The differences between models now becomes a lot more pronounced. Here the fēnix 3 recorded within 1% of the true distance a measly 29.0% of the time in easy conditions compared with 82.6% for the best performer and 47.3% for the group average. This plummets for all models under difficult conditions to 0% for the fēnix 3, 25.7% for the best performer, and 7.3% for the group. In all, at 21.7% across combined conditions the fēnix 3 is under the group average at 33.4%.

fenix3.99.PC

So what gives

When we plug all the factors we record into a model to show us which are associated with distance accuracy we can see the fēnix 3 is strongly effected by tree cover, and mildly effected by by satellite availability (well a general proxy of it). The data also suggests that distance accuracy there is a small benefit by slower average speed (??). On the other hand the raw GPX feed accuracy is predictive of good distance performance. Bottom line is that the fēnix 3 is just not accurate and running under a tree canopy just makes things worse.

Positional

The absolute positional performance of the fēnix 3 is also sub-average. Under easy GPS conditions the median track offset distance was 2.1 metres (average 2.9m), falling to 3.0m (median 3.9m) under moderately difficult conditions. And 3% of points on the surveyed course were 10 or more metres off track where better performing models had <1% greater than 10 metres off track during testing.

fenix3.gpx.offset

Note: positional accuracy is measured as GPS trackpoint to the closest point of a surveyed track (or repeat track where course is un-surveyed), so figures reported here will be a lot better than the actual positional accuracy.

Satellite vs Watch Distance

And for the nerdier types among you, here’s how the watch recorded distance lines up with it’s raw satellite distance. The charts also show how each and every recorded lap distance pans out over the testing.

The recorded distance you see on the watch is in purple whereas the yellow is from the unfiltered gpx track. The vertical lines are the averages of the two. Our general performance expectation is for the filtered recorded distance to be a tighter distribution that pulls the average distance closer to 100%. And indeed this is what happens with the best performing watch we’ve tested.

Garmin’s filtering is a bit of mixed bag. In easy conditions the watch recorded distance is down on the GPX distance while other conditions see little difference between the two. You can also see the deterioration of lap accuracy in the difficult GPS conditions. Also notable is the lack of data here, testing would have benefited from a longer collection period. Though it’s pretty unlikely that things would have drastically improved.

fenix3_measure_dist

When the fēnix 3 was taken into genuinely difficult conditions (heavy tree cover, steep terrain, gorges), this performance improvement was seen again over earlier firmware versions. Previously running repeats of an 8km circuit saw most of the loop as widely spaced parallel tracks or slalom crossed track with an average 12m distance between tracks. In recent firmware on this was down to 6.1m on the same course.

However, on an out-and-back in the same park on a different outing caught the fēnix 3 (4.40 firmware) on a bad day with an average of 19.4m separating a rather sawtooth track. Note: the other three GPS used that day performed normally.

Unsurprisingly variance in accuracy though well down on early firmware is still double that of the better GPS units.

Track Appearance

Unlike early firmware, which were horribly oversmoothed, post 3.60 firmware tracks under reasonable GPS conditions generally appeared reasonably sharp. Gate negotiations and collision avoidance, and overtaking maneuvers are all evident in the tracks.

The trackpoint cloud image below shows the density of GPS position updates from all +3.60 firmware recorded runs against part of surveyed course. While the relatively open sky view conditions (tracks to the right) appear tightly distributed around the track, the limited conditions (track on left) still shows extensive dispersion and shadowing. The really off-track stuff is from one of those fēnix 3 wonky days, get rid of that and the track would look a bit cleaner.

fenix3_cloud_2017The sampling rate on the 1 second setting was, unsurprisingly, very high. To be honest we’d be inclined to leave it on the more efficient smart recording setting for long trail runs as 1s sampling over >6hrs is overkill.

Elevation

Elevation range, if manually calibrated at start, appeared solid. The total climb on the surveyed course with the continuous correction setting on was ok but not stellar coming in at an average of around 84% of the measured course (the average was still dragged down by a few bad-day outliers). Firmware updates had saw little apparent improvement in elevation performance.

An altimeter grumble here is that unlike recorded activities the widget based altimeter/barometer do not continuously correct. I’d often wake to find my house was 40m below sea level which, despite climate change, isn’t yet the case. Also if I run or ride into work which is about 110m higher that my house I’d frequently arrive to a storm alert.

Note: The survey course probably is a bit tough on GPS watches with respect to elevation with lots of rapid 10-30m climbs/descents. Watches would probably perform better on longer more pronounced climbs/descents.

Battery Saving Data Recording

UltraTrac results with the Run HR strap suggests recorded distance can be very good indeed (even without an actual footpod), and broadly comparable with smart GPS recording. Note: we haven’t tested UltraTrac performance extensively though we have tried it various GPS conditions.

The fēnix 3’s strategy here is to cut GPS position updates to once a minute, and derive pace and distance from available accelerometers and sensors (Garmin Run HR Belt, watch accelerometer, or footpod). These, along with HR and elevation are updated as per normal smart recording.

However, when the fēnix 3 (3.60 firmware) was taken into an alpine environment for a three day mountaineering adventure (without the Run HR strap) the story was rather different. The fēnix 3 pretty much lost the plot on the UItraTrac setting. On two of the three days the data generated was rubbish. The map below shows the difference between the fēnix3 track (red) and another brand adventure type running GPS watch (blue) similarly on a 1 minute GPS setting. More recently we saw similarly erratic performance on a heavy bush track with 4.40 firmware, but with the running strap on the unit filtered distance much more conservatively and actually gave a semi-sensible result (only 24% short).

fēnix 3 (red) vs Ambit2 (blue)
fēnix 3 (red) vs another GPS watch (blue)

So sadly the UltraTrac option may not be a sure shot. Though it is capable of accurate recording under difficult GPS conditions we’ve had two outings where it struggled to cough up anything sensible (see below). Its unclear from the extent of our testing what triggers this, and how likely it is to reoccur. Should throw up a red flag to anyone considering it for trail 100 milers though.

Functional Tests

Data Syncing and Sharing

All recorded activities can be automatically synced to a number of 3rd party services via Garmin Connect. Additionally Garmin Connect allows services to sync non Garmin activities into Garmin Connect. Upload options are fantastic with computer-cable, mobile-BLE, and WiFi-direct capability. As a bonus when the fēnix 3 is connected to a computer it is effectively mounted like an USB drive so data is directly accessible for 3rd party desktop apps (even if it’s already been synced via mobile).

Navigation Functions

Multiple options exist for creating an off-road course using ‘follow roads’ type functionality, Garmin Basecamp with OpenStreetMap topo maps, Garmin Connect web, and GPSies.com all work (you can even drop GPX files directly into a folder on the watch). Courses created in Basecamp that have integrated waypoints for cues/alerts are split into separate “Saved Locations” and courses on import to the fēnix 3, more on that below. You can also easily convert historic or friends’ activities into a course.

The fēnix 3 is capable of including an elevation profile on a course, though is only visible when previewing the course. The watch displays the bearing and distance to closest part of course and alerts when you wander back off course (as I often do), nice! However, if a course originally had embedded waypoints, there is no proximity alert/cue functionality when navigating it. Though waypoints will be added and visible in the fēnix 3 map they are no longer part of the course.

Waypoint ETA and alert type functionality only work when navigating to ‘saved locations’ which is direct rather than course based navigation. Be nice to have some ETA functionality that respects course distances. Integrating elevation into the course and waypoint functions would also be good. Finally an auto-lap on course waypoints would be welcome indeed.

The Trackback feature is a great and easy to use ‘get me back to the start’ feature where you are not using course navigation already and need to abort your mission.

Waypoint management itself is good using Garmin Basecamp we can import/export to our hearts content. The on the watch waypoint marking function is also great as it can be setup as a shortcut to a button long-press. Not sure why course navigation doesn’t allow any embedded waypoint functionality but it really is a bit of a deal breaker for us in the navigation feature set.

Pacing Functions

The fēnix 3 has two options on pacing yourself during an activity. The virtual partner one sets off at a constant pace, not slowing for hills or bogs. The better option for trail types is the ability to race previous activities, either your own or your invisible friends. This is races you against a real-time play back of the previously recorded activity and is a much better option for mixed terrain running.

Battery Run Down

On a three day mountaineering trip with 27hrs recorded on the UltraTrac setting, with bluetooth off and no sensors in play the battery had 18% charge (logged temperature was well above zero so frigid conditions weren’t to blame). As noted earlier the positional data was terrible on two of the days, so we assume the GPS was working hard just to get a fix. No chance of getting close to 50hrs endurance when it’s having one of those days.

During the high accuracy exhaustion trail test (1sec recording with HRM on, bluetooth off), the fēnix 3 disappointingly came out bottom of the pool at just under 12.5hrs (the on the box spec states ‘up to 20hrs’). The best in the test went the full 15.5hrs with 19% battery remaining. The run was done in warm conditions with about 50% open sky and 50% heavy bush. The fēnix 3 used was less than 6 months old at time of testing.

The fēnix 3 does have another trick in it’s book – you can run with the charging clip attached to a portable power charger. This gives it potentially limitless battery endurance which gives you serious multiday types no excuse to stop. Still, we’re not overly keen on the power-junkie look in lesser events.

Standard Feature Set

General Trail Running

  • GPS accuracy under canopy : Poor
  • Consistent GPS accuracy : Yes(ish)
  • Rapid GPS Acquisition : Good
  • HRM : Yes
  • Cadence option : Yes
  • Battery 8hr with HRM and high accuracy setting : Yes
  • Barometer : Yes
  • Breadcrumb with waypoint navigation : No
  • Vibration alerts : Yes
  • Trail legible display: Excellent
  • Open data access : Yes
  • Fully ANT+ or BLE Compliant : Yes

Ultra Feature Set (as per trail running plus)

  • Battery 14hr+ with HRM and high accuracy recording : No
  • Battery 24hr+ with HRM and down-sampling : Can’t find testers
  • Electronic compass : Yes

Nice to Have Features

  • Mobile uploads : Yes
  • Cadence (without footpod) : Yes
  • HRV (R-R) recording with recovery estimate/test : Yes
  • Footpod GPS override : No
  • Basic interval workout ability : Yes
  • Pacing function : Yes (fixed pace and past activities)
  • Position/waypoint autolapping : No
  • Custom timer reminders : Yes
  • Everyday watch : Yes
    • Activity Tracking : Yes
    • Mobile Notifications : Yes

Non-Tested Core Features

While functions like heart rate and cadence are core, either we’ve got no means of objectively testing them.

Across all runs with the fēnix 3 cadence and heart rate appeared pretty consistent with the other units. Running with a footpod showed no obvious difference to that of running with the Run HR belt alone. The footpod can be manually calibrated which is always a good idea, so that’s good. We haven’t run without the Run HR belt so can comment on its effect on the fēnix 3’s performance generally.

Oh, and for those running with poles/wizard sticks or whatever you want to call them, the Run HR belt cadence works a treat (unlike wrist based accelerometers which get somewhat confused).

After sporting the fēnix 3 for all activities over a few months (only a subset were part of the formal testing), the fēnix appears rather more optimistic on its recovery checks than Suunto or Polar. I think I only saw something other than the “recovery check good” statement once. And I ran pretty broken on a few days, indeed running home from a 5km race or two days post a 15.5hr ultra certainly didn’t feel like “recovery check good” to me. On the other hand the VO2 max measurement tallied well with the Suunto Ambit3 and Polar V800.

General Useability

The overall design of the fēnix 3 is fantastic, the screen is highly customisable and legible day and night. The alert vibration is strong, beeps audible, and buttons feel positive with instant response. Satellite pickup time is typically rapid after syncing, <10 seconds with a reasonable sky-view.

Pairing the fēnix 3 to a raft of ANT+ sensors, including a Suunto ANT+ footpod was no problem. And by the time we got round to downloading (May 2015) it the Android version of Garmin Communicator worked without issue for syncing, activity reporting, and mobile notifications. What is especially good about the fēnix 3 is that even if you sync activities via mobile, the activity is still available to be picked up by a computer for putting into a 3rd party application (SportTracks in this case).

The Garmin Connect website is as useable as any these days and integrates general health stuff from the fēnix 3 (steps, sleep, blobbing on couch/office chair). A number of services enable syncing activities into and out of Garmin Connect which is great for those with multiple devices. This is in stark contrast to the likes of Polar Flow and Suunto Movescount services which don’t accept any but their own, and make it hard to grab data directly off the watch to boot.

The running features we use most are all easily accessible with minimal faffing. Though the initial setup, being via the watch itself was a bit painful compared to the computer based setup and sync approach of others.

Of the additional features that appeared be potentially useful for runners was the running dynamics screen. Though already a 180+ cadence runner, my ground contact time was still relatively high. Don’t really know what that means, someone might though.

Another feature of note for the mountain runners is the 3D distance/speed. While the concept of Garmin’s IQ Apps is interesting, none really added anything to a trail runner’s arsenal, maybe give it a bit of time?

The charging clip is well designed in comparison to the competitors, and the fact that it can be worn on the fly is an interesting prospect. Though can’t say that packing a charger on a run I.V. style is a solution we’d be keen on if we could avoid it.

Non-running features we liked included smart notifications on the watch, and a music controller. These simply worked with no faffing via the Garmin Connect mobile app on Android.

Like shoes and backpacks, comfort is a personal thing – try before you buy.

Long Term Verdict

Currently around 3,500km clocked on fēnix 3, including short and sharp trail races (2hrs), daily urban run/bike commutes, and some heavy condition trail work. Longest recording was a 15.5hr trail run, though the fēnix 3 died at 12:24hrs. Nothing new identified by way of reliability, or issues outside of what’s already been identified in terms of GPS accuracy, battery life, or navigation limitations.

Firmware updates are frequent, each addressing past performance or functional issues or new features. Unfortunately these updates have been known to break existing features from time to time. On the up side the firmware updates don’t clear the watch history.

The turnaround of the fēnix 3 performance has been pretty impressive. In our first August 2015 based review on pre-3.60 firmware we pretty much wrote it off as a trail running watch. Though still certainly not the most accurate (not even close) or reliable, especially on UltraTrac, it’s compelling feature set at least make it a credible trail watch offering for NZ conditions. Though serious trail racers and ultra types still may want to look elsewhere.

Advertisements