8,105
edits
Changes
From Fellrnr.com, Running tips
no edit summary
{{DISPLAYTITLE:GPS Accuracy of Garmin, Polar, and other Running Watches}}<div style="float:right;">__TOC__</div>
I evaluated the real world accuracy of GPS watches while running over 3,500 miles/5,600Km and recording over 14,000 data points as part of my evaluation of the [[Best Running Watch]]es. Under good conditions most of the watches are remarkably good, but when things get a little tough the differences become more apparent. The table below is a vastly simplified summary of the results, rating the best device as a 10 then comparing the others. (Note that '10' does not indicate perfect, just the best I've measured so far.)
{| class="wikitable"! Device! Trueness! Precision! Overall|-| {{Garmin 910XT}} with [[Footpod]]| 7.8| 10.0| 10.0|-| {{Garmin 205}}| 10.0| 8.7| 9.4|-| {{Garmin 310XT}} with [[Footpod]]| 5.9| 8.3| 8.1|-| iPhone 4s| 4.5| 8.3| 7.5|-| {{Garmin 610}}| 4.1| 8.5| 7.4|-| {{Garmin 310XT}} no [[Footpod]]| 4.7| 7.2| 6.9|-| {{Polar RC3 :GPS}}| 5.9| 6.6| 6.8|Accuracy-| {{Garmin 620}}| 1.9| 7.3| 5.0|-| {{Garmin 10}summary}| 1.7| 6.7| 4.4|}
=Methodology=
Simply taking a GPS watch on a single run does not provide sufficient data to reasonably evaluate its accuracy. So to gather the data for this test I marked my usual running route at quarter-mile intervals, using a [http://www.jonescounter.com/ Jones counter] to provide an accurate measure of distance. The Jones counter is the device used to certify courses, and I followed the [http://www.usatf.org/Products-/-Services/Course-Certifications/USATF-Certified-Courses/Certify-Your-Course.aspx USATF course certification process]. The course I run along is a little challenging for GPS, with lots of twists, tree cover, power lines, and one bridge that I go under. The bridge carries a 4 lane road, so it's wide enough to cause the watches to briefly loose GPS signal. However, I believe that it's reasonably representative of real-world conditions, and probably less challenging than running in the city with skyscrapers. At both ends of the course there is a turnaround, and I set the mark an eighth of a mile from the end. That way I can evaluate how well the watches handle an about turn.
=Accuracy=
I've divided the laps into those that go under the bridge, those that are immediately after a lap that goes under the bridge, those that go next to or under the power lines, those that turn around and then all the rest (normal). I've also included a 'mixed' category that reflects all the sections and a category for the straightest segment. The count field is how many measurements I have for that combination of condition and device.
{{:GPS Accuracy-includestatistics}}
==Interpretation and Conclusions==
What do these statistics mean? This is my interpretation:
* Under normal conditions the GPS accuracy is quite good for most devices.
* The Garmin 620 and Garmin 10 are noticeably poorer than the other devices. I found the accuracy of the 10/620 in general usage to be rather grim, and I did some testing pairing them up with the 610 or the 310XT. In all cases the 10/620 would have poor accuracy compared with the 610 or 310XT on the same run. On one run, the 620 lost over a mile over a 28 mile distance.When Garmin replaced my 620, the new watch would lose over a mile on a 16 mile run!
* The results of the Garmin 610 indicate the problems with the 620 & 10 are not inherent in a smaller device.
* The accuracy of all devices is better in a straight line than on curves or bendy routes. My course is a tough test for GPS devices with many curves and only a few relatively straight sections.
* None of the devices were accurate enough for a runner to trust the display of current pace for training or race pacing.
** For [[Practical Interval Training| interval training]], use a track or measure out the distance using some other mechanism.
** For general training or for races, use a device that supports displaying pace from the Footpodwhile using GPS for distance.
* Adding a Footpod to the {{Garmin 310XT}} improves its GPS accuracy.
* For the {{Garmin 610}} there was no difference with and without the Footpod. (Trueness was 3.33%/3.32%, Precision was 3.54%/3.68%, with/without).
* It takes time for the GPS watches to acquire the satellites. Some watches tended to say they are ready to go before they have an optimal lock. Therefore, to improve accuracy try to give them a little more time. Note that some newer GPS watches such as the {{Garmin 620}} have the ability to be preloaded with the satellite positions, reducing this startup time and start up in accuracy dramatically.
=Even GPS Watches have Bad Days=
Often GPS measurements of races, especially marathons record a longer distance than the race. This is partly because the USATF technique for measuring the distance takes a path that is no more than 12 inches away from the tangent (corner), and few runners are able to run that close. In a large marathon you can be forced to take a line that is a long way from the tangent. The other factor is that on a straight line, the GPS error tends to give a slightly longer measurement.
[[File:GPS Marathon.jpg|none|thumb|500px|Here you can see the GPS line is not following the straight road, giving a longer reading on the Thunder Road Marathon. Notice that the GPS is also cutting the corner at the top (we didn't run through the building).]]
=Detailed Breakdown by Condition=
Below are more detailed statistics for each device under different conditions. This is rather a lot of data, but it allows you to see where various devices seem to have problems.
{{:GPS Accuracy-breakdown}}
=Density Plots=
Below are the density plots for each device.
==Turn Around==
[[File:GPS Turn Around.jpeg|none|thumb|1024px|Density plots of the GPS accuracy for the turn-around condition.]]
=Garmin 620 Issues=
The Garmin 620 has become rather notorious for its poor GPS quality. I raised the issue with Garmin support and they kindly sent me a replacement device, but as you can see below, the replacement was no better. In fact, it was actually worse than my original unit. I've also broken down the readings by firmware version, and you can see some slight improvement going from V2.90 to V3.00, but it's only slight.
{{:GPS Accuracy-g620}}
I have come to suspect that the 620 has two issues. Firstly, Garmin has compromised GPS accuracy for size and battery life. This is then compounded by high levels of smoothing in an attempt to compensate. Secondly, I suspect that the 620 has a problem with its [http://en.wikipedia.org/wiki/Wide_Area_Augmentation_System WAAS] processing. WAAS provides an offset to GPS receivers to correct for errors, so an error in WASS processing could lead to the GPS location being offset. Below are a couple of examples where you can see that it appears that the track has been offset, rather than the 620 simply becoming 'lost'.
{| class="wikitable"
|- valign="top"
|[[File:Garmin620 Offset1.jpg|none|thumb|x500px|Here you can see the last repeat is offset. Starting at lap marker 49, the track follows the same outline as the more accurate tracks, but is offset. So marker 50 should be near 4, 51 near 37, 52 near 2, 53 near 1, and the finish near the start.]]
|[[File:Garmin620 Offset2.jpg|none|thumb|x500px|This is a simple out and back run of ~3 miles/5 Km, but you can see after the turn around the Garmin 620 records a gradulally widening gap, even though it follows the right overall shape. (The outbound track is fairly accurate, the return is messed up.)]]
|}
=Device Specific Notes=
For those interested in some of the details of how devices are configured for testing, here are some additional notes.
* Garmin devices are set to 'smart recording'. I did try an informal test with the 620 using 1-second recording, but it appeared to make no difference.
=Next Steps=
This is an initial analysis of the data I have, and there are a number of further evaluations to do.
