Difference between revisions of "GPS Accuracy"

From Fellrnr.com, Running tips
Jump to: navigation, search
User:Fellrnr (User talk:Fellrnr | contribs)
User:Fellrnr (User talk:Fellrnr | contribs)
Line 1,055: Line 1,055:
 
|[[File:910XT Bad.jpg|none|thumb|x500px|Again, this track is recorded on the same run as the image to the left. The {{Garmin 910XT}} gets a little confused at the start, and then again around lap 27.]]
 
|[[File:910XT Bad.jpg|none|thumb|x500px|Again, this track is recorded on the same run as the image to the left. The {{Garmin 910XT}} gets a little confused at the start, and then again around lap 27.]]
 
|}
 
|}
 +
=Some Devices Are Better Than Others=
 +
Below is a section of two runs showing the same section of the course, both taken at the same time, one from the Garmin 310XT and the other from the Garmin 620. These give a good indication of the accuracy problems I've seen with the Garmin 620.
 +
{| class="wikitable"
 +
|- valign="top"
 +
|[[File:ExampleGarmin310.jpg|none|thumb|x500px| You can see the GPS tracks (thin red line) are close together and the lap markers (yellow diamonds) are clustered nicely. The blue dots on the GPS tracks are the actual GPS recordings.]]
 +
|[[File:ExampleGarmin620.jpg|none|thumb|x500px|By contrast, the 620 has much wider GPS tracks and dispersed lap markers. ]]
 +
|}
 +
=GPS Short and long measurements=
 +
As you can see from the images below, the GPS track tends to take shortcuts around bends, reducing the length of the measured track.
 +
[[File:GPS Shortcuts.jpg|none|thumb|500px|The GPS tracks in red with blue dots representing each GPS point recorded. The arrow points to the actual path taken.]]
 +
Often GPS measurements of races, especially marathons record a longer distance than the race. This is partly because the USATF technique for measuring the distance takes a path that is no more than 12 inches away from the tangent (corner), and few runners are able to run that close. In a large marathon you can be forced to take a line that is a long way from the tangent. The other factor is that on a straight line, the GPS error tends to give a slightly longer measurement.
 +
[[File:GPS Marathon.jpg|none|thumb|500px|Here you can see the GPS line is not following the straight road, giving a longer reading on the Thunder Road Marathon. Notice that the GPS is also cutting the corner at the top (we didn't run through the building).]]
 +
=Density Plots=
 +
Below are the density plots of the 'normal' condition for each device.
 +
[[File:GPS Normal.jpeg|none|thumb|1024px|Density plots of the GPS accuracy.]]
 
=Next Steps=
 
=Next Steps=
 
This is an initial analysis of the data I have, and there are a number of further evaluations to do.
 
This is an initial analysis of the data I have, and there are a number of further evaluations to do.
 
* Gather data from some other [[Best Running Watch| GPS Running Watches]].  
 
* Gather data from some other [[Best Running Watch| GPS Running Watches]].  
−
** I will evaluate the [http://www.amazon.com/Garmin-010-01128-30-Forerunner-620-Bundle/dp/B00FBYYC90 Garmin 620] when it becomes available as it has some promising functionality.
+
* Add in more graphs of the distribution of accuracy, and possibly a [http://en.wikipedia.org/wiki/Q%E2%80%93Q_plot Q-Q plot] (which shows reasonably [http://en.wikipedia.org/wiki/Normal_distribution normal distribution]).
−
* Add in some graphs of the distribution of accuracy, and possibly a [http://en.wikipedia.org/wiki/Q%E2%80%93Q_plot Q-Q plot] (which shows reasonably [http://en.wikipedia.org/wiki/Normal_distribution normal distribution]).
 
 
* Check how GPS accuracy changes over the course of a run, as I've seen a distinct tendency for the watches to say they are good to go when they don't really have an optimal lock on the satellites.
 
* Check how GPS accuracy changes over the course of a run, as I've seen a distinct tendency for the watches to say they are good to go when they don't really have an optimal lock on the satellites.
 
* Look at how accurate the GPS watches are for measuring elevation, and compare with barometric data.
 
* Look at how accurate the GPS watches are for measuring elevation, and compare with barometric data.

Revision as of 16:42, 11 February 2014

I evaluated the real world accuracy of GPS watches while running over 2,500 miles/4,000Km and recording over 10,000 data points as part of my evaluation of the Best Running Watches. Under good conditions most of the watches are remarkably good, but when things get a little tough the differences become more apparent. The table below is a vastly simplified summary of the results, rating the best device as a 10 then comparing the others. (Note that '10' does not indicate perfect, just the best I've measured so far.)

Device Trueness Precision Overall
Garmin 205 10.0 9.0 10.0
Garmin 910XT with Footpod 7.4 8.8 9.0
Garmin 310XT with Footpod 5.8 8.3 7.9
Garmin 610 (preliminary) 4.0 10.0 7.4
iPhone 4s 4.4 8.7 7.2
Garmin 310XT no Footpod 4.6 7.2 6.6
Polar RC3 GPS 5.7 6.2 6.5
Garmin 620 1.9 7.6 4.3
Garmin 10 (preliminary) 1.8 5.1 3.5

The results of the Garmin 10 and 610 are preliminary as I have not gathered an equivalent volume of data to the other devices.

1 Methodology

Simply taking a GPS watch on a single run does not provide sufficient data to reasonably evaluate its accuracy. So to gather the data for this test I marked my usual running route at quarter-mile intervals, using a Jones counter to provide an accurate measure of distance. The Jones counter is the device used to certify courses, and I followed the USATF course certification process. The course I run along is a little challenging for GPS, with lots of twists, tree cover, power lines, and one bridge that I go under. The bridge carries a 4 lane road, so it's wide enough to cause the watches to briefly loose GPS signal. However, I believe that it's reasonably representative of real-world conditions, and probably less challenging than running in the city with skyscrapers. At both ends of the course there is a turnaround, and I set the mark an eighth of a mile from the end. That way I can evaluate how well the watches handle an about turn.

This is the course I use to evaluate the accuracy of GPS Running Watches.

2 Accuracy, Trueness and Precision

For this evaluation I'll use the definition of Accuracy as the combination of trueness and precision:

This is an example of high precision, as all the hits are tightly clustered. However, the trueness is poor as all the hits are off center, so accuracy is low.
This shows good trueness, as all the hits are around the center. On average they are on target, but there is poor precision, as the hits are scattered.

We can look at trueness by measuring the average lap length and precision by measuring the standard deviation.

3 Accuracy

I've divided the laps into those that go under the bridge, those that are immediately after a lap that goes under the bridge, those that go next to or under the power lines, those that turn around and then all the rest (normal). I've also included a 'mixed' category that reflects all the sections.

3.1 Accuracy by Condition then Device

Condition Device Count Trueness (Average Distance Error) Standard Deviation 95% within

Normal

Garmin 10 274

7.740% (408.7 Ft/Mile 77.4 m/Km)

0.066 (350.7 Ft/Mile 66.4 m/Km)

0.133 (701.4 Ft/Mile 132.8 m/Km)
Garmin 205 779

1.381% (72.9 Ft/Mile 13.8 m/Km)

0.031 (165.6 Ft/Mile 31.4 m/Km)

0.063 (331.2 Ft/Mile 62.7 m/Km)
Garmin 620 944

6.913% (365.0 Ft/Mile 69.1 m/Km)

0.040 (213.5 Ft/Mile 40.4 m/Km)

0.081 (427.0 Ft/Mile 80.9 m/Km)
iPhone 4s 668

3.046% (160.8 Ft/Mile 30.5 m/Km)

0.035 (186.0 Ft/Mile 35.2 m/Km)

0.070 (372.0 Ft/Mile 70.5 m/Km)
Polar RC3 GPS 983

1.511% (79.8 Ft/Mile 15.1 m/Km)

0.041 (217.4 Ft/Mile 41.2 m/Km)

0.082 (434.7 Ft/Mile 82.3 m/Km)
Garmin 310XT with Footpod 2786

2.013% (106.3 Ft/Mile 20.1 m/Km)

0.035 (187.1 Ft/Mile 35.4 m/Km)

0.071 (374.3 Ft/Mile 70.9 m/Km)
Garmin 310XT no Footpod 1368

2.708% (143.0 Ft/Mile 27.1 m/Km)

0.041 (214.8 Ft/Mile 40.7 m/Km)

0.081 (429.5 Ft/Mile 81.3 m/Km)
Garmin 610 289

3.124% (164.9 Ft/Mile 31.2 m/Km)

0.031 (166.0 Ft/Mile 31.4 m/Km)

0.063 (331.9 Ft/Mile 62.9 m/Km)
Garmin 910XT with Footpod 514

1.429% (75.4 Ft/Mile 14.3 m/Km)

0.030 (155.8 Ft/Mile 29.5 m/Km)

0.059 (311.5 Ft/Mile 59.0 m/Km)
All 8605

2.811% (148.4 Ft/Mile 28.1 m/Km)

0.042 (223.2 Ft/Mile 42.3 m/Km)

0.085 (446.3 Ft/Mile 84.5 m/Km)

Under Bridge

Garmin 10 24

3.387% (178.8 Ft/Mile 33.9 m/Km)

0.033 (174.3 Ft/Mile 33.0 m/Km)

0.066 (348.6 Ft/Mile 66.0 m/Km)
Garmin 205 66

-1.379% (72.8 Ft/Mile 13.8 m/Km)

0.074 (390.2 Ft/Mile 73.9 m/Km)

0.148 (780.4 Ft/Mile 147.8 m/Km)
Garmin 620 61

4.818% (254.4 Ft/Mile 48.2 m/Km)

0.042 (219.7 Ft/Mile 41.6 m/Km)

0.083 (439.4 Ft/Mile 83.2 m/Km)
iPhone 4s 59

1.402% (74.0 Ft/Mile 14.0 m/Km)

0.034 (178.0 Ft/Mile 33.7 m/Km)

0.067 (356.1 Ft/Mile 67.4 m/Km)
Polar RC3 GPS 86

5.612% (296.3 Ft/Mile 56.1 m/Km)

0.078 (412.7 Ft/Mile 78.2 m/Km)

0.156 (825.3 Ft/Mile 156.3 m/Km)
Garmin 310XT with Footpod 222

3.193% (168.6 Ft/Mile 31.9 m/Km)

0.049 (260.0 Ft/Mile 49.2 m/Km)

0.098 (520.0 Ft/Mile 98.5 m/Km)
Garmin 310XT no Footpod 111

1.420% (75.0 Ft/Mile 14.2 m/Km)

0.066 (348.9 Ft/Mile 66.1 m/Km)

0.132 (697.9 Ft/Mile 132.2 m/Km)
Garmin 610 22

2.761% (145.8 Ft/Mile 27.6 m/Km)

0.038 (201.8 Ft/Mile 38.2 m/Km)

0.076 (403.6 Ft/Mile 76.4 m/Km)
Garmin 910XT with Footpod 43

1.631% (86.1 Ft/Mile 16.3 m/Km)

0.084 (444.5 Ft/Mile 84.2 m/Km)

0.168 (889.0 Ft/Mile 168.4 m/Km)
All 694

2.661% (140.5 Ft/Mile 26.6 m/Km)

0.062 (329.0 Ft/Mile 62.3 m/Km)

0.125 (658.0 Ft/Mile 124.6 m/Km)

Post Bridge

Garmin 10 24

11.616% (613.3 Ft/Mile 116.2 m/Km)

0.037 (194.4 Ft/Mile 36.8 m/Km)

0.074 (388.8 Ft/Mile 73.6 m/Km)
Garmin 205 66

2.157% (113.9 Ft/Mile 21.6 m/Km)

0.024 (125.5 Ft/Mile 23.8 m/Km)

0.048 (250.9 Ft/Mile 47.5 m/Km)
Garmin 620 61

8.900% (469.9 Ft/Mile 89.0 m/Km)

0.049 (257.6 Ft/Mile 48.8 m/Km)

0.098 (515.2 Ft/Mile 97.6 m/Km)
iPhone 4s 54

4.422% (233.5 Ft/Mile 44.2 m/Km)

0.030 (157.4 Ft/Mile 29.8 m/Km)

0.060 (314.9 Ft/Mile 59.6 m/Km)
Polar RC3 GPS 84

7.399% (390.7 Ft/Mile 74.0 m/Km)

0.050 (263.2 Ft/Mile 49.8 m/Km)

0.100 (526.4 Ft/Mile 99.7 m/Km)
Garmin 310XT with Footpod 188

4.095% (216.2 Ft/Mile 41.0 m/Km)

0.037 (197.4 Ft/Mile 37.4 m/Km)

0.075 (394.8 Ft/Mile 74.8 m/Km)
Garmin 310XT no Footpod 106

5.053% (266.8 Ft/Mile 50.5 m/Km)

0.035 (187.0 Ft/Mile 35.4 m/Km)

0.071 (374.1 Ft/Mile 70.8 m/Km)
Garmin 610 22

4.839% (255.5 Ft/Mile 48.4 m/Km)

0.020 (103.1 Ft/Mile 19.5 m/Km)

0.039 (206.1 Ft/Mile 39.0 m/Km)
Garmin 910XT with Footpod 40

4.339% (229.1 Ft/Mile 43.4 m/Km)

0.021 (113.2 Ft/Mile 21.4 m/Km)

0.043 (226.4 Ft/Mile 42.9 m/Km)
All 645

5.287% (279.1 Ft/Mile 52.9 m/Km)

0.043 (228.5 Ft/Mile 43.3 m/Km)

0.087 (456.9 Ft/Mile 86.5 m/Km)

Turn Around

Garmin 10 23

12.971% (684.9 Ft/Mile 129.7 m/Km)

0.038 (198.2 Ft/Mile 37.5 m/Km)

0.075 (396.5 Ft/Mile 75.1 m/Km)
Garmin 205 72

3.645% (192.5 Ft/Mile 36.5 m/Km)

0.026 (136.9 Ft/Mile 25.9 m/Km)

0.052 (273.9 Ft/Mile 51.9 m/Km)
Garmin 620 80

11.302% (596.7 Ft/Mile 113.0 m/Km)

0.043 (228.5 Ft/Mile 43.3 m/Km)

0.087 (457.0 Ft/Mile 86.6 m/Km)
iPhone 4s 62

6.623% (349.7 Ft/Mile 66.2 m/Km)

0.045 (235.6 Ft/Mile 44.6 m/Km)

0.089 (471.3 Ft/Mile 89.3 m/Km)
Polar RC3 GPS 92

6.471% (341.7 Ft/Mile 64.7 m/Km)

0.077 (407.3 Ft/Mile 77.1 m/Km)

0.154 (814.6 Ft/Mile 154.3 m/Km)
Garmin 310XT with Footpod 205

5.573% (294.2 Ft/Mile 55.7 m/Km)

0.042 (219.6 Ft/Mile 41.6 m/Km)

0.083 (439.2 Ft/Mile 83.2 m/Km)
Garmin 310XT no Footpod 123

6.017% (317.7 Ft/Mile 60.2 m/Km)

0.054 (285.9 Ft/Mile 54.1 m/Km)

0.108 (571.8 Ft/Mile 108.3 m/Km)
Garmin 610 26

6.255% (330.3 Ft/Mile 62.6 m/Km)

0.024 (124.7 Ft/Mile 23.6 m/Km)

0.047 (249.4 Ft/Mile 47.2 m/Km)
Garmin 910XT with Footpod 43

5.117% (270.2 Ft/Mile 51.2 m/Km)

0.026 (135.3 Ft/Mile 25.6 m/Km)

0.051 (270.7 Ft/Mile 51.3 m/Km)
All 726

6.523% (344.4 Ft/Mile 65.2 m/Km)

0.053 (279.1 Ft/Mile 52.9 m/Km)

0.106 (558.1 Ft/Mile 105.7 m/Km)

Power Line

Garmin 10 46

7.841% (414.0 Ft/Mile 78.4 m/Km)

0.036 (188.5 Ft/Mile 35.7 m/Km)

0.071 (377.0 Ft/Mile 71.4 m/Km)
Garmin 205 127

1.395% (73.7 Ft/Mile 14.0 m/Km)

0.023 (122.5 Ft/Mile 23.2 m/Km)

0.046 (245.1 Ft/Mile 46.4 m/Km)
Garmin 620 162

7.860% (415.0 Ft/Mile 78.6 m/Km)

0.028 (148.7 Ft/Mile 28.2 m/Km)

0.056 (297.5 Ft/Mile 56.3 m/Km)
iPhone 4s 110

2.560% (135.2 Ft/Mile 25.6 m/Km)

0.029 (152.7 Ft/Mile 28.9 m/Km)

0.058 (305.3 Ft/Mile 57.8 m/Km)
Polar RC3 GPS 167

2.030% (107.2 Ft/Mile 20.3 m/Km)

0.036 (188.8 Ft/Mile 35.8 m/Km)

0.072 (377.7 Ft/Mile 71.5 m/Km)
Garmin 310XT with Footpod 515

2.566% (135.5 Ft/Mile 25.7 m/Km)

0.035 (183.2 Ft/Mile 34.7 m/Km)

0.069 (366.5 Ft/Mile 69.4 m/Km)
Garmin 310XT no Footpod 229

3.360% (177.4 Ft/Mile 33.6 m/Km)

0.036 (187.7 Ft/Mile 35.5 m/Km)

0.071 (375.3 Ft/Mile 71.1 m/Km)
Garmin 610 43

4.038% (213.2 Ft/Mile 40.4 m/Km)

0.026 (136.7 Ft/Mile 25.9 m/Km)

0.052 (273.3 Ft/Mile 51.8 m/Km)
Garmin 910XT with Footpod 90

2.101% (110.9 Ft/Mile 21.0 m/Km)

0.024 (127.2 Ft/Mile 24.1 m/Km)

0.048 (254.4 Ft/Mile 48.2 m/Km)
All 1489

3.281% (173.2 Ft/Mile 32.8 m/Km)

0.038 (198.5 Ft/Mile 37.6 m/Km)

0.075 (397.1 Ft/Mile 75.2 m/Km)

Mixed

Garmin 10 391

8.030% (424.0 Ft/Mile 80.3 m/Km)

0.062 (327.0 Ft/Mile 61.9 m/Km)

0.124 (654.0 Ft/Mile 123.9 m/Km)
Garmin 205 1110

1.411% (74.5 Ft/Mile 14.1 m/Km)

0.035 (185.6 Ft/Mile 35.2 m/Km)

0.070 (371.2 Ft/Mile 70.3 m/Km)
Garmin 620 1308

7.293% (385.1 Ft/Mile 72.9 m/Km)

0.042 (220.1 Ft/Mile 41.7 m/Km)

0.083 (440.2 Ft/Mile 83.4 m/Km)
iPhone 4s 953

3.199% (168.9 Ft/Mile 32.0 m/Km)

0.036 (192.4 Ft/Mile 36.4 m/Km)

0.073 (384.8 Ft/Mile 72.9 m/Km)
Polar RC3 GPS 1412

2.495% (131.8 Ft/Mile 25.0 m/Km)

0.051 (269.9 Ft/Mile 51.1 m/Km)

0.102 (539.9 Ft/Mile 102.3 m/Km)
Garmin 310XT with Footpod 3916

2.439% (128.8 Ft/Mile 24.4 m/Km)

0.038 (199.5 Ft/Mile 37.8 m/Km)

0.076 (399.1 Ft/Mile 75.6 m/Km)
Garmin 310XT no Footpod 1937

3.050% (161.0 Ft/Mile 30.5 m/Km)

0.044 (231.7 Ft/Mile 43.9 m/Km)

0.088 (463.3 Ft/Mile 87.8 m/Km)
Garmin 610 402

3.498% (184.7 Ft/Mile 35.0 m/Km)

0.032 (166.5 Ft/Mile 31.5 m/Km)

0.063 (332.9 Ft/Mile 63.1 m/Km)
Garmin 910XT with Footpod 730

1.900% (100.3 Ft/Mile 19.0 m/Km)

0.036 (188.6 Ft/Mile 35.7 m/Km)

0.071 (377.2 Ft/Mile 71.4 m/Km)
All 12159

3.213% (169.7 Ft/Mile 32.1 m/Km)

0.045 (237.7 Ft/Mile 45.0 m/Km)

0.090 (475.4 Ft/Mile 90.0 m/Km)

3.2 Accuracy by Device then Condition

Device Condition Count Trueness (Average Distance Error) Standard Deviation 95% within

Garmin 10

Normal 274

7.740% (408.7 Ft/Mile 77.4 m/Km)

0.066 (350.7 Ft/Mile 66.4 m/Km)

0.133 (701.4 Ft/Mile 132.8 m/Km)
Under Bridge 24

3.387% (178.8 Ft/Mile 33.9 m/Km)

0.033 (174.3 Ft/Mile 33.0 m/Km)

0.066 (348.6 Ft/Mile 66.0 m/Km)
Post Bridge 24

11.616% (613.3 Ft/Mile 116.2 m/Km)

0.037 (194.4 Ft/Mile 36.8 m/Km)

0.074 (388.8 Ft/Mile 73.6 m/Km)
Turn Around 23

12.971% (684.9 Ft/Mile 129.7 m/Km)

0.038 (198.2 Ft/Mile 37.5 m/Km)

0.075 (396.5 Ft/Mile 75.1 m/Km)
Power Line 46

7.841% (414.0 Ft/Mile 78.4 m/Km)

0.036 (188.5 Ft/Mile 35.7 m/Km)

0.071 (377.0 Ft/Mile 71.4 m/Km)
Mixed 391

8.030% (424.0 Ft/Mile 80.3 m/Km)

0.062 (327.0 Ft/Mile 61.9 m/Km)

0.124 (654.0 Ft/Mile 123.9 m/Km)

Garmin 205

Normal 779

1.381% (72.9 Ft/Mile 13.8 m/Km)

0.031 (165.6 Ft/Mile 31.4 m/Km)

0.063 (331.2 Ft/Mile 62.7 m/Km)
Under Bridge 66

-1.379% (72.8 Ft/Mile 13.8 m/Km)

0.074 (390.2 Ft/Mile 73.9 m/Km)

0.148 (780.4 Ft/Mile 147.8 m/Km)
Post Bridge 66

2.157% (113.9 Ft/Mile 21.6 m/Km)

0.024 (125.5 Ft/Mile 23.8 m/Km)

0.048 (250.9 Ft/Mile 47.5 m/Km)
Turn Around 72

3.645% (192.5 Ft/Mile 36.5 m/Km)

0.026 (136.9 Ft/Mile 25.9 m/Km)

0.052 (273.9 Ft/Mile 51.9 m/Km)
Power Line 127

1.395% (73.7 Ft/Mile 14.0 m/Km)

0.023 (122.5 Ft/Mile 23.2 m/Km)

0.046 (245.1 Ft/Mile 46.4 m/Km)
Mixed 1110

1.411% (74.5 Ft/Mile 14.1 m/Km)

0.035 (185.6 Ft/Mile 35.2 m/Km)

0.070 (371.2 Ft/Mile 70.3 m/Km)

Garmin 620

Normal 944

6.913% (365.0 Ft/Mile 69.1 m/Km)

0.040 (213.5 Ft/Mile 40.4 m/Km)

0.081 (427.0 Ft/Mile 80.9 m/Km)
Under Bridge 61

4.818% (254.4 Ft/Mile 48.2 m/Km)

0.042 (219.7 Ft/Mile 41.6 m/Km)

0.083 (439.4 Ft/Mile 83.2 m/Km)
Post Bridge 61

8.900% (469.9 Ft/Mile 89.0 m/Km)

0.049 (257.6 Ft/Mile 48.8 m/Km)

0.098 (515.2 Ft/Mile 97.6 m/Km)
Turn Around 80

11.302% (596.7 Ft/Mile 113.0 m/Km)

0.043 (228.5 Ft/Mile 43.3 m/Km)

0.087 (457.0 Ft/Mile 86.6 m/Km)
Power Line 162

7.860% (415.0 Ft/Mile 78.6 m/Km)

0.028 (148.7 Ft/Mile 28.2 m/Km)

0.056 (297.5 Ft/Mile 56.3 m/Km)
Mixed 1308

7.293% (385.1 Ft/Mile 72.9 m/Km)

0.042 (220.1 Ft/Mile 41.7 m/Km)

0.083 (440.2 Ft/Mile 83.4 m/Km)

iPhone 4s

Normal 668

3.046% (160.8 Ft/Mile 30.5 m/Km)

0.035 (186.0 Ft/Mile 35.2 m/Km)

0.070 (372.0 Ft/Mile 70.5 m/Km)
Under Bridge 59

1.402% (74.0 Ft/Mile 14.0 m/Km)

0.034 (178.0 Ft/Mile 33.7 m/Km)

0.067 (356.1 Ft/Mile 67.4 m/Km)
Post Bridge 54

4.422% (233.5 Ft/Mile 44.2 m/Km)

0.030 (157.4 Ft/Mile 29.8 m/Km)

0.060 (314.9 Ft/Mile 59.6 m/Km)
Turn Around 62

6.623% (349.7 Ft/Mile 66.2 m/Km)

0.045 (235.6 Ft/Mile 44.6 m/Km)

0.089 (471.3 Ft/Mile 89.3 m/Km)
Power Line 110

2.560% (135.2 Ft/Mile 25.6 m/Km)

0.029 (152.7 Ft/Mile 28.9 m/Km)

0.058 (305.3 Ft/Mile 57.8 m/Km)
Mixed 953

3.199% (168.9 Ft/Mile 32.0 m/Km)

0.036 (192.4 Ft/Mile 36.4 m/Km)

0.073 (384.8 Ft/Mile 72.9 m/Km)

Polar RC3 GPS

Normal 983

1.511% (79.8 Ft/Mile 15.1 m/Km)

0.041 (217.4 Ft/Mile 41.2 m/Km)

0.082 (434.7 Ft/Mile 82.3 m/Km)
Under Bridge 86

5.612% (296.3 Ft/Mile 56.1 m/Km)

0.078 (412.7 Ft/Mile 78.2 m/Km)

0.156 (825.3 Ft/Mile 156.3 m/Km)
Post Bridge 84

7.399% (390.7 Ft/Mile 74.0 m/Km)

0.050 (263.2 Ft/Mile 49.8 m/Km)

0.100 (526.4 Ft/Mile 99.7 m/Km)
Turn Around 92

6.471% (341.7 Ft/Mile 64.7 m/Km)

0.077 (407.3 Ft/Mile 77.1 m/Km)

0.154 (814.6 Ft/Mile 154.3 m/Km)
Power Line 167

2.030% (107.2 Ft/Mile 20.3 m/Km)

0.036 (188.8 Ft/Mile 35.8 m/Km)

0.072 (377.7 Ft/Mile 71.5 m/Km)
Mixed 1412

2.495% (131.8 Ft/Mile 25.0 m/Km)

0.051 (269.9 Ft/Mile 51.1 m/Km)

0.102 (539.9 Ft/Mile 102.3 m/Km)

Garmin 310XT with Footpod

Normal 2786

2.013% (106.3 Ft/Mile 20.1 m/Km)

0.035 (187.1 Ft/Mile 35.4 m/Km)

0.071 (374.3 Ft/Mile 70.9 m/Km)
Under Bridge 222

3.193% (168.6 Ft/Mile 31.9 m/Km)

0.049 (260.0 Ft/Mile 49.2 m/Km)

0.098 (520.0 Ft/Mile 98.5 m/Km)
Post Bridge 188

4.095% (216.2 Ft/Mile 41.0 m/Km)

0.037 (197.4 Ft/Mile 37.4 m/Km)

0.075 (394.8 Ft/Mile 74.8 m/Km)
Turn Around 205

5.573% (294.2 Ft/Mile 55.7 m/Km)

0.042 (219.6 Ft/Mile 41.6 m/Km)

0.083 (439.2 Ft/Mile 83.2 m/Km)
Power Line 515

2.566% (135.5 Ft/Mile 25.7 m/Km)

0.035 (183.2 Ft/Mile 34.7 m/Km)

0.069 (366.5 Ft/Mile 69.4 m/Km)
Mixed 3916

2.439% (128.8 Ft/Mile 24.4 m/Km)

0.038 (199.5 Ft/Mile 37.8 m/Km)

0.076 (399.1 Ft/Mile 75.6 m/Km)

Garmin 310XT no Footpod

Normal 1368

2.708% (143.0 Ft/Mile 27.1 m/Km)

0.041 (214.8 Ft/Mile 40.7 m/Km)

0.081 (429.5 Ft/Mile 81.3 m/Km)
Under Bridge 111

1.420% (75.0 Ft/Mile 14.2 m/Km)

0.066 (348.9 Ft/Mile 66.1 m/Km)

0.132 (697.9 Ft/Mile 132.2 m/Km)
Post Bridge 106

5.053% (266.8 Ft/Mile 50.5 m/Km)

0.035 (187.0 Ft/Mile 35.4 m/Km)

0.071 (374.1 Ft/Mile 70.8 m/Km)
Turn Around 123

6.017% (317.7 Ft/Mile 60.2 m/Km)

0.054 (285.9 Ft/Mile 54.1 m/Km)

0.108 (571.8 Ft/Mile 108.3 m/Km)
Power Line 229

3.360% (177.4 Ft/Mile 33.6 m/Km)

0.036 (187.7 Ft/Mile 35.5 m/Km)

0.071 (375.3 Ft/Mile 71.1 m/Km)
Mixed 1937

3.050% (161.0 Ft/Mile 30.5 m/Km)

0.044 (231.7 Ft/Mile 43.9 m/Km)

0.088 (463.3 Ft/Mile 87.8 m/Km)

Garmin 610

Normal 289

3.124% (164.9 Ft/Mile 31.2 m/Km)

0.031 (166.0 Ft/Mile 31.4 m/Km)

0.063 (331.9 Ft/Mile 62.9 m/Km)
Under Bridge 22

2.761% (145.8 Ft/Mile 27.6 m/Km)

0.038 (201.8 Ft/Mile 38.2 m/Km)

0.076 (403.6 Ft/Mile 76.4 m/Km)
Post Bridge 22

4.839% (255.5 Ft/Mile 48.4 m/Km)

0.020 (103.1 Ft/Mile 19.5 m/Km)

0.039 (206.1 Ft/Mile 39.0 m/Km)
Turn Around 26

6.255% (330.3 Ft/Mile 62.6 m/Km)

0.024 (124.7 Ft/Mile 23.6 m/Km)

0.047 (249.4 Ft/Mile 47.2 m/Km)
Power Line 43

4.038% (213.2 Ft/Mile 40.4 m/Km)

0.026 (136.7 Ft/Mile 25.9 m/Km)

0.052 (273.3 Ft/Mile 51.8 m/Km)
Mixed 402

3.498% (184.7 Ft/Mile 35.0 m/Km)

0.032 (166.5 Ft/Mile 31.5 m/Km)

0.063 (332.9 Ft/Mile 63.1 m/Km)

Garmin 910XT with Footpod

Normal 514

1.429% (75.4 Ft/Mile 14.3 m/Km)

0.030 (155.8 Ft/Mile 29.5 m/Km)

0.059 (311.5 Ft/Mile 59.0 m/Km)
Under Bridge 43

1.631% (86.1 Ft/Mile 16.3 m/Km)

0.084 (444.5 Ft/Mile 84.2 m/Km)

0.168 (889.0 Ft/Mile 168.4 m/Km)
Post Bridge 40

4.339% (229.1 Ft/Mile 43.4 m/Km)

0.021 (113.2 Ft/Mile 21.4 m/Km)

0.043 (226.4 Ft/Mile 42.9 m/Km)
Turn Around 43

5.117% (270.2 Ft/Mile 51.2 m/Km)

0.026 (135.3 Ft/Mile 25.6 m/Km)

0.051 (270.7 Ft/Mile 51.3 m/Km)
Power Line 90

2.101% (110.9 Ft/Mile 21.0 m/Km)

0.024 (127.2 Ft/Mile 24.1 m/Km)

0.048 (254.4 Ft/Mile 48.2 m/Km)
Mixed 730

1.900% (100.3 Ft/Mile 19.0 m/Km)

0.036 (188.6 Ft/Mile 35.7 m/Km)

0.071 (377.2 Ft/Mile 71.4 m/Km)

3.3 Interpretation and Conclusions

What do these statistics mean? This is my interpretation:

  • Under normal conditions the GPS accuracy is quite good for most devices.
  • The 620 is noticeably poorer than the other devices and the preliminary results of the Garmin 10 are even worse. I found the accuracy of the 10/620 in general usage to be rather grim, and I did some testing pairing them up with the 610 or the 310XT. In all cases the 10/620 would have poor accuracy compared with the 610 or 310XT on the same run. On one run, the 620 lost over a mile over a 28 mile distance.
  • The Garmin 610 and Garmin 10 results are preliminary, but I've included them to see if the problems with the Garmin 620 are inherent in a smaller device. The reasonable accuracy I've seen with the Garmin 610 indicates that a small device can have good accuracy.
  • Not surprisingly, accuracy drops going under the bridge.
  • More interestingly the trueness just after the bridge is even lower, suggesting that the GPS watches are struggling to reacquire the satellites.
  • The turnarounds are even less accurate than going under a bridge, but Power Lines do not seem to impact accuracy noticeably.
  • The Footpod improves the accuracy of the 310XT.
    • Note that I'm intentionally using an uncalibrated Footpod (factor = 1.000) to gather data for a comparison of Foodpod and GPS.
  • The Garmin 205 does remarkably well, especially for an older device.

4 Recommendations

Here are some recommendations for GPS watches.

  • GPS watches are accurate enough for casual running.
  • The better devices are accurate enough for most runners if their limitations are understood.
  • None of the devices were accurate enough for a runner to trust the display of current pace for training or race pacing.
    • For interval training, use a track or measure out the distance using some other mechanism.
    • For general training or for races, use a Footpod.
  • Adding a Footpod to the Garmin 310XT improves its GPS accuracy.
  • It takes time for the GPS watches to acquire the satellites. Some watches tended to say they are ready to go before they have an optimal lock. Therefore, to improve accuracy try to give them a little more time. Note that some newer GPS watches such as the Garmin 620 have the ability to be preloaded with the satellite positions, reducing this startup time and start up in accuracy dramatically.

5 Even GPS Watches have Bad Days

While it's tempting to take the various GPS watches on a single run and simply compare the totals, this is a flawed approach. Evaluating the devices GPS accuracy on the basis of a single sample does not tell you much. It's a bit like evaluating an athlete's ability on the basis of one event; everyone has good days and bad days, and that applies to GPS watches as well. To illustrate this, the images below are from two runs, recorded on 9/20 and 9/22. In each run I recorded data on both the 310 and 910 watches, hitting the lap button on both at as close to the same time as is humanly possible. On 9/20 the 910XT was far more accurate than the 310XT, but on 9/22 the situation is reversed. If you were to have evaluated the two watches on the basis of a single run, you would conclude that one is much better than the other. But which device would win would depend on the particular day. This is why I've accumulated a lot of data to do a statistical analysis to work out which is really better.

The Garmin 310XT having a bad day. You can see on the upper half of the course where it got a little confused and off track.
The Garmin 910XT on the same run having no problems, and only the standard, expected level of inaccuracy.
Two days later and it's the turn of the Garmin 310XT to have a good day.
Again, this track is recorded on the same run as the image to the left. The Garmin 910XT gets a little confused at the start, and then again around lap 27.

6 Some Devices Are Better Than Others

Below is a section of two runs showing the same section of the course, both taken at the same time, one from the Garmin 310XT and the other from the Garmin 620. These give a good indication of the accuracy problems I've seen with the Garmin 620.

You can see the GPS tracks (thin red line) are close together and the lap markers (yellow diamonds) are clustered nicely. The blue dots on the GPS tracks are the actual GPS recordings.
By contrast, the 620 has much wider GPS tracks and dispersed lap markers.

7 GPS Short and long measurements

As you can see from the images below, the GPS track tends to take shortcuts around bends, reducing the length of the measured track.

The GPS tracks in red with blue dots representing each GPS point recorded. The arrow points to the actual path taken.

Often GPS measurements of races, especially marathons record a longer distance than the race. This is partly because the USATF technique for measuring the distance takes a path that is no more than 12 inches away from the tangent (corner), and few runners are able to run that close. In a large marathon you can be forced to take a line that is a long way from the tangent. The other factor is that on a straight line, the GPS error tends to give a slightly longer measurement.

Here you can see the GPS line is not following the straight road, giving a longer reading on the Thunder Road Marathon. Notice that the GPS is also cutting the corner at the top (we didn't run through the building).

8 Density Plots

Below are the density plots of the 'normal' condition for each device.

Density plots of the GPS accuracy.

9 Next Steps

This is an initial analysis of the data I have, and there are a number of further evaluations to do.

  • Gather data from some other GPS Running Watches.
  • Add in more graphs of the distribution of accuracy, and possibly a Q-Q plot (which shows reasonably normal distribution).
  • Check how GPS accuracy changes over the course of a run, as I've seen a distinct tendency for the watches to say they are good to go when they don't really have an optimal lock on the satellites.
  • Look at how accurate the GPS watches are for measuring elevation, and compare with barometric data.
  • Check for any correlation between accuracy and running pace.
  • Look for any correlation between accuracy and the use of Heart Rate Monitor. The radio signal from the heart rate monitor could interfere with accuracy.
  • Write up general GPS accuracy.
  • List the sources of potential error.
  • Evaluate the accuracy of the Footpod alone, both trueness and precision.
Retrieved from "http://fellrnr.com/mediawiki/index.php?title=GPS_Accuracy&oldid=4999"