8,105
edits
Changes
From Fellrnr.com, Running tips
→Trail Running and GPS
{{DISPLAYTITLE:GPS Accuracy of Garmin, Polar, and other Running Watches}}<div style="float:right;">__TOC__</div>I evaluated the real -world accuracy of GPS watches while running over 212,500 000 miles/419,000Km and recording over 1050,000 data points as part of my evaluation of the [[Best Running Watch]]es. Under good conditions most of the watches are remarkably good, but when things get a little tough the differences become more apparent. The table below However, '''none of the watches have GPS accuracy that is good enough to be used for displaying your current pace'''. As a vastly simplified summary of result, I've added the test resultsfor various [[Footpod]]s as they can be far more accurate than GPS, rating the best device as but more importantly they tend to have far less moment-to-moment variation so they can give a 10 then comparing the othersfar better display of your current pace. (Note that '10' does my accuracy tests focus on the ability to measure distance, not indicate perfectthe moment in time position, just though the best I've measured so fartwo are obviously related.){| class="wikitable"! Device! Trueness! Precision! Overall|-| {{Garmin 205}}| 10.0| 9.0| 10.0|-| {{Garmin 910XT}} with [[Footpod]]| 7File:GPS Accuracy.4png| 8.8none| 9.0thumb|-800px| {{Garmin 310XT}} with [[Footpod]]| 5An infographic of the accuracy of the GPS running watches.8| 8The top right corner represents the most accurate watches.3| 7(This graphic uses ISO 5725 terminology.9|-| {{Garmin 610}} (preliminary)| 4.0| 10.0| 7.4|-| iPhone 4s| 4.4| 8.7| 7.2|-| {{Garmin 310XT}} no [[Footpod]]| 4The table below is a simplified summary of the results, where a '10' would be a perfect device.6| 7(For an explanation of the ISO 5725 terms 'trueness', 'precision' and 'accuracy', see below.2| 6.6|-)| {{Polar RC3 :GPS}}| 5.7| 6.2| 6.5|Accuracy-| {{Garmin 620summary}}| 1.9| 7.6| 4.3|-| {{Garmin The values used are simply 10}} minus the value for trueness (average) and precision (preliminarystandard deviation from true)| 1.8| 5.1| 3.5|}The results overall is the combination of trueness and precision. Repeatability is how consistent a watch is in providing the same value for the Garmin 10 and 610 are preliminary as I have same course segment. '''Important''': Manufacturers do not gathered an equivalent volume typically release the type of GPS chipset used, so the information in this table is based the best available data to the other devices, but it should be treated with caution.
=Methodology=
''Main article: [[GPS Testing Methodology]]'' Simply taking a GPS watch on a single run does not provide sufficient data to reasonably evaluate its accuracy. So to To gather the data for this test I marked my usual running ran the same route at repeatedly, recording laps every quarter-mile intervals, using a [http://www.jonescounter.com/ Jones counter] to provide an accurate measure of distance. The Jones counter is the device used to certify courses, and I followed the [http://www.usatf.org/Products-/-Services/Course-Certifications/USATF-Certified-Courses/Certify-Your-Course.aspx USATF course certification process]. The course I run along is a little challenging for GPS, with lots of twists, tree cover, power lines, turn arounds and one bridge that I go goes under. The a bridge carries a 4 lane road, so it's wide enough to cause the watches to briefly loose GPS signal. However, I believe that it's reasonably representative of real-world conditions, and probably less challenging than running in the city with skyscrapers. At both ends of the course there is a turnaround, and I set the mark an eighth of a mile from the end. That way I can evaluate how well the watches handle an about turn. [[File:Course Overview.jpg|none|thumb|500px|This is the course I use to evaluate the accuracy of [[Best Running Watch| GPS Running Watches]].]]To prevent startup problems each device was turned on, satellites acquired, and then left for around 5 minutes before the run starts. This is to counter the problem of a device claiming to have acquired the satellites when it only has a minimal lock. The watches are worn on the left wrist or held in the left hand in roughly the same orientation as it would be on the left wrist. (The iPhone was also on the left wrist.) =Accuracy, Trueness and Precision(plus Repeatability)=For this evaluation I'll use the ISO 5725 definition of [http://en.wikipedia.org/wiki/Accuracy_and_precision Accuracy as the combination of trueness and precision]:.
{| class="wikitable"
|- valign="top"
|[[File:High precision Low accuracy.svg|none|thumb|x300pxx200px|This is an example of high precision, as all the hits are tightly clustered. However, the trueness is poor as all the hits are off center, so accuracy is low.]]|[[File:High accuracy Low precision.svg|none|thumb|x300pxx200px|This shows good trueness, as all the hits are around the center. On average they are on target, but there is poor precision, as the hits are scattered.]]
|}
We can look at trueness by measuring the average lap length and precision by measuring the standard deviation. I use the traditional approach to standard deviation (variation from mean) as well as a modified approach that uses variation from the true value. (It is more common in many fields to use "accuracy" to mean closeness to true value and "validity" to mean the combination of accuracy and precision. However, I feel that the meanings used by ISO 5725 are closer to the common usage. If a company sold 'accurate' 12 inch pipes and shipped half of them as 6 inches and half as 18 inches, they would meet the traditional definition of accuracy, but few people would be happy with the product. ) In addition, I calculate a value for "repeatability", which is a measure of how likely a watch is to give the same distance measurement for a specific course. I calculate the standard deviation for each segment of the course, and then take the average. A high repeatability score can mask poor accuracy and can convince users they have a good device.
=Accuracy=
The table below shows summary data for each device. The count field is how many measurements I've divided the laps into those have for that go under the bridgecombination of condition and device, those that are immediately after with each measurement being a lap that goes under the bridgequarter mile distance. I generally aim for over 1, those that go next 000 data points to or under even out the power lineseffects of weather, those that turn around satellite position and then other factors. The Trueness is the absolute of the mean, though nearly all watches tend to read short. The standard deviation is provided based on the variance from the rest (normal)mean and the variance from the known true value. IThe average pace error is shown to give a sense of how much error you've also included re likely to see in the display of current pace. This is an average error not a worst case. The data shown below is a 'mixed' category that reflects summary the accuracy based on all the sections. ==Accuracy by Condition then Device=={| class="wikitable"! Condition! Device! Count! Trueness (Average Distance Error)! Standard Deviation! 95% within|-| rowspan="10"|Normal| {{Garmin 10}}| 274| style="background-color: #FCB079;"|7.740% (408.7 Ft/Mile 77.4 m/Km)| style="background-color: #FB9B75;"|0.066 (350.7 Ft/Mile 66.4 m/Km)| 0.133 (701.4 Ft/Mile 132.8 m/Km)|-| {{Garmin 205}}| 779| style="background-color: #63BE7B;"|1.381% (72.9 Ft/Mile 13.8 m/Km)| style="background-color: #C9DB80;"|0.031 (165.6 Ft/Mile 31.4 m/Km)| 0.063 (331.2 Ft/Mile 62.7 m/Km)|-| {{Garmin 620}}| 944| style="background-color: #FDBC7B;"|6.913% (365.0 Ft/Mile 69.1 m/Km)| style="background-color: #FFE383;"|0.040 (213.5 Ft/Mile 40.4 m/Km)| 0.081 (427.0 Ft/Mile 80.9 m/Km)|-| iPhone 4s| 668| style="background-color: #E5E382;"|3.046% (160.8 Ft/Mile 30.5 m/Km)| style="background-color: #EBE582;"|0.035 (186.0 Ft/Mile 35.2 m/Km)| 0.070 (372.0 Ft/Mile 70.5 m/Km)|-| {{Polar RC3 GPS}}| 983| style="background-color: #6DC07B;"|1.511% (79.8 Ft/Mile 15.1 m/Km)| style="background-color: #FFE183;"|0.041 (217.4 Ft/Mile 41.2 m/Km)| 0.082 (434.7 Ft/Mile 82.3 m/Km)|-| {{Garmin 310XT}} with If you'd like more detailed information, I've split off the [[FootpodDetailed Statistics for GPS Running Watches]]| 2786| style="background-color: #94CC7D;"|2.013% (106.3 Ft/Mile 20.1 m/Km)| style="background-color: #EDE582;"|0.035 (187.1 Ft/Mile 35.4 m/Km)| 0.071 (374.3 Ft/Mile 70.9 m/Km)|-| {{Garmin 310XT}} no [[Footpod]]| 1368| style="background-color: #CADB80;"|2.708% (143.0 Ft/Mile 27.1 m/Km)| style="background-color: #FFE383;"|0.041 (214.8 Ft/Mile 40.7 m/Km)| 0.081 (429.5 Ft/Mile 81.3 m/Km)|-| {{Garmin 610}}| 289| style="background-color: #EBE582;"|3.124% (164.9 Ft/Mile 31.2 m/Km)| style="background-color: #CADB80;"|0.031 (166.0 Ft/Mile 31.4 m/Km)| 0.063 (331.9 Ft/Mile 62.9 m/Km)|-| {{Garmin 910XT}} with [[Footpod]]| 514| style="background-color: #66BF7B;"|1.429% (75.4 Ft/Mile 14.3 m/Km)| style="background-color: #B9D780;"|0.030 (155.8 Ft/Mile 29.5 m/Km)| 0.059 (311.5 Ft/Mile 59.0 m/Km)|-| All| 8605| style="background-color: #D3DE81;"|2.811% (148.4 Ft/Mile 28.1 m/Km)| style="background-color: #FFDE82;"|0.042 (223.2 Ft/Mile 42.3 m/Km)| 0.085 (446.3 Ft/Mile 84for the results under different conditions.5 m/Km)|-| rowspan="10"|Under Bridge| {{Garmin 10}}| 24| style="background-color: #FFEB84;"|3.387% (178.8 Ft/Mile 33.9 m/Km)| style="background-color: #D8DF81;"|0.033 (174.3 Ft/Mile 33.0 m/Km)| 0.066 (348.6 Ft/Mile 66.0 m/Km)|-| {{Garmin 205}}| 66| style="background-color: #63BE7B;"|-1.379% (72.8 Ft/Mile 13.8 m/Km)| style="background-color: #FA8671;"|0.074 (390.2 Ft/Mile 73.9 m/Km)| 0.148 (780.4 Ft/Mile 147.8 m/Km)|-| {{Garmin 620}}| 61| style="background-color: #FED881;"|4.818% (254.4 Ft/Mile 48.2 m/Km)| style="background-color: #FFE082;"|0.042 (219.7 Ft/Mile 41.6 m/Km)| 0.083 (439.4 Ft/Mile 83.2 m/Km)|-| iPhone 4s| 59| style="background-color: #64BE7B;"|1.402% (74.0 Ft/Mile 14.0 m/Km)| style="background-color: #DEE182;"|0.034 (178.0 Ft/Mile 33.7 m/Km)| 0.067 (356.1 Ft/Mile 67.4 m/Km)|-| {{Polar RC3 GPS}}| 86| style="backgroundAccuracy-color: #FECD7F;"|5.612% (296.3 Ft/Mile 56.1 m/Km)| style="background-color: #F97A6F;"|0.078 (412.7 Ft/Mile 78.2 m/Km)| 0.156 (825.3 Ft/Mile 156.3 m/Km)|-| {{Garmin 310XTstatistics}} with [[Footpod]]| 222| style=The "background-color: #F0E683;"|3.193% (168.6 Ft/Mile 31.9 m/Km)| style="background-color: #FECB7E;"|0.049 (260.0 Ft/Mile 49.2 m/Km)| 0.098 (520.0 Ft/Mile 98.5 m/Km)|-| {{Garmin 310XT}} no [[Footpod]]| 111| style="background-color: #66BE7B;"|1.420% (75.0 Ft/Mile 14.2 m/Km)| style="background-color: #FB9C75;"|0.066 (348.9 Ft/Mile 66.1 m/Km)| 0.132 Accuracy (697.9 Ft/Mile 132.2 m/KmCombined)|-| {{Garmin 610}}| 22| style="background-color: #CFDD81;"|2.761% (145.8 Ft/Mile 27.6 m/Km)| style="background-color: #FFE984;"|0.038 (201.8 Ft/Mile 38.2 m/Km)| 0.076 (403.6 Ft/Mile 76.4 m/Km)|-| {{Garmin 910XT}} column has an indication of statistical significance compared with [[Footpod]]| 43| style="background-color: #76C37C;"|1the most accurate entry.631% (86.1 Ft/Mile 16.3 m/Km)| style="background-colorThe key to this indication is: #F8696B;"|† p<0.084 (444.5 Ft/Mile 84.2 m/Km)| 05, * p< 0.168 (889.01, ** p< 0 Ft/Mile 168.4 m/Km)|-| All| 694| style="background-color: #C7DA80;"|2.661% (140.5 Ft/Mile 26.6 m/Km)| style="background-color: #FCA677;"|001, *** p< 0.062 (329.0001, **** p< 0 Ft/Mile 62.3 m/Km)| 00001, ***** p< 0.125 (658.0 Ft/Mile 124.6 m/Km)000001|-| rowspan="10"|Post Bridge| {{Garmin 10}}| 24| style="background-color: #F97C6F;"|11.616% (613.3 Ft/Mile 116.2 m/Km)| styleProgress of newer watches="background-color: #F9E983;"|0.037 (194.4 Ft/Mile 36.8 m/Km)| 0.074 (388.8 Ft/Mile 73.6 m/Km)|-| {{Garmin 205}}| 66| style="background-color: #9FCF7E;"|2.157% (113.9 Ft/Mile 21.6 m/Km)| style="background-color: #87C87D;"|0.024 (125.5 Ft/Mile 23.8 m/Km)| 0.048 (250.9 Ft/Mile 47.5 m/Km)|-| {{Garmin 620}}| 61| style="background-color: #FBA176;"|8.900% (469.9 Ft/Mile 89.0 m/Km)| style="background-color: #FECC7E;"|0.049 (257.6 Ft/Mile 48.8 m/Km)| 0.098 (515.2 Ft/Mile 97.6 m/Km)|-| iPhone 4s| 54| style="background-color: #FFDD82;"|4.422% (233.5 Ft/Mile 44.2 m/Km)| style="background-color: #BCD780;"|0.030 (157.4 Ft/Mile 29.8 m/Km)| 0.060 (314.9 Ft/Mile 59.6 m/Km)|-| {{Polar RC3 I expected GPS}}| 84| style="background-color: #FDB57A;"|7.399% (390.7 Ft/Mile 74.0 m/Km)| style="background-color: #FEC97E;"|0.050 (263.2 Ft/Mile 49.8 m/Km)| 0.100 (526.4 Ft/Mile 99.7 m/Km)|-| {{Garmin 310XT}} watches to improve with [[Footpod]]| 188| style="background-color: #FFE283;"|4time, but the opposite appears to be happening.095% (216.2 Ft/Mile 41.0 m/Km)| style="background-color: #FEEA83;"|0.037 (197.4 Ft/Mile 37.4 m/Km)| 0.075 (394.8 Ft/Mile 74.8 m/Km)|-| {{With the Garmin 310XT}} no [[Footpod]]| 106| style="background-color: #FED580;"|5devices especially, you can see that the older watches generally do far better than the newer ones.053% (266.8 Ft/Mile 50.5 m/Km)| style="background-color: #EDE582;"|0.035 (187.0 Ft/Mile 35.4 m/Km)| 0.071 (374.1 Ft/Mile 70.8 m/Km)|-| {{Garmin 610}}| 22| style="background-color: #FED881;"|4.839% (255.5 Ft/Mile 48.4 m/Km)| style="background-color: #63BE7B;"|0.020 (103.1 Ft/Mile 19.5 m/Km)| 0.039 (206.1 Ft/Mile 39.0 m/Km)|-| {{Garmin 910XT}} with [[Footpod]]| 40| style="background-color: #FFDE82;"|4.339% (229.1 Ft/Mile 43.4 m/Km)| style="background-color: #73C27B;"|0.021 (113.2 Ft/Mile 21.4 m/Km)| 0.043 (226.4 Ft/Mile 42.9 m/Km)|-| All| 645| style="background-color: #FED280;"|5.287% (279.1 Ft/Mile 52.9 m/Km)| style="background-color: #FFDB81;"|0.043 (228.5 Ft/Mile 43.3 m/Km)| 0.087 (456.9 Ft/Mile 86.5 m/Km)|-| rowspan="10"|Turn Around| {{Garmin 10}}| 23| style="background-color: #F8696B;"|12.971% (684.9 Ft/Mile 129.7 m/Km)| style="background-color: #FFEB84;"|0.038 (198.2 Ft/Mile 37.5 m/Km)| 0.075 (396.5 Ft/Mile 75.1 m/Km)|-| {{Garmin 205}}| 72| style="background-color: #FFE884;"|3.645% (192.5 Ft/Mile 36.5 m/Km)| style="background-color: #9ACE7E;"|0.026 (136.9 Ft/Mile 25.9 m/Km)| 0.052 (273.9 Ft/Mile 51.9 m/Km)|-| {{Garmin 620}}| 80| style="background-color: #FA8070;"|11.302% (596.7 Ft/Mile 113.0 m/Km)| style="background-color: #FFDB81;"|0.043 (228.5 Ft/Mile 43.3 m/Km)| 0.087 (457.0 Ft/Mile 86.6 m/Km)|-| iPhone 4s| 62| style="background-color: #FDBF7C;"|6.623% (349.7 Ft/Mile 66.2 m/Km)| style="background-color: #FED881;"|0.045 (235.6 Ft/Mile 44.6 m/Km)| 0.089 (471.3 Ft/Mile 89.3 m/Km)|-| {{Polar RC3 I suspect this is due to compromises to get better battery life and smaller packaging and the cost of GPS}}| 92| style="background-color: #FDC27C;"|6.471% (341accuracy.7 Ft/Mile 64.7 m/Km)| style="background-color: #FA7D6F;"|0.077 (407.3 Ft/Mile 77.1 m/Km)| 0.154 (814.6 Ft/Mile 154.3 m/Km)|-| {{Garmin 310XT}} with [[Footpod]]| 205| style="background-color: #FECE7F;"|5.573% (294.2 Ft/Mile 55.7 m/Km)| styleSmartphone Accuracy="background-color: #FFE082;"|0.042 (219.6 Ft/Mile 41.6 m/Km)| 0.083 (439.2 Ft/Mile 83.2 m/Km)|-| {{Garmin 310XT}} no [[Footpod]]| 123| style="background-color: #FEC87E;"|6.017% (317.7 Ft/Mile 60.2 m/Km)| style="background-color: #FDBD7C;"|0.054 (285.9 Ft/Mile 54.1 m/Km)| 0.108 (571.8 Ft/Mile 108.3 m/Km)|-| {{Garmin 610}}| 26| style="background-color: #FDC47D;"|6.255% (330.3 Ft/Mile 62.6 m/Km)| style="background-color: #86C87D;"|0.024 (124.7 Ft/Mile 23.6 m/Km)| 0.047 (249.4 Ft/Mile 47.2 m/Km)|-| {{Garmin 910XT}} There are various things you will need to do in order to get the level of accuracy I found with [[Footpod]]| 43| style="background-color: #FED480;"|5Smartphones.117% (270.2 Ft/Mile 51.2 m/Km)| style="background-color: #98CD7E;"|0.026 (135.3 Ft/Mile 25.6 m/Km)| 0.051 (270.7 Ft/Mile 51.3 m/Km)|-| All| 726| style="background-color: #FDC17C;"|6.523% (344.4 Ft/Mile 65.2 m/Km)| style="background-color: #FDC17C;"|0.053 (279.1 Ft/Mile 52.9 m/Km)| 0.106 (558.1 Ft/Mile 105.7 m/Km)|-| rowspan="10"|Power Line| {{Garmin 10}}| 46| style="background-color: #FCAF79;"|7.841% (414.0 Ft/Mile 78.4 m/Km)| style="background-color: #EFE683;"|0.036 (188.5 Ft/Mile 35.7 m/Km)| 0.071 (377.0 Ft/Mile 71.4 m/Km)|-| {{Garmin 205}}| 127| style="background-color: #64BE7B;"|1.395% (73.7 Ft/Mile 14.0 m/Km)| style="background-color: #83C77C;"|0.023 (122.5 Ft/Mile 23.2 m/Km)| 0.046 (245.1 Ft/Mile 46.4 m/Km)|-| {{Garmin 620}}| 162| style="background-color: #FCAF79;"|7.860% (415.0 Ft/Mile 78.6 m/Km)| style="background-color: #AED37F;"|0.028 (148.7 Ft/Mile 28.2 m/Km)| 0.056 (297.5 Ft/Mile 56.3 m/Km)|-| iPhone 4s| 110| style="background-color: #BFD880;"|2.560% (135.2 Ft/Mile 25.6 m/Km)| style="background-color: #B4D57F;"|0.029 (152.7 Ft/Mile 28.9 m/Km)| 0.058 (305.3 Ft/Mile 57.8 m/Km)|-| {{Polar RC3 GPS}}| 167| style="background-color: #95CC7D;"|2.030% (107.2 Ft/Mile 20.3 m/Km)| style="background-color: #F0E683;"|0.036 (188.8 Ft/Mile 35.8 m/Km)| 0.072 (377.7 Ft/Mile 71.5 m/Km)|-| {{Garmin 310XT}} with See [[Footpod]]| 515| style="background-color: Running With A Smartphone#BFD880;"|2.566% (135.5 Ft/Mile 25.7 m/Km)| style="background-color: #E6E482;"|0.035 (183.2 Ft/Mile 34.7 m/Km)| 0.069 (366.5 Ft/Mile 69.4 m/Km)|-| {{Garmin 310XT}} no [[Footpod]]| 229| style="background-color: #FDEA83;"|3.360% (177.4 Ft/Mile 33.6 m/Km)| style="background-color: #EEE683;"|0.036 (187.7 Ft/Mile 35.5 m/Km)| 0.071 (375.3 Ft/Mile 71.1 m/Km)|-| {{Garmin 610}}| 43| style="background-color: #FFE383;"|4.038% (213.2 Ft/Mile 40.4 m/Km)| style="background-color: #9ACD7E;"|0.026 (136.7 Ft/Mile 25.9 m/Km)| 0.052 (273.3 Ft/Mile 51.8 m/Km)|-| {{Garmin 910XT}} with [[Footpod]]| 90| style="background-color: #9BCE7E;"|2.101% (110.9 Ft/Mile 21.0 m/Km)| style="background-color: #8AC97D;"|0.024 (127.2 Ft/Mile 24.1 m/Km)| 0.048 (254.4 Ft/Mile 48.2 m/Km)|-| All| 1489| style="background-color: #F7E883;"|3.281% (173.2 Ft/Mile 32.8 m/Km)| style="background-color: #FFEB84;"|0.038 (198.5 Ft/Mile 37.6 m/Km)| 0.075 (397.1 Ft/Mile 75.2 m/Km)|-| rowspan="10"|Mixed| {{Garmin 10}}| 391| style="background-color: #FCAC78;"|8.030% (424.0 Ft/Mile 80.3 m/Km)| style="background-color: #FCA777;"|0.062 (327.0 Ft/Mile 61.9 m/Km)| 0.124 (654.0 Ft/Mile 123.9 m/Km)|-| {{Garmin 205}}| 1110| style="background-color: #65BE7B;"|1.411% (74.5 Ft/Mile 14.1 m/Km)| style="background-color: #EAE582;"|0.035 (185.6 Ft/Mile 35.2 m/Km)| 0.070 (371.2 Ft/Mile 70.3 m/Km)|-| {{Garmin 620}}| 1308| style="background-color: #FDB67A;"|7.293% (385.1 Ft/Mile 72.9 m/Km)| style="background-color: #FFE082;"|0.042 (220.1 Ft/Mile 41.7 m/Km)| 0.083 (440.2 Ft/Mile 83.4 m/Km)|-| iPhone 4s| 953| style="background-color: #F1E783;"|3.199% (168.9 Ft/Mile 32.0 m/Km)| style="background-color: #F6E883;"|0.036 (192.4 Ft/Mile 36.4 m/Km)| 0.073 (384.8 Ft/Mile 72.9 m/Km)|-| {{Polar RC3 Optimizing GPS}}| 1412| style="background-color: #BAD780;"|2.495% (131.8 Ft/Mile 25.0 m/Km)| style="background-color: #FDC57D;"|0.051 (269.9 Ft/Mile 51.1 m/Km)| 0.102 (539.9 Ft/Mile 102.3 m/Km)|-| {{Garmin 310XT}} with [[Footpod]]| 3916| style="background-color: #B5D57F;"|2.439% (128.8 Ft/Mile 24.4 m/Km)| style="background-color: #FFEB84;"|0.038 (199.5 Ft/Mile 37.8 m/Km)| 0.076 (399.1 Ft/Mile 75.6 m/Km)|-| {{Garmin 310XT}} no [[Footpod]]| 1937| style="background-color: #E5E382;"|3.050% (161.0 Ft/Mile 30.5 m/Km)| style="background-color: #FFDA81;"|0.044 (231.7 Ft/Mile 43.9 m/Km)| 0.088 (463.3 Ft/Mile 87.8 m/Km)|-| {{Garmin 610}}| 402| style="background-color: #FFEA84;"|3.498% (184.7 Ft/Mile 35.0 m/Km)| style="background-color: #CBDC81;"|0.032 (166.5 Ft/Mile 31.5 m/Km)| 0.063 (332.9 Ft/Mile 63.1 m/Km)|-| {{Garmin 910XT}} with [[Footpod]]| 730| style="background-color: #8BC97D;"|1.900% (100.3 Ft/Mile 19.0 m/Km)| style="background-color: #EFE683;"|0.036 (188.6 Ft/Mile 35.7 m/Km)| 0.071 (377.2 Ft/Mile 71.4 m/Km)|-| All| 12159| style="background-color: #F2E783;"|3.213% (169.7 Ft/Mile 32.1 m/Km)| style="background-color: #FED680;"|0.045 (237.7 Ft/Mile 45.0 m/Km)| 0.090 (475.4 Ft/Mile 90.0 m/Km)|}==Accuracy by Device then Condition=={| class="wikitable"! Device! Condition! Count! Trueness (Average Distance Error)! Standard Deviation! 95% within|-| rowspan="6"|{{Garmin 10}}| Normal| 274| style="background-color: #FCB179;"|7.740% (408.7 Ft/Mile 77.4 m/Km)| style="background-color: #FB9A75;"|0.066 (350.7 Ft/Mile 66.4 m/Km)| 0.133 (701.4 Ft/Mile 132.8 m/Km)|-| Under Bridge| 24| style="background-color: #FAE983;"|3.387% (178.8 Ft/Mile 33.9 m/Km)| style="background-color: #E1E282;"|0.033 (174.3 Ft/Mile 33.0 m/Km)| 0.066 (348.6 Ft/Mile 66.0 m/Km)|-| Post Bridge| 24| style="background-color: #F97C6F;"|11.616% (613.3 Ft/Mile 116.2 m/Km)| style="background-color: #FFEA84;"|0.037 (194.4 Ft/Mile 36.8 m/Km)| 0.074 (388.8 Ft/Mile 73.6 m/Km)|-| Turn Around| 23| style="background-color: #F8696B;"|12.971% (684.9 Ft/Mile 129.7 m/Km)| style="background-color: #FFE884;"|0.038 (198.2 Ft/Mile 37.5 m/Km)| 0.075 (396.5 Ft/Mile 75.1 m/Km)|-| Power Line| 46| style="background-color: #FCAF79;"|7.841% (414.0 Ft/Mile 78.4 m/Km)| style="background-color: #FBE983;"|0.036 (188.5 Ft/Mile 35.7 m/Km)| 0.071 (377.0 Ft/Mile 71.4 m/Km)|-| Mixed| 391| style="background-color: #FCAD78;"|8.030% (424.0 Ft/Mile 80.3 m/Km)| style="background-color: #FCA677;"|0.062 (327.0 Ft/Mile 61.9 m/Km)| 0.124 (654.0 Ft/Mile 123.9 m/Km)|-| rowspan="6"|{{Garmin 205}}| Normal| 779| style="background-color: #63BE7B;"|1.381% (72.9 Ft/Mile 13.8 m/Km)| style="background-color: #D2DE81;"|0.031 (165.6 Ft/Mile 31.4 m/Km)| 0.063 (331.2 Ft/Mile 62.7 m/Km)|-| Under Bridge| 66| style="background-color: #63BE7B;"|-1.379% (72.8 Ft/Mile 13.8 m/Km)| style="background-color: #FA8571;"|0.074 (390.2 Ft/Mile 73.9 m/Km)| 0.148 (780.4 Ft/Mile 147.8 m/Km)|-| Post Bridge| 66| style="background-color: #9DCE7E;"|2.157% (113.9 Ft/Mile 21.6 m/Km)| style="background-color: #8AC97D;"|0.024 (125.5 Ft/Mile 23.8 m/Km)| 0.048 (250.9 Ft/Mile 47.5 m/Km)|-| Turn Around| 72| style="background-color: #FFE984;"|3.645% (192.5 Ft/Mile 36.5 m/Km)| style="background-color: #9FCF7E;"|0.026 (136.9 Ft/Mile 25.9 m/Km)| 0.052 (273.9 Ft/Mile 51.9 m/Km)|-| Power Line| 127| style="background-color: #64BE7B;"|1.395% (73.7 Ft/Mile 14.0 m/Km)| style="background-color: #85C87D;"|0.023 (122.5 Ft/Mile 23.2 m/Km)| 0.046 (245.1 Ft/Mile 46.4 m/Km)|-| Mixed| 1110| style="background-color: #65BE7B;"|1.411% (74.5 Ft/Mile 14.1 m/Km)| style="background-color: #F6E883;"|0.035 (185.6 Ft/Mile 35.2 m/Km)| 0.070 (371.2 Ft/Mile 70.3 m/Km)|-| rowspan="6"|{{Garmin 620}}| Normal| 944| style="background-color: #FDBC7B;"|6.913% (365.0 Ft/Mile 69.1 m/Km)| style="background-color: #FFE082;"|0.040 (213.5 Ft/Mile 40.4 m/Km)| 0.081 (427.0 Ft/Mile 80.9 m/Km)|-| Under Bridge| 61| style="background-color: #FED981;"|4.818% (254.4 Ft/Mile 48.2 m/Km)| style="background-color: #FFDD82;"|0.042 (219.7 Ft/Mile 41.6 m/Km)| 0.083 (439.4 Ft/Mile 83.2 m/Km)|-| Post Bridge| 61| style="background-color: #FBA176;"|8.900% (469.9 Ft/Mile 89.0 m/Km)| style="background-color: #FEC97E;"|0.049 (257.6 Ft/Mile 48.8 m/Km)| 0.098 (515.2 Ft/Mile 97.6 m/Km)|-| Turn Around| 80| style="background-color: #FA8070;"|11.302% (596.7 Ft/Mile 113.0 m/Km)| style="background-color: #FED881;"|0.043 (228.5 Ft/Mile 43.3 m/Km)| 0.087 (457.0 Ft/Mile 86.6 m/Km)|-| Power Line| 162| style="background-color: #FCAF79;"|7.860% (415.0 Ft/Mile 78.6 m/Km)| style="background-color: #B4D57F;"|0.028 (148.7 Ft/Mile 28.2 m/Km)| 0.056 (297.5 Ft/Mile 56.3 m/Km)|-| Mixed| 1308| style="background-color: #FDB77A;"|7.293% (385.1 Ft/Mile 72.9 m/Km)| style="background-color: #FFDC82;"|0.042 (220.1 Ft/Mile 41.7 m/Km)| 0.083 (440.2 Ft/Mile 83.4 m/Km)|-| rowspan="6"|iPhone 4s| Normal| 668| style="background-color: #E1E282;"|3.046% (160.8 Ft/Mile 30.5 m/Km)| style="background-color: #F6E883;"|0.035 (186.0 Ft/Mile 35.2 m/Km)| 0.070 (372.0 Ft/Mile 70.5 m/Km)|-| Under Bridge| 59| style="background-color: #64BE7B;"|1.402% (74.0 Ft/Mile 14.0 m/Km)| style="background-color: #E8E482;"|0.034 (178.0 Ft/Mile 33.7 m/Km)| 0.067 (356.1 Ft/Mile 67.4 m/Km)|-| Post Bridge| 54| style="background-color: #FFDE82;"|4.422% (233.5 Ft/Mile 44.2 m/Km)| style="background-color: #C3D980;"|0.030 (157.4 Ft/Mile 29.8 m/Km)| 0.060 (314.9 Ft/Mile 59.6 m/Km)|-| Turn Around| 62| style="background-color: #FDC07C;"|6.623% (349.7 Ft/Mile 66.2 m/Km)| style="background-color: #FED480;"|0.045 (235.6 Ft/Mile 44.6 m/Km)| 0.089 (471.3 Ft/Mile 89.3 m/Km)|-| Power Line| 110| style="background-color: #BCD780;"|2.560% (135.2 Ft/Mile 25.6 m/Km)| style="background-color: #BBD780;"|0.029 (152.7 Ft/Mile 28.9 m/Km)| 0.058 (305.3 Ft/Mile 57.8 m/Km)|-| Mixed| 953| style="background-color: #ECE582;"|3.199% (168.9 Ft/Mile 32.0 m/Km)| style="background-color: #FFEB84;"|0.036 (192.4 Ft/Mile 36.4 m/Km)| 0.073 (384.8 Ft/Mile 72.9 m/Km)|-| rowspan="6"|{{Polar RC3 Optimizing Smartphone GPS}}| Normal| 983| style="background-color: #6CC07B;"|1.511% (79.8 Ft/Mile 15.1 m/Km)| style="background-color: #FFDE82;"|0.041 (217.4 Ft/Mile 41.2 m/Km)| 0.082 (434.7 Ft/Mile 82.3 m/Km)|-| Under Bridge| 86| style="background-color: #FECE7F;"|5.612% (296.3 Ft/Mile 56.1 m/Km)| style="background-color: #F97A6F;"|0.078 (412.7 Ft/Mile 78.2 m/Km)| 0.156 (825.3 Ft/Mile 156.3 m/Km)|-| Post Bridge| 84| style="background-color: #FDB67A;"|7.399% (390.7 Ft/Mile 74.0 m/Km)| style="background-color: #FDC67D;"|0.050 (263.2 Ft/Mile 49.8 m/Km)| 0.100 (526.4 Ft/Mile 99.7 m/Km)|-| Turn Around| 92| style="background-color: #FDC27D;"|6.471% (341.7 Ft/Mile 64.7 m/Km)| style="background-color: #FA7D6F;"|0.077 (407.3 Ft/Mile 77.1 m/Km)| 0.154 (814.6 Ft/Mile 154.3 m/Km)|-| Power Line| 167| style="background-color: #94CC7D;"|2.030% (107.2 Ft/Mile 20.3 m/Km)| style="background-color: #FBEA83;"|0.036 (188.8 Ft/Mile 35.8 m/Km)| 0.072 (377.7 Ft/Mile 71.5 m/Km)|-| Mixed| 1412| style="background-color: #B7D67F;"|2.495% (131.8 Ft/Mile 25.0 m/Km)| style="background-color: #FDC37D;"|0.051 (269.9 Ft/Mile 51.1 m/Km)| 0.102 (539.9 Ft/Mile 102.3 m/Km)|-| rowspan="6"|{{Garmin 310XT}} with [[Footpod]]| Normal| 2786| style="background-color: #92CB7D;"|2.013% (106.3 Ft/Mile 20.1 m/Km)| style="background-color: #F8E983;"|0.035 (187.1 Ft/Mile 35.4 m/Km)| 0.071 (374.3 Ft/Mile 70.9 m/Km)|-| Under Bridge| 222| style="background-color: #ECE582;"|3.193% (168.6 Ft/Mile 31.9 m/Km)| style="background-color: #FEC87E;"|0.049 (260.0 Ft/Mile 49.2 m/Km)| 0.098 (520.0 Ft/Mile 98.5 m/Km)|-| Post Bridge| 188| style="background-color: #FFE383;"|4.095% (216.2 Ft/Mile 41.0 m/Km)| style="background-color: #FFE884;"|0.037 (197.4 Ft/Mile 37.4 m/Km)| 0.075 (394.8 Ft/Mile 74.8 m/Km)|-| Turn Around| 205| style="background-color: #FECE7F;"|5.573% (294.2 Ft/Mile 55.7 m/Km)| style="background-color: #FFDD82;"|0.042 (219.6 Ft/Mile 41.6 m/Km)| 0.083 (439.2 Ft/Mile 83.2 m/Km)|-| Power Line| 515| style="background-color: #BCD780;"|2.566% (135.5 Ft/Mile 25.7 m/Km)| style="background-color: #F1E783;"|0.035 (183.2 Ft/Mile 34.7 m/Km)| 0.069 (366.5 Ft/Mile 69.4 m/Km)|-| Mixed| 3916| style="background-color: #B3D57F;"|2.439% (128.8 Ft/Mile 24.4 m/Km)| style="background-color: #FFE784;"|0.038 (199.5 Ft/Mile 37.8 m/Km)| 0.076 (399.1 Ft/Mile 75.6 m/Km)|-| rowspan="6"|{{Garmin 310XT}} no [[Footpod]]| Normal| 1368| style="background-color: #C7DA80;"|2.708% (143.0 Ft/Mile 27.1 m/Km)| style="background-color: #FFDF82;"|0.041 (214.8 Ft/Mile 40.7 m/Km)| 0.081 (429.5 Ft/Mile 81.3 m/Km)|-| Under Bridge| 111| style="background-color: #66BE7B;"|1.420% (75.0 Ft/Mile 14.2 m/Km)| style="background-color: #FB9A75;"|0.066 (348.9 Ft/Mile 66.1 m/Km)| 0.132 (697.9 Ft/Mile 132.2 m/Km)|-| Post Bridge| 106| style="background-color: #FED680;"|5.053% (266.8 Ft/Mile 50.5 m/Km)| style="background-color: #F8E983;"|0.035 (187.0 Ft/Mile 35.4 m/Km)| 0.071 (374.1 Ft/Mile 70.8 m/Km)|-| Turn Around| 123| style="background-color: #FEC87E;"|6.017% (317.7 Ft/Mile 60.2 m/Km)| style="background-color: #FDBB7B;"|0.054 (285.9 Ft/Mile 54.1 m/Km)| 0.108 (571.8 Ft/Mile 108.3 m/Km)|-| Power Line| 229| style="background-color: #F8E983;"|3.360% (177.4 Ft/Mile 33.6 m/Km)| style="background-color: #F9E983;"|0.036 (187.7 Ft/Mile 35.5 m/Km)| 0.071 (375.3 Ft/Mile 71.1 m/Km)|-| Mixed| 1937| style="background-color: #E1E282;"|3.050% (161.0 Ft/Mile 30.5 m/Km)| style="background-color: #FED680;"|0.044 (231.7 Ft/Mile 43.9 m/Km)| 0.088 (463.3 Ft/Mile 87.8 m/Km)|-| rowspan="6"|{{Garmin 610}}| Normal| 289| style="background-color: #E6E482;"|3.124% (164.9 Ft/Mile 31.2 m/Km)| style="background-color: #D3DE81;"|0.031 (166.0 Ft/Mile 31.4 m/Km)| 0.063 (331.9 Ft/Mile 62.9 m/Km)|-| Under Bridge| 22| style="background-color: #CBDC81;"|2.761% (145.8 Ft/Mile 27.6 m/Km)| style="background-color: #FFE683;"|0.038 (201.8 Ft/Mile 38.2 m/Km)| 0.076 (403.6 Ft/Mile 76.4 m/Km)|-| Post Bridge| 22| style="background-color: #FED881;"|4.839% (255.5 Ft/Mile 48.4 m/Km)| style="background-color: #63BE7B;"|0.020 (103.1 Ft/Mile 19.5 m/Km)| 0.039 (206.1 Ft/Mile 39.0 m/Km)|-| Turn Around| 26| style="background-color: #FDC57D;"|6.255% (330.3 Ft/Mile 62.6 m/Km)| style="background-color: #89C97D;"|0.024 (124.7 Ft/Mile 23.6 m/Km)| 0.047 (249.4 Ft/Mile 47.2 m/Km)|-| Power Line| 43| style="background-color: #FFE383;"|4.038% (213.2 Ft/Mile 40.4 m/Km)| style="background-color: #9ECF7E;"|0.026 (136.7 Ft/Mile 25.9 m/Km)| 0.052 (273.3 Ft/Mile 51.8 m/Km)|-| Mixed| 402| style="background-color: #FFEB84;"|3.498% (184.7 Ft/Mile 35.0 m/Km)| style="background-color: #D3DE81;"|0.032 (166.5 Ft/Mile 31.5 m/Km)| 0.063 (332.9 Ft/Mile 63.1 m/Km)|-| rowspan="6"|{{Garmin 910XT}} with [[FootpodAccuracy]]| Normal| 514| style="background-color: #66BF7B;"|1.429% (75for details.4 Ft/Mile 14.3 m/Km)| style="background-color: #C0D980;"|0.030 (155.8 Ft/Mile 29.5 m/Km)| 0.059 (311.5 Ft/Mile 59.0 m/Km)|-| Under Bridge| 43| style="background-color: #76C37C;"|1.631% (86.1 Ft/Mile 16.3 m/Km)| style="background-color: #F8696B;"|0.084 (444.5 Ft/Mile 84.2 m/Km)| 0.168 (889.0 Ft/Mile 168.4 m/Km)|-| Post Bridge| 40| style="background-color: #FFDF82;"|4.339% (229.1 Ft/Mile 43.4 m/Km)| style="background-color: #75C37C;"|0.021 (113.2 Ft/Mile 21.4 m/Km)| 0.043 (226.4 Ft/Mile 42.9 m/Km)|-| Turn Around| 43| style="background-color: #FED580;"|5.117% (270.2 Ft/Mile 51.2 m/Km)| style="background-color: #9CCE7E;"|0.026 (135.3 Ft/Mile 25.6 m/Km)| 0.051 (270.7 Ft/Mile 51.3 m/Km)|-| Power Line| 90| style="background-color: #99CD7E;"|2.101% (110.9 Ft/Mile 21.0 m/Km)| style="background-color: #8DCA7D;"|0.024 (127.2 Ft/Mile 24.1 m/Km)| 0.048 (254.4 Ft/Mile 48.2 m/Km)|-| Mixed| 730| style="background-color: #8AC97D;"|1.900% (100.3 Ft/Mile 19.0 m/Km)| style="background-color: #FBE983;"|0.036 (188.6 Ft/Mile 35.7 m/Km)| 0.071 (377.2 Ft/Mile 71.4 m/Km)|}
==Interpretation and Conclusions==
What do these statistics mean? This is my interpretation:
* Under normal conditions the GPS accuracy is quite good for most devices.
* The 620 accuracy of a calibrated [[Footpod]] is far better than any GPS device. Without calibration the Footpod is noticeably poorer more accurate than any watch currently on the other devices and market with the preliminary results exception of the 310XT/910XT with a Footpod backing up the GPS. * The [[Polar M400]], [[Garmin Fenix 2]], and [[Garmin 10 ]] are even worsenoticeably poorer than the other devices. I found the accuracy of the M400/Fenix2/10/620 in general usage to be rather grim, and I did some testing pairing them up with the 610 or the 310XT. In all cases the Fenix2/10/620 would have poor accuracy compared with the 610 or 310XT on the same run. On one run* The Fenix2 would repeated loose satellite reception, something I've not seen (the M400 has done this once). The statistics do not reflect just how bad the Fenix2 is, as some of the 620 lost over a mile over a 28 mile distancedata is too bad to analyze.* The results of the Garmin 610 and Garmin 10 results are preliminary, but I've included them to see if & 620 indicate the problems with the Garmin 620 10 are not inherent in a smaller device. * The reasonable improvement in GPS accuracy I've seen of the 620 with updated firmware shows just how important the Garmin 610 indicates that software can be. With the earlier firmware the 620 lost over a mile over a small device can have good 20 mile run! * '''The accuracyof all devices is better in a straight line than on curves or twisty routes'''. My course is a tough test for GPS devices with many curves and only a few relatively straight sections. * Not surprisingly, for many devices accuracy drops going under the bridge. However, some devices do great in this section, probably because it's fairly straight.
* More interestingly the trueness just after the bridge is even lower, suggesting that the GPS watches are struggling to reacquire the satellites.
* The turnarounds are even less accurate than going under a bridge, but Power Lines do not seem to impact accuracy noticeably.
* The [[Footpod]] improves the accuracy of the 310XT.
** Note that I'm intentionally using an uncalibrated Footpod (factor = 1.000) to gather data for a comparison of Foodpod and GPS.
* The older Garmin 205 does remarkably well, especially for an older device. =RecommendationsFootpod Accuracy=Here are some recommendations for The accuracy of a Footpod is far higher than GPS watches, as well as more consistent and quicker to react to changes in pace.* GPS watches are accurate enough for casual runningFor any given run, the average pace error from the Footpod is only 7 seconds/mile (at a 9:00 min/mile pace) or 5 seconds/Km (at a 5:30 min/Km pace).* The better devices are accurate enough for most runners if their limitations are understood. * None of the devices were accurate enough for In practical terms, I've found that I always have to use a runner Footpod to trust the display of current pace a marathon or for training or race pacingcritical speedwork. ** For details of how the Footpod calibration was done, see [[Practical Interval Training| interval trainingGPS Testing Methodology]], use a track or measure out the distance using some other mechanism.** For general training or =Trail Running and GPS=Trail running tends to be rather harder for races, use a Footpodwatch to measure accurately. * Adding There are far more twists and turns, and for a [[Footpod ]] your footsteps tend to be uneven. I realized how bad the {{Garmin 310XT}} improves its problem was when running some mountain bike trails and my GPS watch said I'd only been traveling at walking pace. This prompted me to survey and evaluate the accuracyof various devices on these mountain bike trails. * It takes time for The table below is preliminary data, but you'll notice how the results are dramatically worse than my usual GPS testing. The Polar V800, which does really well on my greenway tests has serious problems on trails, though it's still one of the better watches to acquire I've tested so far. The Suunto Spartan Trainer shows its strength more clearly on the satellites. Some mountain bike trails, coming in far ahead of other GPS watches tended to say they are ready to go before they have an optimal lock. ThereforeThe [[Stryd]] footpod is vastly more accurate than GPS, and unlike GPS it could be calibrated to improve its accuracy try to give them a little more timeeven further. Note that some newer GPS watches such as the {{Garmin 620:GPS Accuracy-TrailSummary}} have =Which Chipset? =While the specific chipset used in a GPS watch will impact its accuracy, there are many other factors that come into play. The physical packaging of the chipset, the ability to be preloaded with antenna used, the satellite positionsparticular features that are implemented, reducing this startup time and start up in the software that interprets the raw data will influence the overall accuracy dramatically. It's important to note that the SiRF chipsets such as "SIRFstarIV" are not a single chipset, but rather an overall architecture with several specific chipsets bearing the same name.
=Even GPS Watches have Bad Days=
While it's tempting to take the various GPS watches on a single run and simply compare the totals, this is a flawed approach. Evaluating the devices GPS accuracy on the basis of a single sample does not tell you much. It's a bit like evaluating an athlete's ability on the basis of one event; everyone has good days and bad days, and that applies to GPS watches as well. To illustrate this, the images below are from two runs, recorded on 9/20 and 9/22. In each run I recorded data on both the 310 and 910 watches, hitting the lap button on both at as close to the same time as is humanly possible. On 9/20 the 910XT was far more accurate than the 310XT, but on 9/22 the situation is reversed. If you were to have evaluated the two watches on the basis of a single run, you would conclude that one is much better than the other. But which device would win would depend on the particular day. This is why I've accumulated a lot of data to do a statistical analysis to work out which is really better.
|}
=Some Devices Are Better Than Others=
Below is a section of two runs showing the same section of the course, both taken at the same time, one from the Garmin 310XT and the other from the Garmin 620with the early firmware. These give a good indication of (With the accuracy problems I've seen with later firmware the tracks from the Garmin 620look like the 310XT. )
{| class="wikitable"
|- valign="top"
|}
=GPS Short and long measurements=
As you can see from the images below, the GPS track tends to take shortcuts around bends, reducing the length of the measured track. This cutting of the corners indicates the devices are doing some post-hoc smoothing to try to overcome the GPS errors. The more smoothing they do, the better the accuracy is likely to be in a straight line and the worse it is around corners or twisty courses. In my discussions with engineers working on GPS systems, this type of smoothing is often performed with a[http://en.wikipedia.org/wiki/Kalman_filter Kalman filter]. (When I tested using software without smoothing I found the measurements were long on my course rather than short, which is almost always the case.)[[File:GPS Shortcuts.jpg|none|thumb|500px|The GPS tracks in red with blue dots representing each GPS point recorded. The arrow points showing the tendency to cut the corners on the actual path takencurves.]]
Often GPS measurements of races, especially marathons record a longer distance than the race. This is partly because the USATF technique for measuring the distance takes a path that is no more than 12 inches away from the tangent (corner), and few runners are able to run that close. In a large marathon you can be forced to take a line that is a long way from the tangent. The other factor is that on a straight line, the GPS error tends to give a slightly longer measurement.
{| class="wikitable"
|- valign="top"
[[File:GPS Marathon.jpg|none|thumb|500px|Here you can see the GPS line is not following the straight road, giving a longer reading on the Thunder Road Marathon. Notice that the GPS is also cutting the corner at the top (we didn't run through the building).]]
|[[File:GPS MarketSt.jpg|none|thumb|x300px|Here's another example of running down Market Street in San Francisco, where you can see the errors that would add to the distance. ]]|}=Density PlotsGPS Accuracy and Weather=Below are the density plots GPS Accuracy is slightly better with clear skies than with cloud cover. The difference between completely clear and fully overcast is generally less than 0.1% and my testing includes a similar mix of cloud cover for each devicewatch, so I ignore this difference. However, rain can degrade accuracy by 0.3-3.1%, with the better watches being impacted the least. Because it does not rain that frequently where I test, this has created some potential bias in my testing so I now ignore measurements taken during the rain. This has only made a slight difference to the results, but it ensures consistence. =GPS Accuracy and Seasons=Normal ConditionI run in a wooded area with mostly deciduous trees, so the foliage varies by season. This foliage can have a noticeable impact on GPS accuracy, with better accuracy during the bare winter months than the rest of the year. This difference is mostly 0.1-1.5%, but in some cases can be as large as 2.5%. Because of this, my testing now ignores data from the winter months when the trees are bare. The short winter here in the south of the US means that the impact on the overall results are small, but like the weather impacts noted above, this does ensure greater consistency. =GPS Accuracy and Pace=[[File:GPS NormalAccuracyAndPace.jpegjpg|none|thumb|1024px500px|Density plots A plot of GPS precision against pace. The red line is the correlation.]]There have been reports of GPS accuracy changing with pace, but as you can see from the graph above, my testing does not show this. =GPS and GLONASS=I have found that GPS plus GLONASS produces less accuracy than GPS alone, something that is a little counterintuitive. I have no definitive explanation for this, and I do have a working hypothesis. My thought is that enabling both GPS and GLONASS will increase the number of satellites above the horizon, and a modern chipset can have over 50 channels. This means the chipset will have access to far more satellites with both systems enabled. However, I don't believe that the chipset will use all the available satellites when calculating its position. In an urban, or wooded environment, the satellites nearest the horizon will have the weakest signal, and the satellites closest to directly overhead will have the strongest signal. If the chipset were to use only the strongest 5-6 signals, then it's likely to choose the satellites that are closest to being directly overhead. That means the satellites chosen are relatively close together, which is a poor geometry that reduces accuracy. (In GPS terms this is called Dilution of Precision, or DoP.) I've talked to a GPS specialist who tells me that they have seen this in GPS systems they've tested (though not necessarily consumer grade systems.) What this means in the real world is that if you're in an environment with a partial view of the sky due to tree cover for low buildings then GPS on its own is likely to provide better accuracy. If you're in an environment with a clear view of the sky from horizon to horizon, then it's less clear to me which system is likely to provide better accuracy, and I've not tested this in practice. Given that the theoretical accuracy of GLONASS is not quite as good as GPS I'm not sure that enabling both systems will improve matters. It's possible that GLONASS will do relatively better at extreme polar latitudes due to its different orbital patterns.==Garmin 920XT and GLONASS==The [[Garmin 920XT]] is significantly worse with GLONASS enabled. {| class="wikitable"!Device!Accuracy!Trueness!Precision!Repeatability|-|Garmin 920XT|style="background-color: #FAE983;"|6.6|style="background-color: #D2DE81;"|7.5|style="background-color: #FED680;"|5.9|style="background-color: #D1DD81;"|7.5|-|Garmin 920XT (GLONASS)|style="background-color: #FEC77D;"|5.5|style="background-color: #FAE983;"|6.6|style="background-color: #FCA777;"|4.6|style="background-color: #E1E282;"|7.2|}==Suunto Spartan Ultra and GLONASS==The [[Suunto Spartan Ultra]] seems to do particularly poorly with GLONASS enabled. {| class="wikitable"!Device!Accuracy!Trueness!Precision!Repeatability|-|Suunto Spartan Ultra 1.6.14|style="background-color: #E2E282;"|7.1|style="background-color: #79C47C;"|9.5|style="background-color: #FED881;"|6.0|style="background-color: #D8DF81;"|7.4|-|Suunto Spartan Ultra 1.6.14.GLONASS|style="background-color: #FCB179;"|4.9|style="background-color: #D4DE81;"|7.5|style="background-color: #F9756E;"|3.3|style="background-color: #FDB57A;"|5.0|}==Garmin Epix and GLONASS==The [[Garmin Epix]] has slightly better accuracy with WAAS than without it, and GLONASS didn't degrade the accuracythe way it does with other devices. My belief is that enabling WAAS effectively disables GLONASS, as WAAS is GPS specific (and only available in North America.) There is EGNOS Ground Segment is the equivalent of WAAS for GLONASS/GPS/Galileo in Europe.{| class="wikitable" |- valign="top"!Device!Accuracy!Trueness!Precision!Repeatability|-|Garmin Epix with GLONASS+WAAS|style="background-color: #FFE082;"|6.2|style="background-color: #D6DF81;"|7.4|style="background-color: #FDBE7C;"|5.3|style="background-color: #D6DF81;"|7.4|-|Garmin Epix with WAAS|style="background-color: #FFDF82;"|6.2|style="background-color: #C8DB80;"|7.7|style="background-color: #FDB77A;"|5.1|style="background-color: #F7E883;"|6.7|-|Garmin Epix|style="background-color: #FDC37D;"|5.4|style="background-color: #F3E783;"|6.8|style="background-color: #FB9C75;"|4.4|style="background-color: #F2E783;"|6.8|}==Garmin Fenix 5X and GLONASS==Continuing the theme of poor accuracy with GLONASS enabled, the [[Garmin Fenix 5X]]demonstrates even worse performance than its peers. The values shown below are rather dramatically worse with GLONASS enabled than without. My anecdotal observation is that sometimes the Fenix 5X does a little worse with GLONASS than normal, possibly in line with other Garmin devices, and sometimes it seems to just get lost and produce dramatically worse results. {| class="wikitable sortable"!Device!Accuracy!Trueness!Precision!Repeatability|-|Fenix 5X 4.30|style="background-color: #FEC97E;"|5.6|style="background-color: #EDE683;"|6.9|style="background-color: #FCA377;"|4.5|style="background-color: #F3E783;"|6.8|-|Fenix 5X 4.30 GLONASS|style="background-color: #F97B6F;"|3.5|style=Under "background-color: #FFE082;"|6.2|style="background-color: #F8696B;"|1.6|style="background-color: #F97B6F;"|3.5|}Below you can see a visual representation of the problems. Many of the Bridgetracks are a little worse than normal, but you generally follow the path. A small subset of the tracks are dramatically worse, either showing and offset from the actual path, or sometimes it looks like the sampling frequency has dropped, suggesting that the watch is only periodically able to get a location fix. This latter phenomenon is rather surprising to me and goes against my hypothesis of why GLONASS has worse accuracy. I would expect there would be more satellites available with GLONASS enabled, which would result in the watch selecting the subset with the strongest signal that are more likely to have a narrow angle of separation, which would result in increased Dilution of the Precision. These tracks suggest that the Fenix 5X is unable to get any location fix for brief periods. {| class="wikitable" style="margin-left: auto; margin-right: auto; border: none;"|- valign="top"|[[File:BridgeFenix 5X 4.30 GLONASS.jpg|center|thumb|x300px| The GPS Under Bridgetracks from the Fenix 5X with GLONASS enabled. This diagram has tracks color coded with green indicating good accuracy through to red indicating poor accuracy, and the lap markers as blue dots.]]|- valign="top"|[[File:BridgeFenix 5X 4.30.jpegjpg|none|thumb|1024pxx300px|Here's the tracks from testing with GLONASS disabled for comparison.]]|Density plots }=GPS Accuracy and Sampling Rate=GPS watches default to recording a sample frequently enough that accuracy is not compromised. However, several devices offer the option of recording less frequently to improve battery life at the cost of accuracy. These devices actually turn off the GPS accuracy receiver, turning it on periodically for just long enough to get a fix. The images below are from the segment that goes under [[2014 Badwater 135]] using the bridge.[[Suunto Ambit2| Suunto Ambit2 R]]with recording set to one minute intervals. As you can see, accuracy suffers on curves, but is fine on the straights. For a course like Badwater, the one minute recording interval was fine as the course has few turns. {| class="wikitable" |- valign=Turn Around=="top"|[[File:GPS Sampling Curve.jpg|none|thumb|x300px|On a curve, the infrequent samples tend to 'cut the corners' and are quite inaccurate.]]|[[File:GPS Sampling Straight.jpg|none|thumb|x300px|On the straight sections, the one minute sampling does not lose any accuracy.]]|[[File:GPS Turn AroundSampling Comparison.jpegjpg|none|thumb|1024pxx300px|Density plots Here's a comparison of 1 minute sampling (red) with 1 second sampling (blue). On my GPS testing course the 1 minute sampling lost nearly 2 miles over a 16 mile run.]]|}=GPS Accuracy and Recording Rate (Smart/1-Second)=While the GPS sampling rate mentioned above has a huge impact on GPS accuracy , the same isn't true for recording rate. These two ideas seem to get confused. GPS sampling rate allows a watch to turn off the GPS receiver for short periods to conserve battery life while sacrificing GPS accuracy. Some Garmin watches can be configured to either record every second, or only record when something happens, such a change in heart rate or change in direction, something they call "smart recording." With a smart recording in normal GPS mode, the turn-around conditionGPS system is continually active, so there's no loss in accuracy.To verify this, I tested the [[Garmin Fenix 5X]]in both the smart recording mode I normally use, and one second recording mode for comparison. As you can see, the two modes are virtually identical, and the differences are most likely due to chance (p=0.72). {| class="wikitable sortable"!Device!Accuracy!Trueness!Precision!Repeatability|-|Fenix 5X 4.30 Smart Recording|style="background-color: #FEC97E;"|5.6|style="background-color: #EDE683;"|6.9|style="background-color: #FCA377;"|4.5|style="background-color: #F3E783;"|6.8|-|Fenix 5X 4.30 One Second Recording|style="background-color: #FDBF7C;"|5.3|style="background-color: #E5E382;"|7.1|style="background-color: #FB9073;"|4.0|style="background-color: #FBE983;"|6.6|}
=Device Specific Notes=
For those interested in some of the details of how devices are configured for testing, here are some additional notes.
* Garmin devices are set to 'smart recording'. I did try an informal test with the 620 using 1-second recording, but it appeared to make no difference.
* For details of the calibration of the [[Footpod]] see [[GPS Testing Methodology]].* The Fenix 2 was tested with and without WAAS support activated; WAAS helped slightly. * The [[Garmin 620 tests used 920XT]] was tested with Watch Firmware 2.50 , GPS Firmware 2.70 using smart recording. =Garmin 620 Issues=The Garmin 620 had some notorious problems with its GPS accuracy. The table below shows the changes with various firmware versions, culminating in the GPS-3.30 firmware that resolved the issues. I've including some testing I did without EPO data (NoEPO row below) and with a Footpod (+FP row below). {{:GPS Accuracy-g620}}{| class="wikitable" |- valign="top"|[[File:Garmin620 Offset1.jpg|none|thumb|x500px|Here you can see the last repeat is offset. Starting at lap marker 49, the track follows the same outline as the more accurate tracks, but is offset. So marker 50 should be near 4, 51 near 37, 52 near 2, 53 near 1, and the finish near the start.]]|[[File:Garmin620 Offset2.jpg|none|thumb|x500px|This is a simple out and back run of ~3 miles/5 Km, but you can see after the turn around the Garmin 620 records a gradually widening gap, even though it follows the right overall shape. (The outbound track is fairly accurate, the return is messed up.)]]|}=Garmin Fenix 2 Issues=Like the Garmin 620, I've had similar GPS accuracy issues with the Fenix 2. In fact, the Fenix 2 is the only device I've ever had that has given the "lost satellite reception" message on my usual running route. Because of these issues Garmin replaced my Fenix 2 under warranty, and below are the results for the original and new watches. The replacement watch also gave "lost satellite reception" repeatedly and the error values for the Fenix 2 do not reflect these problems as the data from those runs was useless for analysis. I suspect there are three (possibly related) problems with the Fenix 2:# The MediaTek GPS chipset is not as accurate as the SiRF chipset. The best results from the Fenix 2 are generally mediocre. # The Fenix 2 records the right shape track, but offset by some distance. This does not look like a typical accuracy problem that would manifest itself randomly. # Occasionally the Fenix 2 will report "lost satellite reception", and I have several instances of this where the date and time were wrong after reception was lost. If a GPS Firmware device has the wrong time, then it will expect the satellites to be in different positions and will be unable to acquire a position fix. I have four instances where the workout file was stored with a date in April 2019, indicating that was the date when I terminated the workout and attempted to reacquire satellite lock. In one case I noticed the date and time was set incorrectly on the watch display after the satellite lost message. There are also reports from various users about lost satellite reception and the 2019 date. This problem might also explain the offset track above, but only if the clock was out by a very small amount.{{:GPS Accuracy-Fenix2}}{| class="wikitable" |- valign="top"|[[File:Fenix2 Getting Lost.jpg|none|thumb|x400px|This is an example of just how bad the Fenix 2 can be. This is a short run, with the start and finish in the same place. The track up to marker 18 is not bad, but then the Fenix 2 loses reception for a couple of miles. When it gets reception back, it tracks wildly off course, ending up with a position that's out by around a mile.]]|[[File:Fenix2 Getting Lost3.jpg|none|thumb|x400px|Another example of the Fenix 2 getting lost. You can see marker 41 is a long way off the route, probably about half a mile off. Notice how messy the rest of the track is as well.]]|[[File:Fenix2 Getting Lost4.jpg|none|thumb|x400px|Here you can see the Fenix 2 track is a confused mess.]]|- valign="top"|[[File:Fenix2 Getting Lost5.jpg|none|thumb|x400px| The first part of this run goes okay, but at marker 61 things to go a little astray, and at marker 65 the GPS lock is lost, then briefly regained until marker 70. Not unreasonably, the Fenix 2 assumes straight-line movement until GPS lock is reacquired, but then rather bizarrely seems to assume that the straight-line movement is correct and records a track that is about half a mile/1 Km off.]]|[[File:Fenix2 Short1.jpg|none|thumb|x400px| This is more how the GPS track should look, but even on this run the Fenix 2lost nearly a mile in a 20 mile run.90]]|[[File:Fenix2 Getting Lost6.jpg|none|thumb|x400px|This GPS track looks reasonable until marker #54, and then the track gets offset, but strangely it stays offset until the last marker.]]|}
=Next Steps=
This is an initial analysis of the data I have, and there are a number of further evaluations to do.
* Check how GPS accuracy changes over the course of a run, as I've seen a distinct tendency for the watches to say they are good to go when they don't really have an optimal lock on the satellites. I wait for 5+ minutes between the watches saying they have sufficient satellites locked in, so this should not be a problem with the data shown here, but I could do some tests where I turn on the watch from a cold state, then start running as soon as they claim they have a lock.
* Look at how accurate the GPS watches are for measuring elevation, and compare with barometric data.
* Write up general GPS accuracy.
