Difference between revisions of "High Speed Video Analysis"

From Fellrnr.com, Running tips
Jump to: navigation, search
User:Fellrnr (User talk:Fellrnr | contribs)
User:Fellrnr (User talk:Fellrnr | contribs)
Line 1: Line 1:
High Speed Video is a great way of analyzing your [[Running Form]]. While high end equipment is extremely expensive, there are a number of options for relatively inexpensive cameras with this ability.  
+
[[File:Kinovea1.jpg|right|thumb|300px|An example of the analysis from Kinovea. You can see a number of markers being tracked. The blue elbow marker has the circle of movement shown, and the knee and shoulder have acceleration displayed.]]
 
+
High Speed Video is a great way of analyzing your [[Running Form]]. While high end equipment is extremely expensive, the latest smartphones are quite capable and the software is freely available. I'll focus on using the Kinovea software, though I'll look at some of the mobile apps in the near future.  
=Cameras with High Speed Video=
+
=Equipment=
{| class="wikitable"
+
==Camera==
! Camera
+
You'll need a camera capable of recording high-speed video at a reasonable resolution. In the past, I've used a GoPro, but their fisheye distortion is a less than ideal. The iPhone 6 & 7, and the Samsung Galaxy S7 will all record HD (1280x720) at 240 Frames Per Second (FPS), which is eight times as fast as typical video. I found that this resolution and frame rate is perfectly adequate for analyzing a runner. Using a lower frame rate and a high resolution did not produce as good results. I found that using the zoom lens on my iPhone 7+ from further away produced better results than the wide-angle lens, as there is less perspective distortion. You'll also need a tripod and an adapter for your phone. I'm lucky enough to have a professional grade, heavy-duty tripod, but something far simpler should suffice. I purchased a cheap tripod adapter and Bluetooth remote control at the same time. (<jfs id="B00PMDSEXK" noreferb="true" n="Tripod adapter and remote control"/>)
! Resolution
+
==Treadmill==
! Pixel Count
+
It should be perfectly possible to video a runner outside, but you'll only get a small number of strides, and it can be tricky to analyze the motion as effectively as you can when recording on a treadmill. The main downside of a treadmill is that most runners have a slightly different running form compared with running outside. Ideally, you want a treadmill that doesn't have a side rail that gets in the way of the camera's view of the runner. This is mostly an issue with measuring movement of the hip from the side, so it's not a huge deal.
! Resolution
+
==Markers==
|-
+
The Kinovea software tracks motion using visual markers. These markers should be round, as the software doesn't understand a rotating marker. So, if you put a red square on your ankle, as that square turns with your leg movement, the software will lose track. You also want the marker to be on a plain background that has good contrast with the marker. I've used ping-pong balls attached to black running tights with sticky tape which provides high contrast and the spherical ping-pong ball looks the same to the camera even if there is some twisting motion. For my upper body, I use a white, long-sleeved compression top and I'm experimenting with various markers. I've found that a round piece of tape can work, but the twisting of my upper body can rotate the marker too much. Tracking motion on shoes is a little trickier, as most shoes have complex patterns. I put some black tape over a mostly black shoe, and glued on ping-pong balls which worked reasonably well. I'll add more details as I try different markers. My aim is to track the forefoot, heel, ankle, knee, hip, shoulder, wrist, and head. As I experiment more, I'll give some feedback on what markers are most useful.  
| rowspan="3"|
+
==Room==
[http://www.amazon.com/dp/B004H3X1F2 Casio EX-ZR100] ($360)
+
The room you use to take the measurements is surprisingly important. It's critical you have lots of light, so that the cameras shutter speed is fast enough to freeze the motion. You also need enough distance between the treadmill and where you set up the camera that you're not getting perspective distortion. A plain background behind the treadmill is best, though you can get the software to work against a visually noisy background. (Some functions, like tracking angles, may not work quite right.) The room should also have a fairly solid floor, so that the running doesn't balance the camera. Adding something of a known size into the frame at the same distance as the runner can help calibrate size.
Casio EX-ZR10
+
=Recording =
| 432 x 320
+
It's worth spending a little time to get the camera positioned correctly. You want the runner to fill most of the frame so that you're not wasting resolution. If you want to evaluate the runner with the treadmill at an incline, don't forget to allow for this when positioning the camera. You want the camera positioned perpendicular to the runner, and as level as possible. Ensure the camera has the near edge of the treadmill in view so that it can see each foot strike. I used a Bluetooth remote control to record my running, and I'd recommend using the remote control even if you have a friend helping you. You don't want to touch the phone when starting and stopping recording as any camera shake will cause problems. I'd recommend recording several relatively short clips (~30 seconds), as high-speed video takes up a lot of memory. The more powerful your PC, the longer the segment you'll be able to process.
| 138.2K
+
=Evaluation=
| 240
+
Kinovea is fairly easy to use, and there's a good online help section. That said, there's a few caveats that you should be aware of, so let me walk you through the process.
|-
+
* If you recorded your video in portrait mode, you'll need to rotate it before importing into Kinovea. I used "Rotate & Flip" on the iPhone prior to transferring it, which worked okay. Hopefully the next version of the Kinovea will understand rotated video, as this is a pain.  
| 224 x 160
+
* The next step is to transfer the video to your PC. I used Dropbox, but there are many other solutions.  
| 35.8K
+
* I'd recommend using the experimental version of Kinovea 0.8.25, as it has some nice additional functionality, and appears to be completely stable. Download from [http://www.kinovea.org/]
| 480
+
* The first time you start up Kinovea, I'd recommend changing some of the preferences. (Options menu, then select preferences.)
|-
+
** Click on drawings, then the tracking tab. I find setting the object window to 30x30 pixels and the search window to 90x90 pixels is a good starting point. The default uses a percentage of the overall video window and is too big, causing tracking problems.  
| 224 x 64
+
** If you have a powerful PC, click playback, then the memory tab. Increase the working zone to 30 seconds and memory to 1024 Mb.  
| 14.3K
+
* It's fairly obvious how to open your file.
| 1000
+
* The first thing to do is to adjust playback speed so that you're playing the video in slow motion. There is a slider bar just above and right of the playback buttons, so slide this to the left to slow things down.
|-
+
* You normally set up a "working window" as a subset of the video clip. For performance, only a small subset of the video can be loaded into memory for analysis. Play video, or click on the main timeline to choose your starting point, then click the "[" button to mark the start. Likewise, at the end of your chosen section use the "]" button.  
| rowspan="4"|
+
* To track movement, moved to the beginning of your working window. You'll want to right click on a marker, and select "track path". It's easier to click the middle of your marker if you're zoomed in, which you do using control and the mouse wheel.  
Casio EX-ZR1000
+
* Once you'd set your track point you'll see two concentric rectangles and a small cross. The cross marks of the center of the object being tracked, and the inner rectangle should surround your marker, with a little bit of the background included. The larger rectangle is the search window. For each frame, the software looks for the image in the small rectangle within the area defined by the larger rectangle. Getting these areas right is critical to successful tracking. If you right-click near the cross and select "configuration…" You'll get a dialogue that will allow you to tweak the size of the two rectangles.<br/>[[File:KinoveaConfigure.jpg|none|thumb|400px|]]
Casio EX-ZR700
+
* If you click play, or frame advance, you'll see the line made by the marker recorded on the playback window. If the line doesn't track your marker, but jumps around, it's a sign that the software is finding a better match for your object window within the search window. Tweaking these can often, but not always resolve the problem.
Casio EX-ZR400
+
* Within the configuration window, you can also set the software to display a metric such as acceleration, speed, etc. You can use this to determine the frame with the highest acceleration, or the slowest speed, or whatever else it is you're interested in. For instance, tracking vertical acceleration of a torso marker and looking for the frame with the maximum value would tell you when the greatest landing forces are likely to be.
[http://www.amazon.com/Casio-Exilim-Ex-zr300-Digital-Ex-zr300gd/dp/B0085ME7HC Casio EX-ZR300] ($369)
+
* There is another option on the configuration dialog called "display rotation circle." This will show the circle around which a market is moving.
[http://www.amazon.com/Casio-Exilim-Ex-zr200-Digital-Ex-zr200we/dp/B006596OUA Casio EX-ZR200] ($320)
+
* You can measure angles by clicking the angle icon and then clicking on the video window. You then move the three points so they line up with the joint you're trying to measure, and it will display the angle. With the experimental version, you can right click on one of the points and select "track path" and the software will track the three markers, showing the changing angle as things progress. The problem I found with this functionality is that you can't tweak the object and search windows for each point, so it easily gets confused unless you have a really clear marker.
Casio EX-ZR20
 
Casio EX-ZR15
 
| 640x480
 
| 307.2K
 
| 120
 
|-
 
| 512x384
 
| 196.6K
 
| 240
 
|-
 
| 224x160
 
| 35.8K
 
| 480
 
|-
 
| 224x64
 
| 14.3K
 
| 1000
 
|-
 
| rowspan="2"|
 
[http://www.amazon.com/Nikon-Digital-Camera-System-10-30mm/dp/B005OGQXJW Nikon J1] ($400)
 
| 640 x 240
 
| 153.6K
 
| 400 (5 second max)
 
|-
 
| 320 x 120
 
| 38.4K
 
| 1200 (5 second max)
 
|-
 
| rowspan="3"|
 
[http://www.amazon.com/GoPro-CHDHX-301-HERO3-Black-Edition/dp/B009TCD8V8 GoPro Hero3 Black Edition] ($400)  
 
| 1280x960
 
| 1228.8K
 
| 100
 
|-
 
| 1280x720
 
| 921.6K
 
| 120
 
|-
 
| 848x480
 
| 407K
 
| 240
 
|-
 
| [http://www.amazon.com/GoPro-CHDHN-301-HERO3-Silver-Edition/dp/B009PK9S90 GoPro Hero3 Silver Edition] ($300)
 
| 848x480
 
| 407K
 
| 120
 
|}
 
 
 
=Camera Recommendations=
 
There are pros and cons to the various cameras.
 
* I've used the Casio cameras a number of times, and they are easy to use and work well. The resolution and quality of the higher frame rates it too poor to be of much use, so don't expect to be able to use the 1000 fps mode.  
 
* You can sometimes get [http://go.fellrnr.com/?id=35454X937677&xs=1&url=http://www.ebay.com/sch/Digital-Cameras-/31388/i.html%3F_from%3DR40%26_nkw%3Dcasio%2Bexilim%2Bhigh%2Bspeed%26_sop%3D1 used High Speed EXILIM cameras on ebay] for a reasonable cost.
 
* The Nikon J1 is crippled by its 5 second maximum record time.
 
* The GoPro a good frame rate and a nice resolution, as well as a WiFi remote control
 
* There is no LCD screen for replay of the video on the GoPro, so you need to by the [http://www.amazon.com/GoPro-Touch-BacPac-HERO-Cameras/dp/B009PK9SB8 LCD Screen] for $80.  
 
 
 
=Evaluating the video=
 
The simplest approach is to step through the video frame by frame, which is possible with many software video players. For a more detailed analysis you can extract each frame into a separate photo file using the free open source software [http://www.ffmpeg.org/ FFMpeg]. You can use the following command to extract separate images, but you probably want to extract a short relevant clip before you run this command or you will end up with too many image files to handle.  
 
ffmpeg -i input.avi -f image2 image-%4d.jpeg
 
Where:
 
* Input.avi is the name of your input video file
 
* image-%4d.jpeg is the name of the output files, with %4d replaced by the frame number
 
 
 
=Sample Output=
 
Below are four consecutive frames from a recording at 120 fps to give a sense of the resolution and timing of the available video from a Casio camera. These pictures look identical on first glance, but there is slight movement visible if you rapidly swap between the images.  
 
{| class="wikitable"
 
|- valign="top"
 
|[[File:HSVA-Sample-0413.jpg|none|thumb|x300px]]
 
|[[File:HSVA-Sample-0414.jpg|none|thumb|x300px]]
 
|[[File:HSVA-Sample-0415.jpg|none|thumb|x300px]]
 
|[[File:HSVA-Sample-0416.jpg|none|thumb|x300px]]
 
|}
 
 
 
=Automated Analysis Software=
 
I am hoping to write some software that will perform some automated analysis. Here are my current thoughts on the algorithm that could be used.
 
* Capture:
 
** Shoot so that the runner is shown from the waist down.
 
** Use good light to ensure a fast shutter speed.
 
** The runner should be against a relatively plain background, on flat, level ground.  
 
** Put markers on the runner over the hip, knee, ankle, heel and forefoot.
 
* Processing:
 
** Scan the video, extracting each frame.
 
*** This may be the trickiest part, as working with video in code is rather nasty.
 
** Scan each frame to detect the five markers and reduce them to the x/y coordinates.
 
** For each set of five markers, identify the body positions:
 
*** The hip will be the highest.
 
*** The knee will be next highest.
 
*** The forefoot will be furthest away of the remaining three.
 
*** The heel and ankle can be found based on the angle between then and the forefoot.
 
** Find the frames that represent the first contact. This will be the first frame where either the forefoot or heel ceases to move.
 
** Find the frames that represent the last contact. This will be the first frame that follows first contact where both the forefoot and heel are in motion.
 
** For the first contact frame:
 
*** Measure the angle of the heel/forefoot line to estimate strike angle and therefore strike index. This can be categorized into FFS, MFS, RFS.
 
*** Find the horizontal distance from the middle of the foot to the hip to check for [[Overstriding|overstriding]].  
 
*** Look at the angle between the hip/knee and the knee/ankle to determine how straight the leg is. A straighter leg may indicate a higher impact force.
 
** Evaluate the frames between first and last contact:
 
*** Determine if the heel touches down at any point. This can further categorize FFS.
 
*** Look at the change in knee angle to see how the landing force is absorbed.
 
** Evaluate the frames between two first contact points:
 
*** Measure [[Cadence]].
 
*** Measure the contact time, the float time, and the ratio between the two.  
 
*** Try to determine paw back from the maximum horizontal distance from the midfoot to the hip and the horizontal distance at contact.  
 
*** Try to determine the probable horizontal speed of the foot at contact.
 
*** Try to evaluate the peak deceleration of the hip on landing to determine impact forces.  
 
** Output metrics and possibly key images with overlays of the angles and lines.
 

Revision as of 16:43, 22 January 2017

An example of the analysis from Kinovea. You can see a number of markers being tracked. The blue elbow marker has the circle of movement shown, and the knee and shoulder have acceleration displayed.

High Speed Video is a great way of analyzing your Running Form. While high end equipment is extremely expensive, the latest smartphones are quite capable and the software is freely available. I'll focus on using the Kinovea software, though I'll look at some of the mobile apps in the near future.

1 Equipment

1.1 Camera

You'll need a camera capable of recording high-speed video at a reasonable resolution. In the past, I've used a GoPro, but their fisheye distortion is a less than ideal. The iPhone 6 & 7, and the Samsung Galaxy S7 will all record HD (1280x720) at 240 Frames Per Second (FPS), which is eight times as fast as typical video. I found that this resolution and frame rate is perfectly adequate for analyzing a runner. Using a lower frame rate and a high resolution did not produce as good results. I found that using the zoom lens on my iPhone 7+ from further away produced better results than the wide-angle lens, as there is less perspective distortion. You'll also need a tripod and an adapter for your phone. I'm lucky enough to have a professional grade, heavy-duty tripod, but something far simpler should suffice. I purchased a cheap tripod adapter and Bluetooth remote control at the same time. (Error: Could not parse data from Amazon!)

1.2 Treadmill

It should be perfectly possible to video a runner outside, but you'll only get a small number of strides, and it can be tricky to analyze the motion as effectively as you can when recording on a treadmill. The main downside of a treadmill is that most runners have a slightly different running form compared with running outside. Ideally, you want a treadmill that doesn't have a side rail that gets in the way of the camera's view of the runner. This is mostly an issue with measuring movement of the hip from the side, so it's not a huge deal.

1.3 Markers

The Kinovea software tracks motion using visual markers. These markers should be round, as the software doesn't understand a rotating marker. So, if you put a red square on your ankle, as that square turns with your leg movement, the software will lose track. You also want the marker to be on a plain background that has good contrast with the marker. I've used ping-pong balls attached to black running tights with sticky tape which provides high contrast and the spherical ping-pong ball looks the same to the camera even if there is some twisting motion. For my upper body, I use a white, long-sleeved compression top and I'm experimenting with various markers. I've found that a round piece of tape can work, but the twisting of my upper body can rotate the marker too much. Tracking motion on shoes is a little trickier, as most shoes have complex patterns. I put some black tape over a mostly black shoe, and glued on ping-pong balls which worked reasonably well. I'll add more details as I try different markers. My aim is to track the forefoot, heel, ankle, knee, hip, shoulder, wrist, and head. As I experiment more, I'll give some feedback on what markers are most useful.

1.4 Room

The room you use to take the measurements is surprisingly important. It's critical you have lots of light, so that the cameras shutter speed is fast enough to freeze the motion. You also need enough distance between the treadmill and where you set up the camera that you're not getting perspective distortion. A plain background behind the treadmill is best, though you can get the software to work against a visually noisy background. (Some functions, like tracking angles, may not work quite right.) The room should also have a fairly solid floor, so that the running doesn't balance the camera. Adding something of a known size into the frame at the same distance as the runner can help calibrate size.

2 Recording

It's worth spending a little time to get the camera positioned correctly. You want the runner to fill most of the frame so that you're not wasting resolution. If you want to evaluate the runner with the treadmill at an incline, don't forget to allow for this when positioning the camera. You want the camera positioned perpendicular to the runner, and as level as possible. Ensure the camera has the near edge of the treadmill in view so that it can see each foot strike. I used a Bluetooth remote control to record my running, and I'd recommend using the remote control even if you have a friend helping you. You don't want to touch the phone when starting and stopping recording as any camera shake will cause problems. I'd recommend recording several relatively short clips (~30 seconds), as high-speed video takes up a lot of memory. The more powerful your PC, the longer the segment you'll be able to process.

3 Evaluation

Kinovea is fairly easy to use, and there's a good online help section. That said, there's a few caveats that you should be aware of, so let me walk you through the process.

  • If you recorded your video in portrait mode, you'll need to rotate it before importing into Kinovea. I used "Rotate & Flip" on the iPhone prior to transferring it, which worked okay. Hopefully the next version of the Kinovea will understand rotated video, as this is a pain.
  • The next step is to transfer the video to your PC. I used Dropbox, but there are many other solutions.
  • I'd recommend using the experimental version of Kinovea 0.8.25, as it has some nice additional functionality, and appears to be completely stable. Download from [1]
  • The first time you start up Kinovea, I'd recommend changing some of the preferences. (Options menu, then select preferences.)
    • Click on drawings, then the tracking tab. I find setting the object window to 30x30 pixels and the search window to 90x90 pixels is a good starting point. The default uses a percentage of the overall video window and is too big, causing tracking problems.
    • If you have a powerful PC, click playback, then the memory tab. Increase the working zone to 30 seconds and memory to 1024 Mb.
  • It's fairly obvious how to open your file.
  • The first thing to do is to adjust playback speed so that you're playing the video in slow motion. There is a slider bar just above and right of the playback buttons, so slide this to the left to slow things down.
  • You normally set up a "working window" as a subset of the video clip. For performance, only a small subset of the video can be loaded into memory for analysis. Play video, or click on the main timeline to choose your starting point, then click the "[" button to mark the start. Likewise, at the end of your chosen section use the "]" button.
  • To track movement, moved to the beginning of your working window. You'll want to right click on a marker, and select "track path". It's easier to click the middle of your marker if you're zoomed in, which you do using control and the mouse wheel.
  • Once you'd set your track point you'll see two concentric rectangles and a small cross. The cross marks of the center of the object being tracked, and the inner rectangle should surround your marker, with a little bit of the background included. The larger rectangle is the search window. For each frame, the software looks for the image in the small rectangle within the area defined by the larger rectangle. Getting these areas right is critical to successful tracking. If you right-click near the cross and select "configuration…" You'll get a dialogue that will allow you to tweak the size of the two rectangles.
    KinoveaConfigure.jpg
  • If you click play, or frame advance, you'll see the line made by the marker recorded on the playback window. If the line doesn't track your marker, but jumps around, it's a sign that the software is finding a better match for your object window within the search window. Tweaking these can often, but not always resolve the problem.
  • Within the configuration window, you can also set the software to display a metric such as acceleration, speed, etc. You can use this to determine the frame with the highest acceleration, or the slowest speed, or whatever else it is you're interested in. For instance, tracking vertical acceleration of a torso marker and looking for the frame with the maximum value would tell you when the greatest landing forces are likely to be.
  • There is another option on the configuration dialog called "display rotation circle." This will show the circle around which a market is moving.
  • You can measure angles by clicking the angle icon and then clicking on the video window. You then move the three points so they line up with the joint you're trying to measure, and it will display the angle. With the experimental version, you can right click on one of the points and select "track path" and the software will track the three markers, showing the changing angle as things progress. The problem I found with this functionality is that you can't tweak the object and search windows for each point, so it easily gets confused unless you have a really clear marker.