V4L capturing

This page discusses how to capture analogue video for offline consumption (especially digitising old VHS tapes). For information about streaming live video (e.g. webcams), see the streaming page. For information about streaming digital video (DVB), see TV-related software.

Overview

Analogue video technology was largely designed before the advent of computers, so accurately digitising a video is a difficult problem. For example, software often assumes a constant frame rate throughout a video, but analogue technologies can deliver different numbers of frames from second to second for various reasons. This page will discuss some of the problems you will encounter digitising video and some of the techniques and programs you can use to solve them. You should be able to find similar functionality in whatever program you use.

Recommended process

Digitising videos is a complex problem, and everyone has their own preferred process. Your workflow should look something like this:

1. Set your system up - understand the quirks of your TV card, VCR etc.
2. Encode an accurate copy of the source video - handle issues with the analogue half of the system here. Do as little digital processing as possible
3. Transcode a usable copy of the video - convert the previous file to something pleasing to use
4. Try the video and transcode again - check whether the video works how you want, then transcode again

Analogue-to-digital encoding has all sorts of problems - VCRs overheat and damage tapes, computers drop frames if they use too much CPU, and so on. Creating a digital video also presents challenges - not all software supports all formats, you might want to remove overscan and background hiss, and so on. It's much easier to produce a quality result if you tackle each problem separately.

Choosing formats

When you create a video, you need to choose your audio format (e.g. WAV or MP3), video format (e.g. XviD or MPEG-4) and container format (e.g. AVI or MP4). There's constant work to improve the codecs that create audio/video and the muxers that create containers, and whole new formats are invented fairly regularly, so this page can't recommend any specific formats. Wikipedia's comparisons of audio, video and container formats are a good place to start your research - here are some important things to look for:

• encoding speed - during the encoding stage, using too much CPU load will cause frame-drops as the computer tries to keep up
• accuracy - some formats are lossless, others throw away information to improve speed and/or reduce file size
• file size - different formats use different amounts of disk space, even with the same accuracy
• compatibility - newer formats usually produce better results but can't be played by older software

Speed and accuracy are usually the most important when encoding, but size and compatibility the most important for playback. So it you should use a modern, fast, lossless format for the encoding stage then a format that produces a smaller or more compatible file for the transcoding stage. For example, as of 2015 it might make sense to encode FLAC audio and x264 video into a Matroska file, then transcode MP3 audio and MPEG-4 video into an AVI file. The transcoded file might be larger or lower quality, but it should play on most software, and you can just delete it and try again if your grandmother's DVD player doesn't like it.

Setting up

Before you can record a video, you need to set your system up. This section will guide you through the details.

Connecting your video

	RF Connector	avoid	Usually input #0, these connectors tend to create more noise than the alternatives, and produce snow at the start of a recording
	Composite video connector	use	Usually input #1, these are widely supported and produce a good signal, and produce blackness at the start of a recording
	S-video connector	use if available	Usually input #2, these should produce a good video signal but most hardware needs a converter

Connect your video source (TV or VCR) to your computer however you can. Each type of connector has slightly different properties - try whatever you can and see what works. If you have a TV card that supports multiple inputs, you will need to specify the input number when you come to record.

Determining your video device

Once you have connected your input, you need to determine the name Linux gives it. See all your video devices by doing:

ls /dev/video*

One of these is the device you want. Most people only have one, or can figure it out by disconnecting devices and rerunning the above command. Otherwise, check the capabilites of each device:

for VIDEO_DEVICE in /dev/video* ; do echo ; echo ; echo $VIDEO_DEVICE ; echo ; v4l2-ctl --device=$VIDEO_DEVICE --list-inputs ; done

Usually you will see e.g. a webcam with a single input and a TV card with multiple inputs. If you're still not sure which one you want, try each one in turn:

mpv --tv-device=<device> tv:///<whichever-input-number-you-connected>

(if your source is a VCR, remember to play a video so you know the right one when you see it)

If you like, you can store your device in an environment variable:

VIDEO_DEVICE=<device>

Further examples on this page will use $VIDEO_DEVICE in place of an actual video device

Determining your audio device

See all of our audio devices by doing:

arecord -l

Again, it should be fairly obvious which of these is the right one. Get the device names by doing:

arecord -L | grep ^hw:

If you're not sure which one you want, try each in turn:

gst-launch-0.10 alsasrc do-timestamp=true device=hw:<device> ! autoaudiosink

Again, you should hear your tape playing when you get the right one. Note: always use an ALSA hw device, as they are closest to the hardware. Pulse audio devices and ALSA's plughw devices add extra layers that, while more convenient for most uses, only cause headaches for us.

Optionally set your device in an environment variable:

AUDIO_DEVICE=<device>

Further examples on this page will use $AUDIO_DEVICE in place of an actual audio device

Getting your device capabilities

To find the capabilities of your video device, do:

gst-launch-1.0 --gst-debug=v4l2src:5 v4l2src device=$VIDEO_DEVICE ! fakesink 2>&1 | sed -une '/caps of src/ s/;/\n/gp'

To find the capabilities of your audio device, do:

gst-launch-1.0 --gst-debug=alsa:5 alsasrc device=$AUDIO_DEVICE ! fakesink 2>&1 | sed -une '/returning caps/  s/;/\n/gp'

You will need to press ctrl+c to close each of these programs when they've printed some output. When you record your video, you will need to specify capabilities based on the ranges displayed here.

Video heights

Some devices report a maximum height of 578. A PAL TV signal is 576 lines tall and an NTSC signal is 486 lines, so height=578 won't give you the best picture quality. To confirm this, tune to a non-existent TV channel then take a screenshot of the snow:

gst-launch-1.0 -q v4l2src device=$VIDEO_DEVICE \
    ! video/x-raw,format=UYVY,width=720,height=578 \
    ! imagefreeze \
    ! autovideosink

Here's an example of what you might see - notice the blurring in the middle of the picture. Now take a screenshot with the appropriate height for your TV norm:

gst-launch-1.0 -q v4l2src device=$VIDEO_DEVICE \
    ! video/x-raw,format=UYVY,width=720,height=<appropriate-height> \
    ! imagefreeze \
    ! autovideosink

Here's an example taken with height=576 - notice the middle of this picture is nice and crisp.

You may want to test this yourself and set your height to whatever looks best.

Measuring your video framerate

Due to hardware issues, some V4L devices produce slightly too many (or too few) frames per second. To check your system's actual frame rate, start your video source (e.g. a VCR or webcam) then run this command:

gst-launch-1.0 v4l2src \
    ! video/x-raw,format=UYVY \
    ! fpsdisplaysink fps-update-interval=100000

1. Let it run for 100 seconds to get a large enough sample. It should print some statistics in the bottom of the window - write down the number of frames dropped
2. Let it run for another 100 seconds, then write down the new number of frames dropped
3. Calculate (second number) - (first number) - 1 (e.g. 5007 - 2504 - 1 == 2502)
   • You need to subtract one because fpsdisplaysink drops one frame every time it displays the counter
4. That number is exactly one hundred times your framerate, so you should tell your software e.g. framerate=2502/100

Note: VHS framerates can vary within the same file. To get an accurate measure of a VHS recording's framerate, encode to a format that supports variable framerates then retrieve the video's duration and total number of frames. You can then transcode a new file with your desired frame rate.

Correcting your colour settings

Most TV cards have correct colour settings by default, but if your picture looks wrong (or you just want to check), first capture an image that has a good range of colours:

mpv --tv-device=$VIDEO_DEVICE tv:///<whichever-input-number-you-connected>

Press "s" to take screenshots, then open them in an image editor and alter the hue, saturation brightness and contrast until it looks right. If possible, print a testcard, capture a screenshot of it in good lighting conditions, then compare the captured image to the original. When you find the settings that look right, you can set your TV card.

First, make a backup of the current settings:

v4l2-ctl --device=$VIDEO_DEVICE --list-ctrls | tee tv-card-settings-$( date --iso-8601=seconds ).txt

Then input the new settings:

v4l2-ctl --device=$VIDEO_DEVICE --set-ctrl=hue=0          # set this to your preferred value
v4l2-ctl --device=$VIDEO_DEVICE --set-ctrl=saturation=64  # set this to your preferred value
v4l2-ctl --device=$VIDEO_DEVICE --set-ctrl=brightness=128 # set this to your preferred value
v4l2-ctl --device=$VIDEO_DEVICE --set-ctrl=contrast=68    # set this to your preferred value

Note: you can update these while a video is playing. If your settings are too far off, or if you're able to record a testcard, you might want to change the settings by eye before you bother with screenshots.

Encoding an accurate video

Your first step should be to record an accurate copy of your source video. A good quality encoding can use up to 30 gigabytes per hour, so figure out how long your video is and make sure you have enough space. Most software isn't optimised for analogue video encoding, causing audio and video to desynchronise in some circumstances. Although hard to use, GStreamer has excellent analogue recording support.

The exact steps to encode video will depend on your system and software, but here are some general tips:

Handling desynchronised audio and video

GStreamer should be able to synchronise your audio and video automatically, but some hardware doesn't provide accurate timestamps to make this possible. Before following these instructions, check you're using a raw hw audio device - plughw devices can cause synchronisation issues.

If possible, create clapperboard effects at the start of your videos - hook up a camcorder, start capturing, then clap your hands in front of the camera before pressing play. Failing that, make note of moments in videos where an obvious visual element occurred at the same moment as an obvious audio moment.

Once you've recorded your video, you'll need to calculate your desired A/V offset. You can usually get a good enough result here by opening the video in your favourite video player and adjusting the A/V sync until it looks right to you - different players put this in different places, for example it's Tools > Track Synchronisation in VLC. If you want to get a very precise result, play your video with precise timestamps (e.g. mpv's --osd-fractions option) and open your audio in an audio editor (e.g. Audacity), then find the microsecond when your clapperboard video/audio occurred and subtract one from the other.

Measuring audio noise

Your hardware will create a small amount of audio noise in your recording. If you want to remove this later, you'll need to measure it for each configuration you use (e.g. different input sources or laptop charging/unplugged). You can remove this and other noise sources during transcoding - for now you just need to record a sample.

You'll need a recording of about half a second of your system in a resting state. This can be a silent TV channel or paused tape, but if you're using composite or S-video connectors, the easiest thing is probably just to record a few moments of blackness before pressing play.

Choosing formats

Your encoding formats needs to encode in real-time and lose as little information as possible. Even if you plan to throw that information away during transcoding, an accurate initial recording will give you more freedom when the time comes. For example, your muxer format should support variable frame rates so you can measure your video's frame rate. Once you have that information, you could use it to calculate an accurate transcoding frame rate or to cut out sections where your VCR delivered the wrong number of frames - either way the information is useful even though it was lost from the final video.

Transcoding a usable video

The video you recorded should accurately represent your source video, but will probably be a large file, be a noisy experience, and might not even play in some programs. You need to transcode it to a more usable format.

Cleaning audio

Any analogue recording will contain a certain amount of background noise. Cleaning noise is optional, and you'll always be able to produce a slightly better result if you spend a little longer on it, so this section will just introduce enough theory to get you started. Audacity's equalizer and noise reduction effect are good places to start experimenting.

The major noise sources are:

• your audio codec might throw away sound it thinks you won't hear in order to reduce file size
• your recording system will produce a small, consistent amount of noise based on its various electrical and mechanical components
• VHS format limitations cause static at high and low frequencies, depending on the VCR's settings
• imperfections in tape recording and playback produce noise that differs between recordings and even between scenes

A lossless audio format (e.g. WAV or FLAC) should ensure your original encoding doesn't produce any extra noise. Even if you transcode to a format like MP3 that throws information away, a lossless original ensures there's only one lot of noise in the result.

The primary means of reducing noise is the frequency-based noise gate, which blocks some frequencies and passes others. High-pass and low-pass filters pass noise above or below a certain frequency, and can be combined into band-pass or even multi-band filters. The rest of this section discusses how to build a series of noise gates for your audio.

Identify noise from your recording system by recording the sound of a paused tape or silent television channel for a few seconds. If possible, use the near-silence at the start of your recording so you can guarantee your sample matches your current hardware configuration. Use this baseline recording as a noise profile which your software uses to build a multi-band noise gate. You can apply that noise gate to the whole recording, and to other recordings with the same hardware that don't have a usable sample.

Identify VHS format limitations by searching online for information based on your TV norm (NTSC, PAL or SECAM), your recording quality (normal or Hi-Fi) and your VHS play mode (short- or long-play). Wikipedia's discussion of VHS audio recording is a good place to start. If you're able to find the information, gate your recordings with high-pass and low-pass filters that only allow frequencies within the range your tape actually records. For example, a long-play recording of a PAL tape will produce static below 100Hz and above 4kHz so you should gate your recording to only pass audio in the 100Hz-4000Hz range. If you can't find the information, you can determine it experimentally by trying out different filters to see what sounds right - your system probably produces static below about 10Hz or 100Hz and above about 4kHz or 12kHz, so try high- and low-pass filters in those ranges until you stop hearing background noise. If you don't remove this noise source, the next step will do a reasonable job of guessing it for you anyway.

Identify imperfections in recording and playback by watching the video and looking for periods of silence. You only need half a second of background noise to generate a profile, but the number of profiles is up to you. Some people grab one profile for a whole recording, others combine clips into averaged noise profiles, others cut audio into scenes and de-noise each in turn. At a minimum, tapes with multiple recordings should be split up and each one de-noised separately - a tape containing a TV program recorded in LP mode in one VCR followed by a home video recorded in SP in another VCR will produce two very different noise profiles, even if played back all in one go.

It's good to apply filters in the right order (system profile, then VHS limits, then recording profiles), but beyond that noise reduction is very subjective. For example, intelligent noise reduction tends to remove more noise in quiet periods but less when it would risk losing signal, which can sound a snare drum being brushed whenever someone speaks. But dumb filters silence the same frequencies at all times, which can make everything sound muffled.

You can run your audio through as many gates as you like, and even repeat the same filter several times. If you use a noise reduction profile, you can even get different results from different programs (see for example this comparison of sox and Audacity's algorithms). There's no right answer but there's always a better result if you spend a bit more time, so you'll need to decide for yourself when the result is good enough.

Cleaning video

Much like audio, you can spend as long as you like cleaning your video. But whereas audio cleaning tends to be about doing one thing really well (separating out frequencies of signal and noise), video cleaning tends to be about getting decent results in different circumstances. For example, you might want to just remove the overscan lines at the bottom of a VHS recording, denoise a video slightly to reduce file size, or aggressively remove grains to make a low-quality recording watchable. FFmpeg's video filter list is a good place to start, but here are a few things you should know.

Some programs need video to have a specified aspect ratio. If you simply crop out the ugly overscan lines at the bottom of your video, some programs may refuse to play your video. Instead you should mask the area with blackness. In ffmpeg, you would use a crop filter followed by a pad filter to create the appropriate result.

Analogue video is interlaced, essentially interleaving two consecutive video frames within each image. This confuses video filters that compare neighbouring pixels (e.g. to look for bright grains in dark areas of the screen), so you should deinterleave the frames before using such filters, then interleave them again afterwards. For example, an ffmpeg filter chain might start with il=d:d:d and end with il=i:i:i. You can create a video without re-interleaving the video to better understand how this process works.

Choosing formats

Your transcoding format needs to be small and compatible with whatever software you will use to play it back. If you can't find accurate information about your players, create a short test video and try it on your system. Your video codec may well have options to reduce file size at the cost of encoding time, so you may want to leave your computer transcoding overnight to get the best file size.