V4L channel script

The channel scripts

Introduction

This simple script, with its companion timestamper, is designed to capture television programs on a regular schedule and store the recordings in a dated directory tree along with the closed captioning text files. Because the scripts are so elementary, they are very easy to debug, and the script has been running flawlessly for more than a year now.

The script is designed to handle several capture cards for simultaneous capture, and jobs are scheduled per card in a cron job or issued manually for a single program. Files are saved to the first drive with "tv" in its name and at least 1GB of free space.

The use of bash is elementary and can surely be improved on.

Hardware

My setup is an amd64 motherboard with on-board NICs, sound, and video, and five PCI slots (Gigabyte K8NS Ultra-939), so there's lots of room for grabber cards. It currently uses three LifeView FlyVideo3000FM NTSC and two LifeView FlyTV_Platinum (saa7134) cards. I chose them because I can get sound directly off the cards and don't need a patch cable for each one to a sound card.

The three capture cards are loaded as follows:

saa7134 card=2,54,2,54,2 tuner=39,54,39,54,17 video_nr=1,2,3,4,5 vbi_nr=1,2,3,4,5 radio_nr=1,2,3,4,5 disable_ir=1,1,1,1,1

The CPU is an AMD Athlon 64 dual core 4600+ (939); each mencoder session uses about 20% of the CPU. Results from five simultaneous capture is now excellent. The script assumes you get the signal off the air; cf. the "us-bcast" parameter.

I would like to switch to pcHDTV-3000, but these cards don't have closed captioning support yet.

I capture to SATA drives; the board has support for four of these. The OS is installed on a PATA drive.

Software

The OS is Debian sid with a 2.6.18-1 kernel. I generally leave the OS alone, but occasionally update some of the applications.

Mencoder is used to grab and compress the footage, using DIVX and mp3lame and an avi wrapper (suggestions for improvements are welcome). I'm currently running Christian Marillat's unofficial 1:1.0-pre8cvs20061002-0.0 Debian package.

We don't don't view the footage on the capture machine, but use NFS to view the captured files on other machines. On OSX, you can view the avi files with VLC; Windows is untested, but the files should play after installing the DIVX codec. Users can be given access to the recording machine's crontab. On X11, kcron provides a user-friendly interface. The status of the capture machine can usefully be monitored with gnome-system-monitor.

For listings, we use Freeguide.

Capture script

#!/bin/bash
#
# /usr/local/bin/channel
#
# Script to record analog television programs to .avi files, using mencoder, and
# the associated closed captioning in a separate identically named .txt file
#
# You need to include /usr/local/bin in cron's PATH (see kcron's Variables)
# Use the bash interpreter -- /bin/bash, not /bin/sh
#
# Usage: channel <channel number> <duration> <title> <capture card number>
#
# For instance,
#
#       channel 28, 30min, "BBC World News", 1
#
# Input parameter rules:
#    * commas are optional
#    * give duration (recording time) in minutes 
#    * the word "min" (for minutes) is optional
#    * a space signals a new parameter
#    * enclose multi-word titles in quotes
#    * don't use apostrophes in titles -- messes up searching of captured text
#    * the default card number is 1
#
# Changelog
#    2005-01-12: added $! to terminate ntsc-cc and channel-timestamp
#    2005-04-15: added $DEV input parameter to handle multiple tv cards
#    2005-04-23: added aumix -d /dev/mixer1 -1 R for sound off the saa
#    2005-05-13: added mixer device number 
#    2005-05-16: new directories are created with 775
#    2005-05-22: added busy test
#    2005-10-07: use zvbi-ntsc-cc, soon in libzvbi 0.2.17
#    2006-02-15: removed colon from filename (FIL=)
#    2006-08-18: added drive assignment routine (checks for space)
#    2006-09-04: switched from oss to alsa (2.6.17 kernel)
#    2006-09-14: check /dev/vbi$DEV instead of /dev/dsp$DEV in busy loop
#                (you could use /dev/video$DEV)
#    2006-09-20: added some messages at the end, using 24-hour time
#    2006-09-24: added XDS information to closed captioning
#
# ***************************************************************************

# First some checks triggered by manual mode

if [ "$1" = "" ] ; then
 echo "Use news <channel number> <recording time in minutes>";
 echo "Please give a channel! Valid channels are 2, 4, 5, 7, 9, 11, 13, and 28.";
 exit
fi

if [ "$2" = "" ] ; then
 echo "Use news <channel number> <recording time in minutes>";
 echo "Please give a recording length in minutes!";
 exit
fi

if [ "$3" = "" ] ; then
 echo "Use news <number> <duration> <title>";
 echo "Please give a title to the recording!";
 exit
fi

# Define the video and VBI (vertical blanking interval) device to use
# Default is /dev/video1 and /dev/vbi1
if [ "$4" = "" ] ;
 then DEV=1
 else DEV=$4
fi

# Define variable DRV to contain the first tv drive with more than 1% free
for i in $(df | cut -d'/' -f4 | grep tv | sort); do
  if [ "$(df /$i | grep $i | cut -b 52-54)" -lt 99 ]
      then DRV=$i
      echo "Using drive" $DRV "with" "$(df -h /$i | grep $i | cut -b 35-37)GB free"
      break
  fi
done

# Define variable DIR to contain nested directories for
#  yyyy, yyyy-mm, and yyyy-mm-dd (today's date)
DIR=/$DRV/$(date +%Y)/$(date +%Y-%m)/$(date +%F)

# Create the directory (and any missing parents) if needed
if test ! -d $DIR
  then mkdir -p -m 775 $DIR
fi

# Strip the comma, if any, from the channel name
CH=${1%,*}

# Set the channel (current user needs X11 access)
# You can also set this in mencoder with tv://$CH or channel=$CH
v4lctl -c /dev/video$DEV setchannel $CH

# Convert channel number to name for the base file name
case $CH in
  2) CH=KCBS;;
  4) CH=KNBC;;
  5) CH=KTLA-WB;;
  7) CH=KABC;;
  9) CH=KCAL;;
 11) CH=KTTV-FOX;;
 13) CH=KCOP-UPN;;
 28) CH=KCET;;
esac

# Strip the {min,|min|,} if it's there, from the duration
TIM=${2%m*}; TIM=${TIM%,*}

# Strip the comma, if any, from the title, and convert spaces to underscores
TIT=_`echo ${3%,*} | sed -e "s/ /_/g"`

# Define the file name (escape underscore before $) and print it to screen
FIL=$(date +%F_%H%M)\_$CH$TIT; echo Preparing to record $FIL

# Check if the capture device is busy
BUSY="Capture card $DEV is busy -- will try for another five minutes \n"
START=$(date +%s); TARGET=$[$START + $[5*60]]

while [ "$(sudo lsof /dev/vbi$DEV)" != "" ]; do
  if [ $TARGET -gt $(date +%s) ]
    then echo -ne $BUSY
      sleep 5; BUSY=""
    else echo "Recording of $FIL aborted -- capture card $DEV is not available"
      exit
  fi
done

# Convert minutes to seconds and adjust down five seconds for the hardware to reset
TIM=$[$[TIM*60]-5]

# Adjust the duration down by the delay added by the busy check, exit if no time left
TIM=$[TIM-$[$(date +%s) -$START]]; if [ $TIM -lt 1 ]; then exit; fi

# Start the timestamp script and background
channel-timestamp $CH $TIT $DIR $FIL &

# Get the PID for the timestamper
TMSP=$!

# Start cc capture and background the process
zvbi-ntsc-cc -d /dev/vbi$DEV -cpx >> $DIR/$FIL.txt &

# Get the PID for the closed captioning
NTSC=$!

# Restore default sound sttings and open up the sound channel on the capture card 
/usr/sbin/alsactl restore $DEV

# Set any additional values (none currently needed)
#v4lctl -c /dev/video0 color "50%"

# Record the footage (note this assumes direct audio)

# Use this to control the bitrate -- default is 800
#lavc -lavcopts vbitrate=1800 -oac mp3lame -lameopts cbr:br=128 -endpos $TIM -o \

echo "Initiating recording of $FIL from channel $1"
echo "on capture card $DEV, at $(date +%T), duration $TIM seconds"

mencoder -tv driver=v4l2:device=/dev/video$DEV:fps=30000/1001:chanlist=us-bcast:\
audiorate=32000:adevice=hw.$DEV:alsa:input=0:amode=1:normid=4 -ffourcc DIVX -ovc \
lavc -oac mp3lame -lameopts cbr:br=128 -endpos $TIM -o $DIR/$FIL.avi tv:// > /dev/null

# Stamp the file with receipts to verify capture success
echo -e "\n"
ls -l $DIR/$FIL.*
echo -e "\nCompleted capture on $(date +%A\ %d\ %B\ %Y) at $(date +%T)"
echo -e "Video and audio   file size" $(ls -sh $DIR/$FIL.avi | cut -d' ' -f1)
echo -e "Closed captioning file size" $(ls -sh $DIR/$FIL.txt | cut -d' ' -f1) "\n"

# Terminate the closed captioning and the timestamper
kill $NTSC $TMSP

# EOF

The only part of the script that gave me trouble over a prolonged period was the transition between recordings. Sometimes it takes the kernel several seconds to activate a card, and if the recording length doesn't compensate down, the next scheduled recording bumps into a busy device. However, this problem appears to have been fully solved in the current busy script, which waits for up to five minutes and shortens the recording time by the time waited.

I also tended to lose some programs when a drive was full, since I often missed the exact moment, so I've added some lines to look for a drive with space. I was filling drives at the rate of 4GB a day and had room for four drives, so it would take a year to fill a set of 400GB drives.

Variant: capture to H.264 using alsa

If you want to capture to H.264 instead, and use the new saa7134-alsa module for getting sound, you can use this (and drop aumix from the script, as it doesn't appear to understand alsa's device notation):

mencoder -tv driver=v4l2:device=/dev/video$DEV:fps=30000/1001:chanlist=us-bcast:\
audiorate=32000:alsa:adevice=hw.$DEV:input=0:amode=1:normid=4:width=576:height=480 \
-ovc x264 -x264encopts bitrate=500:bframes=2:subq=2:me=2:frameref=2 \
-oac mp3lame -lameopts cbr:br=64 -endpos $TIM -o $DIR/$FIL.avi tv:// > /dev/null

Time-stamping script

The main script calls this little routine for timestamping the closed captioning files. this is useful only if you want to set up a searchable database of text.

#!/bin/bash
#
# /usr/local/bin/channel-timestamp
#
# This shell script is used to add timestamps to closed captioning
# It is called by the main recording script, /usr/local/bin/channel.
#
# Add -xv to the top line to get verbose messages
#
# Changelog:
#  2005-01-12: The parent script now handles child process terminations
#  2005-01-13: Timestamps could use XDS data -- start with the date and
#              include title (not implemented)
#
# *****************************************************************

# Write the first timestamp
echo $(date +%F_%H:%M:%S)_$1$2 >> $3/$4.txt

# Write a timestamp every ten seconds
while true
 do
  sleep 10
  echo $(date +%F_%H:%M:%S)_$1$2 >> $3/$4.txt
done

# EOF

Crontabs

While programs can be captured manually using the channel script, I use cron for scheduled recordings.

Here are some examples of crontab usage:

# Folders to search for program files.
PATH=/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
# Email output to specified account.
MAILTO=tna
# KCAL 9 News at 9pm
0 21 * * 6      channel 9, 60min, "News at 9pm", 1
# KNBC 4 Meet the Press
0 8 * * 7       channel 4, 60min, "Meet the Press", 2
# KABC 7 World News Tonight
30 17 * * 7     channel 7, 30min, "World News Tonight", 1
# KCBS 2 60 Minutes
0 19 * * 7      channel 2, 60min, "60 Minutes", 1
# Fox 11 Morning News at 6 AM/ Good Day LA
0 6 * * 1,2,3,4,5       channel 11, 60min, "Morning News at 6", 1
# KCET 28 BBC World News
30 5 * * 1,2,3,4,5      channel 28, 30min, "BBC World News", 3

You can edit crontab with kcron or with the "crontab -e" command. It's also possible to use scripts to add jobs:

 crontab -l >> newjob | crontab newjob

If you have a complex crontab you don't want to lose, back it up with "crontab -l > crontab-$(date +%F_%H:%M)".