Video for Linux Two - Driver Writer's Guide

Bill Dirks - December 23, 1999
Modified by Luc Gallant - November 9, 2005 (lucgallant(_AT_)gmail_com)

More content to be added later, when more time is available.

Introduction

Why have this document when there is an API spec that covers all the functions? Because the API spec only covers the user-mode calls, and there are aspects of V4L2 that are only relevant to kernel-mode code. The purpose of this document is to describe how V4L2 driver modules hook into the system, document the V4L2 kernel-mode API, discuss driver implementation issues, and provide any other additional documentation applicable to driver writers.

This document assumes the reader is familiar with the V4L2 user-mode API spec, and the relevant topics of Linux device driver programming. Linux Device Drivers by Allessandro Rubini, ISBN 1-56592-292-1 seems to be a standard reference.
 
 

V4L2 Drivers & videodev

V4L2 is a two-layer driver system. The top layer is the videodev module. When videodev initializes, it registers as char major device 81, and registers its set of char driver method functions. All V4L2 drivers are really clients of videodev, and videodev calls the client drivers through V4L2 driver method functions. When a V4L2 driver initializes, it registers each device it will handle with videodev by passing to videodev a structure which contains the V4L2 driver methods, a minor device number, and a couple other details. The V4L2 methods are very similar to the regular Linux char driver methods, but have different parameters specially for V4L2 drivers. When an application makes a driver call, control passes first to the videodev method, which translates the file and/or inode structure pointers into the corresponding V4L2 structure pointer, and calls the V4L2 driver's method. Thus videodev acts as a thin shell around V4L2 drivers. Videodev is implemented as a module, and all V4L2 drivers are also modules.

The videodev module also includes a set of helper functions which V4L2 driver writers may find useful. For example, some simple queue management, translating a struct file * to the corresponding V4L2 structure, memory mapping helper functions and so on.
 
 

Driver Registration and Method Calls

When a driver initializes, it enumerates the devices in the system which it is going to handle, and for each one it fills in a separate struct v4l2_device structure and passes a pointer to the structure to video_register_device() . video_register_device() will make sure the minor number is available, and call the class_device_register method of the struct video_device. class_device_register is passed the dev member contained in the video_device pointer that was passed to video_register_device() . If class_device_register succeeds then the registration is complete. Videodev will store the pointer in an internal table, and call the method functions as the driver is accessed by applications.

Before calling v4l2_register_device() the driver must fill in the name, type, and minor fields, and the open method. The other methods and fields are optional. The type field uses the same values as the type field in struct v4l2_capability, which are the V4L2_TYPE_* symbols. It is possible to change method fields after registration. When the driver unloads, it uses its release() to call video_unregister_device() with the device pointer.
 

struct video_device

struct file_operations

Design Note: Typically, drivers will internally define a much larger device structure that begins with a struct video_device, followed by all the driver and device state information associated with the particular device. For example, in cx88.h, the bigger struct which contains the struct video_device (s) is a struct cx8800_dev. Proper references to these internal structures must be made in order to access them.

Minor Device Numbers

For V4L2 devices, the minor numbers have to be given as module parameters. Drivers have to declare the parameter variables and export them. Fill in the minor field of each struct video_device, and call video_register_device(). The registration will fail if the minor is already in use. In order to get the first available minor number, simply send -1 as the minor number to the video_register_device().

V4L2 Driver Method Calls

The following is a list of the V4L2 driver method calls, each with a brief description and any relevant notes.

Method: int open(struct inode *inod, struct file *file)

Called when a file descriptor is opened on the device file. Open is passed the stuct inode (ie. /dev/videoX, where videoX is the inode) which is related to the device in question. Using this, the driver can figure out which minor the device is. Then, using this minor number, the driver can access the proper device and retrieve it's private data,which usually contains information specific to that device, and that needs to be passed around frequently. The private data, which is like a file handler, is assigned to the private_data variable of the struct *file pointer which entered the open function. V4L2 devices can be opened more than once simultaneously, either by the same application, or by more than one application. After open, all the other method calls access the private data of the device via the struct *file pointer which comes in to all the functions after open. The return value is returned to the application.

Method: void release(struct inode *inod, struct file *file)

Called when the file descriptor is being closed. Uses the private value of the struct *file in order to get the minor number, which is then used to call video_device_unregister() It then cleans up any parameters it was using. There is no return value. Videodev always returns 0 to the application.

Method: ssize_t read(struct file *file, char *data, size_t count, loff_t *offset)

Called when the application calls read(). Uses the private value of the struct *file in order to get the minor number. Incoming parameters are the char *data (buffer), size_t count (the number of bytes of data requested), and loff_t *offset (offset of the file to start at). The return value is the number of bytes read or a negative error code. The return value is returned to the application.

Method: ssize_t write(struct file *file, char *data, size_t count, loff_t *offset)

Called when the application calls write(). Not many applications use this function. Uses the private value of the struct *file in order to get the minor number. assigned in open() Incoming parameters are the char *data (buffer), size_t count (the number of bytes of data to write), and loff_t *offset (offset of the file to start at).The return value is the number of bytes written or a negative error code. The return value is returned to the application.

Method: int ioctl(struct inode *inod, struct file *file, unsigned int cmd, void *arg)

Called when the application calls ioctl(). Uses the private value of the struct *file in order to get the minor number assigned in the corresponding open(). Return zero for success or a negative error code. The return value is returned to the application. This function basically uses a large case statement on unsigned int cmd in order to determine what command to execute. Drivers should return -EINVAL for ioctl codes in the API which the driver does not support. Drivers should return -ENOIOCTLCMD for all unknown ioctl codes.

Note that the arg parameter will point to kernel memory, and is not the orginal user-space pointer passed in from the application. Videodev will handle the copy_from_user() and copy_to_user() work as needed. Exceptions are if the argument buffer is larger than V4L2_MAX_IOCTL_SIZE (256 bytes), and the clips parameter of a VIDIOC_S_WIN ioctl.

It is important for drivers to handle VIDIOC_QUERYCTRL and to return -EINVAL if the id field is out of range. If your driver does not have any private controls, you need to return -EINVAL if the id field is V4L2_CID_PRIVATE_BASE. It is not necessary to fill in the v4l2_queryctrl category or group fields for pre-defined control IDs; videodev will do it. Videodev fills them in before the driver is called, so the driver can change the default values if desired.

For backward compatibility with applications written for the original API, videodev will translate the old ioctl codes and parameters into V4L2 ioctl calls. To do this, the default case statement would call v4l1_compat_translate_ioctl(), which translates the command into v4l2 calls. All except VIDIOCGBUF can be done. (In other words, VIDIOCGBUF must be implemented in the driver by itself)

Method: int mmap(struct file *file, struct vm_area_struct *vma)

Called when the application calls mmap(). Uses the private value of the struct *file in order to get the minor number assigned in the corresponding open(). Passed the file *file and the virtual memory area structure. The V4L2 specification defines a general-purpose protocol for using mmap() for buffer mapping that all V4L2 devices need to follow. Return 0 for success or a negative error code. The return value is returned to the application. Return -EINVAL for an invalid parameter, or -ENODEV if the buffer could not be allocated.

The V4L2 memory mapping protocol allows for many buffers to be mapped individually, with the buffers organized into independent buffer sets. In the v4l2_buffer structure, the type field identifies the set the buffer belongs to and the index field identifies the individual buffer. It is not possible to pass a v4l2_buffer structure directly to the mmap method, so we use the vma->vm_offset field to identify the buffer to be mapped. You will fill in the offset field of video_buffer with a number that is unique for each buffer that can be mapped. The application will pass that number back to you in the vma->vm_offset field when it calls mmap(). Your mmap handler will look at vm_offset to determine which buffer is to be mapped. You need to invent a numbering scheme for your buffers for this purpose. Warning: the offset numbers should not be based on the length of the buffers or else the kernel might merge vma's leading to trouble. It is recommended that buffers located in system RAM be allocated in mmap. Buffers in system memory should be zero-filled when they are allocated so applications cannot gain access to any data that may be remaining in that memory.

The system does not automatically call the open method (which opens the buffer), so you need to call it from mmap. You should free a system memory buffer in close. Remember to set the V4L2_BUF_FLAG_MAPPED flag when a buffer is mapped, and clear it when it is unmapped. It is not possible to fail a munmap(), so if it is called while streaming is active, you should safely stop the hardware and dequeue the buffer before freeing it.

The videodev mmap wrapper will set up vma->vm_file before calling the V4L2 driver. Videodev will also increment the file usage count after the V4L2 driver's mmap method returns if the mmap is successful. The driver should not increment the file usage count.

The dimensions of a buffer are typically dependent on a data format setting. For example, video capture buffers are dependent on the capture image format. Typically, you would fail an attempt to change the format while a buffer is mapped. An exception is that for v4l backward compatibility, a VIDIOC_S_FMT call is permitted while video capture buffers are mapped, although such a call could not set a format that was larger than would fit in the buffers. The format can be changed after all buffers dependent on the format have been unmapped. After the format has been changed, it is required for the application to call VIDIOC_REQBUFS again, therefore you must make changing the format also undo the effect of VIDIOC_REQBUFS.

If you want to let applications map additional types of memory buffers or memory areas, or have multiple streams of data, this interface is trivially extensible: just add more values for the buffer type field. Values starting from V4L2_BUF_TYPE_PRIVATE_BASE are reserved for driver-private buffer types.

Method: int poll(struct file *file, poll_table *table)

Called when the application calls select(). Uses the private value of the struct *file in order to get the minor number assigned in the corresponding open(). Passed the struct file pointer and the poll_table pointer. The latter two are needed for the poll_wait() system call. Return the regular POLL* flags according to the status. Return POLLERR for a no-I/O open.
 

V4L2 Video Capture Drivers

Some topics specific to video capture drivers.

Non-capturing opens. It is highly recommended to support non-capturing opens on capture devices. This way we can have universal video control panel applications that can run alongside a capturing program. VIDIOC_S_PARM (obsolete soon) can't change the capturemode or timeperframe.

Video for Linux Backward Compatibility. The original API used mmap differently from V4L2. In the original API, a single mmap call mapped all the buffers contiguously in a single, long virtual memory area. It's not possible for the backward compatibility code in videodev to make a V4L2 driver function like a v4l driver in this regard, so if you want your driver to work well with applications that use the v4l API mmap you need to support mapping all the requested buffers in one mmap call. It's only used for video capture, V4L2_BUF_TYPE_CAPTURE, buffers. The compatibility layer will set the flag V4L2_BUF_REQ_CONTIG in the v4l2_requestbuffers type field in the VIDIOC_REQBUFS ioctl. If you support this contiguous mapping of buffers then make sure the flag is set in the type field when VIDIOC_REQBUFS returns. The number of buffers requested this way will most likey be two. Note: the CONTIG flag should not be set in v4l2_buffer structures, or anywhere else-- only v4l2_requestbuffers. On the next mmap call, allocate a block of memory big enough for all the buffers. The vma_close call will free and unmap all the buffers at once. The vm_offset field will be zero when contiguous mapping is used. The v4l2cap.c(old, kernel 2.2) sample capture driver illustrates all of this.

If you don't support the V4L2_BUF_REQ_CONTIG flag, a v4l application will only be able to map one capture buffer, which may not be enough to get the capture performance the user expects.

V4L2 Kernel-Mode API

Videodev exports an assorment of functions for V4L2 drivers. All the functions and data structures have the v4l2_ prefix. The following sections list each function with a brief description.

Driver Registration and Device Structures

int video_register_device(struct v4l2_device *v)

Register the device structure pointer, and assign a minor device number.

void video_unregister_device(struct v4l2_device *v)

Unregister the specified device.

Memory Management

struct page *v4l2_vmalloc_to_page(void * vmalloc_addr)

Returns the page corresponding to a pointer into a buffer allocated with vmalloc(). Useful in the vma->vm_ops->nopage method when memory mapping a buffer allocated with vmalloc(). The return value from this function is suitable for use as the return value from vma->vm_ops->nopage.

Queue Management

[This guide previously used v4l2_q type functions, but these never really existed. For queues relating to video buffers, the video-buf.h file contains queues which most drivers use today.]

Master Clock Functions & Operations on Time Stamps

[This guide previously used v4l2_masterclock and v4l2_timestamp type functions, but these never really existed. For all operations relating to the clock or timestamps, the linux system functions should be used.

Operations on Video Standard Structures

These are the recommended functions for constructing and parsing v4l2_standard structures. If you are only supporting established standard formats then these functions can do almost all the work, and you will probably never need to touch any v4l2_standard fields in your code besides id, or maybe transmission if you deal with RF signals.

unsigned int v4l2_video_std_fps (struct v4l2_standard *vs)

Convert the framerate.numerator and framerate.denominator fields to a frames-per-second value, rounded to the nearest whole number.

int v4l2_video_std_construct(struct v4l2_standard vs, int id, char* name)

Takes the id and name values and fills in all the fields of the v4l2_standard structure. Returns the id value back on success, or a negative error code.