Buffers

Timecodes

A buffer contains data exchanged by application and driver using one of the Streaming I/O methods. In the multi-planar API, the data is held in planes, while the buffer structure acts as a container for the planes. Only pointers to buffers (planes) are exchanged, the data itself is not copied. These pointers, together with meta-information like timestamps or field parity, are stored in a struct v4l2_buffer, argument to the VIDIOC_QUERYBUF, VIDIOC_QBUF and VIDIOC_DQBUF ioctl. In the multi-planar API, some plane-specific members of struct v4l2_buffer, such as pointers and sizes for each plane, are stored in struct v4l2_plane instead. In that case, struct v4l2_buffer contains an array of plane structures.

Dequeued video buffers come with timestamps. The driver decides at which part of the frame and with which clock the timestamp is taken. Please see flags in the masks V4L2_BUF_FLAG_TIMESTAMP_MASK and V4L2_BUF_FLAG_TSTAMP_SRC_MASK in Table 3.4, “Buffer Flags”. These flags are always valid and constant across all buffers during the whole video stream. Changes in these flags may take place as a side effect of VIDIOC_S_INPUT or VIDIOC_S_OUTPUT however. The V4L2_BUF_FLAG_TIMESTAMP_COPY timestamp type which is used by e.g. on mem-to-mem devices is an exception to the rule: the timestamp source flags are copied from the OUTPUT video buffer to the CAPTURE video buffer.

Table 3.1. struct v4l2_buffer

__u32index Number of the buffer, set by the application except when calling VIDIOC_DQBUF, then it is set by the driver. This field can range from zero to the number of buffers allocated with the VIDIOC_REQBUFS ioctl (struct v4l2_requestbuffers count), plus any buffers allocated with VIDIOC_CREATE_BUFS minus one.
__u32type Type of the buffer, same as struct v4l2_format type or struct v4l2_requestbuffers type, set by the application. See Table 3.3, “enum v4l2_buf_type”
__u32bytesused The number of bytes occupied by the data in the buffer. It depends on the negotiated data format and may change with each buffer for compressed variable size data like JPEG images. Drivers must set this field when type refers to an input stream, applications when it refers to an output stream.
__u32flags Flags set by the application or driver, see Table 3.4, “Buffer Flags”.
__u32field Indicates the field order of the image in the buffer, see Table 3.9, “enum v4l2_field”. This field is not used when the buffer contains VBI data. Drivers must set it when type refers to an input stream, applications when it refers to an output stream.
struct timevaltimestamp 

For input streams this is time when the first data byte was captured, as returned by the clock_gettime() function for the relevant clock id; see V4L2_BUF_FLAG_TIMESTAMP_* in Table 3.4, “Buffer Flags”. For output streams the driver stores the time at which the last data byte was actually sent out in the timestamp field. This permits applications to monitor the drift between the video and system clock. For output streams that use V4L2_BUF_FLAG_TIMESTAMP_COPY the application has to fill in the timestamp which will be copied by the driver to the capture stream.

struct v4l2_timecodetimecode When type is V4L2_BUF_TYPE_VIDEO_CAPTURE and the V4L2_BUF_FLAG_TIMECODE flag is set in flags, this structure contains a frame timecode. In V4L2_FIELD_ALTERNATE mode the top and bottom field contain the same timecode. Timecodes are intended to help video editing and are typically recorded on video tapes, but also embedded in compressed formats like MPEG. This field is independent of the timestamp and sequence fields.
__u32sequence Set by the driver, counting the frames (not fields!) in sequence. This field is set for both input and output devices.

In V4L2_FIELD_ALTERNATE mode the top and bottom field have the same sequence number. The count starts at zero and includes dropped or repeated frames. A dropped frame was received by an input device but could not be stored due to lack of free buffer space. A repeated frame was displayed again by an output device because the application did not pass new data in time.

Note this may count the frames received e.g. over USB, without taking into account the frames dropped by the remote hardware due to limited compression throughput or bus bandwidth. These devices identify by not enumerating any video standards, see the section called “Video Standards”.

__u32memory This field must be set by applications and/or drivers in accordance with the selected I/O method. See Table 3.5, “enum v4l2_memory”
unionm  
 __u32offsetFor the single-planar API and when memory is V4L2_MEMORY_MMAP this is the offset of the buffer from the start of the device memory. The value is returned by the driver and apart of serving as parameter to the mmap() function not useful for applications. See the section called “Streaming I/O (Memory Mapping)” for details
 unsigned longuserptrFor the single-planar API and when memory is V4L2_MEMORY_USERPTR this is a pointer to the buffer (casted to unsigned long type) in virtual memory, set by the application. See the section called “Streaming I/O (User Pointers)” for details.
 struct v4l2_plane*planesWhen using the multi-planar API, contains a userspace pointer to an array of struct v4l2_plane. The size of the array should be put in the length field of this v4l2_buffer structure.
 intfdFor the single-plane API and when memory is V4L2_MEMORY_DMABUF this is the file descriptor associated with a DMABUF buffer.
__u32length Size of the buffer (not the payload) in bytes for the single-planar API. This is set by the driver based on the calls to VIDIOC_REQBUFS and/or VIDIOC_CREATE_BUFS. For the multi-planar API the application sets this to the number of elements in the planes array. The driver will fill in the actual number of valid elements in that array.
__u32reserved2 A place holder for future extensions. Applications should set this to 0.
__u32reserved A place holder for future extensions. Applications should set this to 0.

Table 3.2. struct v4l2_plane

__u32bytesused The number of bytes occupied by data in the plane (its payload). Drivers must set this field when type refers to an input stream, applications when it refers to an output stream.
__u32length Size in bytes of the plane (not its payload). This is set by the driver based on the calls to VIDIOC_REQBUFS and/or VIDIOC_CREATE_BUFS.
unionm  
 __u32mem_offsetWhen the memory type in the containing struct v4l2_buffer is V4L2_MEMORY_MMAP, this is the value that should be passed to mmap(), similar to the offset field in struct v4l2_buffer.
 unsigned longuserptrWhen the memory type in the containing struct v4l2_buffer is V4L2_MEMORY_USERPTR, this is a userspace pointer to the memory allocated for this plane by an application.
 intfdWhen the memory type in the containing struct v4l2_buffer is V4L2_MEMORY_DMABUF, this is a file descriptor associated with a DMABUF buffer, similar to the fd field in struct v4l2_buffer.
__u32data_offset Offset in bytes to video data in the plane. Drivers must set this field when type refers to an input stream, applications when it refers to an output stream.
__u32reserved[11] Reserved for future use. Should be zeroed by an application.

Table 3.3. enum v4l2_buf_type

V4L2_BUF_TYPE_VIDEO_CAPTURE1Buffer of a single-planar video capture stream, see the section called “Video Capture Interface”.
V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE 9Buffer of a multi-planar video capture stream, see the section called “Video Capture Interface”.
V4L2_BUF_TYPE_VIDEO_OUTPUT2Buffer of a single-planar video output stream, see the section called “Video Output Interface”.
V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE 10Buffer of a multi-planar video output stream, see the section called “Video Output Interface”.
V4L2_BUF_TYPE_VIDEO_OVERLAY3Buffer for video overlay, see the section called “Video Overlay Interface”.
V4L2_BUF_TYPE_VBI_CAPTURE4Buffer of a raw VBI capture stream, see the section called “Raw VBI Data Interface”.
V4L2_BUF_TYPE_VBI_OUTPUT5Buffer of a raw VBI output stream, see the section called “Raw VBI Data Interface”.
V4L2_BUF_TYPE_SLICED_VBI_CAPTURE6Buffer of a sliced VBI capture stream, see the section called “Sliced VBI Data Interface”.
V4L2_BUF_TYPE_SLICED_VBI_OUTPUT7Buffer of a sliced VBI output stream, see the section called “Sliced VBI Data Interface”.
V4L2_BUF_TYPE_VIDEO_OUTPUT_OVERLAY8Buffer for video output overlay (OSD), see the section called “Video Output Overlay Interface”.
V4L2_BUF_TYPE_SDR_CAPTURE11Buffer for Software Defined Radio (SDR), see the section called “Software Defined Radio Interface (SDR)”.

Table 3.4. Buffer Flags

V4L2_BUF_FLAG_MAPPED0x00000001The buffer resides in device memory and has been mapped into the application's address space, see the section called “Streaming I/O (Memory Mapping)” for details. Drivers set or clear this flag when the VIDIOC_QUERYBUF, VIDIOC_QBUF or VIDIOC_DQBUF ioctl is called. Set by the driver.
V4L2_BUF_FLAG_QUEUED0x00000002Internally drivers maintain two buffer queues, an incoming and outgoing queue. When this flag is set, the buffer is currently on the incoming queue. It automatically moves to the outgoing queue after the buffer has been filled (capture devices) or displayed (output devices). Drivers set or clear this flag when the VIDIOC_QUERYBUF ioctl is called. After (successful) calling the VIDIOC_QBUF ioctl it is always set and after VIDIOC_DQBUF always cleared.
V4L2_BUF_FLAG_DONE0x00000004When this flag is set, the buffer is currently on the outgoing queue, ready to be dequeued from the driver. Drivers set or clear this flag when the VIDIOC_QUERYBUF ioctl is called. After calling the VIDIOC_QBUF or VIDIOC_DQBUF it is always cleared. Of course a buffer cannot be on both queues at the same time, the V4L2_BUF_FLAG_QUEUED and V4L2_BUF_FLAG_DONE flag are mutually exclusive. They can be both cleared however, then the buffer is in "dequeued" state, in the application domain so to say.
V4L2_BUF_FLAG_ERROR0x00000040When this flag is set, the buffer has been dequeued successfully, although the data might have been corrupted. This is recoverable, streaming may continue as normal and the buffer may be reused normally. Drivers set this flag when the VIDIOC_DQBUF ioctl is called.
V4L2_BUF_FLAG_KEYFRAME0x00000008Drivers set or clear this flag when calling the VIDIOC_DQBUF ioctl. It may be set by video capture devices when the buffer contains a compressed image which is a key frame (or field), i. e. can be decompressed on its own. Also know as an I-frame. Applications can set this bit when type refers to an output stream.
V4L2_BUF_FLAG_PFRAME0x00000010Similar to V4L2_BUF_FLAG_KEYFRAME this flags predicted frames or fields which contain only differences to a previous key frame. Applications can set this bit when type refers to an output stream.
V4L2_BUF_FLAG_BFRAME0x00000020Similar to V4L2_BUF_FLAG_KEYFRAME this flags a bi-directional predicted frame or field which contains only the differences between the current frame and both the preceding and following key frames to specify its content. Applications can set this bit when type refers to an output stream.
V4L2_BUF_FLAG_TIMECODE0x00000100The timecode field is valid. Drivers set or clear this flag when the VIDIOC_DQBUF ioctl is called. Applications can set this bit and the corresponding timecode structure when type refers to an output stream.
V4L2_BUF_FLAG_PREPARED0x00000400The buffer has been prepared for I/O and can be queued by the application. Drivers set or clear this flag when the VIDIOC_QUERYBUF, VIDIOC_PREPARE_BUF, VIDIOC_QBUF or VIDIOC_DQBUF ioctl is called.
V4L2_BUF_FLAG_NO_CACHE_INVALIDATE0x00000800Caches do not have to be invalidated for this buffer. Typically applications shall use this flag if the data captured in the buffer is not going to be touched by the CPU, instead the buffer will, probably, be passed on to a DMA-capable hardware unit for further processing or output.
V4L2_BUF_FLAG_NO_CACHE_CLEAN0x00001000Caches do not have to be cleaned for this buffer. Typically applications shall use this flag for output buffers if the data in this buffer has not been created by the CPU but by some DMA-capable unit, in which case caches have not been used.
V4L2_BUF_FLAG_TIMESTAMP_MASK0x0000e000Mask for timestamp types below. To test the timestamp type, mask out bits not belonging to timestamp type by performing a logical and operation with buffer flags and timestamp mask.
V4L2_BUF_FLAG_TIMESTAMP_UNKNOWN0x00000000Unknown timestamp type. This type is used by drivers before Linux 3.9 and may be either monotonic (see below) or realtime (wall clock). Monotonic clock has been favoured in embedded systems whereas most of the drivers use the realtime clock. Either kinds of timestamps are available in user space via clock_gettime(2) using clock IDs CLOCK_MONOTONIC and CLOCK_REALTIME, respectively.
V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC0x00002000The buffer timestamp has been taken from the CLOCK_MONOTONIC clock. To access the same clock outside V4L2, use clock_gettime(2) .
V4L2_BUF_FLAG_TIMESTAMP_COPY0x00004000The CAPTURE buffer timestamp has been taken from the corresponding OUTPUT buffer. This flag applies only to mem2mem devices.
V4L2_BUF_FLAG_TSTAMP_SRC_MASK0x00070000Mask for timestamp sources below. The timestamp source defines the point of time the timestamp is taken in relation to the frame. Logical 'and' operation between the flags field and V4L2_BUF_FLAG_TSTAMP_SRC_MASK produces the value of the timestamp source. Applications must set the timestamp source when type refers to an output stream and V4L2_BUF_FLAG_TIMESTAMP_COPY is set.
V4L2_BUF_FLAG_TSTAMP_SRC_EOF0x00000000End Of Frame. The buffer timestamp has been taken when the last pixel of the frame has been received or the last pixel of the frame has been transmitted. In practice, software generated timestamps will typically be read from the clock a small amount of time after the last pixel has been received or transmitten, depending on the system and other activity in it.
V4L2_BUF_FLAG_TSTAMP_SRC_SOE0x00010000Start Of Exposure. The buffer timestamp has been taken when the exposure of the frame has begun. This is only valid for the V4L2_BUF_TYPE_VIDEO_CAPTURE buffer type.

Table 3.5. enum v4l2_memory

V4L2_MEMORY_MMAP1The buffer is used for memory mapping I/O.
V4L2_MEMORY_USERPTR2The buffer is used for user pointer I/O.
V4L2_MEMORY_OVERLAY3[to do]
V4L2_MEMORY_DMABUF4The buffer is used for DMA shared buffer I/O.

Timecodes

The v4l2_timecode structure is designed to hold a [SMPTE 12M] or similar timecode. (struct timeval timestamps are stored in struct v4l2_buffer field timestamp.)

Table 3.6. struct v4l2_timecode

__u32typeFrame rate the timecodes are based on, see Table 3.7, “Timecode Types”.
__u32flagsTimecode flags, see Table 3.8, “Timecode Flags”.
__u8framesFrame count, 0 ... 23/24/29/49/59, depending on the type of timecode.
__u8secondsSeconds count, 0 ... 59. This is a binary, not BCD number.
__u8minutesMinutes count, 0 ... 59. This is a binary, not BCD number.
__u8hoursHours count, 0 ... 29. This is a binary, not BCD number.
__u8userbits[4]The "user group" bits from the timecode.

Table 3.7. Timecode Types

V4L2_TC_TYPE_24FPS124 frames per second, i. e. film.
V4L2_TC_TYPE_25FPS225 frames per second, i. e. PAL or SECAM video.
V4L2_TC_TYPE_30FPS330 frames per second, i. e. NTSC video.
V4L2_TC_TYPE_50FPS4 
V4L2_TC_TYPE_60FPS5 

Table 3.8. Timecode Flags

V4L2_TC_FLAG_DROPFRAME0x0001Indicates "drop frame" semantics for counting frames in 29.97 fps material. When set, frame numbers 0 and 1 at the start of each minute, except minutes 0, 10, 20, 30, 40, 50 are omitted from the count.
V4L2_TC_FLAG_COLORFRAME0x0002The "color frame" flag.
V4L2_TC_USERBITS_field0x000CField mask for the "binary group flags".
V4L2_TC_USERBITS_USERDEFINED0x0000Unspecified format.
V4L2_TC_USERBITS_8BITCHARS0x00088-bit ISO characters.