Media Controller devices

Media Controller devices
Prev	Chapter 6. Media Devices	Next

struct media_entity_notify — Media Entity Notify
struct media_device — Media device
media_entity_enum_init — Initialise an entity enumeration
media_device_init — Initializes a media device element
media_device_cleanup — Cleanups a media device element
__media_device_register — Registers a media device element
media_device_unregister — Unregisters a media device element
media_device_register_entity — registers a media entity inside a previously registered media device.
media_device_register_entity_notify — Registers a media entity_notify callback
media_device_unregister_entity_notify — Unregister a media entity notify callback
media_device_get_devres — get media device as device resource creates if one doesn't exist
media_device_find_devres — find media device as device resource
media_device_pci_init create and initialize a struct media_device from a PCI device.
__media_device_usb_init create and initialize a struct media_device from a PCI device.
struct media_file_operations — Media device file operations
struct media_devnode — Media device node
media_devnode_register — register a media device node
media_devnode_unregister_prepare — clear the media device node register bit
media_devnode_unregister — unregister a media device node
media_devnode_data returns a pointer to the media_devnode
media_devnode_is_registered returns true if media_devnode is registered; false otherwise.
enum media_gobj_type — type of a graph object
struct media_gobj — Define a graph object.
struct media_entity_enum — An enumeration of media entities.
struct media_entity_graph — Media graph traversal state
struct media_link — A link object part of a media graph.
struct media_pad — A media pad graph object.
struct media_entity_operations — Media entity operations
enum media_entity_type — Media entity type
struct media_entity — A media entity graph object.
struct media_interface — A media interface graph object.
struct media_intf_devnode — A media interface via a device node.
media_entity_id — return the media entity graph object id
media_type — return the media object type
media_id — return the media object ID
media_gobj_gen_id — encapsulates type and ID on at the object ID
is_media_entity_v4l2_video_device — Check if the entity is a video_device
is_media_entity_v4l2_subdev — Check if the entity is a v4l2_subdev
__media_entity_enum_init — Initialise an entity enumeration
media_entity_enum_cleanup — Release resources of an entity enumeration
media_entity_enum_zero — Clear the entire enum
media_entity_enum_set — Mark a single entity in the enum
media_entity_enum_clear — Unmark a single entity in the enum
media_entity_enum_test — Test whether the entity is marked
media_entity_enum_test_and_set — Test whether the entity is marked, and mark it
media_entity_enum_empty — Test whether the entire enum is empty
media_entity_enum_intersects — Test whether two enums intersect
media_gobj_create — Initialize a graph object
media_gobj_destroy — Stop using a graph object on a media device
media_entity_pads_init — Initialize the entity pads
media_entity_cleanup — free resources associated with an entity
media_create_pad_link — creates a link between two entities.
media_create_pad_links — creates a link between two entities.
media_entity_remove_links — remove all links associated with an entity
__media_entity_setup_link — Configure a media link without locking
media_entity_setup_link — changes the link flags properties in runtime
media_entity_find_link — Find a link between two pads
media_entity_remote_pad — Find the pad at the remote end of a link
media_entity_get — Get a reference to the parent module
media_entity_graph_walk_cleanup — Release resources used by graph walk.
media_entity_put — Release the reference to the parent module
media_entity_graph_walk_start — Start walking the media graph at a given entity
media_entity_graph_walk_next — Get the next entity in the graph
media_entity_pipeline_start — Mark a pipeline as streaming
__media_entity_pipeline_start — Mark a pipeline as streaming
media_entity_pipeline_stop — Mark a pipeline as not streaming
__media_entity_pipeline_stop — Mark a pipeline as not streaming
media_devnode_create — creates and initializes a device node interface
media_devnode_remove — removes a device node interface
media_create_intf_link — creates a link between an entity and an interface
__media_remove_intf_link — remove a single interface link
media_remove_intf_link — remove a single interface link
__media_remove_intf_links — remove all links associated with an interface
media_remove_intf_links — remove all links associated with an interface

The media controller userspace API is documented in DocBook format in Documentation/DocBook/media/v4l/media-controller.xml. This document focus on the kernel-side implementation of the media framework.

* Abstract media device model:

Discovering a device internal topology, and configuring it at runtime, is one of the goals of the media framework. To achieve this, hardware devices are modelled as an oriented graph of building blocks called entities connected through pads.

An entity is a basic media hardware building block. It can correspond to a large variety of logical blocks such as physical hardware devices (CMOS sensor for instance), logical hardware devices (a building block in a System-on-Chip image processing pipeline), DMA channels or physical connectors.

A pad is a connection endpoint through which an entity can interact with other entities. Data (not restricted to video) produced by an entity flows from the entity's output to one or more entity inputs. Pads should not be confused with physical pins at chip boundaries.

A link is a point-to-point oriented connection between two pads, either on the same entity or on different entities. Data flows from a source pad to a sink pad.

* Media device:

A media device is represented by a struct media_device instance, defined in include/media/media-device.h. Allocation of the structure is handled by the media device driver, usually by embedding the media_device instance in a larger driver-specific structure.

Drivers register media device instances by calling __media_device_register via the macro media_device_register and unregistered by calling media_device_unregister.

* Entities, pads and links:

- Entities

Entities are represented by a struct media_entity instance, defined in include/media/media-entity.h. The structure is usually embedded into a higher-level structure, such as a v4l2_subdev or video_device instance, although drivers can allocate entities directly.

Drivers initialize entity pads by calling media_entity_pads_init.

Drivers register entities with a media device by calling media_device_register_entity and unregistred by calling media_device_unregister_entity.

- Interfaces

Interfaces are represented by a struct media_interface instance, defined in include/media/media-entity.h. Currently, only one type of interface is defined: a device node. Such interfaces are represented by a struct media_intf_devnode.

Drivers initialize and create device node interfaces by calling media_devnode_create and remove them by calling: media_devnode_remove.

- Pads

Pads are represented by a struct media_pad instance, defined in include/media/media-entity.h. Each entity stores its pads in a pads array managed by the entity driver. Drivers usually embed the array in a driver-specific structure.

Pads are identified by their entity and their 0-based index in the pads array. Both information are stored in the media_pad structure, making the media_pad pointer the canonical way to store and pass link references.

Pads have flags that describe the pad capabilities and state.

MEDIA_PAD_FL_SINK indicates that the pad supports sinking data. MEDIA_PAD_FL_SOURCE indicates that the pad supports sourcing data.

NOTE: One and only one of MEDIA_PAD_FL_SINK and MEDIA_PAD_FL_SOURCE must be set for each pad.

- Links

Links are represented by a struct media_link instance, defined in include/media/media-entity.h. There are two types of links:

1. pad to pad links:

Associate two entities via their PADs. Each entity has a list that points to all links originating at or targeting any of its pads. A given link is thus stored twice, once in the source entity and once in the target entity.

Drivers create pad to pad links by calling: media_create_pad_link and remove with media_entity_remove_links.

2. interface to entity links:

Associate one interface to a Link.

Drivers create interface to entity links by calling: media_create_intf_link and remove with media_remove_intf_links.

NOTE:

Links can only be created after having both ends already created.

Links have flags that describe the link capabilities and state. The valid values are described at media_create_pad_link and media_create_intf_link.

Graph traversal:

The media framework provides APIs to iterate over entities in a graph.

To iterate over all entities belonging to a media device, drivers can use the media_device_for_each_entity macro, defined in include/media/media-device.h.

struct media_entity *entity;

media_device_for_each_entity(entity, mdev) { // entity will point to each entity in turn ... }

Drivers might also need to iterate over all entities in a graph that can be reached only through enabled links starting at a given entity. The media framework provides a depth-first graph traversal API for that purpose.

Note that graphs with cycles (whether directed or undirected) are *NOT* supported by the graph traversal API. To prevent infinite loops, the graph traversal code limits the maximum depth to MEDIA_ENTITY_ENUM_MAX_DEPTH, currently defined as 16.

Drivers initiate a graph traversal by calling media_entity_graph_walk_start

The graph structure, provided by the caller, is initialized to start graph traversal at the given entity.

Drivers can then retrieve the next entity by calling media_entity_graph_walk_next

When the graph traversal is complete the function will return NULL.

Graph traversal can be interrupted at any moment. No cleanup function call is required and the graph structure can be freed normally.

Helper functions can be used to find a link between two given pads, or a pad connected to another pad through an enabled link media_entity_find_link and media_entity_remote_pad

Use count and power handling:

Due to the wide differences between drivers regarding power management needs, the media controller does not implement power management. However, the media_entity structure includes a use_count field that media drivers can use to track the number of users of every entity for power management needs.

The media_entity.use_count field is owned by media drivers and must not be touched by entity drivers. Access to the field must be protected by the media_device.graph_mutex lock.

Links setup:

Link properties can be modified at runtime by calling media_entity_setup_link

Pipelines and media streams:

When starting streaming, drivers must notify all entities in the pipeline to prevent link states from being modified during streaming by calling media_entity_pipeline_start.

The function will mark all entities connected to the given entity through enabled links, either directly or indirectly, as streaming.

The media_pipeline instance pointed to by the pipe argument will be stored in every entity in the pipeline. Drivers should embed the media_pipeline structure in higher-level pipeline structures and can then access the pipeline through the media_entity pipe field.

Calls to media_entity_pipeline_start can be nested. The pipeline pointer must be identical for all nested calls to the function.

media_entity_pipeline_start may return an error. In that case, it will clean up any of the changes it did by itself.

When stopping the stream, drivers must notify the entities with media_entity_pipeline_stop.

If multiple calls to media_entity_pipeline_start have been made the same number of media_entity_pipeline_stop calls are required to stop streaming. The media_entity pipe field is reset to NULL on the last nested stop call.

Link configuration will fail with -EBUSY by default if either end of the link is a streaming entity. Links that can be modified while streaming must be marked with the MEDIA_LNK_FL_DYNAMIC flag.

If other operations need to be disallowed on streaming entities (such as changing entities configuration parameters) drivers can explicitly check the media_entity stream_count field to find out if an entity is streaming. This operation must be done with the media_device graph_mutex held.

Link validation:

Link validation is performed by media_entity_pipeline_start for any entity which has sink pads in the pipeline. The media_entity.link_validate() callback is used for that purpose. In link_validate() callback, entity driver should check that the properties of the source pad of the connected entity and its own sink pad match. It is up to the type of the entity (and in the end, the properties of the hardware) what matching actually means.

Subsystems should facilitate link validation by providing subsystem specific helper functions to provide easy access for commonly needed information, and in the end provide a way to use driver-specific callbacks.