media-workshop

media-workshop@linuxtv.org

113 discussions

media_build fails on Gentoo 4.2 machine....
by Tamer Higazi 24 Oct '15

24 Oct '15

Hi people! I am on a gentoo machine, after updating my kernel, linux header and restarted my machine I wanted to compile "media_build". What could this be ?! That I get this error ? http://pastebin.com/raw.php?i=Ubt5qhP0 output: /storage/downloads/media_build/v4l/videobuf2-core.c:29:30: fatal error: trace/events/vb2.h: No such file or directory #include <trace/events/vb2.h> ^ compilation terminated. scripts/Makefile.build:264: die Regel für Ziel „/storage/downloads/media_build/v4l/videobuf2-core.o“ scheiterte make[3]: *** [/storage/downloads/media_build/v4l/videobuf2-core.o] Fehler 1 Makefile:1386: recipe for target '_module_/storage/downloads/media_build/v4l' failed make[2]: *** [_module_/storage/downloads/media_build/v4l] Error 2 make[2]: Leaving directory '/usr/src/linux-4.2.3-gentoo' Makefile:51: recipe for target 'default' failed make[1]: *** [default] Error 2 make[1]: Leaving directory '/storage/downloads/media_build/v4l' Makefile:26: recipe for target 'all' failed make: *** [all] Error 2 build failed at ./build line 491. thanks! best, Tamer

1 0

media_build fails on Gentoo 4.2 machine....
by Tamer Higazi 24 Oct '15

24 Oct '15

1 0

Draft agenda for the KS media workshop
by Hans Verkuil 19 Oct '15

19 Oct '15

Hi all, This is the draft version of the agenda for the media workshop in Korea. I am assuming that all presenters are subscribed to this mailinglist. Note: the MC topics are tentative. I've placed them at the end, since they might change. They also have a tendency to take over the discussion, so let's do the smaller topics first :-) Mauro, do you know which room we'll be in? I am assuming that we start at 9 am and continue until approx. 5:30 pm. Do you know when the lunch break is? I guessed at the duration of the various topics: please check and let me know if you need more/less time! ----------------------------------------------------------- Draft agenda for the media workshop October 26, 2015, Seoul, South Korea Please be on time! We start at 9 am, we have a full agenda, so we want to make good use of the time. 9:00-9:15: Get laptops hooked up, the projector running, wireless configured, your coffee cup filled, etc. etc. :-) 9:15-9:30: Hans: Introduction 9:30-10:15: Pawel: Update on codec APIs 10:15-10:30: Mauro: kABI documentation improvements 10:30-11:00: Junghak: new uAPI for DVB to use streaming I/O 11:00-11:15: Break 11:15-11:30: Hans: Proposal for adding support for HW colorspace conversion. 11:30-12:30: Hans: Should we replace v4l2_buffer with a new struct that is y2038-safe and that can be cleaned up and extended? (Think proper support for HW timestamps, etc.). See: https://patchwork.linuxtv.org/patch/31413/ And perhaps use this redesign to add support for multiple streams over one video node? I saw an example where the HW combines two streams into one, basically an m2m device, but today you'd need three nodes: two out and one capture in order to implement this. 12:30-13:30: Lunch 13:30-14:00: Hans: Status update of the CEC framework patch series. Discuss if we should really integrate with the RC core (not convinced about this). 14:00-14:30: Hans: Should the v4l2 core just call try_format to validate the format passed in by VIDIOC_CREATE_BUFFERS? Few drivers validate the format and if they do they usually do only partial validation. It's unclear what should happen here. Always calling try_format would also help the vb2 core code. MC discussions: very tentative, and I'm just listing the topics without scheduling them and in no particular order: Mauro: Discuss dynamic subtree removal Hans: How to implement VIDIOC_SUBDEV_QUERYCAP? Shuah: ENT_F_IO appears to be too generic, requiring additional steps to determine what kind of IO function it is really is. I propose preserving and allowing enough uniqueness to enable drivers to tag the functions when they create them. For example, if AUDIO functions are all tagged ENT_F_IO, bridge driver has to determine what kind of IO it really is. This requires finding the Interface entity associated with the function entity or comparing entity name strings. Instead, if a function can be tagged as Audio Capture when it gets created, these additional steps aren't necessary. Shuah: Discuss snd-usb-audio control interface mapping to media. snd-usb-audio creates a control device file which could be exposed in a media graph. In some cases, it is a simple mixer control. Expose ENT_F_MIXER entity with an associated interface node MEDIA_INTF_T_ALSA_CONTROL 17:15-17:30: Hans: Discuss future improvements to the workshop format. Right now (as is clear from this schedule) the discussions are very technical, which makes it hard for non-core developers to attend. Perhaps we should do a split format: a half or full day of status updates, discussions to improve application integration, followed by core-developer-only discussions. Regards, Hans

4 8

V2: Draft agenda for the KS media workshop
by Hans Verkuil 19 Oct '15

19 Oct '15

Hi all, This is the second draft version of the agenda for the media workshop in South Korea. I am assuming that all presenters are subscribed to this mailinglist. Note: the MC topics are still tentative. I've placed them at the end, since they might change. They also have a tendency to take over the discussion, so let's do the smaller topics first :-) I am assuming that we start at 9 am and continue until 5:45 pm (that's when the Korea Linux Forum ends), I've also synced the lunch with the KLF. I guessed at the duration of the various topics: please check and let me know if you need more/less time! Changes since v1: - added poll topic - made time to attend Shuah's ALSA/MC talk ----------------------------------------------------------- Draft agenda for the media workshop October 26, 2015, Seoul, South Korea Room: TBD Please be on time! We start at 9 am, we have a full agenda, so we want to make good use of the time. 9:00-9:15: Get laptops hooked up, the projector running, wireless configured, your coffee cup filled, etc. etc. :-) 9:15-9:30: Hans: Introduction 9:30-10:15: Pawel: Update on codec APIs 10:15-10:30: Mauro: kABI documentation improvements 10:30-11:00: Junghak: new uAPI for DVB to use streaming I/O 11:00-12:15: Break Allows time for attending Shuah's talk: http://korealinuxforum2015.sched.org/event/f59175cc49f866dbc9bebc36f86fe663… 12:15-13:00: Hans: Should we replace v4l2_buffer with a new struct that is y2038-safe and that can be cleaned up and extended? (Think proper support for HW timestamps, etc.). See: https://patchwork.linuxtv.org/patch/31413/ And perhaps use this redesign to add support for multiple streams over one video node? I saw an example where the HW combines two streams into one, basically an m2m device, but today you'd need three nodes: two out and one capture in order to implement this. 13:00-14:30: Lunch 14:30-14:45: Hans: Proposal for adding support for HW colorspace conversion. 14:45-15:00: Hans: poll() behavior for output streams. 15:00-15:15: Hans: Status update of the CEC framework patch series. 15:15-16:00: Hans: Should the v4l2 core just call try_format to validate the format passed in by VIDIOC_CREATE_BUFFERS? Few drivers validate the format and if they do they usually do only partial validation. It's unclear what should happen here. Always calling try_format would also help the vb2 core code. Also discuss DELETE_BUFFERS. 16:00-16:15: Break 16:15-16:30: Mauro: Status update MC discussions: very tentative, and I'm just listing the topics without scheduling them and in no particular order: Mauro: Discuss dynamic subtree removal. The workshop might be too soon to discuss this. Hans: How to implement VIDIOC_SUBDEV_QUERYCAP? (15 min) Shuah: ENT_F_IO appears to be too generic, requiring additional steps to determine what kind of IO function it is really is. I propose preserving and allowing enough uniqueness to enable drivers to tag the functions when they create them. For example, if AUDIO functions are all tagged ENT_F_IO, bridge driver has to determine what kind of IO it really is. This requires finding the Interface entity associated with the function entity or comparing entity name strings. Instead, if a function can be tagged as Audio Capture when it gets created, these additional steps aren't necessary. I estimate 15 min for this topic if we need this. Shuah: Discuss snd-usb-audio control interface mapping to media. snd-usb-audio creates a control device file which could be exposed in a media graph. In some cases, it is a simple mixer control. Expose ENT_F_MIXER entity with an associated interface node MEDIA_INTF_T_ALSA_CONTROL 17:30-17:45: Hans: Discuss future improvements to the workshop format. Right now (as is clear from this schedule) the discussions are very technical, which makes it hard for non-core developers to attend. Perhaps we should do a split format: a half or full day of status updates, discussions to improve application integration, followed by core-developer-only discussions. Regards, Hans _______________________________________________ media-workshop mailing list media-workshop(a)linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/media-workshop

1 1

Topics for workshop
by Hans Verkuil 07 Oct '15

07 Oct '15

Hi all, I have three topics I'd like to discuss at the upcoming workshop: - CEC framework: v9 was recently posted and other than some documentation work I believe it is ready to be merged. I would like to give a presentation on what CEC is and the design of the framework. Est. 30 minutes + questions. - VIDIOC_SUBDEV_QUERYCAP: we never got an agreement about this, so I'd like to discuss it. Est. 30 minutes (but since it touches on the MC, it could just as easily be several hours!) - Support for HW colorspace conversion. I've been working on this on-and-off and I'd like to discuss how to integrate this in V4L2. Est. 30-45 minutes. Regards, Hans

7 18

Today's discussion on IRC at 6pm Helsinki's time
by Mauro Carvalho Chehab 02 Sep '15

02 Sep '15

Hi, As scheduled on IRC, we'll have a followup discussion today in order to address some details that weren't discussed at the MC workshop, and popped up during the last patch series and its review. Items for discussion on today's meeting: 1) Links namespace: From my answer to a Hans review: > > +#define MEDIA_NEW_LNK_FL_ENABLED MEDIA_LNK_FL_ENABLED > > +#define MEDIA_NEW_LNK_FL_IMMUTABLE MEDIA_LNK_FL_IMMUTABLE > > +#define MEDIA_NEW_LNK_FL_DYNAMIC MEDIA_NEW_FL_DYNAMIC > > +#define MEDIA_NEW_LNK_FL_INTERFACE_LINK (1 << 3) Yes, this can be deduced from the type of the objects inside the link. I guess I added it because of some comment from your media.h RFC proposal. Right now, I'm using it at the application to better represent the graph elements: http://git.linuxtv.org/cgit.cgi/mchehab/experimental-v4l-utils.git/tree/con… http://git.linuxtv.org/cgit.cgi/mchehab/experimental-v4l-utils.git/tree/con… But it could, instead, be doing something like: if media_type(link->gobj1.id == MEDIA_GRAPH_PAD) link_is_pad = true; else link_is_pad = false; Btw, I'm using the same type for both data and interface links, as I don't see any reason why to differentiate internally: they all share the same linked list at mdev and the same object ID range. There are two questions here: 1) should we have a flag to distinguish interface links? Not really required, but it makes a little easier for userspace and doesn't consume more than a bit. 2) should we create a new namespace for the V2 API, or should we keep the same namespace? 2) Entities namespace: From Hans review: > > Entities should do something, and just saying 'V4L2_VIDEO' doesn't really convey > that meaning. It is also very easy to confuse with INTF_T_V4L_* types. BTW, we > should decide whether V4L2 or V4L is used here (interfaces now use V4L, entities > V4L2). Since entities already use V4L2, I think the interface defines should > use V4L2 as well. Yes, agreed. We actually need to discuss a little more about namespacing and do some additional renaming stuff. For example, calling a tuner entity as MEDIA_ENT_T_V4L2_SUBDEV_TUNER is wrong, because the tuner can be used only at the DVB side. So, both V4L2 and SUBDEV prefixes there are wrong. Yet, this should be under the V4L2_SUBDEV range to avoid breaking userspace. > > > + > > +/* V4L2 Sub-device entities */ > > +#define MEDIA_ENT_T_V4L2_SUBDEV_SENSOR (MEDIA_ENT_T_V4L2_SUBDEV_BASE + 1) > > +#define MEDIA_ENT_T_V4L2_SUBDEV_FLASH (MEDIA_ENT_T_V4L2_SUBDEV_BASE + 2) > > +#define MEDIA_ENT_T_V4L2_SUBDEV_LENS (MEDIA_ENT_T_V4L2_SUBDEV_BASE + 3) > > + /* A converter of analogue video to its digital representation. */ > > +#define MEDIA_ENT_T_V4L2_SUBDEV_DECODER (MEDIA_ENT_T_V4L2_SUBDEV_BASE + 4) > > + /* Tuner entity is actually both V4L2 and DVB subdev */ > > +#define MEDIA_ENT_T_V4L2_SUBDEV_TUNER (MEDIA_ENT_T_V4L2_SUBDEV_BASE + 5) > > + > > +/* DVB entities */ > > +#define MEDIA_ENT_T_DVB_DEMOD (MEDIA_ENT_T_DVB_BASE) > > After changing DVB_BASE to 0, change this to DVB_BASE + 1, and adjust the other DVB > entity types accordingly. > > This keeps the base defines consistent (i.e. the lower 16 bits are always 0). > > It surprised me when reading this patch, so I'm probably not the only one. This is another thing for discussion: keeping the MEDIA_ENT_T_foo_BASE unused opens space for API abuse. There are several entities at OMAP3 driver, for example, that keeps entity->type undefined. So, they got whatever is the default at v4l2-device.c (before this series: MEDIA_ENT_T_V4L2_SUBDEV, after that: MEDIA_ENT_T_V4L2_SUBDEV_UNKNOWN): $ media-ctl --print-t|grep -B1 Unknown - entity 1: OMAP3 ISP CCP2 (2 pads, 2 links) type V4L2 subdev subtype Unknown flags 0 -- - entity 3: OMAP3 ISP CSI2a (2 pads, 2 links) type V4L2 subdev subtype Unknown flags 0 -- - entity 5: OMAP3 ISP CCDC (3 pads, 8 links) type V4L2 subdev subtype Unknown flags 0 -- - entity 7: OMAP3 ISP preview (2 pads, 4 links) type V4L2 subdev subtype Unknown flags 0 -- - entity 10: OMAP3 ISP resizer (2 pads, 4 links) type V4L2 subdev subtype Unknown flags 0 -- - entity 13: OMAP3 ISP AEWB (1 pad, 1 link) type V4L2 subdev subtype Unknown flags 0 -- - entity 14: OMAP3 ISP AF (1 pad, 1 link) type V4L2 subdev subtype Unknown flags 0 -- - entity 15: OMAP3 ISP histogram (1 pad, 1 link) type V4L2 subdev subtype Unknown flags 0 I guess all the above entities are processing units, so they should have, instead, some type like: MEDIA_ENT_T_V4L2_SUBDEV_PROCESSING Or, even some of the above would actually deserves to have an specific type, like: MEDIA_ENT_T_V4L2_SUBDEV_HISTOGRAM MEDIA_ENT_T_V4L2_SUBDEV_RESIZER ... Let's try to find some time to discuss the entities namespace on IRC during this week. In summary, now that we've split interfaces from entities, MEDIA_ENT_T_V4L2_SUBDEV_* namespace is messy/wrong because: - entities should represent hardware, and not Linux implementation. So, calling them as "V4L2" is wrong, because V4L2 is not a hardware block. - entities are always part of a device. So, calling them as SUBDEV is also wrong. - some entities like a tuner can be used outside V4L2 (and even outside drivers/media). The current patch series removes the type/subtype usage inside the Kernelspace, allowing further patches to fix that mess. Yet, I opted to keep the messy/wrong namespaces there for now, as fixing this can be done on a separate patch series, and won't interfere at the implementation of the new G_TOPOLOGY ioctl or at the implementation of the properties API. However, we should fix this earlier than latter. Back on May, 7 I did a proposal on a 18 patches series: http://www.spinics.net/lists/linux-media/msg89521.html This is a little outdated, but the basic idea there was to split entities by their function. So, we would have things like: MEDIA_ENT_T_CAMERA_foo - for camera stuff (sensor, flash, ...) MEDIA_ENT_T_RF_foo - for things like tuner, LNBf, LNA MEDIA_ENT_T_DTV_foo - for digital TV stuff MEDIA_ENT_T_ATV_foo - for analog TV stuff MEDIA_ENT_T_MEDIA_foo - for common media blocks (scaler, resizer) On my patch series, I actually created numberspace bases, just to keep things easier to maintain: #define MEDIA_ENT_T_DVB_BASE 0x00000000 #define MEDIA_ENT_T_V4L2_BASE 0x00010000 #define MEDIA_ENT_T_V4L2_SUBDEV_BASE 0x00020000 #define MEDIA_ENT_T_CONNECTOR_BASE 0x00030000 but, after thinking a little bit more about that, I think that this may not be a good idea, as userspace programs should not rely on it, and the things at MEDIA_ENT_T_V4L2_SUBDEV_BASE are under that "messy" state: #define MEDIA_ENT_T_V4L2_SUBDEV_UNKNOWN MEDIA_ENT_T_V4L2_SUBDEV_BASE #define MEDIA_ENT_T_V4L2_SUBDEV_SENSOR (MEDIA_ENT_T_V4L2_SUBDEV_BASE + 1) #define MEDIA_ENT_T_V4L2_SUBDEV_FLASH (MEDIA_ENT_T_V4L2_SUBDEV_BASE + 2) #define MEDIA_ENT_T_V4L2_SUBDEV_LENS (MEDIA_ENT_T_V4L2_SUBDEV_BASE + 3) #define MEDIA_ENT_T_V4L2_SUBDEV_DECODER (MEDIA_ENT_T_V4L2_SUBDEV_BASE + 4) #define MEDIA_ENT_T_V4L2_SUBDEV_TUNER (MEDIA_ENT_T_V4L2_SUBDEV_BASE + 5) on the above: - SENSOR/FLASH/LENS are camera related entities - DECODER is an analog TV entity - TUNER is a RF entity shared with analog, digital, software radio and, in the future could even be used on wifi drivers, if we ever need the media controller to be used there A side discussion that we can do if we have time is how to fix those "type V4L2 subdev subtype Unknown" subdevs at OMAP3 (and likely other drivers) that are just putting all non-camera subdevices under a generic umbrella (MEDIA_ENT_T_V4L2_SUBDEV). Regards, Mauro

1 0

[ANNOUNCE] Media Workshop at the Kernel Summit 2015 in Korea
by Mauro Carvalho Chehab 01 Sep '15

01 Sep '15

Hi, As we keep being very active on the development of the Linux Kernel Media subsystem, we'll be doing another workshop this year. This will be the first time we'll be doing it in Asia, so it is a great opportunity for the ones that are there to join us and participate, in order to improve the Linux Kernel support for A/V stream capture, webcam and analog/digital TV. As usual, we'll be using the media-workshop(a)linuxtv.org Mailing List for specific discussions about that, so the ones interested on participate are requested to subscribe it. This time, we'll be doing something different than what we've done on the last years, and use something that other subsystems like ALSA are doing for a while: we'll be using two google docs to track the proposals and the interested parties. If you're interested in participate and wants to be invited for the event, please put your data at this document: https://docs.google.com/spreadsheets/d/1YecCAUo0L1R_i3jPg3S-A5iqX7BYAd2c_Vf… As this is the first time we'll be using Google Docs, I kindly request you to also send an email for media-workshop(a)linuxtv.org if you have any troubles adding things there and/or a backup measure. If you have any proposal for themes, please add it here: https://docs.google.com/spreadsheets/d/1F67VcdhkDlVfRpx-GPhe61adZh81Q26q--_… It is useful if you could also send an email to the workshop ML in order to better explain the theme proposal. Hoping to see you there soon! Mauro

3 3

[DRAFT] Report of the Media Controller workshop in Espoo, Finland – July, 29-31 2015
by Mauro Carvalho Chehab 17 Aug '15

17 Aug '15

Hi, This is the first draft of the Media Controller Workshop report. It was complex to do this report based on the notes we took, as some notes were incomplete. I did my best, but I'm sure some items need some review, in special the presentations. Feel free to review and send me your comments for me to add in the final MC workshop report and send to the main mailing list. Sakari, I don't have the email of the two guys from Intel that participated on our first day. Could you please send it for me to complete at the report? Thanks! Mauro -- Report of the Media Controller workshop in Espoo, Finland – July, 29-31 2015 This is the first workshop dedicated to the Media Controller. We actually did a v4l summit in 2010, also in Finland, that stablished the current foundation for the media controller, and was meant to satisfy the needs of properly reporting the pipelines on the SoC used by smartphones. The focus of this year's workshop is to clarify the kernel→userspace interfaces and extend the Media Controller to be used on other subsystems that need to represent graphs, like DVB, ALSA and IIO. Main event attendees: Antti Laakso Hans Verkuil<hansverk(a)cisco.com> Lars-Peter Clausen <lars(a)metafoo.de> Laurent Pinchart <laurent.pinchart(a)ideasonboard.com> Mauro Chehab <mchehab(a)osg.samsung.com> Sakari Ailus <sakari.ailus(a)iki.fi> Shuah Khan <shuahkh(a)osg.samsung.com> Tuukka Toivonen MC videoconf attendees: Dong-Joo Kim <dj98.kim(a)samsung.com> Geunyoung Kim <nenggun.kim(a)samsung.com> In-Ki Dae <inki.dae(a)samsung.com> Junghak Sung <jh1009.sung(a)samsung.com> Minsong Kim <ms17.kim(a)samsung.com> Ohin Kwon <rany.kwon(a)samsung.com> Seung-Woo Kim <sw0312.kim(a)samsung.com> EDITOR'S NOTE: Usually, we produce the summit reports using a cronologic approach. This time, I opted to move the content of the second day to the beginning of the report, as they're basically several presentations we had covering the Media Controller needs. I opted to put the presentations first, and then put the discussions we had on July 29 and 31 together. 1) Presentations made on July, 30 1.1. Videobuf2 redesign to allow it to support DVB Presentation made by Samsung via videoconf about the scheduled changes that are planned in order to split the videobuf2 core from the V4L2 specific part, in order to allow the videobuf2 core to be used also for DVB 1.2. DVB pipelines Presentation made by Samsung via videoconf about the pipelines. Basically, the pipelines on real usecases for embedded devices would look like: tuner 1 -> -> demux -> Video tuner 2 -> cam 1 -> demux -> Audio tuner 3 -> cam 2 -> demux -> Data -> V4L2 -> ALSA 1.3. ASoC http://linuxtv.org/downloads/presentations/mc_ws_2015/sound_media_controlle… Generic DT machine driver can support around half of DT based devices Not all required data available in ACPI, so ACPI based devices typically have their own drivers DAPM does power management in ASoC much like the current omap3isp driver implementation does Walks the graph Supports large numbers of entities and cyclical graphs Contains ASoC dependencies, but the core of the algorithm is generic Stack based, iterative version in works MC has generic graph walking code as well Iterative Less concerned with performance omap3isp driver has PM code, which could be more generic Ideally we should have single implementation for graph walking and for power management 1.4 Updates to ALSA and au0828 to share resources: 1. state->pads[AU8522_PAD_AUDIO_OUT].flags = MEDIA_PAD_FL_SINK; Need to be MEDIA_PAD_FL_SOURCE; 2. ALSA: to use on a non-atomic context: pcm->substream->nonatomic IRQ safe start/stop pipeline isn't necessary. Not a good idea to change graph_mutex to spinlock. I2C drivers will break. This will reduce the work and patch series v6 becomes smaller. 3. enable_source is necessary at media_device level, but find a better name 4. Write helper function to hde decoder and tuner details from ALSA 5. remove link is needed - implement - Shuah can do that as part of ALSA work 6. Managed Media Controller API - devres - change it to get reference and put reference. It isn't safe to delete media_device device resource. media_device_get() and media_device_put() Keep /dev/media device open and unplug the usb device - kernel would crash 1.4 IIO - Industrial I/O http://linuxtv.org/downloads/presentations/mc_ws_2015/iio_media_controller.… - ADC, DAC, accelerometer, gyro, light sensor, pressure, manometer - typically: low speed < 10kbaud, I2C/SPI - changes: up to 5Gbaud, separate control and data bus (LVDS, JESD204b) - can be used by multi-channel SDR 2) Keysign party at the afternoon of July, 30 We did a Keysign party, where the gpg keys got cross-signed among the participants. 3) Discussions about the Media Controller API and the needs to extend it to support the requirements for DVB, ALSA and IIO 3.1) Discussions took at July, 29 Basic requirements: - Properly represent kernel→userspace API interfaces; - Allow dynamic pipeline changes; - Define the namespace for interfaces and entities; - Support additional properties; - Better represent entities that belong to more than one type It was proposed to use the concept of planes, as defined by ITU and IEEE to represent different layers of a data or media network (ITU-T G.800). So, we would have control and data planes. http://linuxtv.org/downloads/presentations/mc_ws_2015/media_controller_summ… This is a concept largely used on networks, as show at: http://etherealmind.com/controller-based-networks-for-data-centres/ Multiple options to express association between device nodes and entities: - Same IOCTL API as for data links, difference made between data and control links - Different IOCTL API to enumerate interfaces, with associated entities - Properties The decision was, instead, to add a new graph element type (Interface) to represent the elements that controls the entities. It was agreed with the following terminology: - Entity: Functional unit, typically a hardware component or subunit in a hardware component - Pad: Data Input/Output of an entity - Interface: Software control access to a entity (e.g. v4l2-subdev) - Link: Connection between a source pad and a sink pad It was also proposed to use the name “association” for the links between interface and entity, but there was no consensus about that idea. EDITOR'S NOTE: It was decided on July, 31 to use the name “links” also for the connection between an entity and an interface. We used the term “interface links” for such links when we need to distinguish them from “data links” (e. g. pad to pad links). Mauro's concern is that there will be lots of code duplication in the Kernel if we use a different graph type for the interface links. The relationship between Interfaces and entities are n-n: - One interface can control multiple entities - One entity can be controlled by multiple interfaces Media interfaces - A concept of user space facing interfaces - An object of its own right (such as media entities) - At the same level than entities Working proposal (as proposed on July, 29) - Introduce struct media_interface as an object type alongside with entities in the Kernel and in the user space - See how the patches will look like; start with V4L2 and proceed with DVB - How much will there be code duplication? - Target 4.3 at earliest, more likely 4.4 The current API does a crap job on properly defining entities that are coupled to interfaces, and prevented the enablement of the MC DVB support, submitted in December, 2014. That needs to be fixed as soon as possible, as it is causing bad impact on using MC even for other V4L2 device types, like radio. So, Mauro stated that, as the MC API is currently on a broken state, no patches, drivers or framework changes related to the Media Controller will be merged upstream until this is fixed. Fixup patches may be merged, though. - Hans would try to do the work (of proposing the userspace API) at the week after the MC workshop. Userspace API Requirements: - Enumerate Entities + pads/links (exists today) - Enumerate Interfaces + entity associations (and define which interface is the default one) - For entities: enumerate associated interfaces - Should support two application models: Entity-centric and Interface-centric Additional API requirements that should be addressed in the future: - MC: 'versioning number' so apps are informed if the topology has changed. - MC: Notification events for topology changes Points to solve: - entity.type - should be renamed to reflect what's there; - How to represent DVB entities/interfaces? - How to represent DVB network interfaces? - How to represent ALSA entities? - What fields are needed for media interfaces at the public API? - Changes at media-ctl userspace application Some notes to help with network interface definitions: - man 7 netdevice - man 7 rtnetlink - Network interface index is constant for a network device - Network interface index can be converted to name etc. by an IOCTL DRM subsystem uses u64 instead of pointers at the public API: - IOCTL design: http://blog.ffwll.ch/2013/11/botching-up-ioctls.html - Not widely used outside DRM - Most sub-systems have their own conventions TODO: - Add graph topology version to the enumeration (and possibly other) ioctls Question: should we redesign everything in order to support topology changes? 3.2) Discussions took at the afternoon of July30 and on July, 31st It was discussed if we should add a generic graph framework in the kernel. This would require review on linux-kernel and such review may take a long time, as different requirements could pop up. So, perhaps not this time. It was presented some topology for simple hybrid TV hardware (PC-customer hardware), at: http://linuxtv.org/downloads/presentations/mc_ws_2015/media_controller_summ… An agreement was arrived for the UAPI interface. What was agreed: media interfaces will be a separate graph element; links will be used to connect both entities (pads to pads) and media interfaces; PADs will use unique IDs preliminary header changes: http://linuxtv.org/downloads/presentations/mc_ws_2015/media.h V4L2 applications need to know if they can capture from an input/tuner. To make that easy for them, it was proposed to add a new flag to VIDIOC_ENUMINPUT and whether the tuner is in use (and/or the tuner state) through VIDIOC_G_TUNER. This can't be done at the DVB side, however, as there are no reserved fields there. It was discussed about the need to allow bidirectional pads (needed for connectors like a coax that are bidirectional using time or frequency multiplexing) TODO: Connector entities It was proposed the following possible types of entities: Connector, DMA engine, processing block and controller block Whather an entity is a connector needs to be told, as this can't be discovered All others can be derived from other information already in the graph? Entity functional properties Most of the properties at entity level would be used to signal capabilities that can be figured out using sub-system specific interfaces the application already uses Interfaces Agreement: interfaces do have a type Types include V4L2, ALSA, DVB etc. Sub-types are used to tell between different interface types inside the sub-system An interface can be a device node, network interface or a sysfs directory It was proposed two new ioctls: G_TOPOLOGY IOCTL should not be used to convey arbitrary sysfs paths G_INTERFACE to pass information on single interface; there, a single pointer set by the user would be feasible. The struct can be the same. Finding entities Especially device specific applications often need to find an exact entity the application requires This may be a certain scaler in the graph, for instance Configuration may involve e.g. private IOCTLs only a certain sub-device implements Name is only 32 bytes at the moment In practice, external entities have used i2c bus number and i2c address as part of the name This does not scale for DT based devices where the device is uniquely identified by a string that can be very long Blocks that are a part of an ISP there's been just a name of the block without any device specific information This causes that if one has multiple such devices in the system, the name is no longer unique The entity name is not enough to uniquely find an entity in a general case In a general case it is not possible to recognise an entity Hot pluggable buses and entities that have no serial numbers or something equivalent In order to best help the user space, all the information that matters should be provided to the user So, it ended by having device names such as "omap3isp resizer" Other data could be used to enhance the device name, like: Bus Serial number Sysfs name (this may not be stable as sysfs has no official API stability, but may be still better than bus in some cases) This should enable most of the use cases where entities need to be found Properties Array of properties as an IOCTL argument has the problem that the user would have to allocate enough memory for each key-value pair in the array. This is unfeasible. The original proposal of using a text based format requires parsing by the user, albeit the kernel interface would remain simple The new proposal at the new media.h file (http://linuxtv.org/downloads/presentations/mc_ws_2015/media.h) combines the best of both: a single memory area amended with an array of properties that refer to the memory area For the media properties, using strings the key type has benefits, as hierarchy is possible, with is not possible using an u32 integer for instance. Should values be always strings or should other types be accepted? Probably having other types is a good idea. Different options for arrays: As part of the key: entity.0/name As part of the property: add a length field. This is convenient except for strings, where it works as well --- just count the nul characters. Properties could be seen as re-implementing sysfs on the discussions with linux-api people. However they're much more light-weight. Also, there's no atomic access of all sysfs properties and sysfs is limited to page size Properties can be queried by media object, in which case the media object id would be provided The entire tree of properties can be obtained using object id 0 Another option for the kernel interface: key/length/value E.g. type (u32) | length (u32) | value (string) | key (string) Media objects Media objects have an unique ID in the system Objects can be entities, pads or interfaces, for instance A few bits can be reserved for object types, e.g. entity == 0; interface == 1, pad == 2 Action Items: - media_interface/media_topology: RFC for userspace API: Hans - RFC patch series: internal implementation for interface/topology: Mauro - Migration: add v4l-subdev media_interface: Laurent - Migration: add explicit DMA Engine entity: Laurent - media_properties: RFC userspace API: Sakari - MC core support for dynamically adding/removing from the media graph: Samsung - Clarify/implement DVB network interface: Mauro - MC support for Alsa: Lars: Need RFC before Alsa summit during ELCE - MC support for DVB: Mauro - MC default core support for interface-centric V4L2: Hans (?) - MC support for IIO: Lars - VIDIOC_ENUMINPUTS/G_TUNER: tell if the tuner/input is busy: RFC Hans, Shuah has a backend for this. - Fix media_device broken alloc/remove (devres): Shuah

5 9

[RFC] Media graph flow for an hybrid device as discussed at the media workshop
by Mauro Carvalho Chehab 10 Aug '15

10 Aug '15

During the discussions at the Media Workshop, we came with some dot files that would describe a hybrid PC-consumer TV stick with radio, analog video, analog TV and digital TV on it. I consolidated all the dot files we've worked there, and added the connectors for RF, S-Video and Composite. The dot file and the corresponding picture is at: http://linuxtv.org/downloads/presentations/mc_ws_2015/dvb-pipeline-v2.dot http://linuxtv.org/downloads/presentations/mc_ws_2015/dvb-pipeline-v2.png As my plan is to start working on some real driver to produce such graph, please validate if the entities, interfaces, data links and interface links are correct, and if the namespace nomenclature is ok, or if I miss something. Thanks! Mauro

2 4

[PATCH_RFC_v1 0/4] MC initial changes due to MC workshop
by Mauro Carvalho Chehab 05 Aug '15

05 Aug '15

This is an initial set of patches showing the approach I'm taking in order to fulfill all the MC needs that was discussed on the 3 day MC summit in Helsinki. As suggested, I'll be sending incremental patches and avoiding to do a large set of changes into one changeset series. So, this is the first 4 patches of the changes. Along this patch series, I'll be calling "object" as anything "symbol" is part of the media graph. Currently: entity, link and pads. New object types will be needed to represent interfaces. I suspect we'll also need another type to represent group of objects, but this will be covered on future patches. No userspace API changes here, just changes at the internal structs that contains the media graph objects and some new helper functions. The goal of this patchset is: 1) to create a common struct that will be embedded on all internal structs that represents the comon data that will be used by all kinds of objects; 2) to have an unique object ID for each object in the graph. The object ID will be needed by the userspace API in the future, as discussed during the MC workshop; 3) to be sure that no data used by an entity will be freed too early. Latter patches will do similar changes to links and pads. With regards to (3), this patchset is not complete yet; it currently changes only the DVB core, as the changes are simpler there. I'll be working on a similar change for the V4L2 core. Yet, it would be nice to have a feedback earlier in order to avoid rework. Mauro Carvalho Chehab (4): media: Add some fields to store graph objects media: Add a common embeed struct for all media graph objects media: add functions to create/remove entities dvbdev: Use functions to create/remove media_entity struct drivers/media/dvb-core/dvbdev.c | 13 +++--- drivers/media/media-device.c | 4 ++ drivers/media/media-entity.c | 99 +++++++++++++++++++++++++++++++++++++++++ include/media/media-device.h | 4 ++ include/media/media-entity.h | 58 ++++++++++++++++++++++++ 5 files changed, 172 insertions(+), 6 deletions(-) -- 2.4.3

2 8

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

media-workshop