This is the first draft for the Linux Media Summit Report.
Please note that the items 3 to 5 are not in good shape. In special, nobody took Etherpad notes on item 4.
Please review. I'll publish a second (final?) draft after having some feedback.
Regards, Mauro
-
Linux Media Summit - March, 26 2015 - San Jose - CA - USA
Attendees:
Angelos Manousaridis amanous@gmail.com Bob Moragues bob.moragues@lge.com Chris Kohn Guennadi Liakhovetski g.liakhovetski@gmx.de Hans Verkuil hverkuil@xs4all.nl Hyun Kwon Karthik Poduval karthik.poduval@gmail.com Laurent Pinchart laurent.pinchart@ideasonboard.com Mauro Carvalho Chehab mchehab@osg.samsung.com Michal Lebik Mohammed CHERIFI mcherifi@cisco.com Rafael Chehab chehabrafael@gmail.com Ron Birkett Schuyler Patton Shuah Khan shuahkh@osg.samsung.com
1) Media Controller support for DVB Mauro presented a set of slides (add link) showing how the DVB pipelines look like and underlined that several topics needs to be addressed by the Media controller:
a) dynamic creation/removal of pipelines b) change media_entity_pipeline_start to also define the final entity c) how to setup pipelines that also envolve audio and DRM? d) how to lock the media controller pipeline between enabling a pipeline and starting it, in
How to do complex pipelines in DVB?
- The DVB demux can filter MPEG-TS traffic (either in hardware or in software) and can send multiplexed TS to the dvr node, elementary streams to the demux node and can create network interfaces for elementary streams (ES) via the net node. - a given set of elementary streams can go to one of those three options only, or it can be sent directly to a GPU and/or an ALSA pipeline. - there is support for hardware PID filtering at the Kernel, but no support (yet) for a real hw demuxer that splits the MPEG TS into separate DMA MPEG-TS and/or ES streams. - frontend device node is to be attached to the demod entity and it will control the demod, the tuner and a possible LNA via the active Media Controller links. - dvr/net/demux device nodes are attached to the demux entity. - the net interfaces are not (yet) represented via MC: we need the ability to remove entities dynamically for that, and we are not really sure if we want this at all. So, it as agreed to wait for support for removing entities to arrive, then this need can be discussed again. - For now we can safely assume that there is only one Satellite Equipment Control (SEC) in each active data path that goes through a tuner/demod. So each frontend will control just one SEC. Should we encounter really complex scenarios, then we should consider having device nodes for SEC entities.
It was decided that: - The Satellite Equipment Control (SEC) should be an entity, linking them to the connector - Deprecated osd, teletext, video and audio device nodes are only used in av7110. The av7110 driver uses lots of deprecated stuff, we should move this to staging and deprecate the whole driver and see who starts yelling. - Document the high-level overview of DVB (Mauro). Layout needs to be changed to be in line with the other APIs (Hans?). - Mauro will rename “frontend” entity to “demod” at the Media Controller, as the frontend is actually a set of elements. - Laurent will prepare a proposal of reporting device nodes via a new entity properties API addition
2) Media Tokens
Shuah submitted some RFC Patches: https://git.kernel.org/cgit/linux/kernel/git/shuah/linux.git
- Changes from the previous RFC: - simplified after switching au0828 to vb2 - token created by the bridge driver
It was decided that: - Preference for using the Media Controller. That requires that MC support for Alsa is added, the usbaudio driver then needs to find and hook into the MC from the bridge driver. - The RFC patches will help to identify on what places the driver should be touched - Shuag from Samsung is willing to do the changes at ALSA; Rafael is willing to add MC support at au0828/au8522. - Media controler dev (mdev) will have to be created as a dev resource on the parent device to the bridge device similar to media tokens - Need a new media_device_create() interace to allocate it as a device resource. This routine will either return media device if one is created or create it. - Both au0828 and ALSA will first call media_device_create(). Coordinate register/unregister??
3) FPGA/Project ARA: dynamic reconfiguration (http://www.projectara.com/) - partial pipeline removal: controlled removal in the case of FPGA reconfiguration. subdevs/entities will be removed: unsupported today. - no notification in MC when things change: we likely need an event mechanism. - adding entities: doable, might need to add links in a 'pending' state, to be made into a normal link once all streaming is stopped. - subdevs: add refcounting, remove calls subdev_unregister(). Internal release callback when the refcount goes to 0. - Removal of subdevs will lead to holes (missing entities) in the MC graph. - reuse entity IDs? Mauro doesn't like it, Laurent/Hans undecided. - if one entity has a pointer to another it has to take a refcount. Possible locking issues. Needs analysis.
4) Update on ongoing projects
5) Android Camera v3
The Khronos OpenKCam API addresses the same needs as the Android Camera HAL v3 API, and is quite similar in concept. We should make sure that our API can support OpenKCam. The specification isn't public, but it's based on the FCam API (Nokia research project) which should be available publicly.
Hi Mauro,
I could not participate at your Summit, but may have an input to the media-controller in DVB - see below.
On Wed, 22 Apr 2015 15:31:46 -0300 Mauro Carvalho Chehab mchehab@osg.samsung.com wrote:
This is the first draft for the Linux Media Summit Report.
Please note that the items 3 to 5 are not in good shape. In special, nobody took Etherpad notes on item 4.
Please review. I'll publish a second (final?) draft after having some feedback.
Regards, Mauro
Linux Media Summit - March, 26 2015 - San Jose - CA - USA
Attendees:
Angelos Manousaridis <amanous@gmail.com> Bob Moragues <bob.moragues@lge.com> Chris Kohn Guennadi Liakhovetski <g.liakhovetski@gmx.de> Hans Verkuil <hverkuil@xs4all.nl> Hyun Kwon Karthik Poduval <karthik.poduval@gmail.com> Laurent Pinchart <laurent.pinchart@ideasonboard.com> Mauro Carvalho Chehab <mchehab@osg.samsung.com> Michal Lebik Mohammed CHERIFI mcherifi@cisco.com Rafael Chehab <chehabrafael@gmail.com> Ron Birkett Schuyler Patton Shuah Khan <shuahkh@osg.samsung.com>
- Media Controller support for DVB
Mauro presented a set of slides (add link) showing how the DVB pipelines look like and underlined that several topics needs to be addressed by the Media controller:
a) dynamic creation/removal of pipelines b) change media_entity_pipeline_start to also define the final entity c) how to setup pipelines that also envolve audio and DRM? d) how to lock the media controller pipeline between enabling a pipeline and starting it, in
How to do complex pipelines in DVB?
- The DVB demux can filter MPEG-TS traffic (either in hardware or in
software) and can send multiplexed TS to the dvr node, elementary streams to the demux node and can create network interfaces for elementary streams (ES) via the net node.
- a given set of elementary streams can go to one of those three
options only, or it can be sent directly to a GPU and/or an ALSA pipeline.
- there is support for hardware PID filtering at the Kernel, but no
support (yet) for a real hw demuxer that splits the MPEG TS into separate DMA MPEG-TS and/or ES streams.
- frontend device node is to be attached to the demod entity and it
will control the demod, the tuner and a possible LNA via the active Media Controller links.
- dvr/net/demux device nodes are attached to the demux entity.
- the net interfaces are not (yet) represented via MC: we need the
ability to remove entities dynamically for that, and we are not really sure if we want this at all. So, it as agreed to wait for support for removing entities to arrive, then this need can be discussed again.
- For now we can safely assume that there is only one Satellite
Equipment Control (SEC) in each active data path that goes through a tuner/demod. So each frontend will control just one SEC. Should we encounter really complex scenarios, then we should consider having device nodes for SEC entities.
What about demod-diversity: demods of some manufacturers can be used to combine their demodulated symbols and, due to their different antennas and RF-paths, improve the overall reception quality.
If we ever have someone contributing in this area with hardware-drivers, it would be nice to have the user-space possible to select demod-combinations. It should be possible to add and remove a demod to a diversity-chain when and when not being tuned to a channel.
regards, --
Hi Patrick,
Em Thu, 23 Apr 2015 09:06:35 +0200 Patrick Boettcher patrick.boettcher@posteo.de escreveu:
Hi Mauro,
I could not participate at your Summit, but may have an input to the media-controller in DVB - see below.
On Wed, 22 Apr 2015 15:31:46 -0300 Mauro Carvalho Chehab mchehab@osg.samsung.com wrote:
This is the first draft for the Linux Media Summit Report.
Please note that the items 3 to 5 are not in good shape. In special, nobody took Etherpad notes on item 4.
Please review. I'll publish a second (final?) draft after having some feedback.
Regards, Mauro
Linux Media Summit - March, 26 2015 - San Jose - CA - USA
Attendees:
Angelos Manousaridis <amanous@gmail.com> Bob Moragues <bob.moragues@lge.com> Chris Kohn Guennadi Liakhovetski <g.liakhovetski@gmx.de> Hans Verkuil <hverkuil@xs4all.nl> Hyun Kwon Karthik Poduval <karthik.poduval@gmail.com> Laurent Pinchart <laurent.pinchart@ideasonboard.com> Mauro Carvalho Chehab <mchehab@osg.samsung.com> Michal Lebik Mohammed CHERIFI mcherifi@cisco.com Rafael Chehab <chehabrafael@gmail.com> Ron Birkett Schuyler Patton Shuah Khan <shuahkh@osg.samsung.com>
- Media Controller support for DVB
Mauro presented a set of slides (add link) showing how the DVB pipelines look like and underlined that several topics needs to be addressed by the Media controller:
a) dynamic creation/removal of pipelines b) change media_entity_pipeline_start to also define the final entity c) how to setup pipelines that also envolve audio and DRM? d) how to lock the media controller pipeline between enabling a pipeline and starting it, in
How to do complex pipelines in DVB?
- The DVB demux can filter MPEG-TS traffic (either in hardware or in
software) and can send multiplexed TS to the dvr node, elementary streams to the demux node and can create network interfaces for elementary streams (ES) via the net node.
- a given set of elementary streams can go to one of those three
options only, or it can be sent directly to a GPU and/or an ALSA pipeline.
- there is support for hardware PID filtering at the Kernel, but no
support (yet) for a real hw demuxer that splits the MPEG TS into separate DMA MPEG-TS and/or ES streams.
- frontend device node is to be attached to the demod entity and it
will control the demod, the tuner and a possible LNA via the active Media Controller links.
- dvr/net/demux device nodes are attached to the demux entity.
- the net interfaces are not (yet) represented via MC: we need the
ability to remove entities dynamically for that, and we are not really sure if we want this at all. So, it as agreed to wait for support for removing entities to arrive, then this need can be discussed again.
- For now we can safely assume that there is only one Satellite
Equipment Control (SEC) in each active data path that goes through a tuner/demod. So each frontend will control just one SEC. Should we encounter really complex scenarios, then we should consider having device nodes for SEC entities.
What about demod-diversity: demods of some manufacturers can be used to combine their demodulated symbols and, due to their different antennas and RF-paths, improve the overall reception quality.
If we ever have someone contributing in this area with hardware-drivers, it would be nice to have the user-space possible to select demod-combinations. It should be possible to add and remove a demod to a diversity-chain when and when not being tuned to a channel.
It makes sense to map demod diversity via the media controller.
Not sure what would be the best way to map it though, as I don't have a clear understanding about how the hardware pipelines are set for demod diversity.
The way the media controller currently is is that it maps only the data flow. There are discussions about how the control flow should happen.
The data flow for a normal demod is:
IF (or baseband) ---> [demod] ---> MPEG-TS
From dib0700 demod drivers, I remember that several of the demods have a concept of a "slave demod". Are those full demods that can get an IF/baseband input and produce a MPEG-TS output, or are those just IP blocks that have the IF/baseband input but doesn't produce an MPEG-TS output, but, instead, sends some sort of data into the "master demod"?
In other words, would the dataflow be something like
IF TS [tuner] ---> [master demod] ----->[ ] | IF TS [ combiner ] ---> [demux] |--------> [slave demod] ----->[ ]
or: IF TS [tuner] -----------> [master demod] ---> [demux] | IF ^ | | (what sort of data?) |----> [slave demod] ----|
Or is it something else?
Regards, Mauro
Hi Mauro,
On Thu, 23 Apr 2015 07:40:46 -0300 Mauro Carvalho Chehab mchehab@osg.samsung.com wrote:
What about demod-diversity: demods of some manufacturers can be used to combine their demodulated symbols and, due to their different antennas and RF-paths, improve the overall reception quality.
If we ever have someone contributing in this area with hardware-drivers, it would be nice to have the user-space possible to select demod-combinations. It should be possible to add and remove a demod to a diversity-chain when and when not being tuned to a channel.
It makes sense to map demod diversity via the media controller.
Not sure what would be the best way to map it though, as I don't have a clear understanding about how the hardware pipelines are set for demod diversity.
The way the media controller currently is is that it maps only the data flow. There are discussions about how the control flow should happen.
The data flow for a normal demod is:
IF (or baseband) ---> [demod] ---> MPEG-TS
From dib0700 demod drivers, I remember that several of the demods have a concept of a "slave demod". Are those full demods that can get an IF/baseband input and produce a MPEG-TS output, or are those just IP blocks that have the IF/baseband input but doesn't produce an MPEG-TS output, but, instead, sends some sort of data into the "master demod"?
In other words, would the dataflow be something like
IF TS[tuner] ---> [master demod] ----->[ ] | IF TS [ combiner ] ---> [demux] |--------> [slave demod] ----->[ ]
or: IF TS [tuner] -----------> [master demod] ---> [demux] | IF ^ | | (what sort of data?) |----> [slave demod] ----|
Or is it something else?
Let's define a demodulation object:
+---------------------------------------------------+ |RF tuner --> IF|baseband --> demod --> FEC-decoder | +---------------------------------------------------+
RF-tuner: RF downconverter (to a low or zero-IF) IF|baseband: ADC and digital filters demod: synchonization and symbol extraction FEC-decoder: viterbi/ldpc/turbo-code forward error correction
Let's call this object a frontend.
Now imagine a board which has 4 frontends:
high-speed-data-bus --------------------- | | | | FE3 FE2 FE1 FE0
To create a diversity-chain, for simplicity, we can say we connect at least 2 frontends in any order. Both frontends are tuned to the same physical channel and then a combiner will combine symbols decoded at demod-stage.
Any FE can be connected with any FE in any order. Per board different standards can be demodulated at the same time. For example: FE3 and FE2 are doing DVB-T and FE1 and FE0 DVB-T2 .
-- Patrick.
On 04/22/15 20:31, Mauro Carvalho Chehab wrote:
This is the first draft for the Linux Media Summit Report.
Please note that the items 3 to 5 are not in good shape. In special, nobody took Etherpad notes on item 4.
Item 4 was just me presenting on ongoing projects. As far as I remember there was no discussion and a link to my presentation would be enough. I know I mailed it to you.
Please review. I'll publish a second (final?) draft after having some feedback.
Regards, Mauro
Linux Media Summit - March, 26 2015 - San Jose - CA - USA
Attendees:
Angelos Manousaridis <amanous@gmail.com> Bob Moragues <bob.moragues@lge.com> Chris Kohn Guennadi Liakhovetski <g.liakhovetski@gmx.de> Hans Verkuil <hverkuil@xs4all.nl> Hyun Kwon Karthik Poduval <karthik.poduval@gmail.com> Laurent Pinchart <laurent.pinchart@ideasonboard.com> Mauro Carvalho Chehab <mchehab@osg.samsung.com> Michal Lebik Mohammed CHERIFI mcherifi@cisco.com Rafael Chehab <chehabrafael@gmail.com> Ron Birkett Schuyler Patton Shuah Khan <shuahkh@osg.samsung.com>
- Media Controller support for DVB
Mauro presented a set of slides (add link) showing how the DVB pipelines look like and underlined that several topics needs to be addressed by the Media controller:
a) dynamic creation/removal of pipelines b) change media_entity_pipeline_start to also define the final entity c) how to setup pipelines that also envolve audio and DRM? d) how to lock the media controller pipeline between enabling a pipeline and starting it, in
How to do complex pipelines in DVB?
- The DVB demux can filter MPEG-TS traffic (either in hardware or in software) and can send multiplexed TS to the dvr node, elementary streams to the demux node and can create network interfaces for elementary streams (ES) via the net node.
- a given set of elementary streams can go to one of those three options only, or it can be sent directly to a GPU and/or an ALSA pipeline.
- there is support for hardware PID filtering at the Kernel, but no support (yet) for a real hw demuxer that splits the MPEG TS into separate DMA MPEG-TS and/or ES streams.
- frontend device node is to be attached to the demod entity and it will control the demod, the tuner and a possible LNA via the active Media Controller links.
- dvr/net/demux device nodes are attached to the demux entity.
- the net interfaces are not (yet) represented via MC: we need the ability to remove entities dynamically for that, and we are not really sure if we want this at all. So, it as agreed to wait for support for removing entities to arrive, then this need can be discussed again.
- For now we can safely assume that there is only one Satellite Equipment Control (SEC) in each active data path that goes through a tuner/demod. So each frontend will control just one SEC. Should we encounter really complex scenarios, then we should consider having device nodes for SEC entities.
It was decided that:
- The Satellite Equipment Control (SEC) should be an entity, linking them to the connector
- Deprecated osd, teletext, video and audio device nodes are only used in av7110. The av7110 driver uses lots of deprecated stuff, we should move this to staging and deprecate the whole driver and see who starts yelling.
- Document the high-level overview of DVB (Mauro). Layout needs to be changed to be in line with the other APIs (Hans?).
- Mauro will rename “frontend” entity to “demod” at the Media Controller, as the frontend is actually a set of elements.
- Laurent will prepare a proposal of reporting device nodes via a new entity properties API addition
- Media Tokens
Shuah submitted some RFC Patches: https://git.kernel.org/cgit/linux/kernel/git/shuah/linux.git
- Changes from the previous RFC:
- simplified after switching au0828 to vb2
- token created by the bridge driver
It was decided that:
- Preference for using the Media Controller. That requires that MC support for Alsa is added, the usbaudio driver then needs to find and hook into the MC from the bridge driver.
- The RFC patches will help to identify on what places the driver should be touched
- Shuag
Shuag -> Shuah :-)
from Samsung is willing to do the changes at ALSA; Rafael is willing to add MC support at au0828/au8522.
- Media controler dev (mdev) will have to be created as a dev resource on the parent device to the bridge device similar to media tokens
- Need a new media_device_create() interace to allocate it as a device resource. This routine will either return media device if one is created or create it.
- Both au0828 and ALSA will first call media_device_create(). Coordinate register/unregister??
- FPGA/Project ARA: dynamic reconfiguration (http://www.projectara.com/)
- partial pipeline removal: controlled removal in the case of FPGA reconfiguration. subdevs/entities will be removed: unsupported today.
- no notification in MC when things change: we likely need an event mechanism.
- adding entities: doable, might need to add links in a 'pending' state, to be made into a normal link once all streaming is stopped.
- subdevs: add refcounting, remove calls subdev_unregister(). Internal release callback when the refcount goes to 0.
- Removal of subdevs will lead to holes (missing entities) in the MC graph.
- reuse entity IDs? Mauro doesn't like it, Laurent/Hans undecided.
- if one entity has a pointer to another it has to take a refcount. Possible locking issues. Needs analysis.
Update on ongoing projects
Android Camera v3
The Khronos OpenKCam API addresses the same needs as the Android Camera HAL v3 API, and is quite similar in concept. We should make sure that our API can support OpenKCam. The specification isn't public, but it's based on the FCam API (Nokia research project) which should be available publicly.
Regards,
Hans
-
Hans Verkuil -
Mauro Carvalho Chehab -
Patrick Boettcher