<!-- Some styling for better description lists --><style type='text/css'>dt { font-weight: bold;float: left;display:inline;margin-right: 1em} dd { display:block; margin-left: 2em}</style>
svarbanov: <u>mort</u>: both should and must support dmabuf, I think ttomov tried such an encoder pipeline with gstreamer
mort: <u>svarbanov</u>: even 4.9.39? With `v4l2-compliance -d /dev/video0 -s --expbuf-device /dev/video5`, where video0 is a camera and video5 is the encoder, read/write and mmap works, but userptr claims to not be supported, and dmabuf fails
<br> v4l2-compliance sadly doesn't print errno, but exp_q.reqbufs and setupDmaBuf fails in v4l2-test-buffers.cpp
<br> or rather, I suppose that means it's exp_q.reqbufs in the setupDmaBuf function which fails
svarbanov: <u>mort</u>: if you can try some gst pipeline
<br> <u>mort</u>: but most probably you will need some recent gst version (1.14.2 ???)
mort: I have GStreamer 1.12.4 - I'll give it a try with that, and then try to upgrade if that doesn't work
<br> <u>svarbanov</u>: what gstreamer element can I use for the encoder? On https://gstreamer.freedesktop.org/documentation/plugins.html, the only v4l2-related elements I can see are v4l2src, v4l2sink, and v4l2radio, which don't sound too relevant
<br> I know an encoder is sort of just a device which can be used as an output and capture device, but v4l2sink doesn't accept it, presumably because it doesn't set V4L2_CAP_VIDEO_OUTPUT[_MPLANE]
svarbanov: <u>mort</u>: gst-inspect-1.0 | grep v4l2
<br> <u>mort</u>: I'm seeing for example v4l2h264enc
mort: I just see v4l2video4dec; I'll try getting a newer version of gstreamer
***: svarbanov has quit IRC (Ping timeout: 252 seconds)
<br> hverkuil has quit IRC (Ping timeout: 252 seconds)
grohne: I'm sed -i -e '/v4l2_subdev_pad_ops/s/\*video/\*pad/' Documentation/media/kapi/v4l2-subdev.rst
<br> does it make sense to make a patch of such a simple typo or can someone fix it along the way somehow?
ndufresne: mort, just for your interest, stable names (like v4l2h264dec) have been implemented in 1.14
mort: Is there any particular reason why most devices don't support the multi-planar API, and a lot of software (at least Chrome and webrtc and a bunch of online example code) use the single-planar API?
<br> having a ton of `if (is_mplane)` everywhere is kind of annoying and feels somewhat unnecessary
***: ndufresne has quit IRC (Ping timeout: 252 seconds)
tfiga: <u>mort</u>: Chrome uses multiplane for codecs
<br> It only uses single plane for webcams
<br> But AFAIR the UVC driver doesn't expose multiplane
mort: I don't know how it deals with other codecs, but its v4l2 decoder is literally just a wrapper around ffmpeg, where it asks for the default h264 decoder which is a software decoder, and the h264 encoder is just a wrapper around openh264, another software encoder
<br> its h264 decoder*
tfiga: <u>mort</u>: was that about chrome?
mort: yes (well, chromium, because I don't have chrome's source code)
tfiga: That's not true...
<br> And I'm a co-author of some parts of the video stack there
<br> Please search for V4L2VideoDecodeAccelerator for example
mort: I spent a good while patching chromium to use ffmpeg's v4l2m2m h264 decoder instead of using ffmpeg's software h264 decoder (and fixed an ffmpeg memory leak in the process)
tfiga: In cs.chromium.org
<br> Well, chromium can use ffmpeg for software decode
<br> But it's not how it uses hardware
mort: https://cs.chromium.org/chromium/src/third_party/webrtc/modules/video_coding/codecs/h264/h264_decoder_impl.cc is what it has used for me, even though I was running it on a system with a v4l2 encoder
<br> that code also assumes that avcodec_find_decoder returns an encoder with an I420 pixel format by the way, which isn't a safe assumption
tfiga: That's the software thing
mort: yeah, it's the decoder chromium used when asked for h264
tfiga: If you hack ffmpeg to use V4L2 behind the scenes it could obviously use hardware
<br> But that's not a supported workflow
<br> It's expected that ffmpeg does software decoding
mort: it's not "hacking" ffmpeg, it's just compiling ffmpeg with a different set of h264 encoders, which is supported by ffmpeg afaik
tfiga: It's not supported by Chrome
<br> For encoder, there is V4L2VideoEncodeAccelerator class
mort: the solution then is to ask for the software decoder, called "h264", instead of asking for "any h264 decoder", or maybe enumerating the list of encoders until you find one which uses I420, not just hoping to $deity that the user's system's ffmpeg's default h264 decoder uses i420
<br> anyways, I didn't know about v4l2videoencodeaccelerator, I don't know why it's not being used on this system
tfiga: Yeah, that's possibly a bug :)
<br> It's not enabled in Linux builds by default
<br> One of the reasons is that there is too much divergence between how various drivers upstream implement the codec API
<br> But I think we would be happy to see patches to fix any issues on Linux
mort: for some reason ffmpeg's h264v4l2m2m encoder produces green frames on this hardware (as if the luminosity values are correct but the chroma values are all set to 0), I imagine that's due to the divergence you mention :P
<br> what arguments are used to enable v4l2 on linux?
tfiga: I don't remember the exact name, but it had v4l2 in the name
mort: alright
tfiga: I can look up tomorrow
mort: I can search for it with gn
tfiga: Tbh, I'm not sure if it still builds for Linux even...
<br> People haven't tried it for a while
mort: by the way, you don't happen to know if it's possible to reduce chromium's memory usage in any way? We're currently working on moving away from it, because we have a device with 1GB RAM and Chromium's render process eats memory until something is OOM killed
mjourdan: <u>mort</u>: you want "use_v4l2_codec = true". Problem is, it won't build, because some of the v4l2 accelerators depend on kernel headers that are changed in chromeos. You'll want to check Igalia's repository where they added the "use_linux_v4l2_only = true" option. https://github.com/Igalia/chromium
tfiga: Not sure. There is a lot of random knobs which I'm not familiar with
<br> But you may want to file a bug
mort: it's not a memory leak in the web application, I made a super simple file which just sets an <IMG> tag's src to something different every second , and it eats more and more memory
tfiga: There is definitely some elaborate caching scheme involved...
<br> But that goes beyond my level of familiarity with Chrome
<br> I'm just a kernel guy :P
mort: it just cycles through three different URLs, so it's not that it has to cache more and more images
<br> but that's fair, it was a long shot anyways :P
tfiga: <u>mjourdan</u>: thanks for the pointer
mort: changing kernel headers for chromeos sounds really strange, why'd they want to do that
tfiga: In free time, I'm trying to setup some testing of upstream codec drivers with Chromium V4L2 stack
<br> <u>mort</u>: because there was no Request API in upstream until today? :)
<br> And we needed to support a platform that requires such functionality
mort: Google really doesn't make it a priority to make it easy for other people to use their software do they
tfiga: Agreed that we should've made it more conditional...
<br> People sometimes consider build options disabled by default as never-enabled...
mort: there's also the thing with how all your libraries can't really be easily integrated with non-google software
tfiga: I'm not sure about other projects
<br> But not sure what Chromium libraries have this problem
<br> After all, Chromium or its parts is used
<br> With non-Google software
mort: e.g the way to use the webrtc library seems to be to use gn+ninja to generate object files, then run `find src/out/Whatever -name '*.o' -print0 | xargs -0 ar crs libwebrtc.a`, and then find and copy only the headers in src/ and put them somewhere you can actually use them
<br> and try to filter out unnecessary object/header files (for example, one can assume that stuff in obj/examples and obj/rtc_tools is unnecessary)
tfiga: Strange indeed :|
mort: I can sort of understand making a custom build system instead of using cmake/autotools (even though that makes it annoying to integrate with systems like yocto), but not having a simple way to just get a .a or .so and an include/ folder is very odd for something which claims to be a library
tfiga: It kind of feels unbelievable that there is no way
<br> But I'm just a kernel guy :P
<br> Chrome is a huge project and people sometimes have to care about too many things
mort: it's the only way I've found of doing it, and it's the way a couple of other webrtc-building scripts does it too
tfiga: So they might sometimes fail to notice some problems with less common scenarios
mort: I also like that the documentation consists of a couple of nice flow diagrams, and a link to the source code
tfiga: I think you may want to file a bug for this one too
mort: maybe
tfiga: But nobody's perfect :(
<br> But I guess we wouldn't be upset to see patches that fix some of the pain points
<br> Anyway, I'm off to bed for today
<br> Please let me know if I can help anyhow with that V4L2 support
mort: another huge annoyance is that everything Google wants to use Google's sysroot for everything, while people making distros (including us, making a small embedded distro with yocto) want to have their distro's software use the distro's libraries and be compiled with the compilers the rest of the system uses
<br> alright, night :)
tfiga: I'm also trying to bring it up to a better shape as a part of my codec API testing with upstream drivers
<br> But sadly failing to get enough time to move it forward recently
<br> TTYL
***: lucaceresoli has quit IRC (Read error: Connection reset by peer)
<br> benjiG has left
ndufresne: mort, now one picked up the job of porting all the drivers
<br> Though, userspace still need to support single and multi allocation, even with the mplane interface
jdel: hello
<br> are there existing tools for capturing multi-planar video from a v4l device?
<br> ffmpeg specifically does not support multi-planar devices