| ↑back Search ←Prev date Next date→ Show only urls | (Click on time to select a line by its url) |
| Who | What | When |
|---|---|---|
| *** | sigmaris has quit IRC (Ping timeout: 480 seconds) | [00:06] |
| camus has quit IRC () | [00:15] | |
| ........................................... (idle for 3h33mn) | ||
| sigmaris has joined #linux-media | [03:48] | |
| .......................................... (idle for 3h25mn) | ||
| camus has joined #linux-media | [07:13] | |
| ao2 has joined #linux-media
GBenji has joined #linux-media | [07:24] | |
| ...... (idle for 25mn) | ||
| camus has quit IRC (Read error: Connection reset by peer) | [07:52] | |
| GBenji has quit IRC (Quit: Leaving.)
GBenji has joined #linux-media GBenji has left GBenji has joined #linux-media | [08:06] | |
| GBenji has quit IRC ()
GBenji has joined #linux-media | [08:11] | |
| ...... (idle for 26mn) | ||
| djrscally has joined #linux-media | [08:37] | |
| .... (idle for 18mn) | ||
| tfiga_ has joined #linux-media
tfiga_ has quit IRC () | [08:55] | |
| ................... (idle for 1h34mn) | ||
| BrianG61UK has quit IRC (Read error: Connection reset by peer) | [10:30] | |
| mort_ | i asked this in #linux-msm too, but it's relevant here too: I wrote the simplest video4linux decoder test application I can: https://p.mort.coffee/QIy.c - with Linux 4.9, it works as expected: https://p.mort.coffee/Pqh - but with Linux 5.7, the kernel doesn't give me any decoded frames: https://p.mort.coffee/PUH | [10:39] |
| Here's the most detailed log I know how to produce: https://p.mort.coffee/DxJ
that contains all the debug logging from the /dev/video5 device and with verbose output from the decoder test program | [10:45] | |
| ..... (idle for 23mn) | ||
| hverkuil | mort_: the value of bytesused for the output buffers looks really low: 30, 824, 577, 25. I can't imagine that that is enough data to decode even a single frame. | [11:09] |
| mort_ | hverkuil: hmm, good point. As it is, I'm giving it each NAL unit as one buffer, but one NAL unit isn't always one frame | [11:10] |
| hverkuil: what is actually supposed to be input into the decoder though? It's not as if the bitstream is neatly divided into frames which I can give it | [11:16] | |
| hverkuil | It's a stateful decoder, you can just fill up the output buffer and queue it. The decoder will do the parsing. | [11:22] |
| mort_ | alright, let's give it a shot | [11:28] |
| tfiga | hverkuil: I'm not sure this is true actually
mtk-vcodec needs initial buffers to be SPS and PPS in separate buffers and AFAIR venus required the entire frame to be in one buffer although I recall some firmware fixes being made to allow SPS and PPS to be sent separately gnurou might recall more details | [11:31] |
| *** | miqztee has joined #linux-media | [11:34] |
| mort_ | varbanov's example code does parse the bitstream to find nal units fwiw | [11:34] |
| tfiga | yeah, definitely it's not allowed to split NALUs between multiple buffers | [11:35] |
| mort_ | but his code is more complicated than mine, it seems to actually inspect the NAL unit type | [11:35] |
| tfiga | I mean, split the same NALu over multiple buffers* | [11:35] |
| hverkuil | https://hverkuil.home.xs4all.nl/spec/userspace-api/v4l/dev-decoder.html, section 4.5.1.7, has no such restriction. If a driver has additional restrictions, then that should be documented and somehow exposed. | [11:35] |
| mort_ | so I could potentially, depending on the driver, put multiple NAL units in the same buffer, but definitely not put a partial NAL unit into a buffer | [11:35] |
| hverkuil | In practice, userspace apps like gstreamer and (AFAIK) ffmpeg always collect the data for a single compressed frame and pass that to the output buffer. | [11:36] |
| mort_ | where do additional decoder driver restrictions get documented? That would be an immensely useful resource | [11:36] |
| hverkuil | I'm not really sure what the venus status is w.r.t. the stateful API compliance. | [11:37] |
| tfiga | hverkuil: yes, I consider mtk-vcodec broken
I think venus might be behaving much better with current firmware, though probably need to get Stanimir to comment on this I haven't looked at video codecs for a few years now (directly) mort_: I guess you might want to make sure you have the latest firmware | [11:40] |
| hverkuil | tfiga: unfortunately we do not have good compliance tests for this. All the commonly used decoder frameworks use one buffer per compressed frame as far as I know. | [11:43] |
| mort_ | tfiga: I will | [11:43] |
| tfiga | hverkuil: unfortunately we had some internal reorg and I'm not following our video development anymore
but I know there are some Kernel CI efforts on behalf of Chrome OS | [11:44] |
| mort_ | tbh it would have been very nice if I could just read data into the memory mapped buffers and let the driver take care of parsing, it's a bit stupid that my code has to parse the h264 bitstream instead of just being a dumb pipe from video data to driver | [11:45] |
| tfiga | you might want to reach out to frkoenig [at] chromium.org to query for any plans on extended testing coverage
mort_: I think you don't want :) parsing in the kernel is a horrible idea | [11:45] |
| mort_ | eh, I suppose | [11:46] |
| tfiga | but indeed there are decoders which do it in hardware and/or firmware
e.g. s5p-mfc... so I guess the blame goes to the hw vendors | [11:46] |
| mort_ | nah
the blame goes to linux which has no reasonable way to have userspace hardware enc/dec drivers | [11:46] |
| tfiga | if the hardware/firmware is not reasonable, how one can have reasonable drivers? | [11:47] |
| mort_ | if the venus driver was userspace then it could've parsed the bitstream and everything
the hardware seems reasonable to me, if we consider it an accelerator and not a decoder | [11:47] |
| tfiga | well, I think that's what gstreamer and/or ffmpeg is for | [11:47] |
| mort_ | fair | [11:47] |
| tfiga | any reason you're trying to opencode this? | [11:47] |
| mort_ | ffmpeg has no v4l decoder and the v4l encoder is buggy
and it's not *hard* to do manually | [11:48] |
| tfiga | hmm, I was pretty much sure it had a decoder
and for encoder bugs, fix them? :) | [11:48] |
| mort_ | maybe it's the decoder that's buggy and the encoder which doesn't exist
I reported a bug and never got any response iirc that's how it usually goes with these things | [11:48] |
| tfiga | but yeah, I think gstreamer is a much better choice | [11:49] |
| mort_ | probably | [11:49] |
| tfiga | with ndufresne being here :)
although I kind of find it painful that there is no lightweight library to do this and one needs to pull and entire framework to do something as simple as video decoding | [11:49] |
| mort_ | to use gstreamer we would have to install gstreamer to the embedded device this is supposed to run on
which isn't impossible, but it seems quite heavy | [11:50] |
| kbingham | I keep thinking some of the components for libcamera should be made more re-usable. I.e. the v4l2_videodevice/media controller support could be a standalone library to easy interaction with media subsystem.
And we already have m2m support there, (but not encoder formats, just pixel formats :D) | [11:51] |
| tfiga | libv4l2? ;) | [11:51] |
| mort_ | anyways, I'll check out the firmware situation, it's quite possible that it's on an old firmware version | [11:51] |
| kbingham | libv4l2++ ;-) | [11:52] |
| tfiga | tfiga just kidding
but IMHO libv4l2 is something that should exist in the current shape and actually be mandatory and a place to handle hardware quirks like these being discussed here | [11:52] |
| pinchartl | for codecs possibly, but for capture devices, we've decided to deprecate libv4l2 :-)
a userspace codec library would be useful... | [11:53] |
| hverkuil | pinchartl: I agree completely. I would love to have a lightweight codec library that I can use in v4l2-compliance. | [11:55] |
| tfiga | pinchartl: yeah, maybe "in the current shape" is not the right way to put it | [11:56] |
| hverkuil | But it's a non-trivial project. | [11:56] |
| mort_ | http://releases.linaro.org/96boards/dragonboard410c/qualcomm/firmware/ it should be using 1034.2.1, which is the newest firmware from linaro
I don't know if linaro is outdated here or not | [11:56] |
| tfiga | something working more on a single device node level and API compliance quirks that can't be handled in the kernel)
mort_: do you know which venus version that is? | [11:56] |
| mort_ | no idea | [11:58] |
| tfiga | https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/qcom | [11:58] |
| mort_ | the driver says version: 329479
from querycap | [11:58] |
| tfiga | by the way, is it actually an upstream driver you're working with?
I know that Qualcomm had a downstream driver that exposed a somewhat resembling API | [11:58] |
| mort_ | I'm using https://github.com/varigit/DART-SD410-kernel/tree/release/kernel-20.07/drivers/media/platform/qcom/venus
from that branch in that fork of the kernel I can only see commits from svarbanov and greg, no custom patches from the soc vendor. Some may be cherry-picked I suppose though, I'll run a diff | [11:59] |
| tfiga | hmm, that looks like the upstream driver dated to 2020H2 which I guess should have most of the important fixes
so I guess you might want to figure out if any of the firmware blobs from linux-firmware wouldn't work better | [12:01] |
| mort_ | I'll try and see if linux-firmware has firmware for this soc | [12:04] |
| welp, I'm building a system with venus, sdm845 and adreno drivers from linux-firmware. Wish me luck | [12:09] | |
| that made no difference
which is a bit surprising, I would've expected something to break I'll make an image with gstreamer and see if that can decode should help us figure out if it's a driver/firmware/whatever thing or my code that's wrong | [12:15] | |
| *** | camus has joined #linux-media | [12:27] |
| mort_ | tfiga: what's the appropriate gstreamer pipeline element to use v4l2 m2m decoders?
I would guess v4l2videodec but that doesn't seem to exist well derp, I only have coreelements and staticelements, not even plugins-good it seems | [12:29] |
| tfiga | mort_: sorry, I'm not that familiar with the internals there
you might have more luck in a timeframe where ndufresne is here | [12:37] |
| javierm | mort_: you should get a v4l2video?dec for each v4l2 m2m dev you have as /dev/video? (i.e: v4l2video0dec)
mort_: so something like the following: gst-launch-1.0 filesrc location=test.mp4 ! qtdemux ! h264parse ! v4l2video${DEV}dec capture-io-mode=dmabuf ! kmssink | [12:45] |
| mort_ | I don't have a v4l2video?dec, but I have a v4l2h264dec with a device argument which defaults to /dev/video4 (which is the correct device)
that's from the video4linux2 plugin | [12:46] |
| javierm | mort_: ah, I see. But yes, I usually do gst-inspect-1.0 | grep v4l2 | [12:48] |
| mort_ | I don't think I should need qtdemux, I have a raw h264 file not in a container. I don't seem to have h264parse, that seems to be in the bad plugins which I didn't install
I'll install plugins-bad | [12:48] |
| javierm | mort_: right, I was just showing an example gst-launch command using a v4l2 m2m device
got it from my shell history :) | [12:49] |
| mort_ | I hate that you can't even write a h264 bitstream parser without infringing on h264 patents :( | [12:50] |
| well would you look at that
`gst-launch-1.0 filesrc location=basic.h264 ! h264parse ! v4l2h264dec ! videoconvert ! ximagesink` works perfectly maybe it's time to move away from ioctls and use gstreamer | [12:59] | |
| javierm | mort_: great | [13:00] |
| mort_ | oh no... does gstreamer have to be used in a glib application with a glib event loop and everything | [13:07] |
| awesome, my v4l2 decoder doesn't work but gstreamer's v4l2 decoder works, however my v4l2 camera capturer works but gstreamer's v4l2 camera capturer doesn't work
I suspect gstreamer does a VIDIOC_S_FMT on the camera device without changing the format of the corresponding v4l-subdev device | [13:17] | |
| *** | miqztee has quit IRC (Quit: Leaving) | [13:27] |
| .... (idle for 19mn) | ||
| pinchartl | mort_: I recommend libcamerasrc :-)
jokes aside, it will handle the pipeline configuration for you may not work out of the box with camss as I haven't tested it personally and the code may have bitrotten a bit but it's the way forward | [13:46] |
| mort_ | pinchartl: I kind of feel like if I was going to rewrite the whole thing to use gstreamer anyways I would want to use gstreamer's camera source
the fact that gstreamer decoding works is great though, I'll go through debug logs from the driver and see exactly what gstreamer does differently to me and fixing my decoder should be relatively easy (at least compared to the amount of effort involved in rewriting everything to use gstreamer for decoding and encoding and libcamerasrc for camera) | [13:49] |
| pinchartl | libcamerasrc is a gstreamer element | [13:51] |
| mort_ | oh | [13:51] |
| .... (idle for 15mn) | ||
| *** | camus has quit IRC (Read error: Connection reset by peer) | [14:06] |
| ..... (idle for 21mn) | ||
| ndufresne | mort_: gstreamer does not require glib specific mainloop, you are free to driver your GstBus in whatever loop you want
and no, v4l2src is not aware of subdevs, that thing came later in the chronology, and anyway, driving subdevs is hw specific (hence libcamera pipeline manangers) | [14:27] |
| mort_ | the only real difference between gstreamer's use of the driver and my own is that A) gstreamer does VIDIOC_REQBUFS once with memory=mmap, once with memory=dmabuf and once with memory=userptr (even though it's configured to use mmap), and B) gstreamer VIDIOC_CREATE_BUFFERS | [14:35] |
| ndufresne | mort_: these are not real REQBUFS the count is 0 | [14:36] |
| mort_ | my understanding was that CREATE_BUFFERS wasn't necessary, just an optional addition or alternativet o REQBUFS
ah so it's not real CREATE_BUFFERS either then I suppose, since they also have count=0 | [14:36] |
| ndufresne | note that in modern kernel, if you can afford requiring 5.,10+ there is flags for that, no need for REQBUFS(0) anumore
mort_: we can it the probing phase, a trial and error face to map what the kernel and driver can do, gst works on very ancient kernels too | [14:37] |
| mort_ | I'm on 5.7 atm sadly | [14:38] |
| ndufresne | gst is not going to fail because you don't implement all the new and shiny ioctl, it is just calling it at start to check what is supported, to avoid doing it over and over again
of course, libcamerasrc will be much more to the point, since it will trigger a plugin in libcamera that is dedicated for you HW | [14:38] |
| .... (idle for 16mn) | ||
| mort_ | ndufresne: feel like skimming through a couple annotated logs?
this is what the driver says when decoding a video with v4l2h264dec: https://p.mort.coffee/atW.txt this is what it says when decoding with my own decoder program: https://p.mort.coffee/0YE.txt (decoder program at https://p.mort.coffee/0YE.txt) I removed all the trial and error and enumeration and such from the gstreamer output there's a couple small ordering differences but I can't see anything which ought to make a difference well, there's the fact that the gstreamer code seems to combine the two first NAL units into one buffer; gstreamer's first enqueued buffer is 862 bytes, my two first are 30 and 824 that's probably it. I should write a more intelligent bitstream parser and try to send full frames only | [14:55] |
| *** | BrianG61UK has joined #linux-media | [15:01] |
| ..... (idle for 21mn) | ||
| pinchartl | hverkuil: your vb2 qbuf fix has passed testing. do you plan to send a non-RFC patch ? :-)
it would be nice to get it fied in v5.15 (or even better in v5.14 if possible) | [15:22] |
| .... (idle for 17mn) | ||
| hverkuil | pinchartl: yeah, it's on my todo list, as so many other things :-( | [15:39] |
| pinchartl | would you like us to submit the patch ? | [15:40] |
| hverkuil | I am tentatively planning to hoover up patches for a 'Various fixes' PR tomorrow, so I hope to get to it then. If you don't see any activity by the end of tomorrow, then please submit a patch. | [15:42] |
| pinchartl | ok
thank you | [15:42] |
| hverkuil | At least one other patch is needed for v5.14, so let me see if I can get this in as well for 5.14. | [15:43] |
| *** | camus has joined #linux-media | [15:48] |
| hverkuil | pinchartl: I actually prefer to make the patch. There is a bit more involved when combined with Request APIs, and I got an idea for that that I want to look at first. | [15:52] |
| pinchartl | works for me | [15:52] |
| .... (idle for 15mn) | ||
| *** | Epakai has quit IRC (Ping timeout: 480 seconds) | [16:07] |
| Epakai has joined #linux-media | [16:13] | |
| ezequielg | hverkuil: let me know if you have some questions about the tiled pixel format patch (pinchartl you as well, as this is sensitive stuff). i personally think we are currently a bit messy and i think the result is quite nice. | [16:15] |
| pinchartl | ezequielg: we're definitely messy. I can't comment on the result yet, as I haven't read it :-) | [16:15] |
| ezequielg | grab it while it's hot! | [16:18] |
| ...... (idle for 25mn) | ||
| *** | GBenji has quit IRC (Quit: Leaving.) | [16:43] |
| ..... (idle for 21mn) | ||
| ao2 has quit IRC (Remote host closed the connection) | [17:04] | |
| ....... (idle for 30mn) | ||
| jernej | mchehab: would it be possible to fix IRC search at https://linuxtv.org/irc/oftc/irclogger_log_search/linux-media ? it seems agrep is not installed on server | [17:34] |
| ...................... (idle for 1h46mn) | ||
| *** | b-rad has quit IRC (Ping timeout: 480 seconds) | [19:20] |
| ....... (idle for 31mn) | ||
| BrianG61UK_ has quit IRC (Quit: Leaving)
BrianG61UK has quit IRC (Quit: Leaving) | [19:51] | |
| ao2 has joined #linux-media | [20:02] | |
| BrianG61UK has joined #linux-media
BrianG61UK_ has joined #linux-media | [20:10] | |
| ............ (idle for 56mn) | ||
| ao2 has quit IRC (Quit: Leaving) | [21:07] | |
| ...... (idle for 29mn) | ||
| b-rad has joined #linux-media | [21:36] | |
| ............ (idle for 55mn) | ||
| b-rad has quit IRC (Ping timeout: 480 seconds) | [22:31] | |
| ............. (idle for 1h0mn) | ||
| djrscally has quit IRC (Ping timeout: 480 seconds) | [23:31] | |
| ..... (idle for 22mn) | ||
| camus has quit IRC () | [23:53] | |
| ↑back Search ←Prev date Next date→ Show only urls | (Click on time to select a line by its url) |