[00:08] *** b-rad has joined #linux-media
[02:51] *** Epakai has quit IRC (Read error: Connection reset by peer)
[02:52] *** Epakai has joined #linux-media
[06:03] *** miqztee has joined #linux-media
[07:02] *** GBenji has joined #linux-media
[07:22] *** ao2 has joined #linux-media
[08:51] <mort_> are the mainline video decoders and their requirements documented anywhere..?
[08:54] *** djrscally has joined #linux-media
[08:54] <hverkuil> mort_: https://linuxtv.org/downloads/v4l-dvb-apis-new/userspace-api/v4l/dev-mem2mem.html
[08:54] <mort_> hverkuil: I mean the particulars of the different decoder implementations
[08:55] <hverkuil> No, sorry. The point is that they should all follow that spec, but some older drivers predate the spec and may deviate from it.
[08:57] <mort_> alright. Where does the spec document which particular kinds of h264 NAL units a decoder expects in the same buffer
[08:58] <hverkuil> The spec assumes that a stateful decoder has a proper parser, so you can just fill up the buffer to the max and let the HW handle it. Which apparently the venus driver doesn't support.
[08:58] <mort_> do any driver contain bitstream parsers?
[08:59] <hverkuil> coda might have some of that, if memory serves.
[09:01] <hverkuil> You really need to talk to the venus driver maintainer, but he's not here at the moment. You can mail him as well, of course.
[09:05] <mort_> do you know where he is by the way? Did he come over to oftc? I only saw svarbanov on freenode
[09:08] <hverkuil> I thought I saw him here before, but I'm not 100% certain. I saw a post from him on the linux-media mailinglist an hour ago, so he's certainly around :-)
[09:10] <hverkuil> yeah, he's been here before, last on the 19th.
[09:10] <hverkuil> Just mail him.
[09:10] <mort_> I'll do
[09:10] <mort_> I don't understand what gstreamer does; the bitstream contains a 30-byte SPS, a 10-byte PPS, a 690-byte SEI, then a 124-byte IDR frame, totaling 854 bytes, but the first buffer gstreamer queues is 862 bytes, 8 bytes more than 854
[09:12] <mort_> the next NAL unit after that is a 577-byte non-IDR frame, but the next buffer gstreamer queues is 583 bytes, 6 bytes more
[09:12] <mort_> there's clearly something I don't understand here
[09:13] <mort_> I could have a look at the source code of v4l2h264dec and see if I figure anything out
[09:14] <hverkuil> Sorry, not my area of expertise.
[09:15] <javierm> mort_: maybe ndufresne could help 
[09:16] <mort_> maybe. If anyone (such as ndufresne) knows about this stuff and wants to discuss, do ping me :) Otherwise I'll be reading gstreamer source code and maybe emailing varbanov if I can't figure anything out
[09:17] <mort_> the interesting thing is that his example decoder program *also* doesn't work
[09:20] <mort_> I wonder if h264parse messes with the bitstream..
[09:36] *** miqztee has quit IRC (Quit: Leaving)
[09:42] *** camus has joined #linux-media
[09:54] *** camus has quit IRC (Ping timeout: 480 seconds)
[10:36] *** NoGuest1_ has left Textual IRC Client: www.textualapp.com
[10:41] *** camus has joined #linux-media
[10:54] *** camus has quit IRC (Ping timeout: 480 seconds)
[11:04] *** camus has joined #linux-media
[11:12] *** BrianG61UK has quit IRC (Read error: Connection reset by peer)
[11:26] *** camus has quit IRC (Ping timeout: 480 seconds)
[12:08] *** miqztee has joined #linux-media
[13:22] *** ezequielg has quit IRC ()
[13:26] <mort_> Well, I sent that e-mail to stanimir, let's see if he has any ideas
[13:27] <ndufresne> mort_: each nals have a 3 to 4 bytes prefix
[13:27] <ndufresne> its called a start code, is that what you missed ?
[13:28] <ndufresne> mort_: but can't help more without the binary to look at
[13:30] <mort_> ndufresne: It sends the NALs including that prefix
[13:30] <mort_> https://p.mort.coffee/E84.c here's the source for the binary, https://p.mort.coffee/kq9.txt here's its output, https://p.mort.coffee/9Lh.txt here's an annotated log from the driver
[13:32] <mort_> if you follow the read_nal/read_nal_frame logic you'll notice that the first frame it sends starts at the very beginning of the file (including the first 0, 0, 0, 1), until the first NAL unit with type 5
[13:32] <ndufresne> mort_: the differents seems to match the number of emulation bytes, did you notice ?
[13:33] <mort_> what are emulation bytes?
[13:34] <ndufresne> it's an encoding pattern used to replace startcodes pattern withing the entropy coded stream
[13:34] <mort_> oh, I just googled it, so if it would otherwise generate a 0 0 1 sequence it will instead generate a 0 0 3 1
[13:34] <ndufresne> perhaps not the most obvious answer, but it prevent confusing start codes with actual bitstream syntax, and adds 1 bytes per replacement
[13:35] <ndufresne> in the first part, I counted 8 if I'm not miss-leading
[13:36] <mort_> I didn't know about emulation bytes, I did consider this issue but assumed that some clever detail about the encoding would prevent it, for example that exponential golomb would never produce 0 0 1 bytes
[13:36] <ndufresne> decompression staged from Annex B, is to scan for start code , and remove the emulation (sometimes done in-place by some HW), then parse the headers, slice headers, followed my the entropy codeding, which is bitstream parameter decompression, and finally the reconstruction will take place
[13:36] <mort_> though that doesn't make much sense now that I think about it
[13:36] <ndufresne> were did you get these sizes from ?
[13:37] <mort_> I got my sizes from the raw h264 file on disk, I don't do anything special with respect to emulation bytes
[13:37] <ndufresne> are you allowed to share the raw file ?
[13:38] <ndufresne> at least this is a strange coincidence ...
[13:38] <mort_> sure, https://p.mort.coffee/nsd.h264
[13:39] <mort_> it's just the first few frames from that big buck bunny cartoon iirc, been using that as my test h264 file for a while
[13:40] <mort_> how do you know the number of emulation bytes?
[13:40] <mort_> oh you're looking at the output with the first 32 and last 16 bytes of each buffer I'm sending
[13:41] <mort_> I assume that's a coincidence, since the buffer is 854 bytes and you're seeing 48 of them, there has to be more emulation bytes in there right
[13:42] <mort_> if it was a difference in emulation byte handling, then the difference would have to be that gstreamer strips out emulation bytes and I don't; if that was the case, then we would expect gstreamer to enqueue smaller buffers than my implementation
[13:44] <ndufresne> mort_: testing here (gst 1.18), in Annex B (byte-stream), I do get 862/583, and then in AVcC, 856/577
[13:46] *** miqztee has quit IRC (Quit: Leaving)
[13:47] <ndufresne> mort_: ok, nothing very special, your bytestream is missing an AUD nal
[13:48] <ndufresne> because muxing without AUD into MPEG TS is invalid, h264parse (and now H265parse too) will add it
[13:48] <ndufresne> 00 00 00 01 09 f0
[13:48] <ndufresne> and then your first start-code are invalid (3bytes), so h264parse will fix them to 4 bytes
[13:49] <ndufresne> note that decoders are super flexible on 3 vs 4 bytes start code, so not really a bug, but this is just h264parse being a bit pedantic
[13:49] <ndufresne> and the number of emulation bytes is just an accident
[13:49] *** camus has joined #linux-media
[13:50] <mort_> ah, that makes sense
[13:50] <ndufresne> so 6bytes of AUD + two bytes of 00
[13:52] <mort_> I mean, it's worth a shot to just add AUDs everywhere right
[13:52] <mort_> should be easy to write a python script to patch the h264 file
[13:56] <mort_> alright, so I just prepend every frame with an access unit delimiter NAL unit, I think I got it
[13:59] <mort_> ndufresne: I still don't quite understand where the 8 comes from though, the first few bytes in my bitstream is '00 00 00 01 67 64', so if you prepend an AUD to it that would be '00 00 00 01 09 f0 00 00 00 01 67 64' and I don't understand why that's wrong, the start code is still 4 bytes
[14:01] <mort_> so I understand why the rest of the buffers are 6 bytes longer thanks to the AUD, but not why the first buffer is 8 bytes longer
[14:08] <ndufresne> well, AUD needs to be added at frame boundary
[14:09] <mort_> which is at the beginning, + after every NAL unit of type 1 and 5, right?
[14:11] <mort_> oh, or are they added at the end of the buffer and not the beginning
[14:14] <mort_> still wouldn't make it 8 though, looking at the bytes of the first buffer I send
[14:15] <ndufresne> I'd need to check, but I see difference in the number of bytes in start code for each nals
[14:15] <ndufresne> notably SEI
[14:15] <ndufresne> did go further
[14:18] <mort_> hypothesis: h264parse parses the h264 bitstream with a fairly lenient h264 parser, then generates its own equivalent bitstream but which very strictly follows the spec (and whatever requirements not in the spec which hardware might impose on it)
[14:26] <mort_> ndufresne: a pipeline which consists of `filesrc location=before.h264 ! h264parse ! filesink location=after.h264` should in theory give me an h264 file with whatever v4l2h264dec gets, right?
[14:31] <ndufresne> mort_: no, you need to specify the format
[14:32] <ndufresne> ilesrc location=before.h264 ! h264parse ! video/x-h264,stream-format=bytes-stream ! filesink location=after.h264
[14:33] <mort_> aha
[14:36] <mort_> after doing that, I get the same sizes as h264v4l2dec, the first buffer now has a 6-byte AUD + a 1 byte longer SEI + a 1 byte longer IDR frame, the second buffer now has a 6-byte AUD but is otherwise unchanged
[14:37] <mort_> buuut poll still never returns anything other than 0 (timeout)
[14:48] <mort_> I can write my own LD_PRELOADable ioctl dumper and inspect exactly what gstreamer sets its formats to
[15:01] <ndufresne> mort_: decoder get stuck in gst or your app, or both ?
[15:02] <ndufresne> I think Stanimir intentially decided to break backward compat, and strictly depends on SRC_CH event
[15:02] <mort_> the decoder gets stuck in my test app (and in stanimir's test code) but not in gstreamer
[15:02] <ndufresne> I got the event imlement for v4l2src, but didn't find the time to add it to v4l2 decoder
[15:02] <ndufresne> oh, interesting, surprising also
[15:03] <mort_> he's back from vacation on monday
[15:03] <ndufresne> haha, right in time for me going into vacation
[15:03] <mort_> haha
[15:03] <ndufresne> mort_: iirc, the backward hack for non-srch-change event handler is in G_FMT(OUTPUT)
[15:04] <mort_> oh actually, interesting
[15:04] <mort_> https://git.linaro.org/people/stanimir.varbanov/v4l2-decode.git
[15:04] <ndufresne> it will block and wait till the headers has been parsed
[15:04] <mort_> the master branch is from 6 years ago but there's other recent branches
[15:04] <mort_> I should check out those other branches and see if they work
[15:05] <ndufresne> indeed, please notice ctrl_get_min_bufs_for_capture() in the simple branch
[15:06] <ndufresne> ignoring this is a nice way to get stalls
[15:06] <mort_> and the latest commit message is "modernize test app to work properly with latest v4l2 driver"
[15:06] <mort_> my test app is not modernized to work properly with the latest version of the driver
[15:10] <mort_> well would you look at that, the simple branch works flawlessly
[15:21] <mort_> hmmm I can't see that cap_min_bufs being used anywhere though
[15:29] *** GBenji has left 
[15:39] *** djrscally has quit IRC (Quit: Konversation terminated!)
[16:01] *** ezequielg has joined #linux-media
[16:47] *** BigBadBeautifulBoris has joined #linux-media
[17:00] *** ao2 has quit IRC (Remote host closed the connection)
[17:37] <tzanger> hey everying, I'm trying to capture a single image from a camera connected to an i.mx8 CSI port
[17:38] <tzanger> the camera appears to be fine (from v4l2-ctl --device /dev/video0 --all) and I can set the rate/format without it complaining
[17:38] <tzanger> but when I try to capture one frame, it hangs without anything useful in stdout/err or dmesg
[17:38] <tzanger> v4l2-ctl --device /dev/video0 --set-fmt-video=width=2592,height=1944,pixelformat=YUYV --stream-mmap --stream-to=/tmp/out.raw --stream-count=1 is the command I'm using
[17:39] <tzanger> if I use --verbose all the IOCTL responses are "ok" and the last thing I see is "VIDIOC_STREAMON: ok"
[19:46] *** Whoopie_ has joined #linux-media
[19:51] *** Whoopie has quit IRC (Ping timeout: 480 seconds)
[19:51] *** Whoopie_ is now known as Whoopie
[20:09] *** BigBadBeautifulBoris has quit IRC (Remote host closed the connection)
[21:00] *** BrianG61UK_ has quit IRC (Quit: Leaving)
[21:04] *** ao2 has joined #linux-media
[21:04] *** BrianG61UK has joined #linux-media
[21:09] *** BrianG61UK_ has joined #linux-media
[21:51] *** ao2 has quit IRC (Remote host closed the connection)
[23:37] *** camus has quit IRC ()