<!-- Some styling for better description lists --><style type='text/css'>dt { font-weight: bold;float: left;display:inline;margin-right: 1em} dd { display:block; margin-left: 2em}</style>
ndufresne: pinchartl, just testing here with a random logitech camera, without enaling the timestamp, the timestamps indicates 200ms latency, and the first timestamp has 2s latency
<br> pinchartl, after I enabled the HW timestamp, every frame have steady 66ms latency
<br> (just measuring the delta between the frame TS and the monotonic clock here)
gnurou: <u>hverkuil_</u>: thanks, trying to take a look today
<br> paulk-gagarine: the documentation should probably be rewritten from scratch anyway
<br> depending on how much has changed
paulk-gagarine: gnurou, right
hverkuil_: paulk-gagarine: the basics haven't changed, so most of the documentation is still valid. What's missing is documentation for V4L2_CTRL_WHICH_REQUEST and V4L2_BUF_FLAG_REQUEST_FD/IN_REQUEST. But that's easy enough to fix.
paulk-gagarine: hverkuil_, indeed -- I only noticed because it still mentions the VIDIOC-flavour ioctl for creating requests
pinchartl: <u>ndufresne</u>: yes they're disabled by default as they are buggy :-/
<br> I haven't got around to fixing them
ndufresne: pinchartl, but the fallback is sometimes worst, how come we have such a bad fallback, what is it based on ?
<br> I would expect near to 0 delay, but the delay can be huge, specially on the first frame
pinchartl: the "fallback", which is a revert to the previous method, just gets the system time when the driver receives the last packet of the frame
<br> so it's highly dependent on system load
<br> hardware timestamps are much better, but only if they can work reliably
<br> there's a bug something in the implementation
<br> I don't know where
<br> that causes timestamps to jump around sometimes
<br> for as much as several seconds forward
<br> and then back
<br> and that breaks several applications
Whoopie: <u>ndufresne</u>: right, that's what I also found out, but I tested with ld_preload. But I get the next error message then, see https://bugzilla.gnome.org/show_bug.cgi?id=795097
<br> the ld_preload debug output is attached there.
ndufresne: I don't think LD preload will work, libv4l2 is not robust enough
Whoopie: nevertheless, I tested the export command, it's the same behaviour.
ndufresne: pinchartl, there is something happening that does not match that description, because the first TS matches the time I have called S_FMT
<br> that's very reliably the case
Whoopie: <u>ndufresne</u>: here's the error message:
<br> (cheese:11477): cheese-WARNING **: 13:27:19.102: Internal data stream error.: gstbasesrc.c(3055): gst_base_src_loop (): /GstCameraBin:camerabin/GstWrapperCameraBinSrc:camera_source/GstBin:bin18/GstV4l2Src:v4l2src1:
<br> streaming stopped, reason error (-5)
pinchartl: <u>ndufresne</u>: that's weird
ndufresne: pinchartl, indeed, this "first ts" is really the worst, but then the delay is pretty much fixed (except as you said, when it randomly kicks)
<br> but the delays are often much larger then they should, specially that we don't try and account for the real capture time
<br> normally that timestamp should be always later then the HW timestamp
-: ndufresne didn't mananed to understand the code ...
pinchartl: <u>ndufresne</u>: it could be that the driver gets the system time at the start of the frame, not the end, my bad
ndufresne: pinchartl, we'll need to fix that, the "start" is like the request, I think we should try and get the ts when the first payload arrives (or when the last), otherwise we depend on the device response time
<br> e.g. the long setup delay, the light conditions would increase the latency too, which isn't true, it should only make the framerate sporadic
pinchartl: <u>ndufresne</u>: I agree, but we should also fix the hw timestamp support
ndufresne: yep
<br> if that's reassuring, it seems the HW timestamp works on older HW
<br> but I don't have a very large collection of HW to test with
<br> a CI, that have hundreds of webcam would be really nice ;-P
Whoopie: <u>ndufresne</u>: I also installed fedora 28, it's the same bahaviour as under ubuntu 18.04, I exported GST... and ran cheese.
ndufresne: Whoopie, what I need is GStreamer version
Whoopie: <u>ndufresne</u>: it's the 1.14.0, sorry
ndufresne: make sense, with 1.14 we need to enhance cheese, so it can decode the jpeg
<br> for camera's like yours
<br> but as I said, setting the env will bring back the old behaviour, so you can use that as a workaround
<br> while at it, it would be nice to make cheese a tad more robust
Whoopie: <u>ndufresne</u>: no, it doesn't. I gives the error which I posted above. The full debug output is at https://bug795097.bugzilla-attachments.gnome.org/attachment.cgi?id=370679
ndufresne: it should work
Whoopie: it doesn't ;-) neither on fedora 28 nor on ubuntu 18.04
ndufresne: Whoopie, can you translate, fdmemory gstfdmemory.c:127:gst_fd_mem_map: 0x55725546dc00: fd 33: mmap failed: Das Argument ist ungültig
<br> it picks RGB 24bit (packed), which is an very odd format
Whoopie: the argument is not valid
ndufresne: hmm, mmap() on the dmabuf failed
<br> ah, I see, libv4l2 pretends we can export DMABuf, so gst do export dmabuf, but's it's an emulated format
<br> thanks you libv4l2 !
<br> I guess I'll need to add yet another libv4l2 workaround ...
<br> or I just ignore it, and add jpegdec support to cheese, which seems easier, better maintained anyway
hverkuil_: libv4l2 is effectively unmaintained ever since Hans de Goede moved on to other parts of the kernel. Volunteers are welcome!
<br> I'm not sure how relevant it is anymore. The old gspca webcams are almost all replaced by uvc. That leaves the upside-down detection of laptop webcams.
Whoopie: <u>ndufresne</u>: I'm happy to test any patches!
ndufresne: hverkuil_, there is a lot of odd optimization in there, I think that scares away potential maintainers
<br> hverkuil_, I think overall, it could be relevant if it was making programming v4l2 nicer / easier to read, like libdrm, then you'd have usespace that really depends on it, and have more contributor
hverkuil_: I agree. But who would be willing to do that?
ndufresne: no idea, I believe libdrm is born from XFree86 / Xorg people, not from kernel
hverkuil_: The obvious counterpart to that would be gstreamer people.
ndufresne: but these people you know, consultants ;-P
<br> it remains pretty fragmented, specially that v4l2 support is likely going to be replaced by pipewire
<br> Xorg was shared by a lot more groups
<br> anyway, for now I'm adopting the route of not using it
<br> I've looked at the state less codec PoC, and it's worst then VAAPI, double parsing, no way to debug ...
<br> the only possible remaining would be complex cameras, but as it goes, no one is proposing really functional patches, checks the IPU3 stuff
hverkuil_: You're talking about the stateless codec PoC for libv4l2?
<br> that was posted a looong time ago.
Whoopie: <u>ndufresne</u>: what is not clear to me is why libv4l exports dmabuf in newer releases (the working ubuntu has libv4l 1.12.5, the non-working is 1.14.2.
<br> <u>hverkuil_</u>: do you have an idea maybe? I see, you commit a lot to v4l-utils.
hverkuil_: <u>Whoopie</u>: not so much to libv4l2. I can't offhand remember any changes there. That does not necessarily mean much :-)
ndufresne: Whoopie, gstreamer exports dmabuf by default now, that's it, and rely on the emulation to disable it if not supported
<br> if the emulation is incomplete, which is the case here, you can weird failure in gst, because we mainly push dmabuf with jpeg, and pretends it's RGB
<br> I already have hacks the disable CREATE_BUFS if the converter is used, because libv4l2 crashes badly
<br> I tried to fix it, but they allocate memory with a super strange linux specific trick that I didn't understand
<br> and I would have had to drastrically change the way memory is allocated, without any context of why it was done like this
hverkuil_: <u>ndufresne</u>: my recommendation would be to just drop libv4l2 in gstreamer. I think it is no longer very useful and more of a hindrance than a help.
ndufresne: it's not like we have done nothing, thiago provided patches to fix deadlock in the library
hverkuil_: My only worry would be with laptops and upside-down sensors.
ndufresne: hverkuil_, I agree, in Whoopie case, it's camera is jpeg only, so I need to add a jpeg decoder in cheese, though in general, it performs better then libv4l2 (specially if it's ffmpeg jpeg decoder)
<br> hverkuil_, don't we have HFLIP hint we can add as quirks in the kernel ?
<br> I've heard about these, but I don't have any HW that behaves like this
<br> in gst, we have AffineTransform meta that can be used to describe these issues, and previously we'd simply use the orientation tags
hverkuil_: we don't have that, but I think it could be added.
<br> I think such laptops are getting rare. The last addition to the upside down table was early 2016.
Whoopie: how can I override GST_V4L2_IO_AUTO with GST_V4L2_IO_MMAP by default?
<br> <u>ndufresne</u>: ^
ndufresne: Whoopie, depends were you want to hack this
<br> I have to go now
paulk-gagarine: hverkuil_, have you tried calling streamon before queuing buffers on vim2m?
<br> it works fine when calling it after queuing buffers, since the call to v4l2_m2m_try_schedule succeeds
<br> however, when the start_streaming qos is called, v4l2_m2m_try_schedule is never called at any point
<br> so there is no call to v4l2_m2m_try_schedule when streamon is issued before queuing the buffers
<br> hverkuil_, I tried adding v4l2_m2m_try_schedule in the start_streaming qos and it makes a mess
<br> related dmesg: http://paste.debian.net/plain/1019571
hverkuil_: paulk-gagarine: no, that's not tested in v4l2-compliance.
<br> Or at least not in the request-related tests.
paulk-gagarine: hverkuil_, can you confirm that it's a valid use case for m2m?
hverkuil_: Yes, that should work.
<br> It's why the patch series is still RFC :-)
paulk-gagarine: okay, I had to fix a similar case in gnurou's series: https://patchwork.linuxtv.org/patch/47783/
Whoopie: <u>ndufresne</u>: it works! my patch is at http://paste.debian.net/1019568/
paulk-gagarine: not sure I can get a driver callback and submission-time anymore
Whoopie: but next issue:
<br> Device '/dev/video0' does not support bt709 colorimetry: gstv4l2object.c(3780): gst_v4l2_object_set_format_full (): /GstCameraBin:camerabin/GstWrapperCameraBinSrc:camera_source/GstBin:bin18/GstV4l2Src:v4l2src1:
<br> Device wants 2:4:7:1 colorimetry
paulk-gagarine: hverkuil_, I'll go with adding v4l2_m2m_try_schedule in my drivers's req_queue
<br> if that works, we should probably do the same in vim2m to cover that use case
Whoopie: <u>ndufresne</u>: reducing the webcam resolution to 640x480 fixes it.
ndufresne: Whoopie, hmm, that's another bug I suppose, the "defaults" for colorimetry don't match what is valid for RGB
<br> because the format being emulated is JPEG (YCbCr) but the output is RGB
<br> the v4l2convert that transform to RGB should hardcode the colorimetry imho
<br> instead of passing it through
<br> (isnide libv4l2)
<br> Whoopie, it would be probably more constructive to patch libv4l2, return ENOTTY to EXPBUFS when v4l2convert is in use, and forcing SRGB colorspace when we convert to RGB
Whoopie: <u>ndufresne</u>: that's nothing I can do.
ndufresne: haha, as I said, hacking gst is easy ;-P
<br> Whoopie, obviously you patch is not correct, because if you didn't pick an emulated format, you don't want to loose dmabuf support
Whoopie: <u>ndufresne</u>: I see
<br> <u>ndufresne</u>: as I said, I'm happy to test any patches. ;-)
paulk-gagarine: hverkuil_, I don't see a single way to get my m2m context from media_request's request_queue
<br> mhh
<br> nevermind
<br> no, still valid
<br> (sorry)
ndufresne: paulk-gagarine, how does it work in vim2m ?
paulk-gagarine: ndufresne, nah nevermind, I resovled my issue some hours ago
<br> I was selecting for read fds
<br> instead of exception fds
<br> so the result would have been the same with vim2m probably
-: ndufresne will have to look at your vaapi code at some point ...