<!-- Some styling for better description lists --><style type='text/css'>dt { font-weight: bold;float: left;display:inline;margin-right: 1em} dd { display:block; margin-left: 2em}</style>
***: svarbanov has quit IRC (Ping timeout: 480 seconds)
<br> svarbanov has joined #linux-media
<br> eelstrebor has quit IRC (Quit: Ex-Chat)
<br> eelstrebor has joined #linux-media
<br> eelstrebor has quit IRC ()
<br> NiksDev has joined #linux-media
<br> jm_h has joined #linux-media
<br> fleebs has quit IRC (Quit: fleebs)
<br> fleebs has joined #linux-media
<br> jarthur has quit IRC (Ping timeout: 480 seconds)
sailus: <u>jmondi</u>: Buon giorno!
jmondi: <u>sailus</u>: hyvää huomenta sinulle
<br> (according to google translate)
pinchartl: huomenta Jacopo. mitä kuuluu tänään?
sailus: :-)
pinchartl: oho Sakari on myös tässä
sailus: <u>jmondi</u>: Jaakko olisi täydellinen suomalainen nimi sinulle.
pinchartl: haha
jmondi: ah yes, right, indeed
sailus: <u>jmondi</u>: Mitä kuuluu?
pinchartl: Sakari aina antaa suomalaista nimeä
jmondi: I like Jakko actually!
<br> I would have troubles finding an italian sounding name for Sakari
<br> <u>sailus</u>: all good, how about you ?
sailus: <u>jmondi</u>: I'm doing fine. I was just reading a patchset adding a new subdev callback. :-)
jmondi: is it from me ?
sailus: I believe it is.
<br> I was wondering why do you need to call the post_register callback directly from the max9286 driver.
jmondi: because I need all of the remote cameras to have probed first
sailus: It's already being called from v4l2_async_match_notify().
jmondi: yes, but after each single camera has probed
<br> the max9286 has a control for the GMSL bus signal amplitude
<br> which apply to the bus, so to all the connected cameras
<br> in order for the cameras to be able to comply with the increased signal amplitude they have to enable their noise immunity
<br> which happens in probe()
sailus: I remember that. But why the two locations?
jmondi: once all the cameras have probed, and all of them have enabled their noise immunity, the deserializer cycles on all the remotes and calls post_register()
<br> the two locations (I assume you mean in the core and in the driver) are there to maintain operability with receivers the do not specify the DEFER_POST_REGISTER flag
sailus: Ah, ok.
jmondi: so that if the receiver doesn't specify the flag, post_register() is called by the core just after probe()
<br> kudos to hverkuil for that
sailus: I think this is a hack. It's working around bus infrastructure we do not have.
jmondi: yes indeed sir
sailus: Albeit GMSL mostly stays out of the way, apart from this.
jmondi: I very much concur
<br> but I didn't have anything better to propose, if not introducing a new bus with a dedicated set of operations
sailus: You seem to call post_register unconditionally from the driver.
<br> Doesn't that mean two calls to drivers that don't specify the flag?
jmondi: the receiver specifies the flag (in this case the deserializer) at notifier registration time)
<br> so if the receiver has specified the flag, it knows post_register() won't be called by the core, and it's up to it to call it at the opportune time
sailus: Ah, I missed the flag was in the notifier.
jmondi: yes, in the max9286 driver in this case
sailus: How does power cycling affect this configuration?
<br> Say, I suppose you'd have to redo it if you power things off?
jmondi: I presume so, yes
<br> albeit in the setup we have, cameras are powered at probe time
***: ao2 has joined #linux-media
jmondi: <u>pinchartl</u>: is your finnish getting good or did you get quicker with translate ?
sailus: Have you thought how this could work if power management was supported?
pinchartl: <u>jmondi</u>: it's improving. I wouldn't say it's getting good yet :-)
jmondi: <u>sailus</u>: the GMSL serializer we operate in this setup as a sleep mode which retains registers configuration (and a powerdown mode which resets it)
<br> so I think power management should be realized using sleep mode
<br> <u>sailus</u>: I'll have a meeting soon, feel free to ask questions, I'll reply as fast as I can
sailus: How about the sensors?
***: ao2 has quit IRC (Quit: Leaving)
jmondi: back, sorry.. sensors in the camera modules are programmed with a pretty fixed configuration, and if the GMSL deser/ser keep their configuration and the noise immunity protection is still in place, we can safely re-write the long list of registers to the sensor if it's required
***: djrscally has joined #linux-media
sailus: <u>jmondi</u>: I'm back, too. :-)
<br> I was thinking of power management of the sensors.
<br> How does that work?
jmondi: in the camera modules we have, it is controlled by a gpio of the deserializer
<br> sorry, of the serializer, which is embedded in the camera module
sailus: Ok.
<br> I don't know all the details but I'd think you basically needed a GMSL bus and device drivers to implement power management properly.
<br> And sensor drivers would control sensors only.
<br> Whether or not this should be required for upstreaming I don't have an opinion on yet.
<br> There are no DT binding patches in the set anyway, so I guess there's nothing here that couldn't be in principle changed and fixed later on.
<br> In the kernel, that is.
jmondi: no DT changes no
sailus: Many similar systems suffer from problems that are very alike.
jmondi: there's also https://patchwork.linuxtv.org/project/linux-media/list/?series=6117
<br> where the serializer is made a standalone i2c driver and interacted through the subdev API instead of being a library module
<br> that would allow to separate the sensor driver from the camera module driver
<br> I'm still not sure if it's worth it
sailus: Future GMSL systems could have different requirements on power management.
montjoie: <u>hverkuil</u>: I hit the same problem than you now.. module is in use. but I dont know what could use it...
hverkuil: <u>montjoie</u>: videocodec_attach() calls try_module_get(), that's probably the cause. Since it's all compiled into a single module, that no longer makes sense.
<br> (the corresponding module_put also has to be dropped, of course)
***: BrianG61UK has quit IRC (Remote host closed the connection)
montjoie: <u>hverkuil</u>: thanks that was the problem
ezequielg: <u>hverkuil</u>: https://lkml.org/lkml/2021/9/7/221, i'm thinking maybe we should support custom/private v4l2 events?
<br> otoh, private events will surely make gstreamer/ffmpeg life pretty much impossible.
hverkuil: <u>ezequielg</u>: we do support private events, but it only makes sense for really driver-specific events. You need a really good reason to add a private event.
ezequielg: oh, right.
<br> i have to admit those amphion v4l2 events look a bit strange... as well as the buffer flag.
***: eelstrebor has joined #linux-media
<br> svarbanov has quit IRC (Ping timeout: 480 seconds)
ndufresne: <u>hverkuil</u>: V4L2_EVENT_SKIP is completely flawed, it assumes that V4L2 events and buffer are handled in lockstep
<br> if anything, we have timestamp to figure-out wholes, or we can use the sequence for that, like we do for cameras
<br> <u>hverkuil</u>: fyi, dv_ wrote a lib to support their downstream driver, and didn't manage to make it work with that event, he implement a garbage collector instead, using DPB max size of 16, and discard any pictures older that didn't complete
<br> And V4L2_EVENT_CODEC_ERROR is wrong, this is quite clear in the spec that decode error should be handled though the ERROR flag on buffers
<br> And you have two options, ERROR flags with the full bytesused value, meaning some stuff got decoded but it's corrupted, and decode errors with bytesused = 0, meaning nothing was decoded
hverkuil: <u>ndufresne</u>: did we document that last part? (error with bytesused == 0 or != 0)
ndufresne: this is a good question, I would need to check, perhaps its only documented in the leaf part of the V4L2 Capture
<br> this is pretty common behaviour for UVC drivers
***: svarbanov has joined #linux-media
hverkuil: I don't think it is, although it makes sense.
ndufresne: as it can underrun
<br> UVC can underrun 1 packet, which leads to corrupted frames, and will flag the buffer with an error and short size
hverkuil: <u>ndufresne</u>: when you said "dv_ wrote a lib to support their downstream driver,", you were referring to the amphion codec?
ndufresne: I have seen that very often with USB3 cams on slow CPU
dv_: <u>hverkuil</u>: he was
ndufresne: <u>hverkuil</u>: he has been maintaining gstreamer-imx (downstream imx) for quite a while, and this is abstracted with a lib, https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/mxc/hantro_v4l2?h=lf-5.10.y
<br> oops wrong links, even though this is an strange one
dv_: <u>hverkuil</u>: there seem to be _two_ amphion v4l2 drivers. one exists in the nxp linux-imx vendor kernel. the other was posted to the mailing list
<br> the one from linux-imx has some .. interesting behavior.
<br> but yeah, for skipped frames, that event is flawed
<br> ideally, I'd dequeue a v4l2_buffer with something like a SKIPPED flag set, plus the timestamp of the skipped frame
hverkuil: Ah, 'v8' was really the first time it was posted to the mailinglist. I don't have a great memory, but I did expect that I would have remembered seeing v1-v7 :-)
ndufresne: That seems like an option, VB2 whise it's probably a bit annoying, as these flag are bound to the state machine
dv_: <u>ndufresne</u>: I think you meant https://source.codeaurora.org/external/imx/linux-imx/tree/drivers/mxc/vpu_malone?h=imx_5.4.70_2.3.0
ndufresne: yes, sorry, and you lib link, which I forgot were it it again ...
tzanger: not sure if this is part of linux media -- is it possible to open a linux audio device and send/receive data without anything munging the data on the way in/out? I'm thinking of connecting one of hte SAI ports on an ARM SoC to an FPGA for a fast/low latency comm channel
ezequielg: <u>hverkuil</u>: I don't think v8 is the first one to hit the ML. See https://patchwork.linuxtv.org/project/linux-media/list/?series=&submitter=8542&state=*&q=&archive=&delegate=
tzanger: as far as the userspace device is concerned, it's just audio data, but I don't want any smoothing/filtering/volume/mixing going on with the data I write to or read from the device
hverkuil: <u>tzanger</u>: sorry, no. Not part of linux-media.
tzanger: <u>hverkuil</u>: no worries. :-) thanks
ezequielg: <u>hverkuil</u>: but I agree there seems to be a tendency to address small review feedback and post a new version, so that's why version is increasing so rapidly.
<br> back in the old days, we used to be more polite, and wait for more feedback before spamming the ML. but what do i know?
dv_: <u>ndufresne</u>: https://github.com/Freescale/libimxvpuapi/blob/imx8-amphion-backend/imxvpuapi2/imxvpuapi2_imx8_amphion.c
ndufresne: ah, thanks
dv_: now its me who posted the wrong link :) that is the amphion backend
<br> https://github.com/Freescale/libimxvpuapi/ correct link to the library overall
<br> and just for my understanding - nothing in the mem2mem spec makes it OK for drivers to require that the v4l2_buffer timestamp field is set to valid PTS for frame reordering to work, right?
<br> in other words, the timestamp field should purely be treated as opaque user data, and the driver's frame reodering should not rely on it?
ndufresne: correct, the m2m decoders are kind of expected to set V4L2_BUF_FLAG_TIMESTAMP_COPY , which basically means the driver will correctly match the OUTPUT and the CAPTURE buffer and copy the timestamp, but the value of the timestamp will remain opaque
hverkuil: <u>ezequielg</u>: ah, they changed the name from imx8q to amphion. That's what was confusing.
<br> or is it? Now I'm confused.
ndufresne: this task can be difficult, as the HW may not have enought bits to store the value as-is, in which case you need to store the value somewhere in a fifo, and do lookup, but so far there is pretty much always a solution
hverkuil: yes, it's a rename.
ndufresne: <u>hverkuil</u>: yes, that was me asking this
dv_: <u>ndufresne</u>: that's perhaps the biggest "specialty" of the imx amphion driver
<br> it does use the timestamp for frame reordering
<br> its not opaque userdata
<br> from what I saw, nxp did patch your v4l2 decoder elements in the nxp gst fork rather significantly as a result
ndufresne: yeah, I'll check this v8 with that in mind, this is rather problematic, I hope they can patch their firmware to fix it
***: camus has quit IRC ()
ezequielg: <u>dv_</u>: so it means vanilla gstreamer will not work with this amphion driver?
<br> <u>hverkuil</u>: yeah, it was renamed. fwiw, aside from the core changes, the driver looks ok. although it seems worrying that is expects some special userspace.
dv_: <u>ezequielg</u>: yeah, there are errors
<br> but as said, there is another driver in the mailing list. from what I remember, that one is entirely different.
ndufresne: <u>ezequielg</u>: if someone had a board, we could just test it and see, and then report if mainline gstreamer/v4l2 didn't work
<br> but I don't have this type (my time is also rather limited lately)
***: eballetbo_ has joined #linux-media
<br> eballetbo_ has quit IRC ()
<br> BrianG61UK has joined #linux-media
<br> eballetbo has left The Lounge - https://thelounge.chat
<br> eballetbo has joined #linux-media
ndufresne: <u>ezequielg</u>: hverkuil: looks like we'll need to think about V4L2_BUF_FLAG_CODECCONFIG, this is a mimic from OMX, which we did not carry, as none of the drivers trully needed that
<br> Annex B (with started code) decoders, are generally highly flexible, and you can just pass data, and it will take care of parsing, frameing, etc., so for those, this flag can used and ignored without problem
dv_: this would not affect stateless decoders though, right?
ndufresne: But userspace like gstreamer always pass these headers with the first frame, so this flag would need to be strictly optional to keep working
<br> no, stateless decoders gets the headers through a control, already parsed and stored in a C struture
<br> so I don't think this flag is wrong, but I'm not certain either what is the implication for a software using this
***: svarbanov has quit IRC (Ping timeout: 480 seconds)
<br> camus has joined #linux-media
<br> svarbanov has joined #linux-media
<br> GBenji has left
<br> gouchi has joined #linux-media
<br> svarbanov has quit IRC (Ping timeout: 480 seconds)
<br> ao2 has joined #linux-media
<br> camus has quit IRC ()
<br> Ampera has quit IRC (Ping timeout: 480 seconds)
<br> Ampera has joined #linux-media
ezequielg: <u>ndufresne</u>: so far the main problem I see here, is that the driver doesn't seem to work with the stateful spec.
ndufresne: <u>ezequielg</u>: from reading the code or do you actually have HW to test this ?
<br> (my curiosity, if you can find issue from reading the code, that is as good imho)
<br> they should not hide issues they met, and instead discuss them openly
ezequielg: well if you need a new flag .... :)
<br> i'm inclined to think it won't work without that flag.
<br> but no, i haven't checked the code.
ndufresne: that's my point, it's not clear if the driver needs this CODECCONFIG flag, or simply that they need this *if" the first buffer is not combined with a new frame ..
<br> e.g. I'm wondering if they added this because they implement OMX on top
<br> <u>ezequielg</u>: I'll try and ask for clarification, and then I'll give them instructions to drop or how to fix this
<br> since if this is what I think it's kind of valid, but it would need a support flag (so userspace can detect drivers that don't support it) and the doc needs to be extended
ezequielg: it's not only about the flag, is it? but also about mandating a header buffer and then a VLC buffer.
<br> VCL buffer I mean.
***: ao2 has quit IRC (Quit: Leaving)
<br> gouchi has quit IRC (Remote host closed the connection)
<br> Whooa22 has joined #linux-media
<br> KitsuWhooa is now known as Guest6628
<br> Whooa22 is now known as KitsuWhooa
<br> Guest6628 has quit IRC (Read error: Connection reset by peer)
<br> jm_h has quit IRC (Remote host closed the connection)
<br> svarbanov has joined #linux-media
<br> djrscally has quit IRC (Ping timeout: 480 seconds)
<br> NiksDev has quit IRC (Ping timeout: 480 seconds)
<br> BrianG61UK has quit IRC (Ping timeout: 480 seconds)
<br> BrianG61UK_ has quit IRC (Ping timeout: 480 seconds)