| ↑back Search ←Prev date Next date→ Show only urls | (Click on time to select a line by its url) |
| Who | What | When |
|---|---|---|
| *** | djrscally has quit IRC (Ping timeout: 480 seconds) | [00:01] |
| ...... (idle for 25mn) | ||
| ndufresne | Well, can't blame them if the MC design does not cover their use case. Of course you can blame them making it work the wrong way, but they most likely don't really know this.
What I'd blame them, is not testing their ideas by submitting. pinchartl: the libcamera is based on algorithm, based on research paper, the rkaiq is based on trained AI model. The models can be reversed, they are likely off the shelf, but training and training data, that seems very hard | [00:26] |
| pinchartl | how do you know it's based on training AI models ?
don't be fooled by "aiq" in the name | [00:31] |
| ndufresne | The rknn kernel process kicks, I can probably with Tomeu's help dump the commands
(it also crash nicely if you remove rknn) Everything in that bsp default when something is not exactly as expected:-) With NPU on sensors, I can foresee that the stats may no longer come from the ISP in the future | [00:32] |
| *** | danitool has quit IRC (Quit: Cubum autem in duos cubos, aut quadratoquadratum in duos quadratoquadratos) | [00:34] |
| pinchartl | that wouldn't seem to be a very efficient evolution of the ecosystem
and regaring rkaiq | [00:35] |
| ndufresne | S/default/segfault | [00:35] |
| pinchartl | if I were to guess (without having looked at it at all) I'd say they may use inferencing for AWB, and traditional algorithms for the rest | [00:36] |
| ndufresne | I don't see it the same way as you here, having the stats from the sensor would reduce latency
Will be guesses on what exactly they do in AI | [00:36] |
| pinchartl | it won't reduce latency with offline ISPs
and actually won't help much with inline ISPs either | [00:38] |
| ndufresne | In theory, but AI model don't need defined statistics
We'll see how the future goes, meanwhile, you can likely drive this rkisp3 with algo from rkisp1, and get recognisable image, as a starting point And before that, there is 120K loc to upstream | [00:40] |
| pinchartl | which can probably be reduced substiantially :-) | [00:42] |
| ndufresne | (I really hope so, feels pretty large) | [00:43] |
| pinchartl | (if you're talking about using NNs to perform the image processing itself, completely replacing the ISP, that's a different question, but just to get the sensor to generate stats, I don't see that as a path the industry will take)
and regarding reverse engineering unless you're considering decompiling the closed-source daemon whether it uses traditional algorithms or inference to compute the ISP parameters, it won't make a difference | [00:43] |
| ndufresne | We'll do what is needed to progress toward an open driver and userspace | [00:45] |
| pinchartl | one interesting question will be code sharing | [00:46] |
| ndufresne | But I don't see why we'd care much about the daemon, dumping the params and comparing against another implementation in same light conditions seems already enough RE | [00:46] |
| pinchartl | do we want a single kernel driver that will cover the rkisp1 and newer generations ?
I'd recommend starting with a prototype anyway, to get a feel of how things are working you need to find out what the ISP parameters are and how they're expressed. how will you do that ? | [00:46] |
| ndufresne | We will ask?
How do you get that when working with Pi or other vendors other the asking? | [00:48] |
| *** | BrianG61UK has quit IRC (Ping timeout: 480 seconds) | [00:49] |
| pinchartl | many vendors don't want to answer that question
it would be easy otherwise :-) you will likely find out that, at least for a significant set of parameters, you won't get much information sometimes because they simply have no written documentation to start with you're in for a rougher ride than you expect :) of course, the vendor may be more cooperative than I expect, who knows. I love being proven wrong in such cases | [00:51] |
| ndufresne | I like how positive all this sounds (irony), I'm not a fool, and you are not the only one that knows these things
It will be rough, but at least it will have a foundation, this won't affect my specific project | [00:54] |
| pinchartl | sure. step-by-step improvements is the best option | [00:56] |
| ndufresne | And by foundation, I think solving the stuff that Micheal raised with vicap would be a start
The driver itself might not share anything with rk3588, it is pretty different | [00:56] |
| pinchartl | I have a feeling addressing the V4L2 lifetime management issues will be a prerequisite for that | [00:59] |
| ndufresne | Sakari said it's unfixable and must be workaround
Not sure what you expect? | [01:00] |
| pinchartl | nothing is "unfixable", it depends how much effort we're willing to put on it
I agree it's a harsh problem especially with DVB using MC to fix things properly, we would need to also rewrite the whole lifetime management of DVB, which is clearly not worth it I think we should decouple DVB from the rest v4l2-async likely needs to be rewritten too | [01:01] |
| ndufresne | It's clear that hacking DVB is different skill set, I would not know how to test
Changing subject, any updates on the multi-context MC? | [01:02] |
| pinchartl | that's a question for jmondi
I think we have plans to work on a new version but I don't know the details | [01:05] |
| *** | ten157237743246305066182150355 has quit IRC (Remote host closed the connection)
ten157237743246305066182150355 has joined #linux-media | [01:06] |
| pinchartl | btw, we also have plans for a M2M framework where job execution will need to span multiple drivers
(with the input DMA engine, processing block and output DMA engine are all separate IP cores, with separate drivers) | [01:08] |
| ...... (idle for 29mn) | ||
| *** | BrianG61UK has joined #linux-media | [01:37] |
| ............. (idle for 1h0mn) | ||
| mriesch | what are those v4l2 lifetime management issues exactly? | [02:37] |
| and iiuc the multi-context MC framework will facilitate using the time division multiplex operating mode of the newer rk isps
together with the vicap<>isp connection, this would suggest that a new driver for rkisp >= v2.0 is the most suitable option | [02:43] | |
| ......... (idle for 43mn) | ||
| *** | BrianG61UK has quit IRC (Read error: Connection reset by peer) | [03:30] |
| ............. (idle for 1h1mn) | ||
| mvchtz has quit IRC (Remote host closed the connection)
mvchtz has joined #linux-media | [04:31] | |
| ........ (idle for 35mn) | ||
| BrianG61UK has joined #linux-media
BrianG61UK has quit IRC (Read error: Connection reset by peer) | [05:06] | |
| ......... (idle for 43mn) | ||
| Mo has joined #linux-media | [05:51] | |
| mvaittin has joined #linux-media | [06:03] | |
| ............. (idle for 1h1mn) | ||
| dcz has joined #linux-media | [07:04] | |
| djrscally has joined #linux-media | [07:11] | |
| ..... (idle for 20mn) | ||
| GBenji has joined #linux-media | [07:31] | |
| prabhakalad has quit IRC (Quit: Konversation terminated!)
prabhakalad has joined #linux-media | [07:36] | |
| jmondi | ndufresne: it's planned work, likely to happen in Q2
my plan is to talk with you and Hans to see if the new context could be unified with the existing m2m_context | [07:41] |
| *** | tmerciai1 has joined #linux-media
dcz has quit IRC (Quit: Konversation terminated!) dcz has joined #linux-media dcz has quit IRC () | [07:45] |
| .......... (idle for 49mn) | ||
| lplc | pinchartl: hi, should I resend this (https://patchwork.linuxtv.org/project/linux-media/patch/20241023085643.978729-1-laurentiu.palcu@oss.nxp.com/)? It kind of sits unnoticed since Oct... :/ | [08:40] |
| *** | gnuiyl has quit IRC ()
mrpops2ko has quit IRC () | [08:43] |
| mriesch | "Linux-media developers are encouraged to fork $fdo/media-commiters.git under $fdo/users/$username." <- do i need to be in the group for that? | [09:00] |
| pinchartl | lplc: sorry about that. no need to resend, I've dug it up from my inbox, I'll review it | [09:14] |
| lplc | pinchartl: thanks | [09:14] |
| mriesch | ribalda: maybe ^ | [09:20] |
| ............ (idle for 57mn) | ||
| *** | ao2 has joined #linux-media | [10:17] |
| ................... (idle for 1h33mn) | ||
| ndufresne | jmondi: oof, existing m2m is a bit loaded, I've been thinking about it and leaned toward (without really being decided) in forking it into more specialized version for request based / multicore codec
mriesch: you need fdo fork rights, when you click fork you'll get explanation why, and how to request it, you'll want CI-ok rights I believe (it's mostly because of spammers and crypto miners who steal our computation time) | [11:50] |
| *** | mrpops2ko has joined #linux-media | [12:09] |
| .... (idle for 19mn) | ||
| ndufresne | jmondi: one of the thought I had about m2m context is that once you have request the implicit 'job" becomes redundant and confusing, are you aiming for request, or keep all the nodes async? | [12:28] |
| ....... (idle for 33mn) | ||
| *** | dcz has joined #linux-media
hansg has joined #linux-media | [13:01] |
| BrianG61UK has joined #linux-media | [13:17] | |
| mriesch | ndufresne: ok thanks! | [13:19] |
| ....... (idle for 31mn) | ||
| pinchartl | ndufresne: regarding request-based M2M, I think we should redesign the way requests are handled :-D
(not a surprise, given how I tend to want to redesign everything in V4L2) | [13:50] |
| mripard | *only* in v4l2? | [14:01] |
| ndufresne | pinchartl: yeah, but I don't trust you so much to make them nice to use for codecs ;-P | [14:04] |
| pinchartl | :-)
that's the part where I would rely on your expertise of course jokes aside, I have ideas for media pipelines and requests and if we move forward with them, I would want to make sure they would work well for codecs | [14:04] |
| jmondi | ndufresne: if I got your question right, I don't think the plan is currently to use requests for ISPs, no
I know someone tried, not sure where it went | [14:16] |
| *** | mvaittin has quit IRC (Ping timeout: 480 seconds) | [14:17] |
| pinchartl | jmondi: we should consider that
but I'd like to be done on the MC device | [14:17] |
| jmondi | yeah but that's not the immediate goal for introducing contexts, isn't it ?
context will likely make it easier to use requests | [14:17] |
| ndufresne | for me the request make the userspace programming model nicer, and its bigger of importance then the driver side | [14:18] |
| pinchartl | not the immediate goal, no | [14:18] |
| ndufresne | but with some work on the m2m context, we can use request to make multi-core handling nicer | [14:18] |
| pinchartl | but a nearer-than-longer term goal still | [14:19] |
| jmondi | ndufresne: I don't know enough about codec to comment :) for sure the idea is to be allow current users of requests for codecs work the same way as they do today even if we unify the video device contexts and the m2m context
going forward I think that once we have contexts properly in place, going towards a model where requests are issued on a media device and consumed "atomically" will be key to allow proper per-frame-control operations but again, this won't be part of the introduction of video device context which rather paves the way for it | [14:19] |
| ndufresne | jmondi: whatever brings us closer to a command stream makes the codec programming nicer | [14:26] |
| *** | Mo has quit IRC (Ping timeout: 480 seconds) | [14:27] |
| jmondi | I think so too :) (even it won't be exactly like a gpu command stream) | [14:28] |
| ..... (idle for 22mn) | ||
| *** | BrianG61UK has quit IRC (Ping timeout: 480 seconds) | [14:50] |
| ndufresne | pinchartl: about cifv3 + rkisp3, I think these two things don't trully interact with each other, except that if the cif is not setup, the rkisp3 stuff just never produce anything (like when you have no signal over HDMI)
I'm not saying this is what we need to aim for, just describing what I see they most likely break the layers to notify each other, vendors always do that | [14:59] |
| pinchartl | ndufresne: isn't the same CSI-2 RX a shared input between the CIF and ISP ? | [15:03] |
| .... (idle for 16mn) | ||
| ndufresne | the CIF routes the signal, picking from any of the 6 CSI-2 and routing it to one of the 2 ISP, so answer is yes | [15:19] |
| *** | dcz has quit IRC (Quit: Konversation terminated!) | [15:20] |
| ...... (idle for 26mn) | ||
| mvaittin has joined #linux-media | [15:46] | |
| .... (idle for 18mn) | ||
| mvaittin has quit IRC (Ping timeout: 480 seconds) | [16:04] | |
| dcz has joined #linux-media | [16:13] | |
| ........ (idle for 36mn) | ||
| jarthur has joined #linux-media | [16:49] | |
| ............ (idle for 56mn) | ||
| tmerciai1 has quit IRC (Ping timeout: 480 seconds)
GBenji has left | [17:45] | |
| .......... (idle for 46mn) | ||
| BrianG61UK has joined #linux-media | [18:32] | |
| ................ (idle for 1h16mn) | ||
| hansg has quit IRC (Quit: Leaving)
ao2 has quit IRC (Quit: Leaving) | [19:48] | |
| ....... (idle for 31mn) | ||
| dcz has quit IRC (Ping timeout: 480 seconds) | [20:23] | |
| .......... (idle for 49mn) | ||
| dcz has joined #linux-media | [21:12] | |
| sukbeom1 has quit IRC () | [21:17] | |
| sukbeom1 has joined #linux-media | [21:23] | |
| ........ (idle for 38mn) | ||
| Epakai is now known as Ekapai | [22:01] | |
| dcz has quit IRC (Quit: Konversation terminated!) | [22:11] | |
| .............. (idle for 1h6mn) | ||
| danitool has joined #linux-media | [23:17] | |
| ....... (idle for 34mn) | ||
| BrianG61UK has quit IRC (Read error: Connection reset by peer) | [23:51] | |
| ↑back Search ←Prev date Next date→ Show only urls | (Click on time to select a line by its url) |