hverkuil: I find it a bit annoying that in current draft of decoder interface the dynamic resolution change stalls the pipeline
even if you have big enough buffers, the driver will have to wait for the userspace to handle the source change event, before it can decode next frames
tfiga: the main reason for this limitation is that you cannot tell userspace what the resolution of the capture buffer is. I.e. there is no width/height information in struct v4l2_buffer.
What if we would have this information, would we then be able to do this? (Provided the buffer sizes are large enough, of course)
hverkuil: hmm, I guess we could address this limitation incrementally, based on request API?
not only resolution, but also crop
hverkuil: we might not need to pass the width and height back explicitly, if there was a flag in the buffer saying that something has changed and that the application should dequeue the corresponding event and process it before processing the contents of the buffer, it could be enough
The problem with these solutions is that they feel like hacks. The core problem is that struct v4l2_buffer is too small and you really want to be able to communicate (a lot?) more information in it.
I've been of the opinion for a long time now that we need a struct v4l2_ext_buffer that allows such things. I've postponed work on it until the request api was done.
if we pass the information through requests, we might not need to extend the v4l2_buffer structure
I think it would be cleaner to associate buffers with other data through a request instead of trying to stuff the data in the buffer structure
hverkuil: I'm still a big fan of unifying the configuration interface
making all the random interfaces, such as S_FMT, S_PARAM, S_EXT_CTRL, follow the same model
then we could easily attach any kind of data to reuqests
So this would mean that you need to use the request api for capture buffers, also for stateful codecs. I'm not at all convinced. You definitely would have to prototype this first with both a stateful and a stateless codec.
hverkuil: yeah, for stateless this will be also useful, because it's actually the userspace which deals with resolution changes completely
without being able to set format from a request, one would have to stall the pipeline again
My camera capture device is memory mapped, and I export its addresses with vidioc_qbuf. My encoder output device is using dmabuf. All setup seems to work, and no ioctls fail, but when I try to vidioc_qbuf the encoder output device with the dma file descriptor from the recently vidioc_dqbuf'd camera capture device, I get a failure with EFAILand __qb
uf_dmabuf prints "failed to attach dmabuf" and __buf_prepare prints "buffer preparation failed: -14". Any idea what could cause that?
I export its addresses with vidioc_expbuf*
__qbuf_dmabuf also says "buffer for plane 0 changed", I don't know if that's related, or if it's just because I try to qbuf with a different dma fd every time
queueing a different dmabuf every time should be supported, but it can degrade performances
The camera gives me 4 buffers
(though I think I can make it give me just one buffer if I want; I'll investigate that once I have gotten this to work, thanks for the tip)
you should allocate 4 buffers on the encoder side as well then, to keep the v4l2 buffer to dmabuf association
it still doesn't explain why it fails though
what camera and encoder do you use ?
a qualcomm camera subsystem and qualcomm venus video encoder
do they have the same memory constraints ? or does one require for instance physically contiguous memory while the other has an IOMMU ?
how can I check?
that's one common cause of dmabuf import errors, when the importer has tighter memory constraints than the exporter
good question :-)
well, first question, do both drivers use viodebuf2 ?
is there a way to check that?
the encoder is venus_enc, the camera is qcom_camss
ah
videobuf2_dma_sg is used by the modules venus_dec, qcom_camss, and venus_enc
ok
(all three also use videobuf2_core and videobuf2_v5l2 and such, but videobuf2_dma_sg looked the most relevant here)
both the camera and the encoder use nv12 for raw frames, which is one continuous block of memory; both use the multi-planar API, but both just have one plane
you'll have to trace the kernel code, I can't see (in mainline) where the -EFAULT could come from
It's the linaro version of 4.9.39, I'll try reading through the relevant parts of it
mort: did you try to turn on debugging in videobuf2_core? echo 1 >/sys/module/videobuf2_core/parameters/debug
This usually gives useful info in the kernel log.
hverkuil: the "failed to attach dmabuf" likely comes from videobuf2-dma-sg, and it indicates that dma_buf_attach() failed
the function itself shouldn't return -EFAULT
so I assume the exporter's attach function does
but that should also be in videobuf2_dma_sg, and I see no -EFAULT there
so it's likely a QC kernel issue
mort: BTW, is your application 32 bit and the kernel 64 bit?
those messages are from vb2-core
the application and kernel is both aarch64
I see a "failed to attach dmabuf" both in videobuf2-core and in videobuf2-dma-sg
OK, then there are no compat32 issues.
EFAULT usually comes from copying to/from userspace. It's a multiplanar API, so is v4l2_buffer filled in right with the correct length and m.planes values? Where 'length' is the number of element in m.planes.
yeah, buffer.length is 1, and buffer.m.planes is a pointer to exactly one plane
it's just using the multi-planar API even though the devices just have one plane
hverkuil: but "buffer preparation failed: -14" indicates that the error comes from within the kernel, doesn't it ?
Ah
vb2_dma_sg_attach_dmabuf return EFAULT if the dmabuf size is less than the desired size.
I'll bet that that's the cause.
If you check the formats for both the camera and encoder output, are they the same with the same sizes?
I request 176x144 from the camera capture device (because that's what some other component asks for), the camera gives me 640x400. I then request 640x400 from the encoder output device, and the encoder gives me 640x416
is it a problem that the encoder is 640x416 maybe?
It's the actual size of the planes that count.
I see, I'll have it print that too, sec
Check with 'v4l2-ctl -V' for the camera and 'v4l2-ctl --get-fmt-video-out' for the encoder output
what format do you use ?
get-fmt-video-out gives me info for the default resolution which is 1280x720, not the 640x416 I get when asking for 640x400
Ah, it's an mem2mem device. So you need to call it from within the application. Or set the format and get the format on the same line, e.g.:
Actually, just 'try' the format is easier: v4l2-ctl --try-fmt-video-out width=w,height=h,pixelformat=NV12
pinchartl: the camera's capture and the encoder's output's pixelformat is set to V4L2_PIX_FMT_NV12, and other than that and width/height, I don't ask for anything in particular
But I'll bet that the codec expects a larger buffer than the camera gives you.
mort: I assume your encoder needs the line length to be a multiple of 32 bytes
can you try to request 640x400 from the camera, but with bytesperline set to 416 ?
hverkuil: that prints 640 bytes per line and a size of 413696, the camera also has 640 bytes per line but a size of 384000
ah, sorry, it's the height that's rounded
yeah
There you go.
what's the bytesperline value for the encoder ?
640
Either export the buffers from the encoder output, or use VIDIOC_CREATEBUFFERS to create larger buffers in the camera driver.
413696 is really weird then
it can't be divided by 640
unless it is rounding up to some multiple of PAGESIZE (that works, 413696 == 101 * 4096)
indeed
still weird though. 640 * 416 * 1.5 == 97.5 * 4096.
So you'd expect 98 * 4096 or something. Not 101.
I'm sure the answer can be found in the driver code ?
s/\?//
Too lazy. It's Friday afternoon :-)
is the rule that the buffer from the MEMORY_MMAP device has to be bigger than the buffer from the MEMORY_DMABUF device, or do they have to match exactly?
mort: it has to be big enough, they don't have to match exactly
so if I for example made the encoder 640x300, and the resulting buffers were smaller than the camera's buffers, it'd work
yes and no
the encoder expects an image in a particular format
yeah, right, colors would be messed up
possibly with padding at the end of each line to make lines multiple of a hardware constraint (that should be reflected in bytesperline)
possibly with the number of pixels per line also being a multiple of a hardware constraint
(usually the macroblock width)
and the number of lines also being a multiple of a hardware constraint (usually the macroblock height)
all that will give you a buffer size
you need to make sure that the constraints match on both sides
if the camera gives you, let's say, 640x400, and the encoder wants 656x416, even if you pad the buffer at the end to make the overall size match, it encoded image won't look good
so you have to figure out the constraints on both sides
and configure them to make sure they match
padding is configured through bytesperline and sizeimage
the actual size of the image through width and height
I don't remember if sizeimage can be set when calling S_FMT or if it's an output parameter only
if it can, you can modify it, and still allocate buffers with VIDIOC_REQBUFS
if it can't, you need to allocate buffers with VIDIOC_CREATEBUF, as that ioctl takes a size
bytesperline is passed to S_FMT in any case
having too much padding at the end of the image isn't a problem
having different bytesperline values won't give you a good result
interesting, when the camera gives 640x400 frames and the encoder is 640x384, queuing the buffer results in an EINVAL instead
luckily, video frame widths tend to align with the macro block size, so I don't think I'll care about that here
that means __qbuf_dmabuf returns -EINVAL, and the only way I can see that it would return that without printing is if call_bufop returns -EINVAL
(__buf_prepare prints "buffer preparation failed: -22", but nothing else is being pinted, and all cases where __qbuf_dmabuf (called by __buf_prepare) fails without printing is if call_bufop returns non-0))
pinchartl: when I request buffers with VIDIOC_CREATE_BUFS, what should I do with v4l2_create_buffers::format? Should I use fmt.pix_mp, or fmt.pix even though I'm using the multi planar API?
you should set it to the format that you configure with S_FMT
and adjust sizeimage
in practice only the size should be checked by the driver
other format fields should be ignored
it was a bit of a design mistake
what the QC drivers do, however, I don't know
pinchartl: I got it to work - though the camera has 680x400 luminosity bytes, directly followed by the chroma bytes, while the encoder expects 640x416 luminosity bytes directly followed by the chroma bytes, so the colors get screwed up. Is the only solution there to get the camera to capture using the same resolution as the encoder (or not use DMA)
, or is there some way I haven't found yet to tell the camera to add some padding bytes between the luminosity and the chroma?
it feels like the brightness bytes and the colors should've been on different planes, and that would've made this a non-issue