Re: [V4l2-library] Messed up syscall return value

On 07/28/2008 11:49 PM, Gregor Jasny wrote:

Hi,

I've observed strange behavior in the REQBUFS ioctl for the non-emulated case. To reproduce:

• Use a amd64 system (Debian Sid if it matters)
• Linux 2.6.26
• load vivi driver to video0
• Compile attached program w/o libv4l2 and check that it runs fine.
• Compile attached program with libv4l2 and see it fail during REQBUFS ioctl.

I've debugged the problem to the line: result = syscall(SYS_ioctl, devices[index].fd, VIDIOC_REQBUFS, req);

Here the value 2 get stored into result, although the kernel driver returned success (at least it does not complain loudly in the logs).

When stepping assembly instructions the rax value is zero, shortly after it is set to two. Personally I suspect a messed up stack. Both v4l2_ioctl and syscall are variable argument functions.

strace and ltrace will help in this case I guess. Could you provide them for your testcases?