At Wed, 23 Oct 2013 23:31:46 +0100, Mauro Carvalho Chehab wrote:
- Kieran Kunhya: SDI
The biggest problem is separate audio and video file descriptors. Audio data is transfered in the video blanking areas, where should they be separated - in the kernel, using multiple planes, or in the user-space? One possibility is to consider this data as similar to an "uncompressed MPEG" datastream. It might be possible to use hardware-specific time-stamps to synchronise the data.
Separate ALSA and V4L2 nodes are inherently problematic since they do not keep the audio and video together. Another possibility is to offer two APIs: for professional applications, where data is perfectly synchronised, and separate audio and video for "human consumption."
A format for professional applications were VBI/HBI/Video is all captured in one big frame is OK (provided an open source lib is created to parse it).
The proposal for using a multiplanar API where the audio is in a separate plane ran into opposition: the alsa devs should be asked whether it is possible to return the audio data in such a way that it can be exactly associated with the corresponding video buffer. This also requires that the audio can be variable length since for NTSC the size of the audio data will alternate between frames. If this is possible, then the alsa API can be used and things can still remain in sync.
After talks with Mauro and Hans, I took a look back at ALSA code, and actually the recent API addition for "wallclock timestamp" may be of help for such a purpose.
In ALSA kernel driver side, each PCM can add SNDRV_PCM_INFO_HAS_WALL_CLOCK flag to runtime->hw.info. Then wall_clock() snd_pcm_ops will be invoked at each time the position is inquired so that the driver can give its hardware timestamp. The format is the standard timspec, thus the driver must convert to sec/nsec. Other than that, there is no restriction.
In the user-space side, the apps can check the capability via snd_pcm_hw_params_supports_audio_wallclock_ts(), and the timestamp can be obtained via snd_pcm_status_get_audio_htstamp(). (hi-res tstamp).
The addition of timestamp support for compress-offload API is still in discussion. The topic was just raised newly on audio mini summit
HTH,
Takashi