[linux-dvb] libsoftmpeg -- a software mpeg decoding and a/v sync library

[Michael Hunold <hunold@convergence.de>]

Hi,

many people have recently decided to work on software only mpeg 
decoderes. The goal is to have a cheap box with a DVB card and vdr or 
MythTV on it, but only using a so-called budget DVB card without onboard 
mpeg decoder.

There are, however, the following problems to be solved:
- low-delay buffering (for fast channel switches)
- mpeg audio decoding
- audio and video syncronization
- audio rendering
- video rendering
- long term playback stability

In the past, there have been different approaches to solve these 
problems, but they are not quite satisfying yet.

For example, while "mplayer" and "xine" are excellent movie players, 
they use huge buffers for buffering and therefore make channel switching 
nearly unbearable. Additionally, they lack long-term stability, because 
they don't synchronize to the incoming stream and buffer over- and 
underrums quite frequently occur. (see below for details)

The "softdevice" plugin from VDR is on the right way, but unfortunately 
it is bound to VDR. Other projects like "mythtv" and the like need to 
duplicate the code or invent the wheel again,

Because of this, I'd like to invite everyone interested to participate 
on the "linuxtv-softmpeg" project, which tries to provide the 
"libsoftmpeg" library, a thin library on top of existing technologies 
that adresses all the problems mentioned above.

I've already written some parts of "libsoftmpeg" which currently do the 
following:

- low-delay buffering for audio and video, adjustable audio prebuffering 
(currently 500ms, channel switching is usually done in less than a second)

- mpeg audio decoding via libavcodec from "ffmpeg" project

- mpeg video decoding via libavcodec from "ffmpeg" project

- audio and video synchronization using a "sync-to-audio-pts" approach 
and "free flowing video display" (see below for a a link to the file)

- video rendering using DirectFB (http://www.directfb.org), a thin 
library ontop of the Linux framebuffer device. DirectFB hosts a "cle266" 
driver which drives the tv out of the EPIA/cle266 boards very well

- audio rendering using FusionSound 
(http://www.directfb.org/fusionsound.xml), a thin library on top of 
DirectFB and Fusion (fast interprocess communication), which supports 
sound mixing, pitch adjustment and other neat features. Sound mixing is 
very important, other applications can overlay system sounds onto the 
live tv audio.

All this is working fairly well, a demonstration application is included 
with the library that reads from the DVR device (so you need "szap" for 
example to tune to a service and have the data put to the DVR device) 
and displays audio and video. It should be easily possible to use 
"libsoftmpeg" in "vdr", too.

The code was written in the last few weeks and naturally it still lacks 
some features:
- fixed to 48kHz output soundrate, but others can be supported easily
- in some parts, the code is very "hackish" and quick-and-dirty
- lot of clean up code is missing, error paths are missing
- much memory for buffers is wasted
- the api is not very clean and has some "rough" edges
- only a multiplexed TS can be used by the library, support for PS 
streams should be added (DVD playback)
- on some systems, there is a constant audio offset, perhaps this 
depends on the soundcard used

The current code is licensed under the LGPL. If you'd like to 
participate in this project please note that it's not allowed to 
directly copy & paste code from other projects, especially not
from GPL projects.

Because of this, only LGPL libraries can be used in combination with it: 
fortunately libavcodec, DirectFB and FusionSound are LGPL. 8-)

The code is currently part of the LinuxTV.org CVS archive. You can check 
it out like this.
> cvs -z9 -d :pserver:anonymous@linuxtv.org:/cvs/linuxtv co libsoftmpeg

There is currently no webpage, but this will change in the future.

You need:
- DirectFB and a supported gfx adapter (cle266 or Matrox G200/4x0/5x0 is 
fine, vesafb is only enough if you have a horibbly fast machine ;-)
- FusionSound, which needs OSS or ALSA (with oss emulation) sound drivers
- libavcodec, version 0.4.8 from http://ffmpeg.sf.net is fine

You'll get a demo application called "dfb_ts", which reads a transport 
stream from the DVB dvr device. You need to tune your frontend with "szap"
for example and redirect the data to the dvr device with the "-r" 
switch. For example:
> szap -r "Das Erste"
Then you can start the demo application and provide the video and audio 
pids for the service you like to watch (have a look at "szap", it tells 
you the correct pids):
> ./dfb_ts /dev/dvb/adapter0/dvr0 0x7ff 0x800

What's currently completely missing is long-term playback stability. As 
you probably know, the problem arises due to the different clocks used 
for dvb encoding and decoding.

I've added the file "IDEAS" to the CVS, where I try to explain the 
current approach for audio/video synchronization and what can be done 
for long-term a/v sync.

You can visit the file with this link:
http://linuxtv.org/cgi-bin/cvsweb.cgi/libsoftmpeg/IDEAS?rev=1.2&content-type=text/x-cvsweb-markup

The next major step would be to add the ideas of long-term playback 
stability and to change to code that video part currently using 
libavcodec can be replaced for example with the cle266 mpeg decoder 
stuff, sot that cle266 users can use the hw acceleration of their chipsets.

I've created a mailing list for this project.
Write a mail to <ecartis@linuxtv.org> with "subscribe linuxtv-softmpeg" 
in the subject to subscribe.

Please subscribe to the mailing-list if you're interested. Please don't 
reply to this mail on the mailing-lists I crossposted.

If you'd like to actively participate in the project, or use libsoftmpeg 
in your project, please contact me or write to the mailing-list.

CU
Michael


--
Info:
To unsubscribe send a mail to ecartis@linuxtv.org with "unsubscribe linux-dvb" as subject.