From: Klaus Schmidinger Sent: Thursday, 14 October, 2010 00:43

If there's something actually wrong with VDR's DVB subtitle code, it should be fixed in VDR. I don't see why we would need a plugin interface here.

An output device needs to provide a proper STC in order to sync subtitles. I guess I need to change the description of cDevice::GetSTC() to make this a requirement...

Reelbox delivers STC values. Only issue with Reelbox is that STC value seems to be somehow wrong during recording play. STC is correct during live DVB broadcast. The issue is that timing works, but there is 5s offset during recording play. Reelbox do not separate live broadcast and recording play. VDR separate these. One opinion is that VDR recording playback is broken..

Is there any easy place to add delay for subtitling on VDR recording playback code? I would do that hack to my system just to get finally eHD card to use. I assume that I should add some value to STC. Any documentation available about this STC and how it works on VDR?

I did some testing and debugging. First note is that VDR do not do sanity check for STC on delay loop. Now there is on dvbsubtitle.c

if (STC >= 0) {

It should be

if (LimitTo32Bit(STC) >= 0) {

After this delay loop start to work with reelbox-3 plugin. With out that STC int64_t value includes garbage and get most of time negative value. Because same filtering is in use other places for this value, it should be also on if-statement..

This do not fix delay issue on eHD completely, but now subtitles are in sync with AV.

Next step is to add delay for subtitles. This is "dirty trick", it is only for eHD users. Simply add some value to Delta (add second line):

Code: Delta = LimitTo32Bit(sb->Pts()) - LimitTo32Bit(STC); Delta += 500000;

With that 50000 you will get 5.56s delay. With that eHD now works correctly with recording play. Live shows subtitles will be too late with this, but it is minor issue. I mostly watch recordings :)

The root cause is still there, somehow eHD delivers around 500000 too low STC for recording play. For live shows STC value is correct..

On 14.10.2010 14.48, Vesa wrote:

I did some testing and debugging. First note is that VDR do not do sanity check for STC on delay loop. Now there is on dvbsubtitle.c

if (STC>= 0) {

It should be

if (LimitTo32Bit(STC)>= 0) {

After this delay loop start to work with reelbox-3 plugin. With out that STC int64_t value includes garbage and get most of time negative value. Because same filtering is in use other places for this value, it should be also on if-statement..

This do not fix delay issue on eHD completely, but now subtitles are in sync with AV.

Next step is to add delay for subtitles. This is "dirty trick", it is only for eHD users. Simply add some value to Delta (add second line):

Code: Delta = LimitTo32Bit(sb->Pts()) - LimitTo32Bit(STC); Delta += 500000;

With that 50000 you will get 5.56s delay. With that eHD now works correctly with recording play. Live shows subtitles will be too late with this, but it is minor issue. I mostly watch recordings :)

The root cause is still there, somehow eHD delivers around 500000 too low STC for recording play. For live shows STC value is correct..

Thank you! I've tried these kind of hacks myself, but because of "ultimate" coding skills, couldn't produce anything useful...

Too bad, that this is not waf solution yet... I guess, that I'll have to increase the level of the "dirty tricks" and compile couple of vdr instances (one with the delay and one without) and create a command for switching between the instances :D

...hanu

p.s. My 3rd eHD card could be available for testing purposes in Helsinki metropolitan area ...

On 14.10.2010 09:42, crow wrote:

Hi, Well i am not sure if they are fixed or not with .15 and .16 but last time i tested dvbsubtitle on FULL HD tv i got something like like this in [1] and no i don't have eHD hardware but using xine as output device . Yes dvb subtitle are actually showed but on wrong position and small. My biggest problem with dvb subtitle is that they actually not work for TotalTV package on 16.0E all what i sow in logo was this:

Mar 14 23:17:31 vdrbox vdr: [2369] subtitleConverter thread started (pid=2347, tid=2369) But the subtitle arent showed like on zdf [1] .

This may be a problem of provider and not of vdr but I would like to debug it deeper to get this stuff working.. There was a discussion on vdrportal regarding that [2].

But maybe I wrote this in wrong thread here and should actualy open a new one?

[1] http://www.abload.de/img/dvbsubtitle01y8tk.jpg http://www.abload.de/img/dvbsubtitle0257mc.jpg http://www.abload.de/img/dvbsubtitle038jph.jpg [2] http://www.vdrportal.de/board/thread.php?threadid=97744

There is a fix for this from Reinhard Nissl still pending in my pipeline. It will be one of the next things I'll attend to...

Klaus

I wonder if there is an easy way to recompress all recordings that are not already H.264 into H.264 files, making them a lot smaller without losing any quality.

The only way to not lose quality is to keep the bitrate high, especially for recordings with a lot of motion. You should encode with constant quality (CRF) set to whatever the minimum quality you want to preserve. You can also reduce the resolution a bit, just make sure to maintain the aspect ratio. Bottom line is that you will not get smaller filesizes without making sacrifices. Data is lost with every encoding pass.

Would it be easy to write a small script that goes through all recordings, finds out whether the recording is MPEG2, determines the H.264 bitrate that produces approximately the same quality and calls mencoder to convert it?

Has somebody already done that?

Like I said, you aren't going to get smaller files without making a sacrifice. If less filsize is most important to you then you should focus on a target bitrate at the expensive of quality. If quality is most important to you then focus on a CRF value at the expensive of more filesize.

There are way too many variables for a simple one-size-fits-all option so you get your cake and eat it too. Video encoding simply doesn't work that way. Every time data is lost, it's gone forever. The same is true with audio encoding as well. Some people believe in magic, even when there is no magic.

On 29/10/10 00:28, VDR User wrote:

...

I wonder if there is an easy way to recompress all recordings that are not already H.264 into H.264 files, making them a lot smaller without losing any quality.

The only way to not lose quality is to keep the bitrate high, especially for recordings with a lot of motion. You should encode with constant quality (CRF) set to whatever the minimum quality you want to preserve. You can also reduce the resolution a bit, just make sure to maintain the aspect ratio.

And don't try to scale interlaced material without using a good deinterlacing filter first.

Bottom line is that you will not get smaller filesizes without making sacrifices. Data is lost with every encoding pass.

The point of H264 is that it can achieve better quality for a better bit rate. Although some quality will be lost by transcoding, you could probably halve the file size without making a noticeable difference.

Hi and sorry for this long post. I thought to share some of my personal experiences and insights on re-encoding DVB recordings. I might also have a few useful tips that are not always found in these discussions at Linux-specific forums and mailing lists.

Feel free to skip this wall of text if you aren't interested in processing DVB recordings. :)

First a quick response to the previous post.

On 2010-10-29 at 18:50 -0700, VDR User wrote:

On Fri, Oct 29, 2010 at 10:59 AM, Tony Houghton h@realh.co.uk wrote:

...

The point of H264 is that it can achieve better quality for a better bit rate. Although some quality will be lost by transcoding, you could probably halve the file size without making a noticeable difference.

That sounds good when you read the datasheets but real world results are a bit different. Also, what you're referring to is encoding comparisons all from a raw source - not mpeg2 vs. the same mpeg2 reencoded in h264. I can't stress enough that there is no magic to be had here for the reasons in my previous post.

While I agree with this statement in general, a 50 % bitrate reduction is easily possible at visually transparent quality even when the MPEG-2 source is noisy and blocky. Although with x264 this is a bit harder if the video is interlaced: encoding as interlaced is not (yet) as efficient as progressive encoding and bobbing/double-rate deinterlacing will also require more bitrate. Bobbing with a good filter is in my opinion preferable because most realtime deinterlacing solutions that I have encountered are pretty bad compared to the best software methods. I guess the expensive hardware chips aren't much better than Nvidia and Ati GPU deinterlacing implementations.

Benefits of filtering video

If larger changes to the video content such as noise reduction and same-rate deinterlacing are tolerated, it's quite possible to get below 1000 kbps on average for a collection of SD video streams while retaining very good visual quality compared to the originals. This gives the additional benefit of making streaming possible over a 1 or 2 Mbps uplink.

Deinterlacing solutions

It should be noted that good deinterlacing and good noise reduction are more difficult to achieve than one might think. For (bob-)deinterlacing, MPlayer/MEncoder has -vf yadif=1,mcdeint=0:0:10 (add ,framestep=2 for same-rate deinterlacing), which gives acceptable results. This means that the image won't shimmer much and that most sloped edges won't appear jagged. Yadif and mcdeint are far from perfect on these points, but they are still better than most filters. Yadif alone may be good enough for casual viewing, but it shimmers (bobs up and down) and smears details _a_lot_ compared to mcdeint. Nvidia's best "temporal adaptive" deinterlacer in VDPAU also has similar issues although its edge-directed interpolation is slightly better.

The only significantly better freely available deinterlacing solution that I know of is TempGaussMC (TGMC) AviSynth filter combined with NNEDI2/EEDI2/EEDI3. Although AviSynth is Windows-only software (it's GPLed, but tied to VFW), it works perfectly on x86(-64) Linux platforms with avs2yuv and Wine. Only filters that use GPU acceleration (fft3dgpu) can't be used. The y4m output from avs2yuv can be piped directly to x264 and other encoders.

Here's a recent comparison clip that includes output from PureVideo (Nvidia GPU), yadif, TDeint and TGMC: http://forum.doom9.org/showthread.php?p=1433847#post1433847 I did some comparisons of my own and the PureVideo result reflects that of VDPAU's "temporal spatial" deinterlacing method. Note that TGMC has a strong denoising and local stabilization effect that is useful in compression. yadif+mcdeint would probably stand between TGMC and TDeint in a quality comparison, but it's a few orders of magnitude faster than TGMC+NNEDI2. However, unlike MEncoder filters, AviSynth filters can be split evenly to multiple threads so AviSynth may be faster on a multi-core CPU if only one encode is running at a time. If there's enough RAM available (~ 128-512 MB for each SD AviSynth job or 64-128 MB for MPlayer/MEncoder), it's easiest and most efficient to run as many encoding jobs as there are CPU cores.

Denoising and deblocking

The best denoiser in MPlayer/MEncoder is hqdn3d, but it will smooth details out along with the noise. Instead I've been using motion-compensated MDegrain2 from AviSynth's MVTools2. It obliterates most types of grain and noise but still gives a very natural and non-plastic output. This makes further compression much easier. With noisy film sources the bitrate can often be dropped to half from what it would be without denoising.

I'm not going much into deblocking here because I haven't compared the available methods myself. There are many filters available especially in MPlayer/MEncoder. Deblocking should be coupled with the decoder so that information on macroblock quantizers is available. Otherwise it will smooth out actual detail in the video. Oversmoothing is an issue with the good filters too.

Tradeoffs

As Lars Bläser pointed out earlier, all this processing and encoding comes at a high price in CPU time and energy consumption. Whether it's worth it depends on your purposes, so make your own calculations. If you only want a bitrate reduction for storage reasons, CPU time alone may cost more than the required disk space -- unless it gets cold outside and you are heating with electricity...

Efficient encoding still requires lots of manual work because there are no reliable automatic detection schemes for recognizing different types of sources: interlaced, telecined, field-blended and phase-shifted. Personally I'd like a scheme where the encoder frontend would adapt to source type changes on the fly and encode the video with a variable framerate, but this is not always desirable.

Scripts

I wrote a bash script for downloading and installing AviSynth plus some of the essential filters on Linux. I could upload it somewhere if people are interested? There's also a simple command-line based template system that makes using AviSynth slightly easier, but it needs more work to be useful.

Cheers,

Niko Mikkilä

On 30/10/10 02:50, VDR User wrote:

That sounds good when you read the datasheets but real world results are a bit different. Also, what you're referring to is encoding comparisons all from a raw source - not mpeg2 vs. the same mpeg2 reencoded in h264. I can't stress enough that there is no magic to be had here for the reasons in my previous post.

And I wonder is there still much room for improvement in current H264 encoders? The various MPEG 4 part 2 codecs (DivX etc) improved a great deal over time, and ISTR reading that the first H264 encoders offered almost no practical improvement over the newest part 2 codecs at the time.