On Wed, 13 Oct 2010, Vesa wrote:
Similar delay as on ttxsubtitles would be nice addition to VDR. Separate and additional delay for live and record play modes. Suitable range would be 0 - 10s.
Well, IMO that delay code in the ttxtsubs plugin should be removed when PTS code has been merged into the repo.
This would fix also other issue than "non working output". Sometimes
How about adding a STC offset setting in these "non working output" plugins with a little help of latest PCR value and the system clock?
My opinion is that there is several reason to add DVB subtitle delay setting to VDR menu. To fix non working output devices and to fix temporally miss aligned subtitles. This feature is similar as Lipsync set on all modern AV devices. Audio should be in sync with video, but sometimes it is not. It is good to have lipsync on menu for fixing this temporal but totally annoying issue.
I'm against these kind of dirty hacks in the official VDR code tree, but the current dvbsubtitles code in VDR could be refactored to be replaceable with a plugin. Afterwards the reel plugin could reimplement it with any hacks needed.
BR, -- rofa
On 13.10.2010 20:55, Rolf Ahrenberg wrote:
On Wed, 13 Oct 2010, Vesa wrote:
Similar delay as on ttxsubtitles would be nice addition to VDR. Separate and additional delay for live and record play modes. Suitable range would be 0 - 10s.
Well, IMO that delay code in the ttxtsubs plugin should be removed when PTS code has been merged into the repo.
This would fix also other issue than "non working output". Sometimes
How about adding a STC offset setting in these "non working output" plugins with a little help of latest PCR value and the system clock?
My opinion is that there is several reason to add DVB subtitle delay setting to VDR menu. To fix non working output devices and to fix temporally miss aligned subtitles. This feature is similar as Lipsync set on all modern AV devices. Audio should be in sync with video, but sometimes it is not. It is good to have lipsync on menu for fixing this temporal but totally annoying issue.
I'm against these kind of dirty hacks in the official VDR code tree, but the current dvbsubtitles code in VDR could be refactored to be replaceable with a plugin. Afterwards the reel plugin could reimplement it with any hacks needed.
If there's something actually wrong with VDR's DVB subtitle code, it should be fixed in VDR. I don't see why we would need a plugin interface here.
An output device needs to provide a proper STC in order to sync subtitles. I guess I need to change the description of cDevice::GetSTC() to make this a requirement...
Klaus
From: Klaus Schmidinger Sent: Thursday, 14 October, 2010 00:43
If there's something actually wrong with VDR's DVB subtitle code, it should be fixed in VDR. I don't see why we would need a plugin interface here.
An output device needs to provide a proper STC in order to sync subtitles. I guess I need to change the description of cDevice::GetSTC() to make this a requirement...
Reelbox delivers STC values. Only issue with Reelbox is that STC value seems to be somehow wrong during recording play. STC is correct during live DVB broadcast. The issue is that timing works, but there is 5s offset during recording play. Reelbox do not separate live broadcast and recording play. VDR separate these. One opinion is that VDR recording playback is broken..
Is there any easy place to add delay for subtitling on VDR recording playback code? I would do that hack to my system just to get finally eHD card to use. I assume that I should add some value to STC. Any documentation available about this STC and how it works on VDR?
On Wed, Oct 13, 2010 at 11:43 PM, Klaus Schmidinger Klaus.Schmidinger@tvdr.de wrote:
On 13.10.2010 20:55, Rolf Ahrenberg wrote:
On Wed, 13 Oct 2010, Vesa wrote:
Similar delay as on ttxsubtitles would be nice addition to VDR. Separate and additional delay for live and record play modes. Suitable range would be 0 - 10s.
Well, IMO that delay code in the ttxtsubs plugin should be removed when PTS code has been merged into the repo.
This would fix also other issue than "non working output". Sometimes
How about adding a STC offset setting in these "non working output" plugins with a little help of latest PCR value and the system clock?
My opinion is that there is several reason to add DVB subtitle delay setting to VDR menu. To fix non working output devices and to fix temporally miss aligned subtitles. This feature is similar as Lipsync set on all modern AV devices. Audio should be in sync with video, but sometimes it is not. It is good to have lipsync on menu for fixing this temporal but totally annoying issue.
I'm against these kind of dirty hacks in the official VDR code tree, but the current dvbsubtitles code in VDR could be refactored to be replaceable with a plugin. Afterwards the reel plugin could reimplement it with any hacks needed.
If there's something actually wrong with VDR's DVB subtitle code, it should be fixed in VDR. I don't see why we would need a plugin interface here.
An output device needs to provide a proper STC in order to sync subtitles. I guess I need to change the description of cDevice::GetSTC() to make this a requirement...
Klaus
vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Hi, Well i am not sure if they are fixed or not with .15 and .16 but last time i tested dvbsubtitle on FULL HD tv i got something like like this in [1] and no i don't have eHD hardware but using xine as output device . Yes dvb subtitle are actually showed but on wrong position and small. My biggest problem with dvb subtitle is that they actually not work for TotalTV package on 16.0E all what i sow in logo was this:
Mar 14 23:17:31 vdrbox vdr: [2369] subtitleConverter thread started (pid=2347, tid=2369) But the subtitle arent showed like on zdf [1] .
This may be a problem of provider and not of vdr but I would like to debug it deeper to get this stuff working.. There was a discussion on vdrportal regarding that [2].
But maybe I wrote this in wrong thread here and should actualy open a new one?
[1] http://www.abload.de/img/dvbsubtitle01y8tk.jpg http://www.abload.de/img/dvbsubtitle0257mc.jpg http://www.abload.de/img/dvbsubtitle038jph.jpg [2] http://www.vdrportal.de/board/thread.php?threadid=97744
I did some testing and debugging. First note is that VDR do not do sanity check for STC on delay loop. Now there is on dvbsubtitle.c
if (STC >= 0) {
It should be
if (LimitTo32Bit(STC) >= 0) {
After this delay loop start to work with reelbox-3 plugin. With out that STC int64_t value includes garbage and get most of time negative value. Because same filtering is in use other places for this value, it should be also on if-statement..
This do not fix delay issue on eHD completely, but now subtitles are in sync with AV.
Next step is to add delay for subtitles. This is "dirty trick", it is only for eHD users. Simply add some value to Delta (add second line):
Code: Delta = LimitTo32Bit(sb->Pts()) - LimitTo32Bit(STC); Delta += 500000;
With that 50000 you will get 5.56s delay. With that eHD now works correctly with recording play. Live shows subtitles will be too late with this, but it is minor issue. I mostly watch recordings :)
The root cause is still there, somehow eHD delivers around 500000 too low STC for recording play. For live shows STC value is correct..
On Thu, Oct 14, 2010 at 02:48:51PM +0300, Vesa wrote:
The root cause is still there, somehow eHD delivers around 500000 too low STC for recording play. For live shows STC value is correct..
I'll need to have a closer look, but it sounds possible, that the pts is the pts of the last packet put into the video decoder queue and not the latest value of the currently shown frame... That would explain the time lag, the ES queue of the decoder is about 3MB or so.
On 14.10.2010 14.48, Vesa wrote:
I did some testing and debugging. First note is that VDR do not do sanity check for STC on delay loop. Now there is on dvbsubtitle.c
if (STC>= 0) {
It should be
if (LimitTo32Bit(STC)>= 0) {
After this delay loop start to work with reelbox-3 plugin. With out that STC int64_t value includes garbage and get most of time negative value. Because same filtering is in use other places for this value, it should be also on if-statement..
This do not fix delay issue on eHD completely, but now subtitles are in sync with AV.
Next step is to add delay for subtitles. This is "dirty trick", it is only for eHD users. Simply add some value to Delta (add second line):
Code: Delta = LimitTo32Bit(sb->Pts()) - LimitTo32Bit(STC); Delta += 500000;
With that 50000 you will get 5.56s delay. With that eHD now works correctly with recording play. Live shows subtitles will be too late with this, but it is minor issue. I mostly watch recordings :)
The root cause is still there, somehow eHD delivers around 500000 too low STC for recording play. For live shows STC value is correct..
Thank you! I've tried these kind of hacks myself, but because of "ultimate" coding skills, couldn't produce anything useful...
Too bad, that this is not waf solution yet... I guess, that I'll have to increase the level of the "dirty tricks" and compile couple of vdr instances (one with the delay and one without) and create a command for switching between the instances :D
...hanu
p.s. My 3rd eHD card could be available for testing purposes in Helsinki metropolitan area ...
On 14.10.2010 13:48, Vesa wrote:
I did some testing and debugging. First note is that VDR do not do sanity check for STC on delay loop. Now there is on dvbsubtitle.c
if (STC >= 0) {
It should be
if (LimitTo32Bit(STC) >= 0) {
After this delay loop start to work with reelbox-3 plugin. With out that STC int64_t value includes garbage and get most of time negative value. Because same filtering is in use other places for this value, it should be also on if-statement..
"Vesa" did state in his posting dated 2010-10-15/15:39 that the modification to dvbsubtitle.c is not necessary, but I just want to make it clear why that is so.
The check
if (STC >= 0) {
acts on the raw STC value returned by the device, which was allowed to be -1 in case that device is unable to return the STC. Limiting this to 32 bit would break this check.
For version 1.7.17 of VDR I will make it mandatory for devices that can replay to return a valid STC value in GetSTC(), and the above check is then obsolete (since the 'else' branch of that 'if' was never implemented, anyway).
Klaus
On 14.10.2010 09:42, crow wrote:
Hi, Well i am not sure if they are fixed or not with .15 and .16 but last time i tested dvbsubtitle on FULL HD tv i got something like like this in [1] and no i don't have eHD hardware but using xine as output device . Yes dvb subtitle are actually showed but on wrong position and small. My biggest problem with dvb subtitle is that they actually not work for TotalTV package on 16.0E all what i sow in logo was this:
Mar 14 23:17:31 vdrbox vdr: [2369] subtitleConverter thread started (pid=2347, tid=2369) But the subtitle arent showed like on zdf [1] .
This may be a problem of provider and not of vdr but I would like to debug it deeper to get this stuff working.. There was a discussion on vdrportal regarding that [2].
But maybe I wrote this in wrong thread here and should actualy open a new one?
[1] http://www.abload.de/img/dvbsubtitle01y8tk.jpg http://www.abload.de/img/dvbsubtitle0257mc.jpg http://www.abload.de/img/dvbsubtitle038jph.jpg [2] http://www.vdrportal.de/board/thread.php?threadid=97744
There is a fix for this from Reinhard Nissl still pending in my pipeline. It will be one of the next things I'll attend to...
Klaus
Hi,
I have been using VDR for over 10 years now, so I currently have ~4TB of VDR recordings. Some of them are SDTV/MPEG2, many of the newer recordings are HDTV/H.264.
I wonder if there is an easy way to recompress all recordings that are not already H.264 into H.264 files, making them a lot smaller without losing any quality.
Since my HTPC is using the graphics card anyway, I do not rely on the vdr GUI. As long as the result can be played with xine and mplayer, it would be fine.
Would it be easy to write a small script that goes through all recordings, finds out whether the recording is MPEG2, determines the H.264 bitrate that produces approximately the same quality and calls mencoder to convert it?
Has somebody already done that? Any pointers welcome!
Thanks, Carsten.
I wonder if there is an easy way to recompress all recordings that are not already H.264 into H.264 files, making them a lot smaller without losing any quality.
The only way to not lose quality is to keep the bitrate high, especially for recordings with a lot of motion. You should encode with constant quality (CRF) set to whatever the minimum quality you want to preserve. You can also reduce the resolution a bit, just make sure to maintain the aspect ratio. Bottom line is that you will not get smaller filesizes without making sacrifices. Data is lost with every encoding pass.
Would it be easy to write a small script that goes through all recordings, finds out whether the recording is MPEG2, determines the H.264 bitrate that produces approximately the same quality and calls mencoder to convert it?
Has somebody already done that?
Like I said, you aren't going to get smaller files without making a sacrifice. If less filsize is most important to you then you should focus on a target bitrate at the expensive of quality. If quality is most important to you then focus on a CRF value at the expensive of more filesize.
There are way too many variables for a simple one-size-fits-all option so you get your cake and eat it too. Video encoding simply doesn't work that way. Every time data is lost, it's gone forever. The same is true with audio encoding as well. Some people believe in magic, even when there is no magic.
Like I said, you aren't going to get smaller files without making a sacrifice. If less filsize is most important to you then you should focus on a target bitrate at the expensive of quality.
One option is to requantize original MPEG2 stream.
http://code.flexion.org/M2VRequantiser.html
It makes your file smaller with minimal quality loss on video 'focus' area. But it is fairly fast and easy to use via pipes. Demux -> requant -> Remux.
On 29/10/10 00:28, VDR User wrote:
I wonder if there is an easy way to recompress all recordings that are not already H.264 into H.264 files, making them a lot smaller without losing any quality.
The only way to not lose quality is to keep the bitrate high, especially for recordings with a lot of motion. You should encode with constant quality (CRF) set to whatever the minimum quality you want to preserve. You can also reduce the resolution a bit, just make sure to maintain the aspect ratio.
And don't try to scale interlaced material without using a good deinterlacing filter first.
Bottom line is that you will not get smaller filesizes without making sacrifices. Data is lost with every encoding pass.
The point of H264 is that it can achieve better quality for a better bit rate. Although some quality will be lost by transcoding, you could probably halve the file size without making a noticeable difference.
On Fri, Oct 29, 2010 at 10:59 AM, Tony Houghton h@realh.co.uk wrote:
The only way to not lose quality is to keep the bitrate high, especially for recordings with a lot of motion. You should encode with constant quality (CRF) set to whatever the minimum quality you want to preserve. You can also reduce the resolution a bit, just make sure to maintain the aspect ratio.
And don't try to scale interlaced material without using a good deinterlacing filter first.
That's a great point, I should have mentioned it but since I didn't, it's good you did.
Bottom line is that you will not get smaller filesizes without making sacrifices. Data is lost with every encoding pass.
The point of H264 is that it can achieve better quality for a better bit rate. Although some quality will be lost by transcoding, you could probably halve the file size without making a noticeable difference.
That sounds good when you read the datasheets but real world results are a bit different. Also, what you're referring to is encoding comparisons all from a raw source - not mpeg2 vs. the same mpeg2 reencoded in h264. I can't stress enough that there is no magic to be had here for the reasons in my previous post.
Hi and sorry for this long post. I thought to share some of my personal experiences and insights on re-encoding DVB recordings. I might also have a few useful tips that are not always found in these discussions at Linux-specific forums and mailing lists.
Feel free to skip this wall of text if you aren't interested in processing DVB recordings. :)
First a quick response to the previous post.
On 2010-10-29 at 18:50 -0700, VDR User wrote:
On Fri, Oct 29, 2010 at 10:59 AM, Tony Houghton h@realh.co.uk wrote:
The point of H264 is that it can achieve better quality for a better bit rate. Although some quality will be lost by transcoding, you could probably halve the file size without making a noticeable difference.
That sounds good when you read the datasheets but real world results are a bit different. Also, what you're referring to is encoding comparisons all from a raw source - not mpeg2 vs. the same mpeg2 reencoded in h264. I can't stress enough that there is no magic to be had here for the reasons in my previous post.
While I agree with this statement in general, a 50 % bitrate reduction is easily possible at visually transparent quality even when the MPEG-2 source is noisy and blocky. Although with x264 this is a bit harder if the video is interlaced: encoding as interlaced is not (yet) as efficient as progressive encoding and bobbing/double-rate deinterlacing will also require more bitrate. Bobbing with a good filter is in my opinion preferable because most realtime deinterlacing solutions that I have encountered are pretty bad compared to the best software methods. I guess the expensive hardware chips aren't much better than Nvidia and Ati GPU deinterlacing implementations.
Benefits of filtering video
If larger changes to the video content such as noise reduction and same-rate deinterlacing are tolerated, it's quite possible to get below 1000 kbps on average for a collection of SD video streams while retaining very good visual quality compared to the originals. This gives the additional benefit of making streaming possible over a 1 or 2 Mbps uplink.
Deinterlacing solutions
It should be noted that good deinterlacing and good noise reduction are more difficult to achieve than one might think. For (bob-)deinterlacing, MPlayer/MEncoder has -vf yadif=1,mcdeint=0:0:10 (add ,framestep=2 for same-rate deinterlacing), which gives acceptable results. This means that the image won't shimmer much and that most sloped edges won't appear jagged. Yadif and mcdeint are far from perfect on these points, but they are still better than most filters. Yadif alone may be good enough for casual viewing, but it shimmers (bobs up and down) and smears details _a_lot_ compared to mcdeint. Nvidia's best "temporal adaptive" deinterlacer in VDPAU also has similar issues although its edge-directed interpolation is slightly better.
The only significantly better freely available deinterlacing solution that I know of is TempGaussMC (TGMC) AviSynth filter combined with NNEDI2/EEDI2/EEDI3. Although AviSynth is Windows-only software (it's GPLed, but tied to VFW), it works perfectly on x86(-64) Linux platforms with avs2yuv and Wine. Only filters that use GPU acceleration (fft3dgpu) can't be used. The y4m output from avs2yuv can be piped directly to x264 and other encoders.
Here's a recent comparison clip that includes output from PureVideo (Nvidia GPU), yadif, TDeint and TGMC: http://forum.doom9.org/showthread.php?p=1433847#post1433847 I did some comparisons of my own and the PureVideo result reflects that of VDPAU's "temporal spatial" deinterlacing method. Note that TGMC has a strong denoising and local stabilization effect that is useful in compression. yadif+mcdeint would probably stand between TGMC and TDeint in a quality comparison, but it's a few orders of magnitude faster than TGMC+NNEDI2. However, unlike MEncoder filters, AviSynth filters can be split evenly to multiple threads so AviSynth may be faster on a multi-core CPU if only one encode is running at a time. If there's enough RAM available (~ 128-512 MB for each SD AviSynth job or 64-128 MB for MPlayer/MEncoder), it's easiest and most efficient to run as many encoding jobs as there are CPU cores.
Denoising and deblocking
The best denoiser in MPlayer/MEncoder is hqdn3d, but it will smooth details out along with the noise. Instead I've been using motion-compensated MDegrain2 from AviSynth's MVTools2. It obliterates most types of grain and noise but still gives a very natural and non-plastic output. This makes further compression much easier. With noisy film sources the bitrate can often be dropped to half from what it would be without denoising.
I'm not going much into deblocking here because I haven't compared the available methods myself. There are many filters available especially in MPlayer/MEncoder. Deblocking should be coupled with the decoder so that information on macroblock quantizers is available. Otherwise it will smooth out actual detail in the video. Oversmoothing is an issue with the good filters too.
Tradeoffs
As Lars Bläser pointed out earlier, all this processing and encoding comes at a high price in CPU time and energy consumption. Whether it's worth it depends on your purposes, so make your own calculations. If you only want a bitrate reduction for storage reasons, CPU time alone may cost more than the required disk space -- unless it gets cold outside and you are heating with electricity...
Efficient encoding still requires lots of manual work because there are no reliable automatic detection schemes for recognizing different types of sources: interlaced, telecined, field-blended and phase-shifted. Personally I'd like a scheme where the encoder frontend would adapt to source type changes on the fly and encode the video with a variable framerate, but this is not always desirable.
Scripts
I wrote a bash script for downloading and installing AviSynth plus some of the essential filters on Linux. I could upload it somewhere if people are interested? There's also a simple command-line based template system that makes using AviSynth slightly easier, but it needs more work to be useful.
Cheers,
Niko Mikkilä
On Sat, Oct 30, 2010 at 10:03 AM, Niko Mikkilä nm@phnet.fi wrote:
Tradeoffs
As Lars Bläser pointed out earlier, all this processing and encoding comes at a high price in CPU time and energy consumption. Whether it's worth it depends on your purposes, so make your own calculations. If you only want a bitrate reduction for storage reasons, CPU time alone may cost more than the required disk space -- unless it gets cold outside and you are heating with electricity...
Efficient encoding still requires lots of manual work because there are no reliable automatic detection schemes for recognizing different types of sources: interlaced, telecined, field-blended and phase-shifted. Personally I'd like a scheme where the encoder frontend would adapt to source type changes on the fly and encode the video with a variable framerate, but this is not always desirable.
Unfortunately it will always require a lot of manual work, and more importantly knowledge of encoder settings and how they affect the end result. It would be great to have an app which is capable of properly detecting the requirements of each frame and selecting the best way to process it but that's extremely complex and it will never be perfect, which is why you don't see such software. It doesn't help that many editors and broadcasters completely trash their source material. I've delt with video that made me want to slap the person(s) who processed it and I'm guessing I'm not the only one here who can say that. ;)
We can get into a huge discussion about the different filters and methods available but I don't think this thread or mailing list is the right place for that so I'll just say that there are few filters that are 'best' to use across the board. For the best results you have to identify the needs of the source and choose filters that are able to deal with it with the highest quality. Not all interlaced source is the same. Not all telecined source is the same. Etc, etc..
I'd also like to stress to the OP that the time investment, encoding time for TB's worth of sources and energy usage may be far less attractive then simply buying additional storage space. Especially when you consider good 1TB drives can be had for around $50-$70USD these days. Reencoding all that stuff would be the last option on my list to be honest but that's only my opinion.
On 30/10/10 02:50, VDR User wrote:
That sounds good when you read the datasheets but real world results are a bit different. Also, what you're referring to is encoding comparisons all from a raw source - not mpeg2 vs. the same mpeg2 reencoded in h264. I can't stress enough that there is no magic to be had here for the reasons in my previous post.
And I wonder is there still much room for improvement in current H264 encoders? The various MPEG 4 part 2 codecs (DivX etc) improved a great deal over time, and ISTR reading that the first H264 encoders offered almost no practical improvement over the newest part 2 codecs at the time.
On 29.10.2010 00:47, Carsten Koch wrote:
Hi,
I have been using VDR for over 10 years now, so I currently have ~4TB of VDR recordings. Some of them are SDTV/MPEG2, many of the newer recordings are HDTV/H.264. ...
my first suggestion is to upgrade storage if you compare the time you will have to invest and the power consumption of the system that recodes all the stuff then new 2TB disks look much more effective you also will have lots of problems with async sound, lost audio streams (ac3) and picture or just with broken recordings, what will consume additional time
you can read about a conversion script here: http://www.vdr-portal.de/board/thread.php?threadid=82167