New features in 0.1.0: - Support for character set conversion for selected channels. - Support for stripping HTML entities. - Supply user with extra information for each setup menu option using Info key.
Homepage for the plugin: http://projects.vdr-developer.org/projects/plg-epgfixer
Am Samstag, 5. Mai 2012 schrieb Matti Lehtimäki matti.lehtimaki@gmail.com:
New features in 0.1.0:
- Support for character set conversion for selected channels.
- Support for stripping HTML entities.
- Supply user with extra information for each setup menu option using
Info key.
Homepage for the plugin: http://projects.vdr-developer.org/projects/plg-epgfixer
Hi,
thanks for the plugin, I started today to build openSUSE-packages for this plugin in my repository, but for openSUSE Factory it is failing. Perhaps a problem with gcc 4.7?
g++ -g -O3 -Wall -Woverloaded-virtual -Wno-parentheses -fPIC -c - D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE - DUSE_DDEPGENTRY -DUSE_JUMPINGSECONDS -DUSE_LIEMIEXT -DUSE_LIRCSETTINGS - DUSE_PLUGINMISSING -DUSE_TIMERINFO -DUSE_TTXTSUBS -DUSE_WAREAGLEICON - D_GNU_SOURCE -DPLUGIN_NAME_I18N='"epgfixer"' -DHAVE_PCREPOSIX - I/usr/include/vdr/include epgfixer.c In file included from charset.h:11:0, from epgfixer.c:10: tools.h: In instantiation of 'bool cEpgfixerList<T>::LoadConfigFile(const char*, bool) [with T = cRegexp]': tools.h:81:50: required from 'bool cEpgfixerList<T>::ReloadConfigFile(bool) [with T = cRegexp]' epgfixer.c:76:36: required from here tools.h:60:20: error: 'Add' was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation [-fpermissive] tools.h:60:20: note: declarations in dependent base 'cListBase' are not found by unqualified lookup tools.h:60:20: note: use 'this->Add' instead tools.h: In instantiation of 'bool cEpgfixerList<T>::LoadConfigFile(const char*, bool) [with T = cCharSet]': tools.h:81:50: required from 'bool cEpgfixerList<T>::ReloadConfigFile(bool) [with T = cCharSet]' epgfixer.c:78:37: required from here tools.h:60:20: error: 'Add' was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation [-fpermissive] tools.h:60:20: note: declarations in dependent base 'cListBase' are not found by unqualified lookup tools.h:60:20: note: use 'this->Add' instead make: *** [epgfixer.o] Error 1
On 05/06/2012 03:00 PM, Marcel Witte wrote:
Hi,
thanks for the plugin, I started today to build openSUSE-packages for this plugin in my repository, but for openSUSE Factory it is failing. Perhaps a problem with gcc 4.7?
The problem seems to be related to gcc 4.7. Could you try the following patch.
diff --git a/tools.h b/tools.h index dfcee30..bf49fac 100644 --- a/tools.h +++ b/tools.h @@ -57,7 +57,7 @@ protected: cReadLine ReadLine; while ((s = ReadLine.Read(f)) != NULL) { if (!isempty(s)) { - Add(new T()); + this->Add(new T()); cList<T>::Last()->SetFromString(s, true); } }
Am Sonntag, 6. Mai 2012 schrieb Matti Lehtimäki matti.lehtimaki@gmail.com:
On 05/06/2012 03:00 PM, Marcel Witte wrote:
Hi,
thanks for the plugin, I started today to build openSUSE-packages for this plugin in my repository, but for openSUSE Factory it is failing. Perhaps a problem with gcc 4.7?
The problem seems to be related to gcc 4.7. Could you try the following patch.
diff --git a/tools.h b/tools.h index dfcee30..bf49fac 100644 --- a/tools.h +++ b/tools.h @@ -57,7 +57,7 @@ protected: cReadLine ReadLine; while ((s = ReadLine.Read(f)) != NULL) { if (!isempty(s)) {
Add(new T());
this->Add(new T()); cList<T>::Last()->SetFromString(s, true); } }
Using this Patch it is compiling :) Thanks
W dniu 2012-05-05 20:15, Matti Lehtimäki pisze:
New features in 0.1.0:
- Support for character set conversion for selected channels.
- Support for stripping HTML entities.
- Supply user with extra information for each setup menu option using
Info key.
Homepage for the plugin: http://projects.vdr-developer.org/projects/plg-epgfixer
Hello I'm tryin charset recoding but no luck.
1) I currently use VDR headless. It means I can't see OSD and so I don't know if epgfixer needs some "switching on" causing some variables written to VDR conf file?
2) Could you implement some log messages about conversion? I know epgfixer starts, but I don't know if it has read configuration files and tries to use them. My configuration is in charset.conf: 1:iso6937 or iso6937 I've tried also iso8859-2 but it doesn't seem to recode anything.
3)I test on one channel only: :aa TVP 2;CYFRA +:10892:HC34M2S0:S13.0E:27500:167=2:108=pol@4:508:100,1813,500,B00,B01:4808:318:11900:0 Unfortunatelly this channel is encrypted.
VDR alone produces sth like this: May 6 23:52:53 wuwek vdr: [29565] EPG bugfix statistics May 6 23:52:53 wuwek vdr: [29565] ===================== May 6 23:52:53 wuwek vdr: [29565] IF SOMEBODY WHO IS IN CHARGE OF THE EPG DATA FOR ONE OF THE LISTED May 6 23:52:53 wuwek vdr: [29565] CHANNELS READS THIS: PLEASE TAKE A LOOK AT THE FUNCTION cEvent::FixEpgBugs() May 6 23:52:53 wuwek vdr: [29565] IN VDR/epg.c TO LEARN WHAT'S WRONG WITH YOUR DATA, AND FIX IT! May 6 23:52:53 wuwek vdr: [29565] ===================== May 6 23:52:53 wuwek vdr: [29565] Fix Hits Channels May 6 23:52:53 wuwek vdr: [29565] 7 5 TVP 2 It's sign that charset of epg is broken. I would like to know if this messages is produced after or before epgfixer work.
4) I use UTF-8 in system. I have problem how to recognize charset of epg on this channel. I was trying to parse /var/cache/vdr/epg.data but it seems that encoding is broken. Seems like some polish characters are saved on 2 chars (UTF), but it's not proper pair to create polish character. So instead of: 100 tysięcy bocianów I'm getting: 100 tysiЮecy bocianТow
Marx
On 07.05.2012 13:58, Tomasz Bubel wrote:
Hi, For Polish epg I'm using this patch:
Yes, I know it, but I'm trying to reproduce this effect without patch (so without a need of recompiling VDR) only using this new plugin. Patching VDR doesn't allow to use deb packages - you have to compile VDR and all plugins yourself (of course you can produce deb packages for yourself - which doesn't have sense). Having all installed from deb allows for painless upgrade of VDR and plugins. It's easy to add/remove new plugins as needed. Marx
On 05/07/2012 01:22 AM, Marx wrote:
Hello I'm tryin charset recoding but no luck.
- I currently use VDR headless. It means I can't see OSD and so I don't
know if epgfixer needs some "switching on" causing some variables written to VDR conf file?
There is no need to activate the plugin or fixes through OSD. Everything can be done also by just using config files.
- Could you implement some log messages about conversion?
I know epgfixer starts, but I don't know if it has read configuration files and tries to use them. My configuration is in charset.conf: 1:iso6937 or iso6937 I've tried also iso8859-2 but it doesn't seem to recode anything.
I'll add some logging about active fixes to next version. One thing that comes to mind is the need to clear EPG data. The plugin only fixes new EPG data not the already existing data. So before you can see if the conversion works you need to clear old data. You can do that either by manually deleting epg.data, using VDR SVDRP command CLRE or by using this plugin's setup menu.
3)I test on one channel only: :aa TVP 2;CYFRA +:10892:HC34M2S0:S13.0E:27500:167=2:108=pol@4:508:100,1813,500,B00,B01:4808:318:11900:0
Unfortunatelly this channel is encrypted.
VDR alone produces sth like this: May 6 23:52:53 wuwek vdr: [29565] EPG bugfix statistics May 6 23:52:53 wuwek vdr: [29565] ===================== May 6 23:52:53 wuwek vdr: [29565] IF SOMEBODY WHO IS IN CHARGE OF THE EPG DATA FOR ONE OF THE LISTED May 6 23:52:53 wuwek vdr: [29565] CHANNELS READS THIS: PLEASE TAKE A LOOK AT THE FUNCTION cEvent::FixEpgBugs() May 6 23:52:53 wuwek vdr: [29565] IN VDR/epg.c TO LEARN WHAT'S WRONG WITH YOUR DATA, AND FIX IT! May 6 23:52:53 wuwek vdr: [29565] ===================== May 6 23:52:53 wuwek vdr: [29565] Fix Hits Channels May 6 23:52:53 wuwek vdr: [29565] 7 5 TVP 2 It's sign that charset of epg is broken. I would like to know if this messages is produced after or before epgfixer work.
If I read the source of VDR correctly EPG fix 7 has nothing to do with character sets. That message is produced after the plugin since it gives also other plugins and VDR chance to fix EPG. The same fixes can also be activated inside the plugin in VDR's setup.conf to make the basic fixes before the plugin does any user defined fixes. Basically you could disable VDR's EPG fixes and activate them in the plugin to prevent occasional breaking of VDR's EPG fixes. This breaking is caused by the fact that VDR's own fixes are run after the plugin's.
- I use UTF-8 in system. I have problem how to recognize charset of epg
on this channel. I was trying to parse /var/cache/vdr/epg.data but it seems that encoding is broken. Seems like some polish characters are saved on 2 chars (UTF), but it's not proper pair to create polish character. So instead of: 100 tysięcy bocianów I'm getting: 100 tysiЮecy bocianТow
I don't know anything about that kind of problem.
On 07.05.2012 18:10, Matti Lehtimäki wrote:
There is no need to activate the plugin or fixes through OSD. Everything can be done also by just using config files.
good news
I'll add some logging about active fixes to next version. One thing that comes to mind is the need to clear EPG data. The plugin only fixes new EPG data not the already existing data. So before you can see if the conversion works you need to clear old data. You can do that either by manually deleting epg.data, using VDR SVDRP command CLRE or by using this plugin's setup menu.
I'm of course cleaning epg every time I'm testing by deleting /var/cache/vdr/epg.data. I will wait for a version with more logs.
- I use UTF-8 in system. I have problem how to recognize charset of epg
on this channel. I was trying to parse /var/cache/vdr/epg.data but it seems that encoding is broken. Seems like some polish characters are saved on 2 chars (UTF), but it's not proper pair to create polish character. So instead of: 100 tysięcy bocianów I'm getting: 100 tysiЮecy bocianТow
I don't know anything about that kind of problem.
The problem is that probably codepage which I'm trying convert from is not really iso6937. As stated by Klaus in other threads, if provider doesn't broadcast codepage, standard says it's iso6937. If provider broadcast codepage - VDR convert it automatically and it works without any additional steps.
Tests to use epgfixer with converting from iso6937 do nothing - EPG is still broken. So probably iso6937 is not codepage which I had to try to convert from (it's logical - if it really would be iso6937 - VDR would convert it because it's standard). Hence the need to determine in which codepage this EPG is really broadcasted. I was trying "konwert" on epg.data file but no luck. While it suggests a few codepages none of them is usable on that file (converting to thic codepages didn't fix this file). So probably this file has completely messed up encoding which can't be converted. I guess that VDR uses encoding from my system properties - UTF 8. It somehow tries to convert this EPG and fails, yet still save it in UTF-8 which and then file becomes messed up. Another possibility is that this EPG isn't any standard codepage, and so I have to write some regexp to mimic this patch http://pastebin.com/tkNtpTCh
I've just read closer this patch and it's different from what I have used prevoiusly and suggest to try ISO-8859-9. I will try it. Marx
On 08.05.2012 08:27, Marx wrote:
On 07.05.2012 18:10, Matti Lehtimäki wrote:
There is no need to activate the plugin or fixes through OSD. Everything can be done also by just using config files.
good news
I'll add some logging about active fixes to next version. One thing that comes to mind is the need to clear EPG data. The plugin only fixes new EPG data not the already existing data. So before you can see if the conversion works you need to clear old data. You can do that either by manually deleting epg.data, using VDR SVDRP command CLRE or by using this plugin's setup menu.
I'm of course cleaning epg every time I'm testing by deleting /var/cache/vdr/epg.data. I will wait for a version with more logs.
- I use UTF-8 in system. I have problem how to recognize charset of epg
on this channel. I was trying to parse /var/cache/vdr/epg.data but it seems that encoding is broken. Seems like some polish characters are saved on 2 chars (UTF), but it's not proper pair to create polish character. So instead of: 100 tysięcy bocianów I'm getting: 100 tysiЮecy bocianТow
I don't know anything about that kind of problem.
The problem is that probably codepage which I'm trying convert from is not really iso6937. As stated by Klaus in other threads, if provider doesn't broadcast codepage, standard says it's iso6937. If provider broadcast codepage - VDR convert it automatically and it works without any additional steps.
Tests to use epgfixer with converting from iso6937 do nothing - EPG is still broken. So probably iso6937 is not codepage which I had to try to convert from (it's logical - if it really would be iso6937 - VDR would convert it because it's standard). Hence the need to determine in which codepage this EPG is really broadcasted. I was trying "konwert" on epg.data file but no luck. While it suggests a few codepages none of them is usable on that file (converting to thic codepages didn't fix this file). So probably this file has completely messed up encoding which can't be converted. I guess that VDR uses encoding from my system properties - UTF 8. It somehow tries to convert this EPG and fails, yet still save it in UTF-8 which and then file becomes messed up. Another possibility is that this EPG isn't any standard codepage, and so I have to write some regexp to mimic this patch http://pastebin.com/tkNtpTCh
I've just read closer this patch and it's different from what I have used prevoiusly and suggest to try ISO-8859-9. I will try it.
Has anybody ever tried to contact the broadcaster who is sending this broken EPG data, and ask them to fix it? Might be worth a try...
Klaus
On 08.05.2012 10:11, Klaus Schmidinger wrote:
Has anybody ever tried to contact the broadcaster who is sending this broken EPG data, and ask them to fix it? Might be worth a try...
This broadcaster (Cyfra+) sells own decoders which works, so i doubt it will fix it for a few persons using VDR :) but of course I can try do mail them. But first I would like to understand which codepage this EPG use. File /var/cache/vdr/epg.data narrowed to only this channel doesn't seem to use any standard codepage Marx
On 8 May 2012 12:36, Marx acc.for.news@gmail.com wrote:
On 08.05.2012 10:11, Klaus Schmidinger wrote:
Has anybody ever tried to contact the broadcaster who is sending this broken EPG data, and ask them to fix it? Might be worth a try...
This broadcaster (Cyfra+) sells own decoders which works, so i doubt it will fix it for a few persons using VDR :) but of course I can try do mail them. But first I would like to understand which codepage this EPG use. File /var/cache/vdr/epg.data narrowed to only this channel doesn't seem to use any standard codepage Marx
There are numerous defects raised for Cyfra+ over the web against other PVRs (MythTV, Tvheadend etc.).
A quick google seemed to suggest the following encodings are used on Cyfra+:
ISO-8859-2 for source network id 113 ISO-6937 for source network id 318
Hi,
This broadcaster (Cyfra+) sells own decoders which works, so i doubt it will fix it for a few persons using VDR :) but of course I can try do mail them.
Do try. I made the experience that some engineers at these broadcasters even use VDR at home or know folks who do, so they are grateful for information like that.
Regards,
Hanno
On Tue, May 8, 2012 at 6:14 AM, Hanno Zulla abos@hanno.de wrote:
This broadcaster (Cyfra+) sells own decoders which works, so i doubt it will fix it for a few persons using VDR :) but of course I can try do mail them.
Do try. I made the experience that some engineers at these broadcasters even use VDR at home or know folks who do, so they are grateful for information like that.
If they use VDR at home, how come they didn't know their epg was broken?
W dniu 2012-05-08 17:02, VDR User pisze:
On Tue, May 8, 2012 at 6:14 AM, Hanno Zullaabos@hanno.de wrote:
This broadcaster (Cyfra+) sells own decoders which works, so i doubt it will fix it for a few persons using VDR :) but of course I can try do mail them.
Do try. I made the experience that some engineers at these broadcasters even use VDR at home or know folks who do, so they are grateful for information like that.
If they use VDR at home, how come they didn't know their epg was broken?
VDR isn't as popular here in Poland as in Germany, so I doubt anyone in Cyfra+ know what VDR is. Anyway I've tried ISO-8859-9 in epgfixer, but nothing has changed. Next is ISO-8859-2 but I'm afraid that I'm configuring something bad because I see no changes in EPG whatever I put in /etc/vdr/plugins/charset.conf Marx
On 05/08/2012 07:30 PM, Marx wrote:
VDR isn't as popular here in Poland as in Germany, so I doubt anyone in Cyfra+ know what VDR is. Anyway I've tried ISO-8859-9 in epgfixer, but nothing has changed. Next is ISO-8859-2 but I'm afraid that I'm configuring something bad because I see no changes in EPG whatever I put in /etc/vdr/plugins/charset.conf Marx
I don't know why you don't see any changes in EPG data if you change the config since I do, although not the correct characters. I just noticed that due to internal conversion from assumed ISO6937 to whatever your VDR is using causes problem, but these can be circumvented by first reverting VDR's conversion i.e back to ISO6937 and then from for example ISO-8859-9 to VDR's charset. Unfortunately this is not yet possible in the plugin but I have already made the necessary code to fix the problem.
I will try to release a new version hopefully tomorrow. I have also found several other bugs which I will fix before a new release. I have also some new features that I try to include.
Hi,
On 05/08/2012 07:30 PM, Marx wrote:
because I see no changes in EPG whatever I put in /etc/vdr/plugins/charset.conf Marx
I may be way off the mark here - I have come in halfway through the thread and do not use this plugin, but wouldn't the path above be:
/etc/vdr/plugins/epgfixer/charset.conf ?
Regards,
Richard
On 09.05.2012 00:39, Richard Scobie wrote:
Hi,
On 05/08/2012 07:30 PM, Marx wrote:
because I see no changes in EPG whatever I put in /etc/vdr/plugins/charset.conf Marx
I may be way off the mark here - I have come in halfway through the thread and do not use this plugin, but wouldn't the path above be:
/etc/vdr/plugins/epgfixer/charset.conf ?
it may be true, doc shows only cp command: cp -R PLUGINS/src/epgfixer/epgfixer /path/to/vdrconf/plugins/ I will try with subdirectory Marx
W dniu 2012-05-09 08:18, Marx pisze:
On 09.05.2012 00:39, Richard Scobie wrote:
Hi,
On 05/08/2012 07:30 PM, Marx wrote:
because I see no changes in EPG whatever I put in /etc/vdr/plugins/charset.conf Marx
I may be way off the mark here - I have come in halfway through the thread and do not use this plugin, but wouldn't the path above be:
/etc/vdr/plugins/epgfixer/charset.conf ?
it may be true, doc shows only cp command: cp -R PLUGINS/src/epgfixer/epgfixer /path/to/vdrconf/plugins/ I will try with subdirectory
it didn' work I made subderictory and put in /etc/vdr/plugins/epgfixer/regexp.conf text: title=xxx but it didn't change title in any item to "xxx" Marx
On 05/10/2012 06:47 PM, Marx wrote:
it didn' work I made subderictory and put in /etc/vdr/plugins/epgfixer/regexp.conf text: title=xxx but it didn't change title in any item to "xxx" Marx
That is incorrect regexp for the plugin. At the moment all data has to be assigned to EPG fields using regexp backreferences as shown in examples in "epgfixer/regexp.conf" located in the plugin's source directory.
As written earlier I'm packaging this plugin in my vdr repository for openSUSE. But I have a ploblem using the plugin, I got this at start:
vdr: /usr/lib64/vdr/libvdr-epgfixer.so.1.7.27: undefined symbol: pcre_free
The plugin is building without problems. Has anybody any idea how I can fix this? :)
Am Samstag, 5. Mai 2012 schrieb Matti Lehtimäki matti.lehtimaki@gmail.com:
New features in 0.1.0:
- Support for character set conversion for selected channels.
- Support for stripping HTML entities.
- Supply user with extra information for each setup menu option using
Info key.
Homepage for the plugin: http://projects.vdr-developer.org/projects/plg-epgfixer
-- Matti
vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
-
Dominic Evans -
Hanno Zulla -
Klaus Schmidinger -
Marcel Witte -
Marx -
Matti Lehtimäki -
Richard Scobie -
Tomasz Bubel -
VDR User