EPG problems with TVGids_to_xml

[Hebus]

I think TVXB limits the number of channels from which you can retrieve the program description. That's why I stuck to TVgids2xml.

cya Hans

The problem is i can't get anny discription out of the website...
Searching for it now for a week still nothing that works...

[HtV]

From the tvxb.ini

Code:

!------------------------------------------------------------------------------
!
!       *** Be kind to TVGiDS and the other XMLTV users ***
!
! *Remove channels you don't watch to reduce the download time*
!
! If you want full program descriptions for some channels that you watch often,
! then add the following at the end of each of these channels:
!
!    special=fulldescriptions
!    descriptionhtmlstart=<table id="progDetail"
!    descriptionhtmlend=<p class="meerLinks">
!    descriptionhours=1800-0030
!
! *However not for all channels as it will take a very long time to download*
!
!       *** Be kind to TVGiDS and the other XMLTV users ***
!
! Insert channel schedule information below this line
!----------------------------------------------------
[nederland1.gids.nl]
name=Nederland 1
url=http://www.tvgids.nl/zoeken/?trefwoord=&station=1&genre=&interval=[relseq=0,1,2,3,4,5,6]&timeslot=0
htmlstart=[day=w dd Mmm]
start=1
stop=1,-
title=3

Did you add this to the channels you want the descriptions from:

Code:

!    special=fulldescriptions
!    descriptionhtmlstart=<table id="progDetail"
!    descriptionhtmlend=<p class="meerLinks">

Don't forget to delete the exclamation marks at the beginning of each line.

cya Hans

[Hebus]

is this the same for tvgids.be ?

[HtV]

I think so, it's all in the belgian ini file.

[Hebus]

dang xtvb does not work annymore they have changed sommething on there site sombody has a working ini file for belgium ? Its seems i'm the only one from belgium here that has trouble to get a decent TVguide .
Nobody has a decent working system ?

[Wim Fabri]

dang xtvb does not work annymore they have changed sommething on there site sombody has a working ini file for belgium ? Its seems i'm the only one from belgium here that has trouble to get a decent TVguide .
Nobody has a decent working system ?

I have written a perl script to download the tv guide from skynet.be.
Thanks to your post I noticed it stopped working : skynet requires some cookies to be set now.

here is my code : (notice that the channel section is currently hardcoded - you will have to update the path to the firefox cookies file)

Code:

#!/usr/bin/perl -w
#use diagnostics;
use strict;
use IO::File;
use XML::Simple;
use Data::Dumper;
use LWP::UserAgent;
use Encode qw(encode);
use HTTP::Cookies::Netscape;

my $encoding = Encode::resolve_alias("utf-8");

my $channels={};
$channels->{een}{url}="http://tv-nl.skynet.be/index.html?l1=entertainment&l2=tvguide&l3=programs&l4=channels&channelid=7&date=";
$channels->{'Ketnet-Canvas'}{url}="http://tv-nl.skynet.be/index.html?l1=entertainment&l2=tvguide&l3=programs&l4=channels&channelid=9&date=";
$channels->{'la-une'}{url}="http://tv-nl.skynet.be/index.html?l1=entertainment&l2=tvguide&l3=programs&l4=channels&channelid=52&date=";
$channels->{'la-deux'}{url}="http://tv-nl.skynet.be/index.html?l1=entertainment&l2=tvguide&l3=programs&l4=channels&channelid=53&date=";

my $icons={};

my $fh=new IO::File("c:/temp/xmltv.xml","w");
my $log=new IO::File("c:/temp/xmltv.log","w");

print $fh "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n";
print $fh "<tv>\n";
print $fh "  <channel id=\"een\">\n";
print $fh "    <display-name>een</display-name>\n";
print $fh "    <icon src=\"http://images.mds.prd.skynet.be/TVGuideEPG/Channel/Small/42_29_een.jpg\"></icon>\n";
print $fh "  </channel>\n";
print $fh "  <channel id=\"Ketnet-Canvas\">\n";
print $fh "    <display-name>Ketnet-Canvas</display-name>\n";
print $fh "    <icon src=\"http://images.mds.prd.skynet.be/TVGuideEPG/Channel/Small/42_29_canvas.jpg\"></icon>\n";
print $fh "  </channel>\n";
print $fh "  <channel id=\"la-une\">\n";
print $fh "    <display-name>La Une</display-name>\n";
print $fh "    <icon src=\"http://images.mds.prd.skynet.be/TVGuideEPG/Channel/Small/42_29_laune.jpg\"></icon>\n";
print $fh "  </channel>\n";
print $fh "  <channel id=\"la-deux\">\n";
print $fh "    <display-name>La Deux</display-name>\n";
print $fh "    <icon src=\"http://images.mds.prd.skynet.be/TVGuideEPG/Channel/Small/42_29_ladeux.jpg\"></icon>\n";
print $fh "  </channel>\n";

my $timestamp=time();
for (my $d=0;$d<8;$d++) {
    my ($day,$month,$year)=(localtime($timestamp+$d*24*60*60))[3..5];
    my $date=sprintf("%04d%02d%02d",$year+1900,$month+1,$day);
    my ($nday,$nmonth,$nyear)=(localtime($timestamp+($d+1)*24*60*60))[3..5];
    my $nextDate=sprintf("%04d%02d%02d",$nyear+1900,$nmonth+1,$nday);
    foreach my $channel (keys %$channels) {
        my $url=sprintf("$channels->{$channel}{url}%04d-%02d-%02d",$year+1900,$month+1,$day);
        print $log $url,"\n";
    
        my $ua=new LWP::UserAgent;
        my $cookie_jar = HTTP::Cookies::Netscape->new(
            file => "C:/Documents and Settings/userid/Application Data/Mozilla/Firefox/Profiles/profiledir/cookies.txt",
        );
        $ua->cookie_jar( $cookie_jar );

        my $response=$ua->get($url);
        my $html=$response->content;

        my @rows=($html=~m/<li class="(dark|light)"(.+?)<\/li>/g);
        my $prevTime='0000';
        my $startDate=$date;
        my $endDate=$date;
        for (my $i=1;$i<@rows;$i+=6) {
            print $log $rows[$i],"\n";
            my ($startTime)=($rows[$i]=~m/{TIMESTYLE}>(.+?)u<\/span>/);
            print $log $startTime,,"\n";
            $startTime='0'.$startTime if length($startTime)<5;
            substr($startTime,2,1)='';
            next if ($prevTime eq '0000') and ($startTime ge '2200');   # skip entries before midnight
            if ($startTime lt $prevTime) {
                $startDate=$nextDate;
                $endDate=$nextDate;
            }
            $prevTime=$startTime;
            
            my ($descLink,$title,$category)=($rows[$i+2]=~m/<a href="(.+?)" class="listlink">(.+?)<\/a> <span class="subtext">\((.+?)\)<\/span>/);
            print $log $title,"\n";
            $response=$ua->get("http://tv-nl.skynet.be$descLink");
            $html=$response->content;
            my ($desc)=($html=~/<p class="white">(.+?)<\/p>/);        
            $desc=~s/\&/\&amp;/g if defined $desc;
            $desc=~s/<.+?>//g if defined $desc;        # remove html tags
            
            my ($icon)=($html=~/<img src="(.+?)" class="content_border" \/>/);
            $icons->{$title}=$icon if defined $icon;
            
            my ($endTime)=($html=~/tot (.+?)<\/strong>/);
            if (not defined $endTime) {
                ($endTime)=($rows[$i+6]=~m/{TIMESTYLE}>(.+?)u<\/span>/);
            }
            $endTime='0'.$endTime if length($endTime)<5;
            substr($endTime,2,1)='';
            $endDate=$nextDate if $endTime lt $startTime;
            
            my ($subtitle)=($html=~/<h4 class="title">(.+?)<\/h4>/);
            my ($cast)=($html=~/<p><strong>Cast: <\/strong> <span class="white" style="display:inline;">(.+?)<\/span>/);
            my ($regisseur)=($html=~/<p><strong>Regisseur: <\/strong> <span class="white" style="display:inline;">(.+?)<\/span>/);
            my ($jaar)=($html=~/<p><strong>Jaar: <\/strong> <span class="white" style="display:inline;">(.+?)<\/span>/);
             my ($genre)=($html=~/<p><strong>Genre:<\/strong> <span class="white" style="display:inline;">(.+?)<\/span>/);
             my ($land)=($html=~/<p><strong>Land: <\/strong> <span class="white" style="display:inline;">(.+?)<\/span>/);
            
            print $fh "<programme start=\"$startDate$startTime CET\" stop=\"$endDate$endTime CET\" channel=\"$channel\" clumpidx=\"0/1\">\n";
            $title=~s/\&/\&amp;/g;
            print $fh "<title>",encode($encoding,$title),"</title>";
            if (defined $genre) {
                my @genres=split / - /,$genre;
                foreach my $g (@genres) {
                    print $fh "\n<category>",encode($encoding,$g),"</category>" ;
                }
            } else {
                print $fh "\n<category>",encode($encoding,$category),"</category>" ;
            }
            print $fh "\n<sub-title>",encode($encoding,$subtitle),"</sub-title>" if defined $subtitle;
            print $fh "\n<desc>",encode($encoding,$desc) if defined $desc;
            print $fh "\nRegie : ",encode($encoding,$regisseur) if defined $regisseur;
            print $fh "\nMet : ",encode($encoding,$cast) if defined $cast;
            print $fh "\nLand : $land" if defined $land;
            print $fh " - Jaar : $jaar" if defined $jaar;
            print $fh "\n</desc>" if defined $desc;
            print $fh "\n<icon src=\"$icon\"></icon>" if defined $icon;
            print $fh "\n</programme>\n";
        }
    }
}

print $fh "</tv>\n";
$fh->close();
$log->close();

# download icons
$encoding = Encode::resolve_alias("cp1252");
my $targetDir="c:\\Program Files\\devnz\\gbpvr\\media\\art";
system("del \"$targetDir\\*\" /Q");

foreach my $title (keys %$icons) {
    $title=~s/://g;
    
    if (not -e "$targetDir\\$title.jpg") {
        print encode($encoding,"wget -O \"$targetDir\\$title.jpg\" \"$icons->{$title}\" "),"\n";
        system(encode($encoding,"wget -O \"$targetDir\\$title.jpg\" \"$icons->{$title}\" 2>&1"));
    } else {
        print "$targetDir\\$title.jpg\n";
    }
    
}

[Hebus]

Thats sounds good can you give me some more info how to use it (im not a programmer :o ).

I suppose i need to ad extra channels under my channels?

[Wim Fabri]

Thats sounds good can you give me some more info how to use it (im not a programmer :o ).

I suppose i need to add extra channels under my channels?

yes, and also add print statements that write the channel section of the xml. This section could be written from data on the skynet website, but since I only use 4 channels it wasn't worth the effort.

You have to update following line with the path to your firefox cookies file (if you use internet explorer the code will have to be modified).
file => "C:/Documents and Settings/userid/Application Data/Mozilla/Firefox/Profiles/profiledir/cookies.txt"
In firefox, go to the tv guide on the skynet site to store the required cookies.

You also will have to install perl if you didn't do that allready for something else.

[Hebus]

Ok and how does i start it Noob here :p

[Wim Fabri]

Ok and how does i start it Noob here :p

Save the script in a file with extension pl and you should be able to run it as any other program or script if perl is installed.