TVGids.nl actively stopping grabbers?

[-=TJ=-Stefan]

Give the tv_grab_nl_py grabber a try. It works the same as the regular xmltv grabber, but it caches the detailed results so that it does not have to download those every day. It does refresh the time information every time. You can download it from http://graphics.tudelft.nl/~paul/grabber/. It does, however, have a small bug in it that results in wrong stop-time for programs after midnight. PM me if you'd like my fixed version for it.

I think you've already send it to me in my email. But if you have version 60 fixed for me that would be very nice. I presume that is the fixed version. I will install it next to my working tvgids_to_xml and will take a look at it. Can it also download 6 days ahead? Anyway will give a go. If it is less bandwith consuming for tvgids then I will use that version in the future.

At the moment I didn't change anything and it still downloads 6 days ahead. Maybe some system engineer at tvgids who thinks he's funny by closing the site for grabbers...

[hzwaal]

I think you've already send it to me in my email.

Oh, that was you :o. I remember sending it to someone, but I forgot to whom.

But if you have version 60 fixed for me that would be very nice. I presume that is the fixed version.

The one I sent you no longer works after the tvgids.nl change. I'll send you the fixed version 60.

I will install it next to my working tvgids_to_xml and will take a look at it. Can it also download 6 days ahead?

You can configure the number of days to download ahead, so it can download as many days ahead as tvgids.nl provides.

Anyway will give a go. If it is less bandwith consuming for tvgids then I will use that version in the future.

Because of the caching, it will consume a lot less bandwidth than a grabber that grabs everything from scratch. It also tries to be nice to tvgids.nl by pausing 1 or 2 seconds between each page download.

If you're going to run them in parallel you'll of course use more bandwidth

[SuMo]

....... I'll send you the fixed version 60.

any chance you can make this public ?

[hzwaal]

any chance you can make this public ?

NOTE: This is an updated post, the previous post had an error in it (I was too hasty).

Of course I can. Come to think of it I don't know why I didn't do this in the first place.

I've attached the patched version 60 of the Python script. Basically, the fix is one if statement that checks whether the stopdate is before the startdate and a subsequent correction with one day. This is needed because tvgids.nl puts the nightly programs on the page for the previous day. The if statement is:

if stopdate < startdate:
program['stop-time'] = program['stop-time'] + single_day

it needs to be included after the other one-day correction statement in the correct-times function (line 701 for the 60 version).

Installation of the script is simple:
1) Put the .py file somewhere in a directory of your choice
2) Install the Python interpreter (http://www.python.org/ftp/python/2.5.1/python-2.5.1.msi)
3) Schedule the script to be called daily (parameters are the same as for xmltv)

I use the following command line arguments:

.\tv_grab_nl.py --output .\tvguide.xml --config-file .\tv_grab_nl_py.conf --cache .\tv_grab_nl_py.cache --slow --days 8

Good luck with it...