MultiSite Incremental XMLTV Epg Grabber

[jksmurf]

Thanks for the sites ... they are certainly the most easy to do (Chinese) but I will try as soon as I have some time ...

WG++Maker . . Jan

Hi Jan,

Just wondering if you ever got time to look at the inis from TVxB to see if any could be transferred across to WG++?

If nothing else, it would be great to get the schedules from here http://www.setanta.com/hongkong/TV-Listings/ as the Cable Provide takes the show times and rounds them to the nearest HALF hr which means I always miss parts of games.

The Setanta site is not html like TVxB needs, so I can't download the pages and parse the data. I started playing around with an AutoIt script to download the pages as html http://forums.nextpvr.com/showthread.php...ght=autoit but only got so far with it...

k.

[WG++Maker]

Hi jksmurf,

I had a look at http://www.setanta.com/hongkong/TV-Listings/ and , indeed, only the first day has a'normal' grabbable html format. Like you I couldn't figure out how to retrieve the other days. Sorry

As alternative for the setanta sports channel there is http://nowsports.now-tv.com/nowSports/tv...679&time=1. That one is easy to make and has a number of other sports channels.

I found a few other sites in the tvxb ini's you send me. Like :

http://programme.tvb.com/print/pearl/2012-11-18/
http://www.hkatv.com/v5/schedule/schedule.html?c=2

Are you interested in any of these?

WG++Maker .. Jan

[jksmurf]

Hi Jan,

First off, thanks very much for looking at these, much appreciated.
I actually got quite good at figuring out formats and ways to grab and parse html-based sites using TVxB (all the inis are mine) but it falls flat when you go to the ever-increasing number of java-based sites (or ones with no html or possibility to download as html).

Hi jksmurf,

I had a look at http://www.setanta.com/hongkong/TV-Listings/ and , indeed, only the first day has a'normal' grabbable html format. Like you I couldn't figure out how to retrieve the other days. Sorry

Bummer, this was the one I really wanted to work :-) - I did sort of get my crude auto-it script to go to each page one by one like I was "clicking" each tab, but TVxB did not like the downloaded html so would not parse it.

As alternative for the setanta sports channel there is http://nowsports.now-tv.com/nowSports/tv...679&time=1. That one is easy to make and has a number of other sports channels.

Lol - actually I do have those in the ini already - BUT - the show times are only to the nearest half hour (which means I lose start or end of recordings!) and the cable provider just says that's what I get from Setanta .. I have my doubts.

I found a few other sites in the tvxb ini's you send me. Like :

http://programme.tvb.com/print/pearl/2012-11-18/
http://www.hkatv.com/v5/schedule/schedule.html?c=2

Are you interested in any of these?

WG++Maker .. Jan

Actually both of those are working really well with TVxB, so I personally do not need these to be transferred to WG++ format, however other users might; up to you, but you do not need to rush making an ini for me alone.

All in all, I really wonder what the future holds wrt EPGs; if I can't strip it out of my cable box (anotehr therad), and I can no longer get it from a website, NPVR becomes a great program with no data input to use it the way it should be used.

Cheers

k

[WG++Maker]

jksmurf

Good news!! Another (expert) WG++ user 'cracked' the required date /channel entry for the url postdata of setanta.com.asia!!

The working siteini files for it can be downloaded @ http://www.servercare.nl/Docs/SiteIni.Pack.11.rar . You will find it in the 'networks' folder because there are also new variants for Australia and Africa. We are also working on a variant for Ireland.

WG++Maker .. Jan

[jksmurf]

Fantastic, thanks again Jan,

k

[jksmurf]

Fantastic, thanks again Jan,

k

Hi Jan, Found time to really test this and I'm thrilled to say it works really well, thanks again!

Pretty steep learning curve for me, even coming off TVxB, with the ini settings etc, but I finally worked it out.
I only changed the WG++ config.ini for filename location, days and the channel line I got from the xml provided in the list of inis. All good stuff and VERY quick.

A GUI might a good next step to bring it more into the mainstream, for ease of navigation?

I noted a couple of anomalies in the resultant XML, not show stoppers, but it would be quite nice to know how to fix them up?

Unlike the TVxB ini which I know my way around well, the WG++'s setanta.com.ini is pretty much all Greek to me (and that's from someone that can read very limited Greek... ) . This worries me a bit as I have found that as sites change I can change the TVxB ini's to compensate, (TVxB author no longer supports it) whereas for WG++ I'd have to constantly pester you. :o

Two more requests if I may ?
1. Is there a "-noconsole" mode, like TVxBs?
2. Probably "hard-basket" but is there anyway to get the "Live" shows categorised or shown in the description as (Live) or do they just use a jpg or png of live on the webpage?

Cheers,

k.

[WG++Maker]

Hi Jan, Found time to really test this and I'm thrilled to say it works really well, thanks again!

Pretty steep learning curve for me, even coming off TVxB, with the ini settings etc, but I finally worked it out.
I only changed the WG++ config.ini for filename location, days and the channel line I got from the xml provided in the list of inis. All good stuff and VERY quick.

A GUI might a good next step to bring it more into the mainstream, for ease of navigation?

I noted a couple of anomalies in the resultant XML, not show stoppers, but it would be quite nice to know how to fix them up?

Unlike the TVxB ini which I know my way around well, the WG++'s setanta.com.ini is pretty much all Greek to me (and that's from someone that can read very limited Greek... ) . This worries me a bit as I have found that as sites change I can change the TVxB ini's to compensate, (TVxB author no longer supports it) whereas for WG++ I'd have to constantly pester you. :o

Two more requests if I may ?
1. Is there a "-noconsole" mode, like TVxBs?
2. Probably "hard-basket" but is there anyway to get the "Live" shows categorised or shown in the description as (Live) or do they just use a jpg or png of live on the webpage?

Cheers,

k.

Sorry to hear that you didn't understand my Greek. But don't worry too much, the setanta ini has a difficult to understand section that takes care of the construction of the postdata string which is very special and unusual. I think these kind of things aren't even possible in tvxb. So don't blame yourself that you can't read this chapter of Homer right away. Normal ini's are a lot easier to follow.

But if you like to learn the grammar read http://www.servercare.nl/Docs/Documentation.V1.1.1.pdf

I agree about the GUI, and started the development some time ago , but never got the time to finish it. Right now the new website webgrabplus.com takes most of the time.

The anomalies: The (n) at the end of the description is generated by the program to indicate the type of update n, c, g, r that was used to create the program listing. You can disable it when you add n to the mode in the config.
The double period .. is probably from the site itself (not sure) . You can try to add
index_description.modify {cleanup} to the ini. (to see if it works you must do one run in update="f", to force a regrab).
If that doesn't help run patchexe because I changed the way the description is composed and the old way can also generate a .. at the end.

re 1. At the moment there is no 'silent' mode of the program. I have seen that others have constructed batch files from which the program ist started in minimized mode. But I don't exactly remember how (I am terrible with batch files)
re 2. Can you give me an example of a live program? (channel and time) I can have a look in the html how it's listed and how to add it to the xmltv.

Jan

[jksmurf]

Thanks, (n) is a simple fix, will try the ones for ".."

For 2 http://www.setanta.com/asia/TV-Listings/

See 11 am on Sun 16 for e.g.

k.

[WG++Maker]

jksmurf,

to add LIVE! at the beginning of the descriptions :
Under:
index_description.scrub {single|<span class="prog-desc">||</span>|</span>}
Add:
* live ??
index_temp_1.scrub {single|<img id="||</span>|</span>}
index_description.modify {addstart('index_temp_1' ~ "live.gif")|LIVE! }

Translated from the 'Greek'
1st line : index_temp_1 scrub the live.gif image line
2nd line: add to the start of the description the string LIVE! if index_temp_1' contains (~) the string live.gif

WG++Maker .. Jan

[jksmurf]

Cool! Will give it a whirl!

Works well!. If I wanted to add it AFTER the Title, rather than before Description, would I do this?

Under:
index_title.scrub {single|<span class="prog-desc">||</span>|</span>}
Add:
* Check for "Live" shows
index_temp_1.scrub {single|<img id="||</span>|</span>}
index_title.modify {addend('index_temp_1' ~ "live.gif")| (LIVE)}