Zap2it Closing Down

[mediaman]

My 2 Canadian cents (which is worth more in the US than before)

It's inevitable that another source will be found. How reliable or convenient is unknown at this time.

Sub and the other main GBPVR community users have done an incredible job of providing countless people with a free, reliable and damn good piece of software.

The key word is "free". In these forums and the Zap2it forums there are many posts from users willing to pay for the service. That sort of defeats the purpose of free but many users are immediately jumping the gun that there may be no future free service. Probably there will be but at this moment in time...nothing is guaranteed.

Zap2it is dumping the service because it is a financial drain. Their forum spokesperson says

[INDENT]-Continued use of the service to support commercial products, in violation of the agreement.
- Commercial products continued to grow despite previous appeals that this activity stop.
- There are significant changes to the supporting data structure forthcoming and we could not devote resources to the continued upkeep and enhancements of the service.
- Maintenance of the service is impacting our resource pool for other projects.
- We sought alternative options but were unable to find a solution.
- We recognize the hardship this creates for the user community. We are open to alternative solutions and would consider proposals that met the needs of the user community and our company.
- We looked into options to turn this into a paid service however we do not have the infrastructure at this time. [/INDENT]

If GBPVR users (and others who use the service) were to show the amount of potential subscribers then that translates to $. I would bet that if there were a profit to turn then there would be resources to continue the service. It's not that simple as there are regulatory agreements, competition agreements, etc.

Could Sub send a mass email to all forum members and post a $ range poll of willingness to pay? Could do....but that only taps a small portion of the GBPVR users.

Geeks are sometimes fickle. We'll spend thousands on our computers but cry out because of a subscription rate. (again...defeating the wonderful charm of the free aspect of GBPVR)

I do know that some alternative will appear but, on the other hand, Sub could, if he tracks it, propose to Zap2it that there have been x number of downloads of GBPVR and extrapolate the potential market. (I wouldn't be opposed to Sub and the crew getting a cut for all their hard work). If x number of GBPVR users time x number of $ subscriptions and it means a profit for Zap2it...they'll consider it.

But it would be wise to make that proposal well before September 1st.

That's my 2c

[sub]

Instead of p2p, another idea along the same lines is using a usenet news group like "alt.tv" for making this data available to others.

The client app that produces xmltv files for use in GB-PVR would first check the users ISP's "alt.tv" newsgroup for chunks applicable to them, downloading any that are useful. If it has gaps, then scrape at least a bit more data from tv.yahoo.com or where ever, before constructing a new chunk and posting to alt.tv. (include a text signature in the post so that you know they've authentic)

Sorry, I know these are outside the scope of a simple grabber, but some to think about if the need arises in the future

Why is this better than every one pulls their own data? less traffic for big Yahoo!?

I'm really thinking about a scenario I have first hand experience with. I was nightly running web scraping to get my listing, which resulted in a several thousand hits on their webserver in a short period (one hit for each 2 hour period for a week * one hit for every show listed in that 2 hour period * about 90 odd channels = about 30,000 web server hits). Thats about 30,000 hits on the web server just for my listings. I stupidly didnt have a pause between requests, which resulted in a large noticable spike occuring on their web server logs every night at the same time. This web site banned my IP address, and explained to me why when I rang them. Luckily my IP address was changed by the cable company short time later.

This type of activity is much harder on a system than real users just browsing the website to look at whats on the tube. If the hundreds of thousands of previous zap2it users all start using some other website, each submitting thousands of requests to get their listings, then even a big player like yahoo will take notice.

zap2it originally created the the zap2it service because of the high load the old xmltv grabber produced. This was serveral years ago now - there has been alot of growth in this market in recent years, and there will be a significantly higher number of people trying to do this.

(of course you'd come up with a more efficent scheme which only updates the next couple of days worth of listings, caching the rest from previous requests, but you get the idea...it's still a lot of hits)

[zed]

This discussion only affects the free data service zap2it was offering. SageTV and BeyondTV will continue to get listings from zap2it, and pay thousands of dollars per month for the pleasure. ...

I wonder if one could purchase Sage or Beyond and use it only to get the listings? Then siphon the file(s) off and use them in GB-PVR, probably after some conversion? 70 or 80 bux is kinda steep but if you got long term access to listings it might be worth it...or is there a listings fee in addition to the purchase?

Or perhaps install the free trial and reverse-engineer the listings loader...but that might not be legal...

Hmmm...I have an MCE 2005 disk around here somewhere...I wonder...

[Ryland]

Is there a way for me to get GBPVR to use BeyondTV's tv listings since I already have a valid account with them?

There is not a separate listings fee for BTV.

[madcat]

these sites may be of some use for grabbing information
http://tv.entertainment.excite.com/index.html

Doesn't do Canada

http://couchville.com/guide

But this one seems to! and I think can be easily grabbed.

I already foudn that the line they use inside the code to get shows titles is something like

Code:

/GuideDataService.ashx/GetEpisodeTiles?TileSpecs=2037%2C0%3B2038%2C0%3B2039%2C0%3B2040%2C0%3B2041%2C0%3B2037%2C1%3B2038%2C1%3B2039%2C1%3B2040%2C1%3B2041%2C1%3B2037%2C2%3B2038%2C2%3B2039%2C2%3B2040%2C2%3B2041%2C2%3B2037%2C3

which is basically

Code:

/GuideDataService.ashx/GetEpisodeTiles?TileSpecs=2037,0;2038,0;2039,0;2040,0;2041,0;2037,1;2038,1;2039,1;2040,1;2041,1;2037,2;2038,2;2039,2;2040,2;2041,2;2037,3

.

results look like

Code:

[
    [
    {"Id":"EP7886020023","S":244440,"E":244470,"T":"Rick Mercer Report","SE":false,"HD":false,"CId":"10091","M":false,"IN":false},
    {"Id":"SH5320160000","S":244440,"E":244500,"T":"Omni News: Italian Edition","SE":false,"HD":false,"CId":"10105","M":false,"IN":false},
    {"Id":"SH8611820000","S":244440,"E":244500,"T":"Agenda with Steve Paikin","SE":false,"HD":false,"CId":"10119","M":false,"IN":false},
    {"Id":"EP8537860005","S":244440,"E":244500,"T":"Very Bad Men","SE":false,"HD":false,"CId":"10130","M":false,"IN":false},
    {"Id":"EP8819060002","S":244470,"E":244500,"T":"Little Mosque on the Prairie","SE":false,"HD":false,"CId":"10091","M":false,"IN":false},
    {"Id":"SH5371520000","S":244500,"E":244560,"T":"CBC Winnipeg Comedy Festival","SE":false,"HD":false,"CId":"10091","M":false,"IN":false},
    {"Id":"SH7618990000","S":244500,"E":244530,"T":"Vivere II","SE":false,"HD":false,"CId":"10105","M":false,"IN":false},
    {"Id":"EP6883590055","S":244500,"E":244560,"T":"House","SE":false,"HD":false,"CId":"10130","M":false,"IN":false},
    {"Id":"SH2992650000","S":244500,"E":244680,"T":"TV Guide","SE":false,"HD":false,"CId":"27779","M":false,"IN":false},
    {"Id":"EP0800320068","S":244500,"E":244560,"T":"A Touch of Frost","SE":false,"HD":false,"CId":"10119","M":false,"IN":false},
    {"Id":"SH6048400000","S":244530,"E":244560,"T":"Canada Contacto","SE":false,"HD":false,"CId":"10105","M":false,"IN":false}],
    [
    {"Id":"SH2992650000","S":244500,"E":244680,"T":"TV Guide","SE":false,"HD":false,"CId":"27779","M":false,"IN":false},
    {"Id":"EP9259280005","S":244560,"E":244620,"T":"The Best Years","SE":false,"HD":false,"CId":"10130","M":false,"IN":true},
    {"Id":"EP0186930014","S":244560,"E":244590,"T":"The Simpsons","SE":false,"HD":false,"CId":"10105","M":false,"IN":false},
    {"Id":"SH2836270000","S":244560,"E":244620,"T":"CBC News: The National","SE":false,"HD":false,"CId":"10091","M":false,"IN":false},
    {"Id":"EP8750020003","S":244560,"E":244620,"T":"Amazing Planet","SE":false,"HD":false,"CId":"10119","M":false,"IN":false},
    {"Id":"EP2070020114","S":244590,"E":244620,"T":"King of the Hill","SE":false,"HD":false,"CId":"10105","M":false,"IN":false},
    {"Id":"EP6873470097","S":244620,"E":244655,"T":"Metropia","SE":false,"HD":false,"CId":"10105","M":false,"IN":false},
    {"Id":"SH8611820000","S":244620,"E":244680,"T":"Agenda with Steve Paikin","SE":false,"HD":false,"CId":"10119","M":false,"IN":false},
    {"Id":"SH8622260000","S":244620,"E":244680,"T":"The Hour","SE":false,"HD":false,"CId":"10091","M":false,"IN":false},
    {"Id":"SH8072670000","S":244620,"E":244650,"T":"News Final","SE":false,"HD":false,"CId":"10130","M":false,"IN":false},
    {"Id":"SH7763470000","S":244650,"E":244680,"T":"Entertainment Tonight Canada","SE":false,"HD":false,"CId":"10130","M":false,"IN":false},
    {"Id":"EP0768383438","S":244655,"E":244717,"T":"Late Show With David Letterman","SE":false,"HD":false,"CId":"10105","M":false,"IN":false}],
    [
...
...

(format - newlines, not in original response, but who cares).

[nitrogen_widget]

When you try it in a machine with a card, log the network traffic with Ethereal/Wireshark. It'd be great if this network interaction isnt encrypted.

It gets it's EPG data from
Zap2it.
but i'll be darned as to where it's storing all it's EPG data.:confused:

[wtg]

sub,

Have you ever contacted TitanTV regarding their developer program? They used to have a link with contact info about it on their site - maybe still do but I couldn't find it exactly. I did find a link about becoming a business partner anyway. Maybe it costs a small fortune like Zap2It does, but then again, you won't know until you ask.

Tim

[whurlston]

OK. Simple test time. US users, go to http://tv.yahoo.com and set up your zip code, service type and service provider. Click on the "Jump to: Now" or "Primetime Tonight" links at the top of the grid. In the address bar, the url will look something like this:

tv.yahoo.com/listings?starttime=1182384000&showFavorites=&headend=us_KY16673d

Note the bold, underlined portion. You will need the underlined portion for your provider/zip.

Download and extract the attached zip file.
Run the YahooTVGrabber.exe.
Enter your headend ID in the input box (It is prepopulated with mine for anyone who wants to test it.)
Click the "Get Data" button. The top window will show the Yahoo TV listing grid starting at the time you opened the program.
Once the page downloads, there is a slight delay then the bottom boxes will be populated with your channel list and the show schedules.

The delay is because the page does not actually contain the channel list, just the show listings. It pulls the channel list inline after the page loads so the delay is there to give it time to pull that data. If your channel list does not populate, it probably means that a longer timeout will be needed.

This will NOT save any data. It is just a test to make sure the parsing is correct.

The duration times may look strange. This is because it is showing the duration LEFT in that particular show.

Let me know how it works.

[whurlston]

madcat: I am looking into couchville. They are using data from SnapStream, and I am somewhat familiar with their guide because of the work I am doing for the new QAM Mapper for the HDHomeRun.

[whurlston]

Mind if I ask how you are parsing the data? I parse data for the sportsscores plugin and am curious.

I pretty much just pull webpage source then pass it through a couple of functions to parse the data I need.