Free XMLTV EPG scraper

0 Vote(s) - 0 Average

Free XMLTV EPG scraper

JerryFan
Offline

Junior Member

Posts: 5
Threads: 0
Joined: Sep 2009

#201

2009-09-05, 05:42 AM

Yes, you are right LewE. After running the "ChangeEPGSources.bat" from the wiki, I was able to fix my problem. The listings are accurate and detailed. But the MSN scraper was nice too. And easy to use. I'm still deciding whether or not to subscribe longer to SD. I beginning to think I might not have a choice.

wscrivens
Offline

Member

Posts: 85
Threads: 21
Joined: Jul 2005

#202

2009-09-05, 11:42 AM

Back to the topic of the MSN scraper...

I did some further investigation yesterday. As I said, I was able to get a good download by running the MSN scraper from the command line. When I tried later in the day, I had to do it four or five times before I got a good connection.

Then I looked in more detail at the errors I was getting in the e-mail. By clicking on the URL the prigram was trying to reach I got the following from firefox:

Code:
The page isn't redirecting properly

Firefox has detected that the server is redirecting the request for this address in a way that will never complete.

    *   This problem can sometimes be caused by disabling or refusing to accept

          cookies.

I have cookies set at the default setting, which IIRC refuses third party cookies but accepts session cookies. I haven't changed that setting in years.

The URL used contains identifying information to get the correct lineup. Is this something that can be adjusted/changed in some way? I'm thinking that msn may have made some internal changes that we need to reflect in the URL. For comparison with JerryFan's messages, here is the URL that mine is using:

Code:
http://tv.sympatico.msn.ca/DiscoveryWS/ProfileManagement.ashx?action=create&id=&zipCode=33445&headend=FL09541-&mkt=en-US&dts=1252130407000

BTW, I tried looking at the Planet Replay forum, and there is a similar post over there:
http://planetreplay.com/phpBB2/viewtopic...19e23e39bf

Any help or ideas would be welcome!

Walt

Hauppague 150, and 1600, Redeon HD5670 Video, 2.91 GHz Athalon CPU, ASUS M2N68-AM MoBo, Winows 7 Professional, Samsung LNS2641DX/XAA display

Jeckler
Offline

Junior Member

Posts: 42
Threads: 9
Joined: Dec 2006

#203

2009-09-10, 07:27 PM

From the Planet Replay forum....
I haven't had a chance to try it yet as I'm still at work.

meedio_user Wrote:I've been using MSN scraper for the last few years and have recently hit the same problem that others are having -- but I've found a solution that works for me.

As PG said above, the first time the scraper runs, it gets listings correctly but subsequent attempts fail. This is because the first time it runs, it downloads listings from tvintl.msn.com. Afterwards, it always uses tv.sympatico.msn.ca which no longer exists. The solution is to point tv.sympatico.msn.ca to tvintl.msn.com on the computer downloading the listings. tvintl.msn.com points to the IP address, 65.54.195.254, so you'll need to update your hosts file to point to the new location:

1. Open C:\WINDOWS\system32\drivers\etc\hosts with Notepad.
2. Add the following line at the bottom:

65.54.195.254 tv.sympatico.msn.ca

I've been using this for the last couple of days and so far it works

mikeneron
Offline

Member

Posts: 77
Threads: 9
Joined: May 2006

#204

2009-09-10, 08:06 PM

Once I made that change, I am now getting no satellite providers.

All Steamed Out
Offline

Junior Member

Posts: 16
Threads: 6
Joined: Jun 2007

#205

2009-09-11, 02:58 AM

Originally Posted by meedio_user
I've been using MSN scraper for the last few years and have recently hit the same problem that others are having -- but I've found a solution that works for me.

As PG said above, the first time the scraper runs, it gets listings correctly but subsequent attempts fail. This is because the first time it runs, it downloads listings from tvintl.msn.com. Afterwards, it always uses tv.sympatico.msn.ca which no longer exists. The solution is to point tv.sympatico.msn.ca to tvintl.msn.com on the computer downloading the listings. tvintl.msn.com points to the IP address, 65.54.195.254, so you'll need to update your hosts file to point to the new location:

1. Open C:\WINDOWS\system32\drivers\etc\hosts with Notepad.
2. Add the following line at the bottom:

65.54.195.254 tv.sympatico.msn.ca

I've been using this for the last couple of days and so far it works

I implemented the fix above and it seems to be working fine. Based on earlier posts I unchecked the "Download as Canadian" box but still used my local PC. Seems to be working fine, but I only have the Shaw analog channels so the comment of the poster above about missing the satellite channels doesn't apply to me.

Jeckler
Offline

Junior Member

Posts: 42
Threads: 9
Joined: Dec 2006

#206

2009-09-11, 05:47 PM

I don't have satellite, so I don't have the issue, but adding the IP to the hosts file worked to get my Cox listings back. Time will tell if it stays fixed or the author updates the program.
I may try mc2xml if it stops working again.

mikeneron
Offline

Member

Posts: 77
Threads: 9
Joined: May 2006

#207

2009-09-12, 02:25 AM

What I find weird is if I actually go to the website and go to the TV listings and enter my postal code, the satellite options are there. No clue why it won't come up when I try running the MSN scraper program.

[Image: TVGuide-1.jpg]

Jeckler
Offline

Junior Member

Posts: 42
Threads: 9
Joined: Dec 2006

#208

2009-09-12, 07:03 PM

Post the full address that the scraper is trying to get to, from XMLOut.log

mikeneron
Offline

Member

Posts: 77
Threads: 9
Joined: May 2006

#209

2009-09-12, 07:39 PM

Right now it doesn't show an address because it can't find any satellite providers.

Quote:9/11/2009 9:38:07 PM: MSN_XMLTV_Downloader V0.65
9/11/2009 9:38:08 PM: XML version 6 detected.
9/11/2009 9:38:08 PM: Loading.....Settings.xml loaded.
9/11/2009 9:38:12 PM: Getting Antenna TV channels for XxXYyY Market: en-CA.
9/11/2009 9:38:13 PM: Getting Cable TV providers for XxXYyY.
9/11/2009 9:38:13 PM: Getting Satellite providers for XxXYyY.
9/11/2009 9:38:13 PM: No results returned for satellite provider list, skipping.
9/11/2009 9:41:00 PM: Exited normally. Runtime: 2 m, 53 s. 1.95 kB downloaded total.

This is what I was getting before I added the hosts entry (removed my postal code for this post):

Code:
9/11/2009 9:27:53 PM: MSN_XMLTV_Downloader V0.65

9/11/2009 9:27:53 PM: XML version 6 detected.

9/11/2009 9:27:53 PM: Loading.....Settings.xml loaded.

9/11/2009 9:27:56 PM: Settings saved in Settings.xml.

9/11/2009 9:27:56 PM: No channels selected for Bell TV-Western Canada, skipping.

9/11/2009 9:27:56 PM: Sending report email...Report email sent successfully.

9/11/2009 9:28:03 PM:  Getting Antenna TV channels for XxXYyY Market: en-CA.

9/11/2009 9:28:05 PM: Http response not OK. Was: 404 to request [url]http://tv.sympatico.msn.ca/DiscoveryWS/GetHeadends2.ashx?zipCode=XxXYyY&providerType=TER&mkt=en-CA&dts=1252722483000[/url]

9/11/2009 9:28:05 PM: Download failure, retry 1...

9/11/2009 9:28:06 PM: Http response not OK. Was: 404 to request [url]http://tv.sympatico.msn.ca/DiscoveryWS/GetHeadends2.ashx?zipCode=XxXYyY&providerType=TER&mkt=en-CA&dts=1252722483000[/url]

9/11/2009 9:28:06 PM: Download failure, retry 2...

9/11/2009 9:28:06 PM: Received data not gzip compressed. .

9/11/2009 9:28:06 PM: GZip header incorrect. Was:  should have been 1F8B.

9/11/2009 9:28:06 PM: Download failure, retry 3...

9/11/2009 9:28:07 PM: Received data not gzip compressed. .

9/11/2009 9:28:07 PM: GZip header incorrect. Was:  should have been 1F8B.

9/11/2009 9:28:07 PM: Download failure, retry 4...

9/11/2009 9:28:07 PM: Download retries failed from [url]http://tv.sympatico.msn.ca/DiscoveryWS/GetHeadends2.ashx?zipCode=XxXYyY&providerType=TER&mkt=en-CA&dts=1252722483000[/url]

9/11/2009 9:28:07 PM: Error downloading antenna channel list, skipping.

9/11/2009 9:28:07 PM:  Getting Cable TV providers for XxXYyY.

9/11/2009 9:28:07 PM: Received data not gzip compressed. .

9/11/2009 9:28:08 PM: GZip header incorrect. Was:  should have been 1F8B.

9/11/2009 9:28:08 PM: Download failure, retry 1...

9/11/2009 9:28:08 PM: Received data not gzip compressed. .

9/11/2009 9:28:08 PM: GZip header incorrect. Was:  should have been 1F8B.

9/11/2009 9:28:08 PM: Download failure, retry 2...

9/11/2009 9:28:09 PM: Received data not gzip compressed. .

9/11/2009 9:28:09 PM: GZip header incorrect. Was:  should have been 1F8B.

9/11/2009 9:28:09 PM: Download failure, retry 3...

9/11/2009 9:28:09 PM: Received data not gzip compressed. .

9/11/2009 9:28:09 PM: GZip header incorrect. Was:  should have been 1F8B.

9/11/2009 9:28:09 PM: Download failure, retry 4...

9/11/2009 9:28:09 PM: Download retries failed from [url]http://tv.sympatico.msn.ca/DiscoveryWS/GetHeadends2.ashx?zipCode=XxXYyY&providerType=CAB&mkt=en-CA&dts=1252722487000[/url]

9/11/2009 9:28:09 PM: Error downloading cable provider list, skipping.

9/11/2009 9:28:09 PM:  Getting Satellite providers for XxXYyY.

9/11/2009 9:28:10 PM: Received data not gzip compressed. .

9/11/2009 9:28:10 PM: GZip header incorrect. Was:  should have been 1F8B.

9/11/2009 9:28:10 PM: Download failure, retry 1...

9/11/2009 9:28:11 PM: Received data not gzip compressed. .

9/11/2009 9:28:11 PM: GZip header incorrect. Was:  should have been 1F8B.

9/11/2009 9:28:11 PM: Download failure, retry 2...

9/11/2009 9:28:11 PM: Received data not gzip compressed. .

9/11/2009 9:28:11 PM: GZip header incorrect. Was:  should have been 1F8B.

9/11/2009 9:28:11 PM: Download failure, retry 3...

9/11/2009 9:28:12 PM: Received data not gzip compressed. .

9/11/2009 9:28:12 PM: GZip header incorrect. Was:  should have been 1F8B.

9/11/2009 9:28:12 PM: Download failure, retry 4...

9/11/2009 9:28:12 PM: Download retries failed from [url]http://tv.sympatico.msn.ca/DiscoveryWS/GetHeadends2.ashx?zipCode=XxXYyY&providerType=DBS&mkt=en-CA&dts=1252722489000[/url]

9/11/2009 9:28:12 PM: Error downloading satellite provider list, skipping.

9/11/2009 9:28:12 PM: No providers found in XxXYyY.

9/11/2009 9:28:12 PM: Check your zipcode, firewall or internet explorer connection settings.

9/11/2009 9:28:12 PM: Exited normally. Runtime: 0 m, 19 s. 0.00 kB downloaded total.

All Steamed Out
Offline

Junior Member

Posts: 16
Threads: 6
Joined: Jun 2007

#210

2009-10-13, 04:24 AM

Hi to all,

The scraper is still downloading but only up till 5pm(PDT) on Thursday, Oct 15th. I noticed this last Thursday when I tried to download and update the EPG.

Event though the scraper is set to download listings for 14 days and details for 10 days (maximum) it only gets them to the date above.

It is almost like the provider has not released the program list.

Anybody have any ideas?

Thanks in advance

« Next Oldest | Next Newest »

Users browsing this thread: 2 Guest(s)

Pages (22): « Previous 1 … 18 19 20 21 22 Next »

Possibly Related Threads…
Thread		Author	Replies	Views	Last Post
	How to extract M3U8 and get matching XMLTV guide data from NPVR	almightyj	0	2,941	2018-10-23, 07:24 AM Last Post: almightyj
	XMLTV Channel Number Option	gdogg371	12	6,056	2018-07-18, 04:32 PM Last Post: sub
	Merge xmltv data with xslt	Graham	4	3,233	2014-01-02, 12:45 PM Last Post: Graham
	XMLTV data	mvallevand	6	2,382	2011-04-16, 03:08 AM Last Post: sub
	zap2xml - a free zap2it scraper	jdg	193	111,158	2011-04-13, 03:01 PM Last Post: fla
	XmlTv Structure	psycik	2	2,037	2008-12-29, 10:53 PM Last Post: psycik
	Content Provider: How to publish XMLTV data?	skoka123	6	3,140	2008-09-03, 05:48 AM Last Post: skoka123
	XMLTV Questions	-Oz-	34	9,542	2008-03-24, 01:19 AM Last Post: zehd
	New XMLTV Enhancer and more...	zehd	388	79,347	2008-03-05, 07:45 PM Last Post: zehd
	I-xmltv updated for YahooXMLTv (adds a new show marker and more)	Jim_	78	22,986	2008-01-02, 10:33 AM Last Post: tvshowman

Forum Jump: