Cutting The Vader EPG Chord

[snaitaz]

Good Day and Merry Christmas to all…

Vaders has had its ups and downs over the years but it has always rebounded and I personally have been happy with their services until now. The most recent issue that has plagued this service provider has been their URL’s for both the m3u and EPG data and has been driving me bonkers. In the past one could use for the m3u http://api.vaders.tv/vget?username=xxxx&...s/m3u/m3u8 or just leave off the “&format=” and for the EPG you would use http://vaders.tv/p2.xml. Now due to some maintenance going on over at the Vaders shop these URL’s are no longer valid. Using a browser to validate the “currently working” URL for the m3u pull as noted above, I get the following response
[ATTACHMENT NOT FOUND]

However, using the following URL in a browser with either &format=m3u, m3u8, ts, or blank yields me a valid m3u file; http://vaders.tv/vget?username=xxx&passw...ormat=m3u8. The difference is that the “api” at the beginning must be omitted. I don’t know why but this is what works for me in NPVR now. As of this morning it looks like either http://vaders.tv/p2.xml.gz or http://vaders.tv/p2.xml are working now in the browser when only yesterday the http://vaders.tv/p2.xml.gz was the working model. In NPVR http://vaders.tv/p2.xml is also working but it doesn’t auto map the channels. This is most likely due to “tvg-name” from m3u file not finding a match with “channel id” in xml file. However, I did notice that the format of the xml file has gone through a massive restructure. Here is an example of what channels 17398 and 14863 looked like on 2018/12/17

<channel id="I197.17398.schedulesdirect.org">
<display-name>CBC</display-name><display-name>CBCT</display-name>
<display-name>197 CBCT</display-name>
<display-name>197</display-name>
<icon height="270"
</channel>
<channel id="I201.14863.schedulesdirect.org">
<display-name>CTV Halifax HD</display-name><display-name>CJCH</display-name>
<display-name>201 CJCH</display-name>
<display-name>201</display-name>
<display-name>CTV Halifax</display-name>
<icon height="270"
</channel>

versus what they look like today

<channel id="I197.17398.schedulesdirect.org">
<display-name>CBCT</display-name>
</channel>
<channel id="I201.14863.schedulesdirect.org">
<display-name>CTV Halifax</display-name>
</channel>

So, I have decided to cut the EPG chord from Vaders and construct my own so I don’t have to go through this experience again. As of now Vaders currently does not have a working EPG that plays nice with NPVR. I will be using NPVR (of course), mc2xml, Notepad++, Excel, Schedules Direct (SD) and MS notepad.

The first step was to pull the m3u and xml files and load them into NPVR so that I would know what channels I had access to. Next, without doing anything else I exported the channels from NPVR to its default xml file. Next, I opened the exported file with Excel (yes that’s right Excel). Excel has a very kool way of parsing the file into its logical components as shown below.

[ATTACHMENT NOT FOUND]



Column B indicates the tvg-name from the m3u file while column H indicates the channel id from the xml file. You can readily see in this depiction the mismatch between the two columns therefore, necessitating a manual/auto channel mapping. This is also the way I can see what channels I don’t want or need in order to eliminate them in the next step.

If you are able to pull in your EPG using NPVR and you can auto map all of your channels you can skip this step which uses mc2xml. For me I performed all the steps indicated thus far but could not auto map because of the mismatch. I used mc2xml to pull specific channels and edited the xml channel id to match the tvg-name in the m3u file. For those of you interested in a statistic, I pulled 850 channels and renamed the channel id in 13 seconds. Crazy huh!

Using mc2xml and SD, http://mc2xml.awardspace.info/ I executed mc2xml which gave me this

[ATTACHMENT NOT FOUND]

If you have an SD account (free trial also) you can enter your creds and continue. I used their JSON service with the following selections for my UK channels. Their “schedules direct” service does not give you the opportunity to enter a postal or country code selection. USA channels and other country channels were queried in the same manner. When it completes its query, you will get your xml file.

For the best control (IMO), I chose to use a batch file to call mc2xml.exe with parameters that told the query what channels I wanted specifically and how to rename them so that they would match my m3u file in NPVR. Using MS notepad or Notepad++ I entered this within my UpdateEPG.bat… “mc2xml -J user:password -c gb -g cf31 -D uk.dat -C uk.chl -R uk.ren -a -U -u -d 10 -o uk.xml”. I won’t take the space here to explain the switches because you can read all about this proggy at http://mc2xml.awardspace.info/. However, here is a sample of my channel request which is my uk.chl file…

[ATTACHMENT NOT FOUND]

The rename file uk.ren looks like this…

# rename the following

I0.17153.schedulesdirect.org:bbc one hd
I0.17154 schedulesdirect.org:bbc two hd
I0.83282 schedulesdirect.org:bbc four hd
I0.17468.schedulesdirect.org:itv 1
I0.20630.schedulesdirect.org:itv 2 hd
I0.44643.schedulesdirect.org:itv 3 hd
I0.48020.schedulesdirect.org:itv 4 hd
I0.17155.schedulesdirect.org:channel 4

I chose to use the channel id in lieu of display-name or some other field to match on because not every xml file has those fields for every channel. However, every channel has a specific channel id that is unique to it. So, “How do I know what the channel id is for every channel I want to pull you ask”? Remember that Excel file pic I showed you earlier? Well if you look at the “guide id” column they are listed there, not all but most. We have 85 channels in the UK which is why I chose to work on those first. I have started the USA pull but only 1/3 of the way done.

For those channels that do not have a number listed and instead have some alpha characters indicating the station call name, you will use this name to search within your xml file you pulled from Schedules Direct. The search will yield you a channel id that you can use. You may encounter more than one possible choice.

I hope this information helps you out. For me I now no longer am tied to Vaders EPG or any other EPG for that matter. My gratitude goes out to all of you that helped me on this especially sub, Graham and Martin. Cheers and Merry Christmas to all.

[Coulen]

Hey...had very similar issues with Vader and the EPG. With the latest updates the episode and season were no longer in the metadata and therefore NPVR just recorded all episodes regardless if i had them already.
I had so many duplicates.

I started following your process to build my own and came across this one.
https://www.iptv-epg.com/

This one here you load your M3u file and then it auto maps known channels, and then you can go in and manually map any unknown.
The reliablility has been unreal and very quick.

Its more expensive than SD but its fully automated and you never need to go in and map again.
Very easy to update/change etc. You can also remove channels from the m3u file should you not want them.
I mapped all the english channels on Vader, and was roughly 350 channels. I purchased the 1000 channel package but the 24/7 and MatchCenter do not have an automatic 3rd party epg.

You can do a trial for 5 days to see if you get what you want out of it.
Havent had a problem with epg since.



It spits out a new M3u and XMLTV link for you.

Saved so much time.

[johnsonx42]

how nice to see them stealing EPG data from schedules direct

[jrockow]

snaitaz,

with your custom EPG, do you get more than 3 days of schedule for VADERS?

[snaitaz]

Good Day to all...@Coulen ...I did give them (iptv-epg.com) a go after seeing your suggestion and was disappointed in the results. I sent them 100 USA, 50 UK and 50 Canadian channels in my m3u file. IPTV-epg.com was only able to map the USA (some) channels and very little of the UK/Canadian ones so I decided to stay with my process. Also, I also decide to drop Vaders once and for all and went with a different provider which pretty much had everything Vaders had plus more. This new service also allowed me to use my process in obtaining the channels I want and only those for easier channel mapping.

@johnsonx42...Idon't know who you were addressing this to but I would hope that iptv-epg.com would be paying a vendor's price to SD in order to do what they do. But I have no bloody idea nor is it in my wheelhouse to.

@jrockow...yes, when I was using vaders, I was able to get more than 14 days of EPG data but since SD was providing the data I stuck with the 14 days. The batch file which calls mc2xml has a parameter to tell it how many days you want to pull for. Read the documentation for mc2xml for a real good understanding of all the parameters you can use.

UPDATE>>>I have started to use epg123 and to be honest with you, this makes creating the EPG that much faster. It also uses the JSON service of SD but instead of writing out all of the channels you want as I did in my .chl file, you pick and choose the channels from a drop down list. Eazy peazy. unfortunately, you still have to do a somewhat manual channel mapping process as there is no opportunity to rename the channel-id tags to match. However, I am working on something to get this done after the fact.

[jrockow]

Also, I also decide to drop Vaders once and for all and went with a different provider which pretty much had everything Vaders had plus more. This new service also allowed me to use my process in obtaining the channels I want and only those for easier channel mapping.

Can I ask who you went with after dropping Vaders?

Also, I'm just getting started with building my own EPG.
One quick dumb question; why can't this be done in SD?
Why do you need to use another app in addition? (epg123)

[snaitaz]

[ATTACHMENT NOT FOUND]

[Coulen]

was only able to map the USA (some) channels and very little of the UK/Canadian ones so I decided to stay with my process.

@snaitaz - i think all but 10 UK channels with Vader mapped. 90% of Canadian and US mapped and I manually went in and mapped those that didnt. You just type in the channel name to map.
A couple Us channels (DIY is manually mapped to DoItYourself) You only have to do this once.

The Red indicates it couldnt map the channel to the guide.
[ATTACHMENT NOT FOUND]

Ive started to type in the Channel name, and bbc4.uk shows up. Now its mapped always in your links.
[ATTACHMENT NOT FOUND]

Once saved, it turns white like the other mapped channels and you dont have to do it again.
Something else pretty cool with these guys is i got an email the other day saying changes were made and channels were added to my m3u from Vader.
When i logged back into iptv-epg it showed me the new channels and what it mapped them to.

Also would like to know what you moved to as Vader buffering is getting to me.
This would work for all iptv sites im sure as well.