NextPVR Forums

Full Version: Tempest EPG Generator
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Pages: 1 2 3 4 5 6 7 8
Yes I can create the a_0.siteconfig.php file

Code:
<?php
/*     Tempest EPG Generator (made by Kvanc)
https://github.com/K-vanc/Tempest-EPG-Generator.git  */
return array (
  'filename' => 'a',
  'creation_date' => '2022-02-12',
  'rev_no' => 'R0',
  'requestOption1' => '1',
  'ccrequestOption1' => '1',
);
?>

NextPVR has built in Schedules Direct support and there is no real need to compete with the several alternatives that already do this for those few users that need independent xmltv files. Perhaps zap2it.com is a better choice.

Try the new changes now. No change still not updating tempest_config.new Having the name is nice though.

For tvpassport no channel block message using the Canada code for my city.

Martin
(2022-02-12, 02:59 PM)mvallevand Wrote: [ -> ]Yes I can create the a_0.siteconfig.php file

Code:
<?php
/*     Tempest EPG Generator (made by Kvanc)
https://github.com/K-vanc/Tempest-EPG-Generator.git  */
return array (
  'filename' => 'a',
  'creation_date' => '2022-02-12',
  'rev_no' => 'R0',
  'requestOption1' => '1',
  'ccrequestOption1' => '1',
);
?>

NextPVR has built in Schedules Direct support and there is no real need to compete with the several alternatives that already do this for those few users that need independent xmltv files.  Perhaps zap2it.com is a better choice.

Try the new changes now.  No change still not updating tempest_config.new  Having the name is nice though.

For tvpassport no channel block message using the Canada code for my city.

Martin

Can you please write your code name? I will test. It has some 6000+ location code so impossible to check all one by one but should work
We are in the news these days because of a few ignorant people taking over our downtown.

For flixed ottawa/on
For tv passport on/12088/ottawa

Here is the channel I am trying to load in flixed

<channel site="[ENC][EX]flixed.io[CA]_0" site_id="58494" xmltv_id="[Local Over the Air Broadcast] CBOTDT - CBC">[Loc

Martin
(2022-02-12, 03:56 PM)mvallevand Wrote: [ -> ]We are in the news these days because of a few ignorant people taking over our downtown.

For flixed ottawa/on
For tv passport  on/12088/ottawa

Here is the channel I am trying to load in flixed

<channel site="[ENC][EX]flixed.io[CA]_0" site_id="58494" xmltv_id="[Local Over the Air Broadcast] CBOTDT - CBC">[Loc   

Martin
I listened that news while driving back home from work on radio. It is strange reaction but some groups are available in everywhere  Smile


I am almost pretty sure there is something in your server which is not like my code  Smile
And last photo. Are sure  you updated the new files?
Good news, for flixed.io it turns out it wasn't the server, it was the browser. Changing from Firefox to Chrome configuration worked as expected.

www.tvpassport.com was not working because my provider DNS is blocking www.tvpassport.com but not tvpassport.com. Creating a hosts file fixed that. The tvpassport data is nice because it includes the channel number however it doesn't download the channels and hangs I think because of a TLS v1.2 error

The lists are still too large to work with and it will take a lot of work for a users to create the proper names to make the display-name match the tuner name to map in NextPVR, but once it is set up the scrapping is fast.

I also tested the cli interface and that worked very well.

Code:
[D:\php]php tempest.php engine=Generate tempconf=tempest_config\tempest.config.xml
[                   ]
[                   ]-------------------------------------
[                   ]        Tempest EPG Generator
[                   ]
[                   ]   The Most Advanced Programmable
[                   ]       GUI XMLTV EPG Generator
[                   ]     by Kivanc Altug aka "Kvanc"
[                   ]-------------------------------------
[                   ]
[12-02-2022 22:14:09] Tempest Version : rv1.3.1
[12-02-2022 22:14:09] System Info : Windows NT ACERWIN10 10.0 build 22000 (Windows 11) AMD64
[12-02-2022 22:14:09] PHP Version : 8.1.2
[12-02-2022 22:14:09] Server Info : Command Line Interface (Cli)
[12-02-2022 22:14:09] Initializing...
[12-02-2022 22:14:09] Checking required modules...
[12-02-2022 22:14:09] xmltv.xml generated...
[12-02-2022 22:14:09] Grabbing started for 8 days
[                   ]
[12-02-2022 22:14:09] (1/17) Site: [ENC][EX]flixed.io[CA]_0
[12-02-2022 22:14:09] Channel id: 10130
[12-02-2022 22:14:09] Xmltv id: [Local Over the Air Broadcast] CIII - GLOBAL
[12-02-2022 22:14:11] 258 shows grabbed in 1.94 seconds
....
[12-02-2022 22:14:32] Grabbing completed...
[12-02-2022 22:14:32] 4090 shows grabbed in 22.90 seconds
[     Completed     ]
[12-02-2022 22:14:32] Tempest Overall Memory Usage: 4229KB
[12-02-2022 22:14:32] Tempest Peak Memory Usage: 10383KB

Martin
(2022-02-12, 10:43 PM)mvallevand Wrote: [ -> ]Good news, for flixed.io it turns out it wasn't the server, it was the browser. Changing from Firefox to Chrome configuration worked as expected. 

www.tvpassport.com was not working because my provider DNS is blocking www.tvpassport.com but not tvpassport.com.  Creating a hosts file fixed that.  The tvpassport data is nice because it includes the channel number however it doesn't download the channels and hangs I think because of a TLS v1.2 error

The lists are still too large to work with and it will take a lot of work for a users to create the proper names to make the display-name match the tuner name to map in NextPVR, but once it is set up the scrapping is fast.

I also tested the cli interface and that worked very well.

Code:
[D:\php]php tempest.php engine=Generate tempconf=tempest_config\tempest.config.xml
[                   ]
[                   ]-------------------------------------
[                   ]        Tempest EPG Generator
[                   ]
[                   ]   The Most Advanced Programmable
[                   ]       GUI XMLTV EPG Generator
[                   ]     by Kivanc Altug aka "Kvanc"
[                   ]-------------------------------------
[                   ]
[12-02-2022 22:14:09] Tempest Version : rv1.3.1
[12-02-2022 22:14:09] System Info : Windows NT ACERWIN10 10.0 build 22000 (Windows 11) AMD64
[12-02-2022 22:14:09] PHP Version : 8.1.2
[12-02-2022 22:14:09] Server Info : Command Line Interface (Cli)
[12-02-2022 22:14:09] Initializing...
[12-02-2022 22:14:09] Checking required modules...
[12-02-2022 22:14:09] xmltv.xml generated...
[12-02-2022 22:14:09] Grabbing started for 8 days
[                   ]
[12-02-2022 22:14:09] (1/17) Site: [ENC][EX]flixed.io[CA]_0
[12-02-2022 22:14:09] Channel id: 10130
[12-02-2022 22:14:09] Xmltv id: [Local Over the Air Broadcast] CIII - GLOBAL
[12-02-2022 22:14:11] 258 shows grabbed in 1.94 seconds
....
[12-02-2022 22:14:32] Grabbing completed...
[12-02-2022 22:14:32] 4090 shows grabbed in 22.90 seconds
[     Completed     ]
[12-02-2022 22:14:32] Tempest Overall Memory Usage: 4229KB
[12-02-2022 22:14:32] Tempest Peak Memory Usage: 10383KB

Martin
Thanks Martin. Good to hear finally it worked for you. But changing browser also fixed your config save issue ? Normally I tested Tempest on Chrome/Edge/Opera for Windows; Chromium, Midori, Vivaldi for Linux and Chrome,Samsung,Opera,Puffin for android and never faced and issue. If it was the problem, I may focus on it to fix. For grabbing, some websites are checking your user agent string as a precaution for robots. User agent string may be set in siteconfigs(which is not possible for enc ones) or from tempest configuration file. If both values are empty, Tempest will look and use your servers user agent. Also if it is not set for server then it will use hard coded string inside of its code which a line for Windows Chrome. So basicly setting user agent in configuration should do the same .

Tvpassport is using detail pages for shows unlike flixed which all data is in api. This means Tempest will make a request first to get shows then for each show, it will make another seperate request to get details such as actor, director etc. As expected it is a little slower than flixed to finish 1 channel. Tempest using curl extension for requests and if there is any tls error, it will be both sent to your screen and to your log if enabled. As long as your server updated, it shouldn't be a tls issue. You may try to increase timeout setting in your config. If it is too low and operation needs more time, Tempest will stop and proceed to next channel. As retry "2" and timeout "15" are good values for starting.

For the lists, unfortunately Tempest is taking what it is there. I can add some replace commands for manual changes but source format should be in a specific logic which usually N.American sites dont have.

Just a note about cli command. "tempconf" shall be only name of config file without path. If Tempest cannot locate given file in its fixed path, it will run default config which is "tempest.config.xml". In your case, it cannot find the file you asked and running default file which is what you want  Smile  So your line shall be "tempconf=tempest.config.xml" or you can directly omit this "tempconf=tempest.config.xml" parameter to run default config.
Yes everything worked changing browsers

If tv passport is several minutes per channel it won't be worth using.

Martin
(2022-02-13, 04:08 AM)mvallevand Wrote: [ -> ]Yes everything worked changing browsers

If tv passport is several minutes per channel it won't be worth using.

Martin
Code:
[                   ]
[                   ]-------------------------------------
[                   ]        Tempest EPG Generator
[                   ]
[                   ]   The Most Advanced Programmable
[                   ]       GUI XMLTV EPG Generator
[                   ]     by Kivanc Altug aka "Kvanc"
[                   ]-------------------------------------
[                   ]
[13-02-2022 07:24:28] Tempest Version : v1.3.0
[13-02-2022 07:24:28] System Info : Windows NT WDS4JL3144 10.0 build 18363 (Windows 10) AMD64
[13-02-2022 07:24:28] PHP Version : 7.3.8
[13-02-2022 07:24:28] Server Info : PHP 7.3.8 Development Server
[13-02-2022 07:24:28] Initializing...
[13-02-2022 07:24:28] Checking required modules...
[13-02-2022 07:24:28] Tvpass[CA].xml generated...
[13-02-2022 07:24:28] Grabbing started for 2 days
[                   ]
[13-02-2022 07:24:28] (1/15) Site: tvpassport.com[CA]_0
[13-02-2022 07:24:28] Channel id: 91
[13-02-2022 07:24:28] Xmltv id: [Broadcast - Toronto, ON] NBC (WGRZ) Buffalo, NY (2-1)
[13-02-2022 07:24:59] 31 shows grabbed in 30.80 seconds
[                   ]
[13-02-2022 07:24:59] (2/15) Site: tvpassport.com[CA]_0
[13-02-2022 07:24:59] Channel id: 12409
[13-02-2022 07:24:59] Xmltv id: [Broadcast - Toronto, ON] Antenna (WGRZ2) Buffalo, NY (2-2)
[13-02-2022 07:26:42] 96 shows grabbed in 102.82 seconds
[                   ]
[13-02-2022 07:26:42] (3/15) Site: tvpassport.com[CA]_0
[13-02-2022 07:26:42] Channel id: 10051
[13-02-2022 07:26:42] Xmltv id: [Broadcast - Toronto, ON] True Crime Network (WGRZ3) Buffalo, NY (2-3)
[13-02-2022 07:27:29] 49 shows grabbed in 47.18 seconds
[                   ]
[13-02-2022 07:27:29] (4/15) Site: tvpassport.com[CA]_0
[13-02-2022 07:27:29] Channel id: 32423
[13-02-2022 07:27:29] Xmltv id: [Broadcast - Toronto, ON] Quest (WGRZ4) Buffalo, NY (2-4)
[13-02-2022 07:28:25] 63 shows grabbed in 56.52 seconds
[                   ]
[13-02-2022 07:28:25] (5/15) Site: tvpassport.com[CA]_0
[13-02-2022 07:28:25] Channel id: 36093
[13-02-2022 07:28:25] Xmltv id: [Broadcast - Toronto, ON] Twist (WGRZ5) Buffalo, NY (2-5)
[13-02-2022 07:29:35] 79 shows grabbed in 70.02 seconds
[                   ]
[13-02-2022 07:29:35] (6/15) Site: tvpassport.com[CA]_0
[13-02-2022 07:29:35] Channel id: 36
[13-02-2022 07:29:35] Xmltv id: [Broadcast - Toronto, ON] CTV2 Toronto, ON (3-1)
[13-02-2022 07:30:43] 68 shows grabbed in 67.66 seconds
[                   ]
[13-02-2022 07:30:43] (7/15) Site: tvpassport.com[CA]_0
[13-02-2022 07:30:43] Channel id: 88
[13-02-2022 07:30:43] Xmltv id: [Broadcast - Toronto, ON] CBS (WIVB) Buffalo, NY (4-1)
[13-02-2022 07:31:41] 52 shows grabbed in 58.08 seconds
[                   ]
[13-02-2022 07:31:41] (8/15) Site: tvpassport.com[CA]_0
[13-02-2022 07:31:41] Channel id: 34
[13-02-2022 07:31:41] Xmltv id: [Broadcast - Toronto, ON] CBC (CBLT) Toronto, ON (5-1)
[13-02-2022 07:32:04] 25 shows grabbed in 22.87 seconds
[                   ]
[13-02-2022 07:32:04] (9/15) Site: tvpassport.com[CA]_0
[13-02-2022 07:32:04] Channel id: 6
[13-02-2022 07:32:04] Xmltv id: [Broadcast - Toronto, ON] Global (CIII) Ontario (6-1)
[13-02-2022 07:33:01] 63 shows grabbed in 56.87 seconds
[                   ]
[13-02-2022 07:33:01] (10/15) Site: tvpassport.com[CA]_0
[13-02-2022 07:33:01] Channel id: 89
[13-02-2022 07:33:01] Xmltv id: [Broadcast - Toronto, ON] ABC (WKBW) Buffalo, NY (7-1)
[13-02-2022 07:34:01] 61 shows grabbed in 60.39 seconds
[                   ]
[13-02-2022 07:34:01] (11/15) Site: tvpassport.com[CA]_0
[13-02-2022 07:34:01] Channel id: 15123
[13-02-2022 07:34:01] Xmltv id: [Broadcast - Toronto, ON] Bounce (WKBW-TV2) Buffalo, NY (7-2)
[13-02-2022 07:34:50] 45 shows grabbed in 48.23 seconds
[                   ]
[13-02-2022 07:34:50] (12/15) Site: tvpassport.com[CA]_0
[13-02-2022 07:34:50] Channel id: 15172
[13-02-2022 07:34:50] Xmltv id: [Broadcast - Toronto, ON] Grit TV (WKBW-TV3) Buffalo, NY (7-3)
[13-02-2022 07:35:33] 40 shows grabbed in 43.32 seconds
[                   ]
[13-02-2022 07:35:33] (13/15) Site: tvpassport.com[CA]_0
[13-02-2022 07:35:33] Channel id: 31910
[13-02-2022 07:35:33] Xmltv id: [Broadcast - Toronto, ON] Court TV Mystery (WKBW-TV4) Buffalo, NY (7-4)
[13-02-2022 07:36:31] 54 shows grabbed in 58.41 seconds
[                   ]
[13-02-2022 07:36:31] (14/15) Site: tvpassport.com[CA]_0
[13-02-2022 07:36:31] Channel id: 36559
[13-02-2022 07:36:31] Xmltv id: [Broadcast - Toronto, ON] HSN (WKBW-TV5) Buffalo, NY (7-5)
[13-02-2022 07:37:13] 48 shows grabbed in 41.49 seconds
[                   ]
[13-02-2022 07:37:13] (15/15) Site: tvpassport.com[CA]_0
[13-02-2022 07:37:13] Channel id: 10996
[13-02-2022 07:37:13] Xmltv id: [Broadcast - Toronto, ON] MeTV (WHEC-TV2) Rochester, NY (10-2)
[13-02-2022 07:38:34] 75 shows grabbed in 81.09 seconds
[                   ]
[13-02-2022 07:38:34] Grabbing completed...
[13-02-2022 07:38:34] 849 shows grabbed in 14 minutes 5.78 seconds (845.78 seconds)
[     Completed     ]
[13-02-2022 07:38:34] Tempest Overall Memory Usage: 2074KB
[13-02-2022 07:38:34] Tempest Peak Memory Usage: 5353KB

Very strange since firefox is the father of some of these alternative browsers I tested. I will try to install and reproduce problem whenever I have some free time.

For tvpassport, I choosed first 15 channels to test on 2 days grabbing. Overall result is 0.996sec/per show which seems quite good to me if you consider I am currently in Middle East region for work and connected from a restricted network so values should be %20-%35 better for you. Total time for a channel is related with how many shows it has. For example if you check 2nd channel in log, you will see 96 shows grabbed in 102.82 secs. For these 96 shows, total of 97 requests done, scrapped, modified and saved  in xml file which is very good and I claim that it cannot be done faster than Tempest.

If you need just generic info and faster grabbing, there is an option in config file as index-only mode. If you enable it, Tempest will ignore detail pages and just use available index data. For tvpassport, you can get start, stop, title, subtitle, category, channel logo and new information with index-only mode.

Edit:
Oops, I tried to add xml and log.txt but seems like not allowed so;
(2022-02-13, 05:07 AM)kvanc Wrote: [ -> ]Oops, I tried to add xml and log.txt but seems like not allowed so;
Any attachments need to be zipped.
Pages: 1 2 3 4 5 6 7 8