2016-02-11, 05:20 PM
My 2 cents: Is the drive badly fragmented?
"I'd rather have a bottle in front of me than a frontal lobotomy"
|
"I'd rather have a bottle in front of me than a frontal lobotomy"
2016-02-11, 06:12 PM
Graham Wrote:But, maybe ... just maybe ... you could try this ... https://sqlite.org/lang_vacuum.html ... immediately before an update. I tried that, as sub explained it is the inserts not the deletes causing the problem. The likely sqlite change to speed this up is to make the update one transaction with the commit or rollback at the end. I did this with MB3 and you can do thousands of inserts in the time it takes to do a couple. Martin
2016-02-11, 06:18 PM
mvallevand Wrote:The likely sqlite change to speed this up is to make the update one transaction with the commit or rollback at the end. I did this with MB3 and you can do thousands of inserts in the time it takes to do a couple.It does already do this, but per channel. ie, 250 or so inserts, then committing the transaction. On my machine it's very quick to save the listings, probably because my database is on an SSD drive, so I'd never noticed any need to change, but I could make a change in the next release to do mulitple channels in the same transaction.
2016-02-11, 07:56 PM
Wow... A most informative exchange.
A would like to raise my voice for a parameter that instructs the EPG Updater how often to commit transactions: two options 1)default 2)all updates in a single transaction. It would certainly be on the owner for option two. If things go south, a complete EPG rebuild would be necessary. The user would probably notice when that would be needed. But even if the update was corrupted, another update would only take 4 minutes. I can vote for that. Thanks to all contributors. A great tutorial on the SQLite transactions and NextPVR info on how things work. Um...sub...any chance of getting a tryout version? Single transaction?
Later...JohnC
System Status: Humming nicely!  Desktop: Intel DX38BT MB - Intel Q9650 3GHz Quad - NVIDIA GeForce GTX 1050 Ti Graphics Software: Win 10 Pro(Build 1909) - NextPVR - WinTV8 Video: Verizon FiOS - Hauppauge DCR-3250 - Hauppauge 1512 HDPVR2 RaspberryPi 4 Raspbian and FLIRC
2016-02-11, 08:13 PM
jcole998 Wrote:Um...sub...any chance of getting a tryout version? Single transaction?Sorry, I can't change this until the next release.
2016-02-11, 08:23 PM
I am now only about 0.3 seconds a transaction on my SSD drive with 184 channels and 67000+ rows
2016-02-11 06:48:10.277 [DEBUG][10] Saving EPG listings 2016-02-11 06:48:10.277 [DEBUG][10] Deleting old listings from Channel OID: 6865 (TVO) 2016-02-11 06:48:10.897 [DEBUG][10] Deleting old listings from Channel OID: 6866 (GLOBAL) 2016-02-11 06:48:11.306 [DEBUG][10] Deleting old listings from Channel OID: 6867 (TVGCAN) *no guide info for this barker channel 2016-02-11 06:48:11.425 [DEBUG][10] Deleting old listings from Channel OID: 6868 (CBOFT) 2016-02-11 06:48:11.798 [DEBUG][10] Deleting old listings from Channel OID: 6869 (CHRO) 2016-02-11 06:48:12.191 [DEBUG][10] Deleting old listings from Channel OID: 7957 (CIII-HD) ... 2016-02-11 06:49:16.054 [DEBUG][10] Deleting old listings from Channel OID: 7074 (102.1 The Edge) Frankly I do it before I wake up and it is only an issue when I do a manual update. It used to be much slower before SD JSON and the SSD drive and I don't know if it worth making and change just for this. Martin
2016-02-11, 08:24 PM
I have made the change for the next release. Hopefully it'll be a notable difference for those users with lots of channels and no SSD drive
2016-02-11, 08:30 PM
What's scary is 184 channels is no longer considered lots. https://www.youtube.com/watch?v=Pu1AOImIFME
Martin
2016-02-11, 10:13 PM
Thank you, sub, I'll be on the lookout.
Martin, my subscription with Verizon pretty much dictates how many channels I have. I probably watch 60 of the 562(?) they supply. That's 180,000+ rows for a two-week TV guide. And, I run the update automatically at 4am. It's just an exposure to sneaky problems to have an update run so long. SSDs are off in the distance. Thanks, again, to all who have helped.
Later...JohnC
System Status: Humming nicely! Desktop: Intel DX38BT MB - Intel Q9650 3GHz Quad - NVIDIA GeForce GTX 1050 Ti Graphics Software: Win 10 Pro(Build 1909) - NextPVR - WinTV8 Video: Verizon FiOS - Hauppauge DCR-3250 - Hauppauge 1512 HDPVR2 RaspberryPi 4 Raspbian and FLIRC
2016-02-11, 10:15 PM
jcole998 Wrote:Martin, my subscription with Verizon pretty much dictates how many channels I have. I probably watch 60 of the 562(?) they supply. That's 180,000+ rows for a two-week TV guide. And, I run the update automatically at 4am. It's just an exposure to sneaky problems to have an update run so long. SSDs are off in the distance.If you only watch 60 of those, you know you could delete those other 500-ish channels, and you EPG update will be quick from then on? |
|
|
