[WIP] New Confluence Skin for XNEWA

[pkscout]

I like your fix for midnight but not keen on moving time fixes out of xnewa_connect. I want it to be the glue to newa having it do all the data normalization. I might move it to lib but I definitel want it to b e a lib any future xbmc addon could use. I still have to move player out of details to isolate things better but web client and x-newa should in theory be able to work as two addons with proper planning.

That doesn't look too difficult. I can move the stuff out of xnewa_datetime.py and into xnewa_connect. I think it would make sense rename the public functions to formatDate and formatTime so that the call would be something like self.xnewa.formatDate( self.detailData['start'], withyear=True ) and self.xnewa.formatTime( self.detailData['start'] ).

On a completely unrelated front, I kind of fell down a rathole around the image storage and cache stuff. I was trying to troubleshoot an image display issue and spent way longer than I wanted to finding all the places XNEWA stores images. As it currently stands, XNEWA stores the following things (I think):

1- channel images are downloaded from NPVR and stored in script.xbmc.xnewa/fanart/Channels
2- show images are downloaded from NPVR and stored in special://temp/x-newa/covers
3- if there are images in script.xbmc.xnewa/fanart/Shows they are used instead of special://temp/x-newa/covers
4- genre images are read from script.xbmc.xnewa/fanart/Genres, but it doesn't look like they are download from NPVR

I'd like to propose (and I already having working code for this) the following:

1- channel images downloaded from NPVR would be stored in special://temp/x-newa/fanart/Channels
2- show images downloaded from NPVR would be stored in special://temp/x-newa/fanart/Shows
3- genre images downloaded from NPVR (if such a thing ever happens) would be stored in special://temp/x-newa/fanart/Genres
4- there would be an 'override folder' in addon_data/script.xbmc.xnewa called fanart with subdirectories for Channels, Shows, and Genres
5- if there are images in addon_data/script.xbmc.xnewa/fanart/ they are used instead of the ones in special://temp

A couple of nice things happens when you do this. First, updating the addon doesn't blow away the images. That would mean XNEWA wouldn't have to download the channel images again after every update *and* a user wouldn't lose any custom show images. I'm not saying the latter has happened to me, but the latter has happened to me. Second, the cache and override folders have consistent naming and all the image types have the same behavior (right now Genre Icons and Channel Icons have no cache, but Show Icons do).

I'm also wondering if, based on your desire to have XNEWA_Connect be the call point for all the central functions, the stuff in fanart.py should just be moved into XNEWA_Connect. Right now XNEWA_Connect handles the downloads and fanart.py handles the reads. That's not necessarily a big problem, but some constants have to be manually synced up between the two (like cache location) in order for everything to work right.

I'm happy to work on the image code, since I can sort of stretch and say it's within the scope of the skinning work.

[mvallevand]

That doesn't look too difficult. I can move the stuff out of xnewa_datetime.py and into xnewa_connect. I think it would make sense rename the public functions to formatDate and formatTime so that the call would be something like self.xnewa.formatDate( self.detailData['start'], withyear=True ) and self.xnewa.formatTime( self.detailData['start'] ).

Ok, some of this will depend on what UJB changes on the NEWA end, but the JSON call will deal withe that.

On a completely unrelated front, I kind of fell down a rathole around the image storage and cache stuff. I was trying to troubleshoot an image display issue and spent way longer than I wanted to finding all the places XNEWA stores images. As it currently stands, XNEWA stores the following things (I think):

Yeah, fanart is a mess even the downloading is done way too many times. You should also note there is a NEWA bug and downloads are saved in 500,0000 byte segment UJB added my fix for that in 2.13

1- channel images are downloaded from NPVR and stored in script.xbmc.xnewa/fanart/Channels.

They aren't optional so they are designed to be uninstalled with the addon. There is no refresh so I figured the update would clean it.

2- show images are downloaded from NPVR and stored in special://temp/x-newa/covers

They are optional so I decided they could be persist

3- if there are images in script.xbmc.xnewa/fanart/Shows they are used instead of special://temp/x-newa/covers

That was in case I gave an option not to persist them

4- genre images are read from script.xbmc.xnewa/fanart/Genres, but it doesn't look like they are download from NPVR

Not supported, The intent was to have some genre art in the addon itself, for DVB that works a lot better.

I'd like to propose (and I already having working code for this) the following:

I'm very interested if a clean uninstall is possible.

I'm also wondering if, based on your desire to have XNEWA_Connect be the call point for all the central functions, the stuff in fanart.py should just be moved into XNEWA_Connect. Right now XNEWA_Connect handles the downloads and fanart.py handles the reads. That's not necessarily a big problem, but some constants have to be manually synced up between the two (like cache location) in order for everything to work right.

I'm happy to work on the image code, since I can sort of stretch and say it's within the scope of the skinning work.

Sure x-newa pickle cache files are there in special://temp too and having this in common code makes sense

Martin

[pkscout]

OK, here's 2.4.1~beta2.

changelog
- made icon and channel caching consistent
- now use addon_data dir instead of addon dir for override fanart
- moved date/time format functions into XNEWA_Connect
- moved fanart functions into XNEWA_Connect

Hopefully this was the direction you wanted me to go with this. I've eliminated the separate fanart.py and xnewa_datetime.py files and refactored all the other code to find those functions in the XNEWA_Connect object (i.e. self.xnewa). I haven't really looked much yet at your concern that xnewa was downloading images too much. It looks like to me xnewa will only download channel images once the first time it runs, and show images are only downloaded when details.py is called and then only if there is no cached or override image available. I could add something that would tell xnewa not to try again to download an image from NPVR for a couple of weeks for cases where it doesn't find an image the first time. I actually do something similar with Artist Slideshow.

We should probably continue the conversation about the right place to put the cached images. Right now they are in special://temp/x-newa/fanart (Shows, Channels, and Genres), and if there are any images in addon_data/script.xbmc.xnewa/fanart (Shows, Channels, and Genres) then those are used instead. This probably doesn't get you a "clean" uninstall in the way you want, but I'm also not sure we can get a 100% clean uninstall and have the user be able to put override files someone and not have them disappear after every update.

Right now if you uninstall xnewa, the stuff in addon_data will remain as well the stuff in special://temp. XBMC appears to assume that add on devs will manage the files in those locations, but there is no way to clean them during an uninstall (at least not that I'm aware of). So even with the shipping 2.4.0 version, images downloaded to the covers folder in special://temp/x-newa and any of the pickled cache files will remain after an uninstall.

We could move the entire cache into a directory in the add on folder. There doesn't appear to be any performance downside to this, and that cache would automatically get cleared during an update or reinstall then. I'd still like to have the user override stuff in addon_data even though this means that stub of stuff would be left after an uninstall.

The good news is that it is very simple to change the cache and override locations. Lines 52 and 53 of XNEWA_Connect.py now define those locations once, so you only have to change it there and restart NEWA for those folders to live somewhere else. On instantiation XNEWA_Connect will even create the directories if they don't exist. So we can continue to chat about the right place for the folders without having to constantly search and replace throughout the code as we change our minds.

P.S. I think the zip I uploaded has the recurring recordings list defaulted to sort by title. My wife really doesn't like having to sort it once just to get that list in the "right" order, so I change my local copy here. I try and remember to change it back before I upload anything, but it's late and I think I forgot.

[mvallevand]

All good. I will add a setting option to opt for a permanent cache noting the manual uninstall. The pickle cache needs to follow the same rule.

Recurring list priority order makes more sense when there is a priority in 2.13 I like that order because new adds go to the bottom of the list. I will consider a settings options or maybe do like sub and leave it as the last sort.

Martin

[pkscout]

Recurring list priority order makes more sense when there is a priority in 2.13 I like that order because new adds go to the bottom of the list. I will consider a settings options or maybe do like sub and leave it as the last sort.

Last sort would be great (and, as you noted, consistent with NextPVR). When there is a priority, I do think that might be the right initial choice. It's just that right now it makes the list look random.

[mvallevand]

Last sort would be great (and, as you noted, consistent with NextPVR). When there is a priority, I do think that might be the right initial choice. It's just that right now it makes the list look random.

Ok I have that working plus I have resolved the time issues using the new NEWA and this library https://labix.org/python-dateutil It helps over other methods because ilocattime ican be DST aware. I'll keep you method in the non-JSON call since that can work pre NEWA 2.13

Martin

[pkscout]

Ok I have that working plus I have resolved the time issues using the new NEWA and this library https://labix.org/python-dateutil It helps over other methods because ilocattime ican be DST aware. I'll keep you method in the non-JSON call since that can work pre NEWA 2.13

Awesome. I look forward to trying this with the new NEWA.

[mvallevand]

Awesome. I look forward to trying this with the new NEWA.

You now can http://forums.nextpvr.com/showthread.php...to=newpost

Attached is the beta with your fanart and cache changes. I've modified the setup to allow storing the permanent cache outside the plugin an option.
- The default sorts for recordings and recurring schedules are also options.
- Stores the last sort order
- added dateutil.parsor as a library that can be used for better time zone handling.

New features in NEWA can now be tested

- shows the recording priority sort order
- modify recording priority up/down
- cover art with non-filename characters now available.
- new JSON call for All Recordings so SOAP can be removed in the next version of NextPVR

Martin

[mvallevand]

@pkscout, could you help me out and do a bit of research on your Confluence skin and maybe update nextpvr_recent.xml? I have a written a small change to recent.py to multi-select recordings (for delete, archive, watch, unwatch) and the default skin works because it has a selectedcolor tag. Could you add the tag to the list item and it likely should match the colours in File Manager http://wiki.xbmc.org/index.php?title=Fil...ting_Files

Thanks,.

Martin

[pkscout]

@pkscout, could you help me out and do a bit of research on your Confluence skin and maybe update nextpvr_recent.xml? I have a written a small change to recent.py to multi-select recordings (for delete, archive, watch, unwatch) and the default skin works because it has a selectedcolor tag. Could you add the tag to the list item and it likely should match the colours in File Manager http://wiki.xbmc.org/index.php?title=Fil...ting_Files

I'd be happy to. Are the changes you made to recent.py in the beta3 code, or do I need something more up-to-date so I can test? Same question for the nextpvr_recent.xml for the default skin.