Recording Logic

[TRiddle]

Hello all,

I came across NPVR from the Kodi forums when I began looking for an alternative to WMC due to the recent guide changeover just absolutely butchering my data. Prior to WMC, I was a long time SageTV (and still have all of my licenses) until the Google buyout. Prior to trying NPVR, I had setup MediaPortal with less than stellar results from my perspective. Let me say that I love just how simple it was to "just make it work" with NPVR. My Kodi boxes around the house are running beautifully right now.

Seeing as how I was a previous user of both WMC and SageTV, I have become accustomed to the intelligent recording logic that both used (I still to this day believe that SageTV has some of the best recording logic I've ever seen). I have 5 tuners, one colossus for use with my Dish box and 4 hdhr tuners for ATSC. What I'm having a hard time wrapping my head around is the logic that NPVR is using for recordings. Again, due to what I've used extensively in the past, I'm accustomed to adding a favorite, telling it to record "new", and letting the logic figure out how to deal with conflicts. That does not appear to be how NPVR does things. When I look at my web interface for managing recordings, I see conflicts that don't seem to have resolutions. A prime example would be tonight: I have 5 satellite shows that air on Sunday nights, all are setup as recurring favorites, record only new, any time slot, any channel. All of these shows have multiple airings of the same "new" episode. Obviously the "new" episodes for each are in conflict with each other. The prior softwares I've used appear to have this logic: if conflict, then take top priority show and record it and find an alternate airing of the lower priority show and record that episode. In the past, this didn't always work and recordings were missed, but that was the very rarest of occasions.

So, how does the recording logic for NPVR work? I think that I can work my way around the conflicts if it doesn't behave as I've described above, that will just require more vigilance on my part with the web interface. I've also done quite a bit of searching on the forums regarding this, and I'm coming up empty. Maybe I'm not searching for the right combination of terms. It definitely would not be the first time that I've done that!

Any insight would be greatly appreciated. Thanks.

Tyler

P.S. SageTV is about to go open source, so it might be worthwhile to taking a peak at how their recording engine works when the time comes. I'm by no means a coding expert (mostly relegated to visual basic in excel), but getting that type of logic into NPVR would be something that I'd definitely be willing to take a crack at.

[johnsonx42]

you told it to record "only new". later showings are not new. it's doing what you told it.

(actually, now that I think about it, the recording type you say you are using "record only new, any time slot, any channel" doesn't exist (unless you created it by hand using SQL matchers). There's a "NEW Episodes, this channel" and an "All Episodes, All Channels", but no "NEW episodes, All channels".)

[johnsonx42]

to explain a little further, to give some insight as requested:

the npvr scheduler makes no explicit effort to resolve conflicts during scheduling, rather it lets them resolve themselves.

when it starts scheduling, it starts with your highest priority recurring recording and searches the epg database in chronological order for an episode that matches the criteria (usually show title and channel, sometimes the first-run or "new" flag, sometimes more criteria); when a match is found it then checks, based on the unique_id of the episode, whether the episode is already recorded, already scheduled to record, or recently deleted; if the unique_id is not found in any of those locations, it then attempts to find a tuner that is available to record the episode; if a tuner is available, it schedules the episode and goes on.

if no tuner is available, it marks THAT SHOWING of the episode as a conflict and goes on.

now here's where it "lets the conflicts resolve themselves": if a later showing of an episode is found and scheduled successfully, then the original conflicting showing is simply deleted and no record of it remains (this is again based on the "unique_id" of the episode in the database, usually provided explicitly by your EPG data source). Note however that the later showing still has to match ALL of the criteria of the recurring recording request; if the recording request specifies things like a particular days or days, timeslot, or first-run only (aka "new"), then later showings may not match and will not be recorded.

[TRiddle]

Thanks for the explanation. I had garnered from playing around with the recording logic today on how it was handling things. I think I can craft up a working example using excel (it's what I know) on how other programs are doing it. It is nothing more than if-then structure: If there is a conflict with this episode marked as "new" in the guide, then find another timeslot with this recording and record it, unless there is again a conflict, then keep searching. If it exhausts all options within the guide, then the conflict will not be recorded. To my knowledge, Sage did it with id's, and also had a setting for "record series, new episodes only", and it just found a way to get all of the recordings done. WMC does this (albeit worse than Sage) as well.

Just as an FYI: I can set a recurring recording using the web interface as "record this series, any timeslot, any channel, only new recordings". Now, whether that is actually executing in that manner is another question, but it is rather easy to set up a recurring recording to do what you say cannot be done. I just set one up using American Ninja Warrior to prove to myself that I wasn't crazy in being able to choose these settings.

[johnsonx42]

Just as an FYI: I can set a recurring recording using the web interface as "record this series, any timeslot, any channel, only new recordings". Now, whether that is actually executing in that manner is another question, but it is rather easy to set up a recurring recording to do what you say cannot be done. I just set one up using American Ninja Warrior to prove to myself that I wasn't crazy in being able to choose these settings.

Sorry, I always forget about the web interface... I never use it. UJB has cleverly added features to NEWA that are not present in the main program.

As to the more advanced logic for more explicitly resolving conflicts by a direct attempt to locate an alternate showing, I'm sure sub could add such logic if he wants to. The issue will always be whether it's worth it: the existing logic handles probably 98% of conflicts for 98% of the users, is it worth his time to implement an additional layer of logic for the remaining 2%? Especially when there is an existing solution for those with many channels on a single capture source that air multiple repeats of the same episodes: don't use the "only new" option. Sure, you'll get some re-runs of older episodes too from time to time, so you'll have to delete a few and they'll go in the recently deleted list and not come back for awhile.

[TRiddle]

Challenge Accepted

Attached is a little spreadsheet I crafted this morning that will perform conflict management beautifully. Sure, this is excel, but the logic does not change from language to language. Knock yourself out. As per standard operating procedure, enable macros. Code is accessed by pressing alt + F11 (in case you didn't know). Priority is from highest to lowest. The EPG can be configured with shows of the types (N) and regular: (N) indicating a new airing and the regular (without the (N)) being of the same showID, but not the first airing of said show. "Run Me" runs the logic, and the "Clear" buttons should be self-explanatory. Total time coding this: 3 hours (give or take a few minutes).

As to the more advanced logic for more explicitly resolving conflicts by a direct attempt to locate an alternate showing, I'm sure sub could add such logic if he wants to. The issue will always be whether it's worth it...

Yeah...this is standard logic in Sage and WMC (and probably MediaPortal with the Argus Scheduler, although I didn't get far enough to experiment). Also, it's not difficult logic as I hope my spreadsheet proves.

...the existing logic handles probably 98% of conflicts for 98% of the users, is it worth his time to implement an additional layer of logic for the remaining 2%?

You're about to be brought to task on this. Being an engineer myself with a criminal defense attorney for an older brother, I have to ask: you have some evidence to back up that claim? Again, I've been in the homebrew/DIY HTPC club for going on 12 years now. Most of those years were spent with Sage, and the past few years have been spent with WMC. Again, in both programs, we call that type of recording logic STANDARD. Seeing as how I'm now at NPVR due to 1: SageTV being long ago purchased by Google (even though open source is coming) and 2: Microsoft all but killing WMC future development (and recently upgrading guide services which all but destroyed my EPG), this was typical recording logic behavior that I was expecting once I pulled the trigger to re-configure. I bet I'm probably not the only one that is jumping ship from WMC either (that would be speculation on my part, not fact). Further speculating, I bet I'm not the only one that ended up finding NPVR to be an "almost" perfect 1:1 transfer as far as features that I need. I say all of this to say the claim of "98% for 98%" is quite possibly erroneous and flat untrue.

Especially when there is an existing solution for those with many channels on a single capture source that air multiple repeats of the same episodes: don't use the "only new" option. Sure, you'll get some re-runs of older episodes too from time to time, so you'll have to delete a few and they'll go in the recently deleted list and not come back for awhile.

No thanks. I'll go hack in my Schedules Direct data into WMC, breaking all kinds of ToS agreements or get a (shudder) commercial solution before I accept that I'll have to delete some recordings of older episodes (that I've already watched) and hope that they don't come back "for awhile". It's been a few years since I've done Dish installation (college job), but I'm guessing even the hopper system has advanced conflict logic by this point (speculation). Nor do I plan on buying a second satellite receiver to fix the problems (it easily would). I've been able to run a system with 5 tuners, 4 ATSC and 1 satellite box on a HD-PVR/Colossus, for years, and I can count on one hand the amount of "not recorded due to a conflict" I've had on one hand. The logic can do better.

At this point, you probably think I'm attacking you. That really isn't the tone I'm trying to take at all. I see a problem, and I think of ways to solve it. That is why I took the time to craft up a proof of concept working example in excel this morning. It works, and it's far from complicated. Furthermore, this looks to be a prime opportunity to attract new members seeing as how WMC is on life support. I should probably also state that I worked out my conflicts by setting timeslots for recordings last night, so nothing went unrecorded. However, that was work that I honestly felt should've been handled by better recording/epg logic. Also, should one of those shows change timeslots in the future, I get to go back and do all of that over again. That doesn't sound too ideal to me.

I'd love some feedback on my proof of concept as well.

Tyler

[TRiddle]

FYI...

http://forums.sagetv.com/forums/showthread.php?t=62315

Now officially open source.

[sub]

I'm a reasonable guy, and I always happy to look into improving things. It just takes someone to clearly articulate the possible improvement they like, and suggestions for how it'd work certainly help make it more likely.

FYI, to give you a little more perspective on why NextPVR works the way it does. WMC and SageTV were both commercial applications, providing their own EPG, and had logic around the data they had available in their EPG, and had a very North American centric view. NextPVR on the other was developed in New Zealand, and was designed around the fact that majority of users were getting their listings from the DVB EPG, or xmltv etc, and things like unique show identifiers were just not available, and no indication of a show being new or a repeat. With these EPG sources, even show descriptions/subtitles/etc were often just generic descriptions, not useful for identification. So for the majority of users, if a particular episode couldn't be recorded, there was no way in hell to for NextPVR to find another airing of it.

In NextPVR's defense, there is some scheduling things NextPVR does worlds better than WMC or SageTV. Stuff like scheduler handling multi-record, allowing multiple shows on different channels to recorded simultaneously on a single tuner, making it less likely that something cant be record. For example, here in NZ, I've had a single dual-tuner device recording 18 different shows at the same time, across 18 channels. It's also able to do stuff like padding on both recordings in back-to-back recordings scenarios, without tying up additional tuners.

[TRiddle]

Thanks for the reply Sub. Reading through my post, even I think the tone sounds bad, and I didn't intend for that. I can remember first setting up Sage and first setting up WMC, and when I compare it to setting up NPVR, it is night and day. NPVR was just that user friendly to setup. I was shocked, especially since I had spent the previous two days trying to get MediaPortal what I wanted it to do without success. The multi-record functionality was definitely impressive when I tinkered with it, so definite props on that as well. I also understand bad guide data as it's one of the main reasons, if not the singular reason that I'm no longer with WMC. Doesn't do much good to have that type of recording logic if the data is unreliable. I've always kept a SD subscription lying around, so that was another feather in the cap for NPVR on setup in that it was just so easy to integrate my SD lineup.

Again, I don't know much about how the coding for NPVR works. All I know is logic. I knew that I could craft the logic using what I hope was an easy to follow proof of concept. To break it down to the simplest concept, I just want the ability to use two flags for the purposes of recording: the new flag and the showID flag for when the new flag throws a conflict between two or more recordings. Again, thanks for the reply and keep up the good work!

[mvallevand]

I have a particular problem that I have articulated several times that happens mainly on Sundays on AMC. At 10pm and 11pm they air the same new show and I want to record the 11pm show. I always end up with a conflict for the 10pm because NextPVR tries to record it and then then 11pm show is usually recorded. I think NextPVR should respect the time and only fall back to the 90 minute window when it fails.

Martin