Newbie - Understanding the Recording Process

[daveslee]

Hi Everyone,

First off, thanks to the community for Next PVR and Kodi providing a replacement for Windows Media Centre now that I've gone to Windows 10. My wife makes heavy use of Windows Media Centre so I need to thrash out the bugs in the Next PVR/Kodi system so that this all becomes "wife-friendly".

All that follows is for:

NextPVR: 3.4.8

NextPVR add-ons and utilities: TV Listings, TV Recordings (could live without these); XmktvExtras.exe, XmlTvGUI (scraping from Radio times)

TV Tuners: Hauppauge WinTV MOD7700 DVB-T dual tuner; PCTV single tuner recognised as Hauppauge MSi2500 all taking a very weak signal from UK, Northumberland, Pontop Pike with significant amplification and distribution sometimes resulting in poor signal (which we have to accept living in a rural area).

Kodi: "Isengard" 15.0 Git:20150721

Kodi add-ons (other than default): MCEREmote, NextPVR PVR Client, XEM

I've read the first 20 pages of so of the Forum, checked out the FAQs and the Wiki, but I'm still cluless on what follows. Apologies if this has been asked a million times before - in which case point me at the forum posts that I can't find (even with search).

===

Setting recordings/timers.

Kodi seems to provide all of the interface that we want with the exception that "series link" type functionality seems a bit hit and miss. The record request for the "base" program seems to go back to NextPVR from Kodi quite quickly and reliably, but the "linked" recordings necessary seem hit and miss. Sometimes they appear instantly, sometimes they appear after delay and sometimes never (if you see what I mean). Setting the recordings directly in NextPVR through "TV Guide" always works; sometimes the add-in "TV Listings" doesn't behave as I'd expect (a bit like setting timers through Kodi).

Can anyone explain or point me at the wiki/forum articles that explain how the recordings database works so I can understand what (if anything) is going wrong. If all else fails, I'll just keep using NextPVR "TV Guide" only to set recordings.

Noisy Recordings

We seem to suffer from two types of noise in the .ts files that are recorded (any the WTV and dvr-me files that various Media Centre editions have provided):

(i) "Shot noise" from passing traffic, tree movements etcetera. This produces mpeg blocking (as expected - on our dedicated TV sets too) and is sometimes so severe that it upsets software playback of the transmission streams.

(ii) Startup noise: Whenever the tuners start-up or change channel they seem to take some time to lock and quite often there are errors in the first few frames of mpeg video that upset some software decoders.

I'm using the LAV decoders and they seem more susceptible to damaged .ts than other decoders. For a give damaged .ts file, LAV decoders perform worst, then the internal decoders in VLC media player. Best performing are the decoders for .ts inside my WD-Live stand-alone players.

Not really understanding the way that NextPVR makes recordings then makes them available to e.g. Kodi, I tried to insert ProjectX in the "signal chain". Running those dodgy .ts files through a stand-alone compled version of ProjectX 0.91.0.10 set to demux. drop all except valid video and audio and remux to a .ts cures the crashing of the LAV and VideoLAN converters and makes the skip forward/back on the WDLive boxes of these files a bit more reliable.

But I'm obviously doing this wrong. I looped ProjectX in using a "PostProcessing.bat" script which acts on the original .ts file passed as %1 into the script. ProjectX gives a file with [remux] added into the filename, so I rename the original .ts to .bak and rename the new ts to remove the [remux].

Somehow though, NextPVR notices that I fiddle with the .TS file. Timekeeping in playback is hosed in NextPVR and likewise in Kodi->TV->Recordings. Curiously, the file plays as expected if I go Kodi->TV Shows->(e.g.) Recently Added then all is well. I guess I don't understand the way files are processed in NextPVR and I'm "intercepting" at the wrong stage. What is causing NextPVR to not just play the new (remixed) .ts file - what is instructing it to do something different?

Can anyone explain or point me at a description of how recordings are made so I can fix this for myself?

In the meantime, I'll manually run ProjectX over any skipping/crashing .ts files, but this is not "wife friendly"...

Thanks in advance everyone.

D

[sub]

If you're getting missing data etc due to signal problems, but these best solution is always going to be to try fixing those signal issues if possible.

expected if I go Kodi->TV Shows->(e.g.) Recently Added then all is well. I guess I don't understand the way files are processed in NextPVR and I'm "intercepting" at the wrong stage. What is causing NextPVR to not just play the new (remixed) .ts file - what is instructing it to do something different

I've never used ProjectX, so I don't know what it might be doing to break these recordings. If you want to supply a short recording, I can check for anything obvious.

Can anyone explain or point me at a description of how recordings are made so I can fix this for myself?

I'm not sure what you want to know. These are a bit-for-bit dump of the broadcast data for that channel, delivered by the device's drivers. NextPVR is not manipulating it.

[daveslee]

If you're getting missing data etc due to signal problems, but these best solution is always going to be to try fixing those signal issues if possible.

Thanks - we are making the best of a bad job at the moment and I'm considering putting in a head-end for DVB-S with six to 8 channels of recording to serve the whole house. But this will require "wife-unfriendly" wiring to get hard-wired gigabit Ethernet to the relevant places.

I've never used ProjectX, so I don't know what it might be doing to break these recordings. If you want to supply a short recording, I can check for anything obvious.

I was expecting ProjectX to repair the recordings. In manual tests, I took the .ts file pointed to by %1 in the PostProcessing script and ran it through ProjectX set to de-mux, discard any invalid frames and remux back together as a new .ts. These remixed files don't crash the LAV filters or VLC like the original .ts do. I'll do some 5 second samples for you of the "before" and "after" so you can perhaps see what ProjectX is changing.

I'm not sure what you want to know. These are a bit-for-bit dump of the broadcast data for that channel, delivered by the device's drivers. NextPVR is not manipulating it.

That's really odd. All my post processing script was doing was renaming the .ts file that NextPVR pointed to with %1 to .ts.bak and then (a little while later - after ProjectX had completed its demux-mux pass) copied the resulting Projectx .ts file back to the same place as was pointed at by %1. If I play that file with Windows Media Player with the LAV filters or point Kodi at the file, it plays just fine (without the garbage causing the filter crashes). But if I let NextPVR or Kodi play the file "pointed to" by the NextPVR database, this is clearly a different file or is being commanded somehow to skip bits. Perhaps that is a clue - I wonder if ProjectX is hiding away some file with the cut points of the damaged section of .ts and NextPVR might be interpreting these as skip points for e.g. the ComSkip add-in?

So what I was looking to understand was the relationship between the files in the filing system and the entries in the database because somehow manual playback of the files is different from playback based on the information in the database...

But thanks anyway for thinking about this. I'll probably go the satellite road for decent signal strength, or just bite the bullet and manually repair any .ts files that give problems rather than just dumping everything through ProjectX.

[sub]

That's really odd. All my post processing script was doing was renaming the .ts file that NextPVR pointed to with %1 to .ts.bak and then (a little while later - after ProjectX had completed its demux-mux pass) copied the resulting Projectx .ts file back to the same place as was pointed at by %1. If I play that file with Windows Media Player with the LAV filters or point Kodi at the file, it plays just fine (without the garbage causing the filter crashes).

I'm pretty sure if you've replaced the existing .ts file with one of the same name, then it'll be trying to play that file. If you're uncertain, we can always look at a log, which will show the file name. You can also manually try playing one of these files by right clicking and selecting 'open file' and just browsing to it, to see if it plays normally.

But if I let NextPVR or Kodi play the file "pointed to" by the NextPVR database, this is clearly a different file or is being commanded somehow to skip bits. Perhaps that is a clue - I wonder if ProjectX is hiding away some file with the cut points of the damaged section of .ts and NextPVR might be interpreting these as skip points for e.g. the ComSkip add-in?

Maybe. For commercial skip points it'll look for the same file name but with .edl instead of .ts.

[daveslee]

Here's some file samples so you can see what I'm trying to achieve:

https://www.dropbox.com/sh/8i15xzd5qgxn7...E_j3a?dl=0

Wheeler Dealers_20150805_14001500_log.txt and the corresponding "biglog" file show ProjectX chopping the first 10MB or so off the original .ts file as recovered by NextPVR.

Wheeler Dealers_20150805_14001500(0)[copy]00.ts is the resulting shortened and still unplayable file.

Wheeler Dealers_20150805_14001500(0)[copy]00[remux].ts is the result of running that unplayable file through the ProjectX default filters, demuxing the audio and video apart and then remuxing them back together into the new .ts file.

Wheeler Dealers_20150805_14001500(0)[copy]00_log.txt and the corresponding "biglog" show the three changes that ProjectX made to the file to make it playable. I have no idea what the jargon means.

I've tried turning each setting in ProjectX on and off, but there is more than one correction going on sop I haven't found out which correction algorithms taken together give me the sort of fix that I want.

Thanks again for your patience with me on this. Just shout in here if I've got the DropBox permissions wrong for the samples.

D

[daveslee]

One further thing on this. The problem files most often occur when I have two recordings both coming from the same bouquet on a single multiplex - only one tuner is in play and the result is two .ts files (somehow by magic and hand-waving). Could this have anything to do with the weirdness I'm seeing, or is this another red-herring?

Cheers

D

[sub]

Can you supply a longer sample? 30 seconds or so would be idea.

[daveslee]

Happy to provide a longer sample, but those two samples (the first of which crashes out) are 51 seconds long...

Please stand-by...

D

[sub]

Ah, right you are. The files looked pretty small, and when I played the remuxed one in WMP, it loops back to the beginning after about 4 seconds. I can see VLC playing it's full length though.

[daveslee]

Hi, yes, this is the sort of "intermediate" level of corruption I see. It's enough to really upset the LAV codecs, but the ones built into VLC seem to cope.

I've reverted back to not tinkering with files and I'll leave NextPVR/Kodi alone for a few days to collect fresh and unambiguous logs and some more damaged recordings (if they happen - a watched kettle never boils), whilst deleting all the guff that has built up whilst tinkering.

Thanks again for all your input - far above that which most developers put into their code.

For the next time around - is there a sensible way to cut the log files down to size or do I just zip up a multi-megabyte package of everything?