TS and Comskip

[petercooperjr]

I think this may be something that sub already knows about, but I wanted to check and I guess see if it was on a to-do list somewhere.

When using GB-PVR with my HVR-1600 digital recordings with the dvr-ms mux, commercial detection worked really well. I recently switched to a HDHomeRun, but I needed to switch to the TS mux to get that to work. With the TS files from the HDHomeRun, it seems to jump early, like skipping over the last 5-10 seconds of show and resuming with 5-10 seconds of commercial left. I think the comskip files are correct, and it's just not matching them up with the playback right. I know that sub had said something at some point about not being able to know where one was in TS files as easily. So, I tried postprocessing with ffmpeg to turn the TS file into an MPEG PS file, which seemed to play just fine, but surprisingly to me had the same issues in the same way with the skipping not being quite right.

So, I've been trying to switch to NPVR, hoping that since it's designed from the ground up to work with TS files, that it would work better. But it seems to be the same. Where it skips just doesn't quite match up with where the commercials are.

I am a programmer, but I don't really know about the internals of MPEG, but I'm curious what's so hard about taking a frame number from the output of comskip and seeing when that frame number comes up. But I'm sure it must just be much more complicated than that.

I guess my questions are just (1) is this on the list of things to be worked on, (2) would there be any log files or comskip files or anything that I should attach to see if there's something I'm doing wrong, and (3) is there any way I could help this get fixed?

Thank you.

[johnsonx42]

I've been wondering for awhile whether re-muxing a .ts file to MPEG would make comskip work better. I kept meaning to try it, but never got to it. One question though, did you re-mux before running comskip, or after? If you re-muxed after the comskip file was already created it may not have helped. I'm not sure it will help the other way either though.

I think the fundamental issue with .ts, comskip and GB-PVR/NPVR is that time relative to frame count is a bit variable. Comskip reads the file and figures out which frames to mark as commercials. When the file is played back in GB-PVR/NPVR, the relative time position in the file isn't known exactly so the resulting skips are a little off from where comskip calculated them. GB-PVR/NPVR apparently does positioning in the file using time and framerate rather than direct frame count (and frame rate is the variable in .ts files). I recall sub saying the only solution would be to count every frame in the file, and do the skips based on exact frame number. I'm not sure though whether actually doing this is easily accomplished, or whether sub intends to do so.

Finding a solution perhaps hasn't been all that high on sub's list:

I didnt bother building in any special direct running of comskip.exe, because I havnt used it for years, and have kind of lost interest in it.

[petercooperjr]

I've been wondering for awhile whether re-muxing a .ts file to MPEG would make comskip work better. I kept meaning to try it, but never got to it. One question though, did you re-mux before running comskip, or after? If you re-muxed after the comskip file was already created it may not have helped. I'm not sure it will help the other way either though.

With GB-PVR, I had tried re-muxing to a program stream MPEG and using the same comskip files, as well as re-running comskip on the program stream MPEG afterward, and either way seemed to get the same results. There could be something wrong with how I'm doing it, too.
I'm running (from memory)

Code:

ffmpeg -i file.ts -acodec copy -vcodec copy -f dvd file.mpg

which seems to give me a good file although it gives a lot of buffer warnings, I assume because the dvd format isn't designed to deal with a HD MPEG file.

If someone's had success getting comskip to work with a better/different remuxing, in either GB-PVR or N-PVR, maybe that would be a good solution.

But it seems like one ought to be able to get it to work with transport streams, too.

Thanks.

[johnsonx42]

That is, as far as I know, the correct ffmpeg command to remux the file. The only way I think you could get it to work without changes to NPVR is to re-encode the file to MPEG rather than just re-muxing it; this would take a lot more time, and could degrade the quality. Just re-muxing it preserves the frame rate issues of the .ts file, so the end result is the same.

I think we need to hope sub decides to fix this for us, though it won't be trivial.

Anyone else have any ideas?

[peter_h]

I'm a bit of newbie to .TS files and PVR, and being a bit of a techie too, I've been doing a lot of reading lately, trying to figure things out.

From what I've read (so if anybody wants to expand, please do), it's very common to have sync issues with transcoding .TS files that come from broadcasts (DVB-T). Something to do with it being packetized and packets arriving corrupted and/or because the audio may not necessarily arrive at the same time as the audio, particularly when the broadcaster switches streams (e.g. cuts from studio to outside broadcasts during live; inserts commercials).

It seems that many people recommend the first thing to do before any processing is fix any errors in the .TS stream, using tools like Quickstream Fix in VideoRedo v4 beta*; TS-Doctor; or TS Packet Editor (Unfortunately, all of which are US$50-$75. But trials are available).

* As yet, I've only tried VideoRedo v4 beta, but found that though it does handle h.264 but still has a problem writing out correct streams with HE-AAC, so I have deemed it not suitable yet for our DVB-T files.

Some TS theory: http://www.bitstreamtools.com/Theory/index.php

[pBS]

try: ffmpeg -y -i %1 -vcodec copy -acodec copy -f vob "%~dpn1.mpg"

you'll notice it takes a bit longer to produce and has slightly different file sizes...
and may fix your timing issues...

[petercooperjr]

try: ffmpeg -y -i %1 -vcodec copy -acodec copy -f vob "%~dpn1.mpg"

Thanks for the tip, but it didn't seem to help. It produces a slightly different file as you say, but after running that and then comskip on the new file, N-PVR still skips and resumes about 10 seconds early or so.

[Reddwarf]

try: ffmpeg -y -i %1 -vcodec copy -acodec copy -f vob "%~dpn1.mpg"

you'll notice it takes a bit longer to produce and has slightly different file sizes...
and may fix your timing issues...

would it be possible to include any subtitle streams to the .mpg file?

[petercooperjr]

I don't think there are any "subtitle" streams in the file, although the Closed Captioning data is certainly in there somewhere (and can get played by N-PVR after the remuxing, so it presumably gets through the process okay.

I don't think my original questions have been answered, on whether this is something sub plans on working on (he said he's not most interested in it, but I don't know if that's an outright "no, never"), or if there's something I can do to help. I'd be happy to try and take a look through the source code if sub wants to do that route (although I totally understand if he doesn't), and I'd be happy to contribute money toward the time to get this feature completely working. It looks like I'm not the only one experiencing issues with comskip skipping on TS files, so hopefully other people would be willing to contribute time and/or money as well.

Thank you very much. (And this is the only thing wrong with my setup now, and I'm loving the new program.) This is just the one thing that's not perfect yet.

[sub]

I don't think my original questions have been answered, on whether this is something sub plans on working on (he said he's not most interested in it, but I don't know if that's an outright "no, never")

I'm always looking to have better skipping and duration reporting, and there will always be on going improvements in this area over time.

Its not on the top of my priority list though, with so much other stuff still to be done in NPVR, and with skipping being "good enough" for most manual skipping.