Comskip timing with TS files and EDL

[sub]

You probably need the EDL with .ts files. It has logic to use the frame numbers for .mpg and .avi files, specifically looking in the file to determine the frame rate from the headers inside the file - I dont think this was extended to look for the frame rate in transport stream files.

It wont really improve your accuracy though, since all it does is use 'frame number' divided by the 'frames per second' when loading the .txt file to convert it into times. Its not counting frames as they play or anything like that.

[ceandra]

OK, thanks for the clarification.

I did not get to timing the show. Kids were busy using the system for a movie. I'll do it tomorrow.

Chuck

[ceandra]

OK, I did some timing with a stopwatch. I deleted the Comskip output files so that GBPVR would not skip commercials, and ran the NCIS show (same one that has zip file above) from beginning to end with a stopwatch.

It looks like the issue is in GBPVR's TS reader and its interpretation of the timeline.

When I look in Comskip GUI, the commercial breaks are hitting correctly as far as frame number (txt file). I also found that with stopwatch, the COmskip times in the EDL file are correct.

Here are some times I pulled with the stopwatch. Note that the TV (GBPVR) started at 7 seconds. I could not get it all the way to 0, so there is an offset throughout.

Stopwatch GBPVR
0 0:00:07
1:00 1:38
2:00 2:45
3:00 3:38
5:00 5:41
10:00 10:24
(commercial at about 11:00)
14:00 14:55
20:00 20:40
Commercial at 20:22 stopwatch, as predicted in EDL at 1222, but 21:02 on timeline
Commercial end at:
23:55 25:08
31:00 32:02 (mid commercial)
43:00 43:13
54:00 53:54
59:00 58:40
1:01:57 1:01:58

Several notes:
1. The end time (1:01:58) did not change at all form beginning to end
2. The timeline always picked up time during commercials, sometimes very dramatically. During post-show padding commercials, it was going almost double time.
3. The timeline was slower than stopwatch during show.
4. Initial commercial set things back about 30 seconds, hard to catch up in first half hours
5. Result is commercial cuts are coming 30 to 60 seconds early

COmskip seems to locate the commercials correctly, both in real time (EDL) and in frame number (checked in Comskip GUI). It appears the timeline does not flow consistently during the full hour. It is likely there is a different frame rate in the commercials (though, it might appear that the frame rate must be faster in commercial?).

Erik appears to have this issue licked. I know it was dead on with Haali too.

Well, the internal reader is definitely improving in stability, and the end time looks to be pretty good. But the intermediate positioning of the timeline does not seem to match reality very well.

Any other logs or files that might help?

I am using the latest TS patch (see above), the latest public Comskip (80.007). The channel is SD. I have kept the show above in case we need to go back and try anything, so we can keep a consistent frame of reference to help work this through. I doubt I could post the whole show, though. Too big.

Thanks
Chuck

[erik]

Could you set in your comskip.ini
output_timing=1
and rerun comskip , or the gui, does not matter which
Then there will be a large _timing.csv file that lists per frame the PTS,DTS values.
It can be used to see discontinuities in the timeline or changes in framerate
ZIp it and mail it to me (or post here)

[ceandra]

Erik:

Apparently comskip always makes the timing.csv files, as I have a bunch of them, and the setting was not in the ini.

However, I set the output timings, and ran again. I am posting all comskip output from that run. Sorry, I thought I had posted a zip above, but must have been late at night.

Oh. Now I get it. The zip is too big. I'll post it elsewhere and post you a link...

Chuck

[ceandra]

Erik (and Sub):

You can find the zip file at http://www.ncffafunfly.org/gbpvr/NCIS_20...002100.zip

This contains all of the comskip output for this latest run. Hope this helps in finding the issue.

Please let me know when it is OK to delete the file off the server. Zipped it is about 3.5MB, so its not a big deal on storage space.

Chuck

[erik]

Thanks, I downloaded the zip.
No relevant discontinuities
However at certain points in the recording the recording changes and uses repeated frames (don't know what they are actually called but the mpeg decoder tells me to repeat a frame).

This means that, to have a correct timeline, frames from the decoder are repeated to the display.

In total 4217 frames, about 140 seconds of video of repeated frames and 3575 seconds of real frames
Together its 1:01:58
Sub, does that make sense to you?

[sub]

GB-PVR doesnt count frames or anything like that. It needs to know how long the file is prior to processing the whole file, so it just has to estimate based on the size of the file and the average bitrate. This scheme generally works well but can be screwed up if the ad breaks during the show are using a lower bitrate etc.

Things are much easier when you're dealing with file types other than transport stream files, where you can count on the time line in the file. With transport streams the timeline is unreliable - the time line often wraps, or goes to a entirely different timeline for ads or when new programs start or broadcasts are stitched together.

You dont have this problem when recordings come out of analog hardware encoder, or when you use a mux that regenerates the timeline line (Cyberlink, DVR-MS muxes etc, but these have other problems). In these you have a consistent timeline from the beginning to end of the file so its pretty easy to know where you are and how long things are etc.

Like comskip, I know a lot about the file after I've finished playing it, but unfortunately its too late by then.

[ceandra]

This behavior does appear to be consistent on all shows on this channel, and also appears to happen (though not investigated as much) on other channels. Every show on CBS now ends up with timeline issues that cause commercial skips to start early. So, it is not isolated to this one example.

I beleive it worked OK with Haali. Should I try DVR-MS again? I really did not like that format, but it may be worth taking a look at it...

Thanks for taking a look, Erik. Hopefully this example will help improve the internal reader.

Chuck

[ceandra]

Sub:

Speaking as a naive observer....

When you get to teh commercial break time, if you are using EDL, you know at that point how much "real" time has passed, since you are playing the file. For example, at the 1222 commercial, the timeline is at 21:02 or so, but real time and EDL time are at 1222 (20:22). Could you not look at real time elapsed up to that point, and adjust the timeline based on that?

Of course, if one skipped ahead in the file, perhaps that information is lost.

Alternatively, could you use any of COmskip's information (say in the csv file) to "know" more about the file?

I have no real interest in having the timeline correct for timeline sake. It is only for COmskip, so perhaps someting that "fixes" the timeline based on Erik's output information?

Or, are frames identified (again, I don't know what I am talking about) as to frame number when reading the TS? IF so, could you use the .txt file and break for commercials based on frame number?

Chuck