Tuning of Comskip Part 2, ver 0.69

[reboot]

After running tests for 8 hours, on various channels, I find 0.69 does indeed run faster (than previous versions), however, I had to tweak my comskip.ini black frame detections slightly. The comskip.ini included in the download doesn't work for me at all, and I had to manually merge the newer detctions, with my old ones as well.
Overall, a good upgrade, including better logo detection, and the aspect ratio detection does help.
Well done Erik

[erik]

Thanks :o

Any speed figures?

On my 1.8 GHz laptop it runs a 2:15 hours movie at 350 fps in 9.6 minutes with detect_method=43 and verbose=10
My 2.4GHz Pentium is slower :confused:

[King Mob]

I have a question. I'm using comskip v.69, and my log shows this:

After Sorting - 31
--------------
Block 6 - 0.001 4128 18021 33.5
Block 0 - 0.100 1 2021 37.2
Block 16 - 0.100 28024 17433 69.6
Block 28 - 0.178 49524 1142 71.7
Block 34 - 1.500 53609 192 72.1
Block 35 - 1.500 53801 64 72.2
Block 33 - 1.688 53433 176 72.5
Block 21 - 1.944 47715 399 73.3
Block 22 - 1.944 48114 60 73.4
Block 11 - 2.000 24860 466 74.3
Block 12 - 2.000 25332 779 75.7
Block 13 - 2.000 26111 63 75.8
Block 14 - 2.000 26174 950 77.6
Block 23 - 2.569 48174 788 79.0
Block 24 - 2.569 48962 61 79.2
Block 25 - 2.569 49023 73 79.3
Block 29 - 2.676 50666 65 79.4
Block 1 - 3.750 2028 759 80.8
Block 26 - 4.390 49096 119 81.0
Block 27 - 4.588 49215 309 81.6
Block 5 - 4.845 3828 300 82.2
Block 15 - 5.143 27124 900 83.8
Block 7 - 5.625 22156 84 84.0
Block 8 - 5.625 22240 367 84.7
Block 2 - 5.664 2787 224 85.1
Block 3 - 5.664 3011 367 85.8
Block 17 - 5.833 45465 760 87.2
Block 18 - 5.833 46225 139 87.5
Block 20 - 5.833 47264 451 88.3
Block 10 - 7.500 23957 903 90.0
Block 19 - 7.778 46365 899 91.6
Block 32 - 9.000 52083 1350 94.1
Block 9 - 10.000 22607 1349 96.7
Block 30 - 13.500 50731 452 97.5
Block 31 - 13.500 51183 900 99.2
Block 4 - 14.636 3378 450 100.0
The 63.00 percentile of 53835 frames is 0.10
Threshold used - 0.1000 After rounding - 0.1000

In the ini percentile=0.67, so shouldn't this be the 67th percentile? This is affecting commercial deletion, as block 28 of this recording is a show segment. Actually what I'd like to do is raise this to 0.71. How do I change this setting? I've tried changing percentile from 0.67 to 0.71, and adding score_percentile=0.71, with no change. What am I doing wrong?

[erik]

You don't do anything wrong.

I added a feature that turns out to be a bug.
When a lot of logo is found I use the logo to set the percentile.
This works if almost all of the show has logo. Yours does abviously not have that

My new version will allow you to do a hard setting of the percentile or use auto setting from the logo.

As a workaround you could use
global_threshold=0.5
in the ini file which will do a hard setting of the scoce level above which it is assumed to be a commercial.
However is just a temporary workaround untill I release the new version.
Hopefully tomorrow.

[reboot]

I have my percentile=0.70 which is slightly less than the 0.71 default. I seem to get less commercials this way, and better detection of the logo.
I also use two_pass_logo=1, which works better than a single pass.
Using logo_present_modifier = 0.01, to increase the priority of a frame containing the logo, as well as severly tested black frame detection numbers, I'm getting 100% detection through the show, but 0 detection of pre and post padding commercials.

Questions:
1) What does this mean, and how can I use it, what setting modifies it?: "and uniform of 152953"

2) In the "After Sorting" list, what do all these numbers mean?

Block 25 - 0.001 37916 16824 58.2

I know the first set is the scoring...what's the 37916, 16824 and 58.2?

How can I use the ar_split=1.45 to my advantage, when 99% of everything I record is full screen, but many commercials are not?
What about a movie that is letterboxed, yet commercials are fullscreen?

For input, here's my comskip.ini file. I realize that my black detection levels are high, and this may result in oversegmenting, but I do get much better detection overall, using these numbers, even when the ACTUAL brightness/black level of a truly black frame is about 17.

Code:

detect_method=47    ;1=black frame, 2=logo, 4=scene change, 8=fuzzy logic, 16=closed captions, 32=aspect ratio, 255=all
verbose=10        ;show extra info
fps=29.970        ;frames per second (NTSC=29.970, PAL=25)
border=20         ;border around edge of video to ignore
two_pass_logo=1        ;run comskip a second time, once the logo is correctly identified
max_brightness=140      ;frame not black if any pixels checked are greater than this (scale 0 to 255)
test_brightness=65      ;frame not pure black if any pixels checked are greater than this, will check average brightness (scale 0 to 255)
max_avg_brightness=60    ;maximum average brightness for a dim frame to be considered black (scale 0 to 255)
max_commercialbreak=421    ;maximum length in seconds to consider a segment a commercial break
min_commercialbreak=19    ;minimum length in seconds to consider a segment a commercial break
max_commercial_size=331    ;maximum time in seconds for a single commercial or multiple commercials if no breaks in between
min_commercial_size=9  ;mimimum time in seconds for a single commercial
min_show_segment_length=121
logo_present_modifier = 0.01
live_tv=0
ccCheck=0
delete_logo_file=0
output_default=1    ;output the default .txt file
output_videoredo=0    ;output a video redo file
min_black_frames_for_break=1;
play_nice_start=0720    ;at what time of the day should comskip slow itself down
play_nice_end=0730    ;ending time for self slowing
REM new things added here
percentile=0.70
ar_split=1.45
output_framearray=0
output_womble=0
output_mpgtx=1
output_zoomplayer_chapter=0
output_zoomplayer_cutlist=0
after_logo=0
before_logo=0

[erik]

I have my percentile=0.70 which is slightly less than the 0.71 default.

In the 0.69 version the percentile is not used when sufficient logo is found.
The percentile of logo is used to determine the percentile

I also use two_pass_logo=1, which works better than a single pass.

This is now the default and it indeed works better

1) What does this mean, and how can I use it, what setting modifies it?: "and uniform of 152953"

I am working on the detection of uniform brightness frames (like black but also dark blue or pure white frames)
In the next version it will be enabled.
The figure for the pure dark frames is a measure for the noise level.

Block 25 - 0.001 37916 16824 58.2

0.001 The score of block 25 (just like in the block list above)
37916 The start frame of the block (just like in the block list above)
16824 The length of the block
58.2 The running total of the percentage of the total runtime of the recording. The block that exceeds the percentile*100 will be the last block in this list to be regarded as show.

How can I use the ar_split=1.45 to my advantage, when 99% of everything I record is full screen, but many commercials are not?

Aspect Ratio looks only to the black bars above and below the picture
full screen is 1.3
DVD wide screen is 1.7
Anything above ar_split will be regarded as WIDESCREEN, anything below as FULLSCREEN
This figure used to be fixed but commercials are broadcasted with all kind of strange aspect ratios.

Now on the black frame detection:
There are two kind of measures
A single pixel brightness
The total average frame brightness.

Any single pixel above max_brightness will make the frame not black
More then a certain number of pixels with brighness above test_brightness will make the frame not black
If the average brightness of the frame is above max_avg_brightness then it is not black.

So the order should be
max_avg_brightness < test_brightness < max_brightness

Default values are
25 < 60 < 100

Your value for max_avg_brightness seems very high.
Could you mail me a log file with verbose set to 10?
The histogram in the log file will tell you how to set the max_avg_brightness
An example

Show Histogram - 0.06534
0 - 0 - 0.00000 *
1 - 0 - 0.00000 *
2 - 0 - 0.00000 *
3 - 0 - 0.00000 *
4 - 0 - 0.00000 *
5 - 0 - 0.00000 *
6 - 0 - 0.00000 *
7 - 0 - 0.00000 *
8 - 0 - 0.00000 *
9 - 0 - 0.00000 *
10 - 0 - 0.00000 *
11 - 0 - 0.00000 *
12 - 0 - 0.00000 *
13 - 0 - 0.00000 *
14 - 1 - 0.00001 *
15 - 140 - 0.00145 **********
16 - 455 - 0.00612 ******************************
17 - 121 - 0.00737 ********
18 - 54 - 0.00792 ****
19 - 150 - 0.00946 **********
20 - 106 - 0.01055 *******
21 - 136 - 0.01195 *********
22 - 82 - 0.01279 ******
23 - 144 - 0.01427 **********
24 - 373 - 0.01810 *************************
25 - 221 - 0.02037 ***************
26 - 244 - 0.02288 ****************

455 frames have average brightness of 16.
The figure after the count is the running sum of the percentile of frames with that brightness.
If you have put the max_avg_brightness very high then the default for black_percentile = 0.0073 is used to determine where to cut off the black frames.
In this case anything above 16 will not be used as black frame.
This save you from getting way too many black frames
Look at the line

Setting brightness threshold to xx

below the histogram.
The figure in xx is the actual cut off value for black frames.
There should be no reason to put your max_avg_brightness much above this value.

Ido hope someone is collecting all my posts and constructing a FAQ

[erik]

I have build version 0.70.
It can be downloaded here

Uniform brightness frames such as black, blue or white frames, can now be used as cutpoints.
I can use some feedback if this actually works because it will depend on the TV signal noise levels around the world.
please set verbose=10 in the ini file and mail me a log file

[reboot]

Going by the histogram, I should set my max_avg_brightness to 17, however...If I do this, I get poor results on channels WITH a logo. Leaving it at 65 or so, gives more segmenting, but more accuracy. On one particular channel (the most important one), I get 100% with my black frame detection numbers this high.
On channels without a logo, I can't seem to get anything working correctly.

I do have version 0.64 tuned to work extremely well for me, so I still have a backup that works almost perfectly on all channels, logo or not.

I will try 0.70 though, and let you know the results later, or tomorrow.
Instead of mailing you a log, I'll just do some tests with different numbers, and let you know how the segmenting versus actual detection is going.
On channels with a logo, ver 0.69 worked rather well, with some tweaking of black frame detection to slightly lower numbers, much lower than 0.64 needed for accuracy.
Without a logo, 0.69 failed miserably, probably because of my black frame numbers, so I'm relying far more on logo detection. Combine this with aspect detection, and I should have near-perfect, with black frame as a backup, instead of the primary detection method.
If I must use black frame as the primary detection, then a lot more tuning is needed on my part, and obviously the high numbers I have won't work well.

EDIT: Just ran a test, and with detect method=47, verbose=10, it only detected one commercial break in a 2 hour movie. It also only included scene change detection in the log. Why isn't there black frame, logo, aspect, and fuzzy included in the log?
Bug?

[reboot]

Verbose 10 just doesn't show log stuff like I'm used to.
According to the histogram, my black level (on this recording) is 21. I currently have max_avg_brightness=25 and it seems to work.

Code:

cut in----------
14 -      0 - 0.00000 *
15 -      3 - 0.00006 *
16 -    173 - 0.00335 ************
17 -     26 - 0.00385 **
18 -     14 - 0.00411 *
19 -     91 - 0.00585 *******
20 -     41 - 0.00663 ***
21 -     36 - 0.00731 ***
cut------------
Setting brightness threshold to 21

The block list after weighing, has a very low threshold, but did accurately detect 3 commercial blocks.

Code:

After Sorting - 23
--------------
Block  12 - 0.001      8576     16825    32.1
Block  23 - 0.001     29062     20737    71.7
Block   0 - 0.100         1      5831    82.9
Block   3 - 1.485      6197       101    83.0
Block  22 - 1.485     28936       103    83.2
Block   1 - 1.492      5846       187    83.6
Block  20 - 1.492     28584       188    84.0
Block  25 - 1.498     51612       884    85.6
Block  13 - 1.498     25409       924    87.4
Block  24 - 1.499     49823      1789    90.8
Block  11 - 2.096      8051       512    91.8
Block   2 - 2.235      6033       149    92.1
Block  21 - 2.235     28772       149    92.4
Block   9 - 2.429      7869       116    92.6
Block   7 - 2.432      7650       139    92.9
Block   8 - 2.450      7791        78    93.0
Block  10 - 2.450      7988        63    93.1
Block  17 - 2.983     27457       180    93.5
Block  18 - 2.994     27637       495    94.4
Block  14 - 4.042     26333       521    95.4
Block   4 - 4.495      6298       853    97.0
Block   5 - 5.250      7151        48    97.1
Block  19 - 5.612     28134       449    98.0
Block  16 - 6.720     27234       223    98.4
Block   6 - 8.149      7199       451    99.3
Block  15 - 9.088     26854       380    100.0
The 71.00 percentile of 52375 frames is 0.00
Threshold used - 0.0010    After rounding - 0.0100

This works extremely well, on channels that have a logo.

Code:

Logos detected on the following frames
--------------------------------------
Logo start -    203    end -   5684    length -  0: 3:2.88    before:6.7 s     after:4.9 s
Logo start -   8642    end -  10150    length -  0: 0:50.32    before:2.2 s     after:508.8 s
Logo start -  10614    end -  25346    length -  0: 8:11.56    before:68.0 s     after:1.8 s
Logo start -  29087    end -  49706    length -  0:11:27.99    before:0.8 s     after:3.1 s
Logo detected on 80.66 percent (42344 out of 52496) of the frames.

The percentile is high, because this was a show in progress when I started the recording, yet comskip figured it out nicely.

[reboot]

I have one show, which I have been using for testing. It has a distinct logo, and comskip find's it accurately, and easily.
It accurately finds all commercials, except one block.
Show length is an hour, and here's the cut list:

Code:

1 - start:      8       end:   1830     length:  0: 1:0.79
2 - start:   8015       end:  11637     length:  0: 2:0.85
3 - start:  30190       end:  33799     length:  0: 2:0.42
4 - start:  53946       end:  57554     length:  0: 2:0.39
5 - start:  70100       end:  73722     length:  0: 2:0.85

There is a commercial block, beginning and ending with black frames, from frames 91482 to 95096, and it cannot detect this block as a commercial block, no matter what settings I try and use.
There is a logo in the frames pre, and post this block.

Which setting should I adjust, and how, to get this detection?