ComClean cuts file size, but not length....

[erik]

Can SA cut on AR changes or only on black frames?

[reboot]

SA doesn't cut, it just detects
AFAIK (check with Jere), it uses AR, amongst other things, to detect "border" frames, then checks against the length, audio volume, etc., to determine if it's a commercial or not, then scores the block.
I know it does much more than just black frame detection!

[erik]

You are completely right!!!

Next to that I have implemented in comskip the ability to use AR changes to generate addition cutpoints in those cases where there is a logo on the show only.

In a typical broadcast here in Europe there are only some commercials and teasers separated by blackframes. And there may be anywhere upto 60 seconds of teaser/commercials before or after the show without black frame.
However the broadcasters use AR changes to tell us its "different" then the show.
So I use AR detection (more then just WIDESCREEN/FULLSCREEN) to determine where these teasers/commercials start and stop.
Without this I would and up with on average 2 minutes of leftover commercials/teasers which substantially reduces the WAF

I have explained all this to Jere and I assume he is considering to add this to SA, which I don't mind because I would like to use SA because it is faster then comskip but today the accuracy of SA is for me not good enough.

Now I can step down from my soap box and continue to debug the next comskip release :o

[reboot]

Have you run one of your more troublesome recordings through Schoolhouse?
I took the worst of the worst I could find, that comskip had serious trouble with, and using schoolhouse, I have it completely commercial free.
This is now the profile I use on US stations, because their logo usage is inconsistent, shows may be fullscreen and commercials not (instead of the other way around), and they never use black frames between.

[erik]

Let me first state that everyone should use what they prefer.
Next to that I am convinced that Jere did a very good job with comskip and now with SA. And I am sure SA will become even much better.
The real purpose for me continuing this discussion is to convince Jere to add these improvements to SA.

Yes, I did use schoolhouse, and I like it very much!!!!
Please take a look at the AR transitions of this recording

Code:

Block list after weighing
----------------------------------------------------
#     sbf  bs   be     fs     fe    sc     len    scc    scr  cmbnd   ar     cut  bri  logo        cc
  0)+    2   1    1      2  17689   0.02  707.480   0    0.00     0  1.64     a b   82  0.99              
  1)-    3   1    2  17691  18443   6.00   30.080   0    0.00     0  1.30       b   98  0.00              
  2)-    4   3    1  18444  18976   2.92   21.280   0    0.00     2  1.30     a    153  0.00              
  3)-    2   1    1  18977  19023   2.92    1.840   0    0.00     2  1.86     a     50  0.00              
  4)-    4   1    3  19024  19075   3.97    2.040   0    0.00     4  1.30       b  184  0.00              
  5)-    4   4    0  19076  19811  14.07   29.400   0    0.00     4  1.30       b   57  0.00              
  6)-   12   9    3  19825  20336  24.23   20.440   0    0.00     6  1.30       b  146  0.00              
  7)-    6   3    3  20337  21092  27.08   30.200   0    0.00     8  1.30       b   45  0.00              
  8)-    5   4    1  21093  21160  20.96    2.680   0    0.00    12  1.30     a     35  0.00              
  9)-    2   1    1  21161  21299  20.96    5.520   0    0.00    12  1.70     a     40  0.00              
10)-    4   1    3  21300  21598  15.40   11.920   0    0.00    10  1.30       b   62  0.00              
11)-    6   3    3  21599  21854  16.74   10.200   0    0.00     8  1.30       b  134  0.00              
12)-    6   3    3  21855  22610  25.64   30.200   0    0.00     6  1.30       b  149  0.00              
13)-    8   4    4  22611  22869  11.63   10.320   0    0.00     6  1.30      ub   88  0.00              
14)-    6   5    1  22870  23610  54.00   29.600   0    0.00     8  1.30     a b   90  0.00              
15)-    4   1    3  23612  23989  99.99   15.080   0    0.00    10  1.58     a b  116  0.00              
16)-    4   1    3  23993  24370  99.99   15.080   0    0.00    12  1.79     a b  102  0.00              
17)-    3   1    2  24374  24501  16.19    5.080   0    0.00    10  1.64     a b  120  0.00              
18)-    3   1    2  24505  25258  44.35   30.120   0    0.00     8  2.10     a b   89  0.00              
19)-    2   1    1  25262  25640  30.37   15.120   0    0.00     6  1.30       b   91  0.00              
20)-    2   1    1  25641  26270  11.63   25.160   0    0.00     4  1.30       b  117  0.00              
21)-    4   2    2  26271  26775  23.26   20.160   0    0.00     4  1.30       b  103  0.00              
22)-    6   3    3  26776  27156  15.09   15.200   0    0.00     4  1.30       b  105  0.00              
23)-    3   3    0  27157  27234   2.92    3.080   0    0.00     2  1.30       b   59  0.00              
24)-    2   1    1  27235  28019   2.92   31.360   0    0.00     2  1.30       b  103  0.00              
25)-    2   1    1  28020  28214   2.00    7.760   0    0.00     0  1.30     a     62  0.00              
26)-    2   1    1  28215  28669   1.00   18.160   0    0.00     0  1.64     a     42  0.00              
27)-    1   1    0  28670  28813   2.00    5.720   0    0.00     0  1.30     a b   81  0.00              
28)+    2   1    1  28815  45694   0.02  675.160   0    0.00     0  1.64     a     66  0.99              
29)-    4   1    3  45695  45806   2.00    4.440   0    0.00     0  1.47     aub   69  0.00              
30)-    1   1    0  45806  46524   2.00   28.720   0    0.00     0  1.30       b   82  0.00              
31)-    3   1    2  46525  47279  18.38   30.160   0    0.00     4  1.30       b   95  0.00              
32)-    5   2    3  47280  47659  45.58   15.160   0    0.00     8  1.30     a b  173  0.00              
33)-    4   1    3  47663  48540  17.45   35.080   0    0.00     8  1.58     a b   97  0.00              
34)-    7   1    6  48544  48675  34.68    5.240   0    0.00    12  1.30       b  131  0.00              
35)-    7   1    6  48686  49423  23.12   29.480   0    0.00    12  1.79     a b   65  0.00              
36)-    3   1    2  49431  49559  38.75    5.120   0    0.00    14  1.30       b  140  0.00              
37)-    3   2    1  49560  49742  12.46    7.280   0    0.00    10  1.30     a    102  0.00              
38)-    2   1    1  49743  49772   6.23    1.160   0    0.00    10  1.64     a     47  0.00              
39)-    4   1    3  49773  50066  12.46   11.720   0    0.00    10  1.30       b  102  0.00              
40)-    5   3    2  50067  50569  19.90   20.080   0    0.00     6  1.30       b   92  0.00              
41)-    6   3    3  50570  51325  10.34   30.200   0    0.00     2  1.30       b   81  0.00              
42)-    5   4    1  51326  51403   2.00    3.080   0    0.00     0  1.30     a     58  0.00              
43)-    2   1    1  51404  51991   2.00   23.480   0    0.00     0  1.70     a     48  0.00              
44)-    2   1    1  51992  52053   1.00    2.440   0    0.00     0  1.64     a     45  0.00              
45)+    1   1    0  52054  59729   0.30  307.000   0    0.00     0  1.30     a b   87  0.86

In particular pay attention to the transition from block 28 to 29 where the aspect ratio changes from 1.64 to 1.47.
When SA can detect this I can stop working on comskip.

[Jere_Jones]

FWIW, I am paying attention and will add the methods discussed and more. However, in about 2 hours my wife is flying in after being in Spain for the last couple months.

Jere

[reboot]

@ Jere_Jones, anxiously awaiting the next version...however, I suppose I can wait a few hours longer, while you get "reacquainted"

@ Erik, did you run the same file through schoolhouse, and what did it have to say about that block (too long/short, medium probability or???).
I wish I had a copy of that file, so I could try some of my SA profiles on it...
Edit: Would it be possible to meet me on IRC to upload it to me...unless you can think of a better way?

[erik]

Reboot,

You can download the file from my ftp site (you know the username/password already)
Its called lin_tester.mpg
I can not run it through SASchoolhouse because it causes SASchoolhouse to hangup my computer. The only method to stop SASchoolhouse is to reboot my computer. I hate rebooting :mad:

The choise to include or exclude the leadin/leadout with different aspect ratio is a matter of taste. Linfor wants them to be included, I would like them to be excluded because they are always the same. So its good to have an option to select what you want.

Jere, I will PM you the info of my ftp site so you can download the file and test SA with it.

[reboot]

Downloading now, thanks Erik
I'll let you know what settings I used to get everything removed.

[reboot]

Here's the successful settings.
First set: Includes the "teaser" at the beginning of the show, with it's corresponding w_teaser.txt so you can check the cut times.

Code:

Longest Possible Commercial Segment = 381
Longest Possible Individual Commercial = 91
Shortest Possible Commercial Segment = 9.5
Shortest Possible Show Segment = 180.5
[Score Modifiers]
Length Too Long = 0.1
Low Probability Length = 1.1
Length Too Short = 1
Medium Probability Length If Combined = 2
Medium Probability Length = 0.8
High Probability Length If Combined = 1.5
High Probability Length = 2
Edge Profile Positive Match = 1
Edge Profile Negative Match = 2
Edge Profile Unknown Match = 2
[Options]
Aggressive Segmentation = 0

Here are the settings to exclude the teaser at the beginning, with it's corresponding "noteaser.txt" for you to check the cuts.

Code:

Longest Possible Commercial Segment = 381
Longest Possible Individual Commercial = 91
Shortest Possible Commercial Segment = 9.5
Shortest Possible Show Segment = 180.5
[Score Modifiers]
Length Too Long = 0.1
Low Probability Length = 1.1
Length Too Short = 1
Medium Probability Length If Combined = 2
Medium Probability Length = 0.8
High Probability Length If Combined = 1.5
High Probability Length = 2
Edge Profile Positive Match = 0.1
Edge Profile Negative Match = 2
Edge Profile Unknown Match = 2
[Options]
Aggressive Segmentation = 0

Note that the only difference, is the edge profile matching.