ComClean cuts file size, but not length....

[erik]

Can SA cut on AR changes or only on black frames?

[reboot]

SA doesn't cut, it just detects
AFAIK (check with Jere), it uses AR, amongst other things, to detect "border" frames, then checks against the length, audio volume, etc., to determine if it's a commercial or not, then scores the block.
I know it does much more than just black frame detection!

[erik]

You are completely right!!!

Next to that I have implemented in comskip the ability to use AR changes to generate addition cutpoints in those cases where there is a logo on the show only.

In a typical broadcast here in Europe there are only some commercials and teasers separated by blackframes. And there may be anywhere upto 60 seconds of teaser/commercials before or after the show without black frame.
However the broadcasters use AR changes to tell us its "different" then the show.
So I use AR detection (more then just WIDESCREEN/FULLSCREEN) to determine where these teasers/commercials start and stop.
Without this I would and up with on average 2 minutes of leftover commercials/teasers which substantially reduces the WAF

I have explained all this to Jere and I assume he is considering to add this to SA, which I don't mind because I would like to use SA because it is faster then comskip but today the accuracy of SA is for me not good enough.

Now I can step down from my soap box and continue to debug the next comskip release :o

[reboot]

Have you run one of your more troublesome recordings through Schoolhouse?
I took the worst of the worst I could find, that comskip had serious trouble with, and using schoolhouse, I have it completely commercial free.
This is now the profile I use on US stations, because their logo usage is inconsistent, shows may be fullscreen and commercials not (instead of the other way around), and they never use black frames between.

[erik]

Let me first state that everyone should use what they prefer.
Next to that I am convinced that Jere did a very good job with comskip and now with SA. And I am sure SA will become even much better.
The real purpose for me continuing this discussion is to convince Jere to add these improvements to SA.

Yes, I did use schoolhouse, and I like it very much!!!!
Please take a look at the AR transitions of this recording

Code:

Block list after weighing ---------------------------------------------------- # sbf bs be fs fe sc len scc scr cmbnd ar cut bri logo cc 0)+ 2 1 1 2 17689 0.02 707.480 0 0.00 0 1.64 a b 82 0.99 1)- 3 1 2 17691 18443 6.00 30.080 0 0.00 0 1.30 b 98 0.00 2)- 4 3 1 18444 18976 2.92 21.280 0 0.00 2 1.30 a 153 0.00 3)- 2 1 1 18977 19023 2.92 1.840 0 0.00 2 1.86 a 50 0.00 4)- 4 1 3 19024 19075 3.97 2.040 0 0.00 4 1.30 b 184 0.00 5)- 4 4 0 19076 19811 14.07 29.400 0 0.00 4 1.30 b 57 0.00 6)- 12 9 3 19825 20336 24.23 20.440 0 0.00 6 1.30 b 146 0.00 7)- 6 3 3 20337 21092 27.08 30.200 0 0.00 8 1.30 b 45 0.00 8)- 5 4 1 21093 21160 20.96 2.680 0 0.00 12 1.30 a 35 0.00 9)- 2 1 1 21161 21299 20.96 5.520 0 0.00 12 1.70 a 40 0.00 10)- 4 1 3 21300 21598 15.40 11.920 0 0.00 10 1.30 b 62 0.00 11)- 6 3 3 21599 21854 16.74 10.200 0 0.00 8 1.30 b 134 0.00 12)- 6 3 3 21855 22610 25.64 30.200 0 0.00 6 1.30 b 149 0.00 13)- 8 4 4 22611 22869 11.63 10.320 0 0.00 6 1.30 ub 88 0.00 14)- 6 5 1 22870 23610 54.00 29.600 0 0.00 8 1.30 a b 90 0.00 15)- 4 1 3 23612 23989 99.99 15.080 0 0.00 10 1.58 a b 116 0.00 16)- 4 1 3 23993 24370 99.99 15.080 0 0.00 12 1.79 a b 102 0.00 17)- 3 1 2 24374 24501 16.19 5.080 0 0.00 10 1.64 a b 120 0.00 18)- 3 1 2 24505 25258 44.35 30.120 0 0.00 8 2.10 a b 89 0.00 19)- 2 1 1 25262 25640 30.37 15.120 0 0.00 6 1.30 b 91 0.00 20)- 2 1 1 25641 26270 11.63 25.160 0 0.00 4 1.30 b 117 0.00 21)- 4 2 2 26271 26775 23.26 20.160 0 0.00 4 1.30 b 103 0.00 22)- 6 3 3 26776 27156 15.09 15.200 0 0.00 4 1.30 b 105 0.00 23)- 3 3 0 27157 27234 2.92 3.080 0 0.00 2 1.30 b 59 0.00 24)- 2 1 1 27235 28019 2.92 31.360 0 0.00 2 1.30 b 103 0.00 25)- 2 1 1 28020 28214 2.00 7.760 0 0.00 0 1.30 a 62 0.00 26)- 2 1 1 28215 28669 1.00 18.160 0 0.00 0 1.64 a 42 0.00 27)- 1 1 0 28670 28813 2.00 5.720 0 0.00 0 1.30 a b 81 0.00 28)+ 2 1 1 28815 45694 0.02 675.160 0 0.00 0 1.64 a 66 0.99 29)- 4 1 3 45695 45806 2.00 4.440 0 0.00 0 1.47 aub 69 0.00 30)- 1 1 0 45806 46524 2.00 28.720 0 0.00 0 1.30 b 82 0.00 31)- 3 1 2 46525 47279 18.38 30.160 0 0.00 4 1.30 b 95 0.00 32)- 5 2 3 47280 47659 45.58 15.160 0 0.00 8 1.30 a b 173 0.00 33)- 4 1 3 47663 48540 17.45 35.080 0 0.00 8 1.58 a b 97 0.00 34)- 7 1 6 48544 48675 34.68 5.240 0 0.00 12 1.30 b 131 0.00 35)- 7 1 6 48686 49423 23.12 29.480 0 0.00 12 1.79 a b 65 0.00 36)- 3 1 2 49431 49559 38.75 5.120 0 0.00 14 1.30 b 140 0.00 37)- 3 2 1 49560 49742 12.46 7.280 0 0.00 10 1.30 a 102 0.00 38)- 2 1 1 49743 49772 6.23 1.160 0 0.00 10 1.64 a 47 0.00 39)- 4 1 3 49773 50066 12.46 11.720 0 0.00 10 1.30 b 102 0.00 40)- 5 3 2 50067 50569 19.90 20.080 0 0.00 6 1.30 b 92 0.00 41)- 6 3 3 50570 51325 10.34 30.200 0 0.00 2 1.30 b 81 0.00 42)- 5 4 1 51326 51403 2.00 3.080 0 0.00 0 1.30 a 58 0.00 43)- 2 1 1 51404 51991 2.00 23.480 0 0.00 0 1.70 a 48 0.00 44)- 2 1 1 51992 52053 1.00 2.440 0 0.00 0 1.64 a 45 0.00 45)+ 1 1 0 52054 59729 0.30 307.000 0 0.00 0 1.30 a b 87 0.86

In particular pay attention to the transition from block 28 to 29 where the aspect ratio changes from 1.64 to 1.47.
When SA can detect this I can stop working on comskip.

[Jere_Jones]

FWIW, I am paying attention and will add the methods discussed and more. However, in about 2 hours my wife is flying in after being in Spain for the last couple months.

Jere

[reboot]

@ Jere_Jones, anxiously awaiting the next version...however, I suppose I can wait a few hours longer, while you get "reacquainted"

@ Erik, did you run the same file through schoolhouse, and what did it have to say about that block (too long/short, medium probability or???).
I wish I had a copy of that file, so I could try some of my SA profiles on it...
Edit: Would it be possible to meet me on IRC to upload it to me...unless you can think of a better way?

[erik]

Reboot,

You can download the file from my ftp site (you know the username/password already)
Its called lin_tester.mpg
I can not run it through SASchoolhouse because it causes SASchoolhouse to hangup my computer. The only method to stop SASchoolhouse is to reboot my computer. I hate rebooting :mad:

The choise to include or exclude the leadin/leadout with different aspect ratio is a matter of taste. Linfor wants them to be included, I would like them to be excluded because they are always the same. So its good to have an option to select what you want.

Jere, I will PM you the info of my ftp site so you can download the file and test SA with it.

[reboot]

Downloading now, thanks Erik
I'll let you know what settings I used to get everything removed.

[reboot]

Here's the successful settings.
First set: Includes the "teaser" at the beginning of the show, with it's corresponding w_teaser.txt so you can check the cut times.

Code:

Longest Possible Commercial Segment = 381 Longest Possible Individual Commercial = 91 Shortest Possible Commercial Segment = 9.5 Shortest Possible Show Segment = 180.5 [Score Modifiers] Length Too Long = 0.1 Low Probability Length = 1.1 Length Too Short = 1 Medium Probability Length If Combined = 2 Medium Probability Length = 0.8 High Probability Length If Combined = 1.5 High Probability Length = 2 Edge Profile Positive Match = 1 Edge Profile Negative Match = 2 Edge Profile Unknown Match = 2 [Options] Aggressive Segmentation = 0

Here are the settings to exclude the teaser at the beginning, with it's corresponding "noteaser.txt" for you to check the cuts.

Code:

Longest Possible Commercial Segment = 381 Longest Possible Individual Commercial = 91 Shortest Possible Commercial Segment = 9.5 Shortest Possible Show Segment = 180.5 [Score Modifiers] Length Too Long = 0.1 Low Probability Length = 1.1 Length Too Short = 1 Medium Probability Length If Combined = 2 Medium Probability Length = 0.8 High Probability Length If Combined = 1.5 High Probability Length = 2 Edge Profile Positive Match = 0.1 Edge Profile Negative Match = 2 Edge Profile Unknown Match = 2 [Options] Aggressive Segmentation = 0

Note that the only difference, is the edge profile matching.