Comclean plugin : automatic deletion of commercials

[erik]

This is the first release of comclean, it can be downloaded here
It is a utility to be used inside your postprocessing.bat to automatically delete the commercial segments found by comskip.
The cutter used (mpgtx) is very fast and very small and it runs with a low CPU load
The only limitation is that cutting is done at GOP boundaries (accuracy about 1 second)
Please read the readme.txt inside the zip file for more details.
Have fun!

[reboot]

Why 30fps? NTSC is 29.97fps. One would think that 3/100ths doesn't make much difference (if any), but on long recordings (3+ hours), something will probably go wrong.
Can this be made to work with comskiphelper?
Is there any way to increase the number of I frames in the recording?
I know a standard GOP is 18 frames, with one I frame, which would leave a cut mark a little more than every half second, but it would be nice to have more accuracy.

I'm going to try this with a framerate of 29.97 and see if it works.

[erik]

I have to admit that I was too layzy to test NTSC. :o
You are right, the framerate is 29,97 but...... the cmd processor only can do math on integers so I will change the script to work with the framerate times 100.

So set framerate to 2997 and change line 62 to
set /A position=%1 * 100 / %framerate%

Comskiphelper searches and runs the comskip.exe file with the .mpg file as a parameter. ZaDDaz could add the running of comclean to Comskiphelper.
Or you could patch comskiphelper.exe at offset 0x26c3 to bat in unicode.
This will run comskip.bat instead of comskip.exe allowing you to do whatever you want.

I am not aware of any method to decrease the size of the GOP, that's why videoredo and MPEG-VCR can do frame precise cutting by re-encoding.
I like to have that too but mpgtx is the only fast cmdline cutter I found.
Any suggestions for a better cutter are welcome.

[reboot]

I had to name the actual path to mpgtx as c:\Program Files\devnz\gbpvr\mpgtx.exe to get it to run.
I also modified the rename recording bit, so that the original is maintained to compare (for testing).

Rem -----------------------------------------Renamerecoding fix -----------------------------------------
Rem RENAME the old .mpg file and rename the new one to the old name
Rem Uncomment next 2 lines to make renamerecoding work!!!!!!!
move "%~dpn1.mpg" "%~dpn1_original.mpg"
move "%~dpn1_clean.mpg" "%~dpn1.mpg"
Rem ------------------------------------------Renamerecoding fix -----------------------------------------

Even with the I frame cutting, it seems to work extremely well, and is also super fast!
Comskip takes about 7 minutes per hour of recording, yet mpgtx takes less than a minute. I doubt you'll find a better cutter, and for the fussy ones, we can still manually edit the file to get that half second precision if needed.

[reboot]

Just did a test on Stargate SG-1 (Space network).
Excellent results. It caught the beginning of the show perfectly, didn't leave any commercial bit's within the show. Didn't cut ANY of the actual show, but left about 30 seconds of junk at the end, leading into the next show.
I think this is a comskip thing though, as it tends to use the logo more on this channel. I think I can compensate by decreasing the pre and post overlap a bit.

[jorm]

I have not had a chance to try it yet but it looks very useful.

Good job Erik.

Now all we need is a plugin to allow you to check and modify the comskip cut points.

I actually have a request to colin whom has a tool that gets the iframe from a mpeg as a image, if I can get that to work I might be able to build an editor.

[reboot]

I think my results were so good, because Space always fades to black before a commercial, then fades from black after.
Comskip seems to select a cut one I frame later than needed before a commercial, and cut one I frame earlier at the end of the commercials. This is probably on purpose. It's easy to edit if needed, but still very accurate in general (the mpgtx cuts).
Yes, there always seems to be some residue at the end of the recording...again, adjusting the overlap to 0 minutes in config, fixed this up nicely. When scheduling, I will have to remember which channels NEED the extra minute or two (because their clocks are all wrong in the US), and which don't (Canadian stations)

I'm sticking with this for now, instead of the convoluted womble method.
I'm going to write a .bat that uses this method on most recordings (it'll be default), and use the womble method on others (probably based on recording length).
This way, half, and one hour shows will get the default, but anything longer will get the womble method. This guarantees more accurate cuts for those shows that will (probably) be burned to DVD.

Tipster, I don't understand why you have to press Play to skip commercials. The mpgtx output should have no commercials, except maybe at the end.

[erik]

Tipstir
Comclean reads the .txt file generated by comskip to find the cut points.
So if you can skip the commercials while playing (and they are indeed the original commercials as indicated by comskip) than you are not looking to the cleaned file but to the original file.
If you use the .txt file to skip commercials on the cleaned file you will skip to the mid of a show segment for the first cut, and the other skips are all wrong.
So indeed the first test was done on the non cleaned files!
Comskip is not perfect, therefore it missed some commercial in your second test.
In this second test you looked at the cleaned file because you used the rename adaptation from reboot.

[reboot]

Yeah, what he ^ said

I ran another test, perfect results.
Ran another, a 1 second commercial bit in the middle, and some at the end (different channel!)
Running another 3 tests, on different channels now.
I'll let you know how it all works out.

The cutting is flawless. It's comskip's output that's flawed.
Time to tweak the comskip.ini

[erik]

For you convenience I have created version 0.0.1 with the adaptations suggested by reboot.
It can be dowloaded here