TV intro skipping

[mvallevand]

I've been looking into automatically skipping intros since I know this is a feature that Plex users like.  Some of you have probably used the feature on Netflix where you get to intro and there is a skip button.  I researched this and there are a few ways to do it but I found is using an open source netcore 6 library on github called soundfingerprinting it is actually not as hard to implement as I originally thought.

To be fair I have only tested this with this two shows, "Star Trek: Picard" which is an annoying at almost 2 minutes and "The Big Bang Theory" which is probably too short to reach for the remote at 20 seconds but all tests have been spot one.

The issues I see would be

- creating the mp3 for the intro.  I got "The Big Bang Theory" from here https://www.televisiontunes.com/ and it was pretty easy manually creating one is a little bit harder.  It would need to be a wav for the database but the mp3 is easier to distribute.

- distributing the mp3s

- matching the program title (especially with OTA data)

- It won't need comskip but I will need to be able merging the skip into the comskip edl file.  Note for Kodi I'd have have option to have it a manual or automatic skip.

Is this something any of you are interested in?   If so I'll post some more instructions on how I'd like some sample files to test this.

Martin

[sub]

The funny thing is, I work with the soundfingerprinting project creator regularly in my day job. He’s a good guy and very smart.

It’s a great idea to use it this way to skip intros.

You may not need to distribute the actual mp3/wav files of these intros, you might be able to generate data files containing the pre-generated fingerprint data. (avoiding any copyright issues etc)

[mvallevand]

We will see how it goes for now I am using Sergui's Emysound free service https://emysound.com/pricing/ since for my own needs 10 units isn't bad. I could probably host it on VPS docker if there is any interest here. I have the addon doing a mix of local and remote lookups too so I can handle a few shows like TBBT, GoT, Star Trek's for testing.

I'd have to figure out how to distribute the tracks with the fingerprint and metadata but could do that too, but there still is a distribution issue.

I guess I will see if there is any demand at all.

Martin

[BrettB]

Is this something any of you are interested in?   If so I'll post some more instructions on how I'd like some sample files to test this.

Martin,

I normally don't find the "intros" too bad since I can typically just hit the right arrow once or twice to skip over them. (Plus, these days, it seems like most shows don't even have theme songs/intros any more.) As you said, there are a couple of shows where it could be very nice.

That said, I do think that it is an interesting feature. And I'd be more than willing to help test, provide data, etc.

-Brett

[mvallevand]

It does depend on the show for sure, because of start with comksip I typically will have the remote in my hand but I have a minute skip and I find most of the intros are under 30 seconds. I think NCIS is about 22 seconds, so there really isn't a remote friendly way to skip that. This guy's Plex example was how I feel https://www.youtube.com/watch?v=wzGNXirUj0c&t=290s

"Star Trek: Picard" was the one that got me thinking about it because it is long just under 2 minutes and it can be up to 10 minutes into the show when I am not holding the remote anymore.

Do you have any shows you might binge watch in the next few weeks or do you keep up to date? I'll see if I can record some for comparison.

Martin

[mvallevand]

I have decided to increase the scope of this to include credits, if I find credits in the prepadding area it will autoskip the end of the credits.

Martin

[mvallevand]

So it works out pretty well I started testing with Star Trek Discover today and it is like Picard I also tested outro processing since I see too uses for that, skipping the outro in prepadding and skip to end on outtro new post padding. I also include a couple of tests for little sound bites on these broadcasts (This broadcast is in described video, this broadcast has coarse language etc) and that works most of the time.

Serializing and de-serializing the fingerprints for distribution works too.

Code:

Star Trek Discovery\Season 04\Star Trek Discovery.S04E01.Kobayashi Maru.ts
Track Discovery matched. Started: 718.0841674804688 Coverage: 88.52 seconds
Track DV matched. Started: 58.39187240600586 Coverage: 5.76 seconds
Track Coarse matched. Started: 68.62118530273438 Coverage: 5.94 seconds

Star Trek Discovery\Season 04\Star Trek Discovery.S04E02.Anomaly.ts
Track Discovery matched. Started: 825.126220703125 Coverage: 88.71 seconds
Track Coarse matched. Started: 71.1291732788086 Coverage: 5.94 seconds
Track DV matched. Started: 60.91146469116211 Coverage: 5.94 seconds
Track STDS9 Outtro matched. Started: 12.656023979187012 Coverage: 46.54 seconds

Star Trek Discovery\Season 04\Star Trek Discovery.S04E03.Choose to Live.ts
Track Discovery matched. Started: 517.84033203125 Coverage: 87.87 seconds
Track DV matched. Started: 64.94049072265625 Coverage: 5.57 seconds
Track STDS9 Outtro matched. Started: 16.52249526977539 Coverage: 46.44 seconds

Star Trek Discovery\Season 04\Star Trek Discovery.S04E04.All Is Possible.ts
Track Discovery matched. Started: 646.0609741210938 Coverage: 88.34 seconds
Track Coarse matched. Started: 58.11320877075195 Coverage: 5.57 seconds
Track DV matched. Started: 58.00870895385742 Coverage: 6.22 seconds
Track STDS9 Outtro matched. Started: 11.64586353302002 Coverage: 42.91 seconds

Star Trek Discovery\Season 04\Star Trek Discovery.S04E05.The Examples.ts
Track Discovery matched. Started: 478.4557189941406 Coverage: 88.62 seconds
Track DV matched. Started: 63.47750473022461 Coverage: 5.48 seconds
Track STDS9 Outtro matched. Started: 15.326560974121094 Coverage: 46.63 seconds

Star Trek Discovery\Season 04\Star Trek Discovery.S04E06.Stormy Weather.ts
Track Discovery matched. Started: 675.5645751953125 Coverage: 88.52 seconds
Track DV matched. Started: 59.34397888183594 Coverage: 9.47 seconds
Track Coarse matched. Started: 59.460086822509766 Coverage: 6.32 seconds

Star Trek Discovery\Season 04\Star Trek Discovery.S04E07.But to Connect.ts
Track Discovery matched. Started: 308.8533935546875 Coverage: 88.71 seconds
Track DV matched. Started: 59.1698112487793 Coverage: 5.85 seconds
(failed recording)

Star Trek Discovery\Season 04\Star Trek Discovery.S04E08.All In.ts
Track DV matched. Started: 58.252540588378906 Coverage: 5.76 seconds
Track STDS9 Outtro matched. Started: 9.915817260742188 Coverage: 46.26 seconds
Track Coarse matched. Started: 68.47024536132812 Coverage: 5.67 seconds

Star Trek Discovery\Season 04\Star Trek Discovery.S04E09.ts
Track Discovery matched. Started: 734.1422729492188 Coverage: 87.69 seconds
Track STDS9 Outtro matched. Started: 10.25253677368164 Coverage: 46.26 seconds
Track DV matched. Started: 59.27431106567383 Coverage: 9.57 seconds
Track Coarse matched. Started: 59.390419006347656 Coverage: 9.57 seconds

Star Trek Discovery\Season 04\Star Trek Discovery.S04E10.ts
Track Discovery matched. Started: 923.0421142578125 Coverage: 88.62 seconds
Track STDS9 Outtro matched. Started: 12.249637603759766 Coverage: 47.00 seconds
Track DV matched. Started: 60.563133239746094 Coverage: 6.22 seconds
Track Coarse matched. Started: 60.67924118041992 Coverage: 6.32 seconds

Star Trek Discovery\Season 04\Star Trek Discovery.S04E11.Rosetta.ts
Track Picard Outtro matched. Started: 605.6893920898438 Coverage: 58.24 seconds
Track DV matched. Started: 665.5906982421875 Coverage: 6.32 seconds
(failed recording)

Star Trek Discovery\Season 04\Star Trek Discovery.S04E12.Species Ten-C.ts
Track Discovery matched. Started: 605.2017822265625 Coverage: 88.52 seconds
Track DV matched. Started: 65.31204986572266 Coverage: 5.76 seconds
Track Picard Outtro matched. Started: 4.957912445068359 Coverage: 6.50 seconds

Star Trek Discovery\Season 04\Star Trek Discovery.S04E13.Coming Home.ts
Track Discovery matched. Started: 660.6444091796875 Coverage: 88.43 seconds
Track DV matched. Started: 66.2757568359375 Coverage: 6.13 seconds
Track Picard Outtro matched. Started: 6.095790863037109 Coverage: 5.94 seconds

[sub]

Nice!

[gEd]

Sounds like a very neat idea and I'd certainly use it to add "slickness" to NPVR as long as there are no fees involved.
However my TV recordings are all transcoded to work with the UI client on my TV so I don't know if that would present any potential issues.
Would the idea be to have a skip intro button on the remote to user can decide if they want to skip or not? (there are some shows that my wife watches that have short intros that I think she likes to leave running while she gets "comfy"

[mvallevand]

No fees planned which means downloading the fingerprints is manual. My utility is a bit like comskip so it runs standalone and creates/merges edl files so far I basically I have tested with ts an mp4 files. It would need to run after comskip for the merged since comskip would trash it.

If you aren't using comskip you shouldn't have a problem buit NextPVR doesn't really handle EDL files to spec, it is either auto skip or not. For intro skip, there would have to be a few changes to make it work. Ideally sub could give an Intro OSD but this is far more in the prototype stage at this point for that. The other change is to support EDL actions

0 - Cut
1 - Mute
2 - Scene Marker (if start and end times are specified, the end time is used)
3 - Commercial Break

I see the beginning and end of the intro as being screens.

The best thing to PM me the first 5 or 10 minutes of a show (-t 00:mm:00 with ffmpeg) and I can see what I can do with it. Alternatively if the show has standard end credits just PM the whole thing.

Martin