Getting Pluto extra to work

[mvallevand]

@bouilliaba, thanks for the info on the recent streamlink change. I have updated the wiki with these. It turns out the version I had did have some S/E logic so I included that for everyone.

I also included some changes I had been working on for the xmltv-update minutes option. You might want to test this out. Originally I didn't want to add another python dependency but I figured what the heck it doesn't work well anyway. If you follow the wiki and update PostStartup it will survive reboots too.

Martin

[bouilliabai]

Remuxing a recording of the 2022 Stockyards Rodeo (which is not far from here) from a little while ago fixed a lot of the playback issues. Albeit, there is still a mismatch in the total time being reported in various players, so advancing beyond the actual length of the recording can cause problems. I seem to recall there being a switch in FFmpeg to increase the length of the segment being analyzed for duration, but have slept since then.

The attached PostProcessing script builds a decision tree around a simple remux, where one can add devices (Pluto Extra is 21) and/or channels (PBR RidePass = 2520.1) to be processed. I used the latest version of Streamlink, with FFmpeg 7.0 copied into the NextPVR\Other program folder, and the changes described above to pluto.py so that Streamlink could find the channel. It's crude code - I am not a programmer, but it appears to work.

Curiously, the Pluto API provides up to 10 hours of EPG now, and a it looks like only one change is required to line #89 of pluto.py in order to account for it.

    et = st + timedelta(hours=10)

The real culprit here is FFmpeg and its inability to correctly handle PTS discontinuities. Until that is fixed, everything is just a workaround, in some cases with more success than others.

[bouilliabai]

@bouilliaba, thanks for the info on the recent streamlink change.  I have updated the wiki with these.  It turns out the version I had did have some S/E logic so I included that for everyone. 

I also included some changes I had been working on for the xmltv-update minutes option.  You might want to test this out.  Originally I didn't want to add another python dependency but I figured what the heck it doesn't work well anyway.  If you follow the wiki and update PostStartup it will survive reboots too.

Martin

Glad to be of service!

No doubt your solution is far more elegant than what I'd come up with, also using PostStartup and a batch Ping command to customize the wait time, with a regular interrupts to check for stop conditions. Another thing PostStartup allows for is an initial run of xmltv-update when the server is started, which could be a desirable feature.

Will have a look at the latest version of the extra and provide feedback as needed, though it might be a day or 2.

[mvallevand]

First, I am very excited to see you 10 hour guide discovery and sure enough that worked well here and it looks like 12 hours work, so I will update the file on the wiki shortly (done).

However the recording problem has nothing to do with FFmpeg, this is not a remux problem, I can deconstruct a recording and put it together with FFmpeg but I can't automate the process. The source is designed to have commercials on a custom player and that source is an m3u8 with lots of potential streams for their player. The output we want is a single mpeg-ts file constructed from segments. Do you have problems using the PlutoTV app?

If anyone is looking for a golden bullet, I wouldn't bother with that script. I was watching the Twilight Zone channel one time and 2 shows played perfectly through the flying PlutoTV commercials before the third one crapped out. There is no logic that can work this out except possibly some pure channels which I haven't found.

The change in PostStartup will adjust for time that might be remaining while there service was shutdown so it might not be immediate after startup.

Martin

[bouilliabai]

The wiki was a little unclear about the incremental updates running from just PostStartup. Excellent idea to keep it on schedule independent of the server status.

It's been a while since I've been over that monstrosity calling itself FFmpeg documentation, but best guess is that Streamlink is concactenating the individual HLS stream segments, and possibly using FFmpeg to do it. FFmpeg has always been quirky about concactenating by remuxer, which is straightforward to do programmatically, vs. concactenating by protocol, which preserves the sequence a lot better. Without knowing more about what Streamlink is doing, there's no way to tell what's getting through to NextPVR. But it's a whole lot more manageable than using m3u8 links as IPTV devices or in custom extras.

To be honest, yesterday was the first time in a while I'd fired up the actual Pluto app. Switching to a Sanford & Son live TV feed, the first thing I noticed, other than Aunt Esther truly being a sore for sight eyes, was that the quality was quite poor. Perhaps there's a setting in the app for that, or perhaps it was just luck of the draw. The ability to select the highest quality stream may be an immediate benefit of using Streamlink. And a simple post-processing remux, while certainly limited in its scope, seems better than nothing. As an added bonus, in some cases it also appears to reject the discontinuities at commercial boundaries, cutting the playback time accordingly.

[mvallevand]

Streamlink does very little except cleaning things out of the m3u8 and then it mainly leaves it up to ffmpeg to stream. If you want something better I know that ChannelsDVR has an mpegts rewriter for this type of stream and their Docker is fancier too, but I haven't actually used it.

Martin

[bouilliabai]

Remuxing a recording of the 2022 Stockyards Rodeo (which is not far from here) from a little while ago fixed a lot of the playback issues. Albeit, there is still a mismatch in the total time being reported in various players, so advancing beyond the actual length of the recording can cause problems. I seem to recall there being a switch in FFmpeg to increase the length of the segment being analyzed for duration, but have slept since then.

The options were -probesize nnnn -ss 0.000000 before the input and -ss 0.000000, where nnnn is the size of the video in bytes. I tried this on the PBR RidePass recording and all it did was shift and delay the start of the video. What's happening is that a simple remux is shifting the PTS values to start at or near zero as intended, which leads to a reasonable value for the total time, making the recording far more robust to playback operations. Since the PTS values in the commercials are so out of whack, these are getting rejected somehow, either by Streamlink in the capture or the player honoring the PTS schedule linearly, as it should. The problem this creates is that the total duration is less than what is being shown by NextPVR and reflected in other clients, such as the Kodi addon. This, in turn, is likely being calculated from the start and end times in recording-backup.xml. Attempts to seek or play past the actual recording time will then cause playback to terminate. Otherwise the remuxing process can greatly improve the playback characteristics in at least some cases.

The placement of Streamlink in between live Pluto HLS streams and NextPVR provides some time-shifting capability within a program, far more than the max 30 seconds allowed by the app at any rate, however, it is not resilient when program boundaries are encountered. Earlier, a method was suggested to add a second buffer via external script, but I'm missing something on how that might be implemented, and even then have no idea how the result could be passed back to NextPVR for writing to the tempfile. It's too bad Streamlink doesn't allow the output to be directed to named pipes, which could then be buffered and redirected to stdout for use by NextPVR.

Well, whaddaya want for nuttin'? Rubber biscuit?

[mvallevand]

Your analysis doesn't sound right because it assumes a constant PTS which a simple remux won't give you and these streams don't necessary have. HLS seeking doesn't need it plus adaptive streams mix and match resolution bit rates etc for the connections speed so it is not a linear file.

An extra to call a batch file is shown here https://forums.nextpvr.com/showthread.ph...#pid581970 and below is the the batch file to give you a pipe from ffmpeg you are talking about check from the command line and make your paths are good.

Code:

streamlink %1 best -O | ffmpeg -i pipe: -c:v copy -c:a copy -c:s copy -

However that's enough time wasted on streamlink for me for awhile, thanks again for pointing out the streamlink changes and the longer guide data.

Martin

[bouilliabai]

Ah. I hadn't tried that as an extra, but rather from the console, with nearly the exact same command (but including the -re switch) with the output going to a file, and the result was unplayable. Again, I've only been using NextPVR and Streamlink for a couple of weeks or so. Another lesson for the memory banks. Thanks.

A note about the re-installation of the PlutoTV extra: if the procedure in the forum is followed, there will be a lot of duplicate channels created. To avoid that, the extra-pluto.xml file has to be deleted, then the channels cleared in the web GUI before running python pluto.py extra, then scanning for channels. That method does not appear in the wiki either.

For some reason, after following that procedure, neither the EPGUpdate nor python pluto.py xmltv-update method loaded the EPG in the web UI, even though pluto-xmltv.xml had been updated. Completely deleting all PlutoTV extra related files, clearing channels, deleting the device in the UI and using the python pluto.py install method did. I probably did something wrong.

And as long as we're on the subject of the wiki, a suggestion: the placement of the psutil module installation instructions in the VOD section of the wiki is a little confusing, as is the statement "In PostLoadEPG add PostStartUp add the following" in the setup of incremental EPG updates. And with an immediate update now simply being an incremental update with a delay of 0, it requires psutil as well.

Of course PTS isn't constant, but rather ideally increments with each displayed frame of a video. FFmpeg looks for that and will generate an error if a video contains timestamps which are not monotonically increasing. The point was that remuxing reindexes the timestamps at both the stream and container levels. For the case of the bull riding video, an ffprobe of the original file contained the following.

start_pts=294884640
start_time=3276.496000
duration_ts=8305934192
duration=92288.157689

For the remux, however.

start_pts=134640
start_time=1.496000
duration_ts=88640640
duration=984.896000

So instead of being reported as a 25 hour video or whatever, the times are a lot more reasonable, and seeking becomes much easier to manage.

Something like GSpot, which provides the ability to look at the timestamps of individual frames would be helpful, but for some reason, that isn't working with any of these recorded videos, even remuxed to mp4 containers. What I suspect is happening is that the commercial boundaries are marked by large PTS increases, and Streamlink is not passing them through. Therefore NextPVR writing to the tempfile ends and playback stops when it gets to that point. Only when a timed recording is set up does it resume at the other side, and since those PTS values are in line with what came before, they play back normally in a recording. This is verified by the remux of the remux generating no errors in FFmpeg.

The key to uninterrupted and seekable playback of live Pluto streams then, at least in instances like these, could be forcing NextPVR to continue writing to the tempfile, even when no data is being passed from Streamlink, and the player pausing playback as needed until data starts streaming again.

[mvallevand]

I meant the PTS may not increment as expected, you can check the timing file that is saved with recordings as it already shows the timestamps for each second. NextPVR records the mpeg-ts stream as-is so it never rewrites the stream. I referred you to ChannelsDVR if you want more control.

We already know remuxing the PBR channels is not sufficient seems like a waste of time to continue. If you want to persist trying things I suggest looking at the end of this thread from ChannelsDVR https://community.getchannels.com/t/plut...nels/24870 I suspect there is something in the maddox Docker that help. I added the two lines to the streamlink pluto.py file after the jwt line and things are better with the CLI version you might find even more suggestions there.

Code:

            params["jwt"] = data["sessionToken"]
            params['serverSideAds'] = 'true'
            params['clientID'] = 'c63f9fbf-47f5-40dc-941c-5628558aec87',

However NextPVR can't (and shouldn't) handle the potentially very long no data periods that show on the CLI as

Code:

[stream.hls][info] Filtering out segments and pausing stream output
[stream.hls][warning] Encountered a stream discontinuity. This is unsupported and will result in incoherent output data.

[stream.hls][info] Resuming stream output
[stream.hls][info] Filtering out segments and pausing stream output
[stream.hls][warning] Encountered a stream discontinuity. This is unsupported and will result in incoherent output data.

[stream.hls][info] Resuming stream output
[stream.hls][info] Filtering out segments and pausing stream output
[stream.hls][warning] Encountered a stream discontinuity. This is unsupported and will result in incoherent output data.

[stream.hls][info] Resuming stream output

Most of my testing is with 'MTV Spankin' New' which is pretty much the only channel that interests me and the CLI works with a lot of pregnant pauses.

For the wiki, if you want to download the .md file and edit it I can upload any changes. Re-installing using the install command line is not documented nor do I plan on doing it. I believe if you delete all channels with the trashcan and run the command things might work. In general rescanning Extras in NextPVR is a bit tricky. The new section for psutil isn't really in the VOD section is just doesn't have a header.

Martin