Streams stopping intermittently

[Langelus]

Hi!

Tearing my hair out on this issue that I can't pinpoint the cause if anyone wise might take a look, the steam gets stopped anywhere from a couple of minutes to roughly 2 hours and I can not find any pattern as a loaded up network or such in regards to if it will work for long or not.

The setup is NextPVR version 6.1.5.231022 installed on Ubuntu 22.04, source is minisatIP with a TBS quad tuner running as a docker on the same server. Behaviour is the same regardless if the streams are to Kodi or Jellyfin as client. If I in Kodi set the streaming method to transcode it does seem to be alot more stable but that's not a viable solution right now as I don't have hardware transcoding.

Code:

2023-12-18 08:14:26.026    [DEBUG][54]    Got request [192.168.0.1]: /
2023-12-18 08:14:26.713    [DEBUG][72]    Have received 1577432612 bytes from rtsp source (291 dropped packets)
2023-12-18 08:14:27.028    [DEBUG][77]    Got request [192.168.0.1]: /
2023-12-18 08:14:28.038    [DEBUG][77]    Got request [192.168.0.1]: /
2023-12-18 08:14:28.179    [DEBUG][41]    Renewed lease... (200064)
2023-12-18 08:14:28.179    [DEBUG][41]    Renew lease ok (0x200064)
2023-12-18 08:14:29.040    [DEBUG][38]    Got request [192.168.0.1]: /
2023-12-18 08:14:29.800    [DEBUG][35]    Have received 3297216192 bytes from rtsp source (3659 dropped packets)
2023-12-18 08:14:30.057    [DEBUG][77]    Got request [192.168.0.1]: /
2023-12-18 08:14:30.771    [DEBUG][30]    Renewed lease... (1F0066)
2023-12-18 08:14:30.771    [DEBUG][30]    Renew lease ok (0x1F0066)
2023-12-18 08:14:31.069    [DEBUG][54]    Got request [192.168.0.1]: /
2023-12-18 08:14:32.073    [DEBUG][55]    Got request [192.168.0.1]: /
2023-12-18 08:14:33.088    [DEBUG][77]    Got request [192.168.0.1]: /
2023-12-18 08:14:33.228    [DEBUG][41]    Renewed lease... (200064)
2023-12-18 08:14:33.228    [DEBUG][41]    Renew lease ok (0x200064)
2023-12-18 08:14:34.090    [DEBUG][55]    Got request [192.168.0.1]: /
2023-12-18 08:14:35.093    [DEBUG][38]    Got request [192.168.0.1]: /
2023-12-18 08:14:35.822    [DEBUG][30]    Renewed lease... (1F0066)
2023-12-18 08:14:35.822    [DEBUG][30]    Renew lease ok (0x1F0066)
2023-12-18 08:14:36.098    [DEBUG][55]    Got request [192.168.0.1]: /
2023-12-18 08:14:36.714    [DEBUG][72]    Have received 1585228408 bytes from rtsp source (291 dropped packets)
2023-12-18 08:14:37.102    [DEBUG][38]    Got request [192.168.0.1]: /
2023-12-18 08:14:37.979    [DEBUG][57]    Sending rtsp keep alive
2023-12-18 08:14:38.117    [DEBUG][55]    Got request [192.168.0.1]: /
2023-12-18 08:14:38.229    [DEBUG][41]    Renewed lease... (200064)
2023-12-18 08:14:38.229    [DEBUG][41]    Renew lease ok (0x200064)
2023-12-18 08:14:39.119    [DEBUG][55]    Got request [192.168.0.1]: /
2023-12-18 08:14:39.775    [ERROR][30]    Unexpected error streaming channel: System.NullReferenceException: Object reference not set to an instance of an object.
   at NShared.Utility.RollingFile.get_Length()
   at NShared.Streaming.StreamingHelper.StreamNative(ServiceResponseStream responseStream, Dictionary`2 parameters)
2023-12-18 08:14:39.800    [DEBUG][35]    Have received 3317801064 bytes from rtsp source (3659 dropped packets)
2023-12-18 08:14:40.126    [DEBUG][38]    Got request [192.168.0.1]: /
2023-12-18 08:14:40.776    [DEBUG][30]    StopStream(2031718)...
2023-12-18 08:14:40.776    [DEBUG][30]    clientHandle[jellyfin.12] = 2031718
2023-12-18 08:14:40.776    [DEBUG][30]    Calling StopStream()
2023-12-18 08:14:40.776    [DEBUG][30]    Stop SATIP stream
2023-12-18 08:14:40.776    [DEBUG][30]    StopStream handle: 102
2023-12-18 08:14:40.776    [DEBUG][30]    Closing TSWriter....: LIVE&/home/user/recordings/live-TV4 HD-1460-89.ts
2023-12-18 08:14:41.133    [DEBUG][77]    Got request [192.168.0.1]: /
2023-12-18 08:14:41.176    [DEBUG][30]    Close@2
2023-12-18 08:14:41.176    [DEBUG][30]    Closing mapAccessor
2023-12-18 08:14:41.408    [DEBUG][30]    Closing mapMMF
2023-12-18 08:14:41.408    [DEBUG][30]    Closing mapFileStream
2023-12-18 08:14:41.408    [DEBUG][30]    deleting /home/user/recordings/live-TV4 HD-1460-89.ts.map
2023-12-18 08:14:41.408    [DEBUG][30]    Close@exit
2023-12-18 08:14:41.408    [DEBUG][30]    StopStream@exit
2023-12-18 08:14:41.422    [DEBUG][35]    Shutting down RTSP input stream
2023-12-18 08:14:41.423    [DEBUG][30]    No more streams. Stopping SATIP device
2023-12-18 08:14:41.423    [DEBUG][30]    StopStream() done
2023-12-18 08:14:41.803    [DEBUG][30]    Cleaning up: /home/user/recordings/live-TV4 HD-1460-89.ts
2023-12-18 08:14:41.803    [DEBUG][30]    Cleaning up: /home/user/recordings/live-TV4 HD-1460-89.ts
2023-12-18 08:14:41.803    [DEBUG][30]    Cleaning up: /home/user/recordings/live-TV4 HD-1460-89.ts
2023-12-18 08:14:41.803    [DEBUG][30]    Cleaning up: /home/user/recordings/live-TV4 HD-1460-89.ts
2023-12-18 08:14:41.803    [DEBUG][30]    Cleaning up: /home/user/recordings/live-TV4 HD-1460-89.ts
2023-12-18 08:14:42.138    [DEBUG][55]    Got request [192.168.0.1]: /
2023-12-18 08:14:43.147    [DEBUG][30]    Got request [192.168.0.1]: /
2023-12-18 08:14:43.254    [DEBUG][41]    Renewed lease... (200064)
2023-12-18 08:14:43.254    [DEBUG][41]    Renew lease ok (0x200064)
2023-12-18 08:14:43.901    [DEBUG][24]    Sending rtsp keep alive

[mvallevand]

That is two channels streaming so very confusing. It is impossible to diagnose this with just snippets please send the zipped full logs. These logs might help but the problem could be the source.

Martin

[Langelus]

Hi Martin,

I'll attach the full zip here.

Thanks!

[mvallevand]

It looks like Jellyfin has added some kind of stupid crap keepalive, logging every second.

I suspect the quality though, could explain all those missed packets

2023-12-17 19:00:44.053 [DEBUG][77] FE: 1 Locked: 1 Level: 255 Quality: 15

If you open the minisatip web page do things look good while it is streaming.

Martin

[Langelus]

TBH I'm not sure what to look for in minisatip that should indicate a problem, looks fine to me(?)

Started two streams and here's what it looks like in the webinterface

[mvallevand]

I would check when the problems occur to see if there are server errors, but STR looks low to me. Do you have a booster on to get that high SNR?

Martin

[Langelus]

No booster, just straight from the cable connection in the wall. I did an experiment today, started a channel and had the stream both directly from minisatip to an VLC and through NextPVR to Jellyfin. VLC had no issues holding the stream for 8+ hours but NextPVR stopped as before, I think that this might rule out minisatip as the culprit

[mvallevand]

Sub might need to put in more logging around

Code:

2023-12-18 08:14:39.775    [ERROR][30]    Unexpected error streaming channel: System.NullReferenceException: Object reference not set to an instance of an object.
   at NShared.Utility.RollingFile.get_Length()
   at NShared.Streaming.StreamingHelper.StreamNative(ServiceResponseStream responseStream, Dictionary`2 parameters)

Does it ever happen on recordings?

If not, I am not sure if there is extra logging if you enable verbose logging and let NextPVR play (Settings then Ctrl-click the Log button) There might be a collision when the file rolls. Perhaps increase the timeout on the scrollback buffer to reduce the number of rolls.

Martin

[Langelus]

I pretty much never use recording so I don't know of the top of my head but I've put a recording on now and we'll see

[Langelus]

Tried with a recording this morning that was fairly long and it seems like something happened around the two hour mark (picture is just frozen) and if I try to play it in the webinterface I get transcoder exited, so yeah something seems corrupt with the recording.