Windows Service Hanging

[BrettB]

A couple of times recently, I've had my NextPVR server apparently "hang". Windows shows that the service is running, but it does not respond to connection attempts, recordings don't happen, the EPG update doesn't happen. Looking at the Nrecord log file, it appears to have been "cut off" in mid message (maybe that is just from the buffering).

I've attached a set of logs after restarting the service. So the "hung" log is the .1 one. I also had Windows do a process dump of the service process before I restarted it. I'll pm sub with a link to the dump file.

[mvallevand]

Is there anything in the Window Event viewer.

I'd be equally worried about all the failed recordings and how UI client is handling that. Do you see anything in your HDHomerun logs?

Martin

[BrettB]

I'd be equally worried about all the failed recordings and how UI client is handling that.  Do you see anything in your HDHomerun logs?

Sorry! I attached the wrong set of log files to that original post. That's from the server here. And the recordings here are failing because I cancelled the cable service earlier this week. So the HDHR isn't on the network here any more. (Packed up to go back to MN with me today.)

Good call on the Event Viewer. The System log didn't show anything around that time. But the Application log does:

Code:

- <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
- <System>
  <Provider Name=".NET Runtime" />
  <EventID Qualifiers="0">1023</EventID>
  <Version>0</Version>
  <Level>2</Level>
  <Task>0</Task>
  <Opcode>0</Opcode>
  <Keywords>0x80000000000000</Keywords>
  <TimeCreated SystemTime="2021-11-09T02:54:03.0816322Z" />
  <EventRecordID>1958</EventRecordID>
  <Correlation />
  <Execution ProcessID="0" ThreadID="0" />
  <Channel>Application</Channel>
  <Computer>WALLY</Computer>
  <Security />
  </System>
- <EventData>
  <Data>Application: NextPVRServer.exe CoreCLR Version: 4.700.21.41101 .NET Core Version: 3.1.19 Description: The process was terminated due to an internal error in the .NET Runtime at IP 00007FF9E7EA1EE7 (00007FF9E7D10000) with exit code c0000005.</Data>
  </EventData>
  </Event>

and

Code:

Log Name:      Application
Source:        Application Error
Date:          11/8/2021 8:54:04 PM
Event ID:      1000
Task Category: (100)
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      WALLY
Description:
Faulting application name: NextPVRServer.exe, version: 1.0.0.0, time stamp: 0x611b1159
Faulting module name: coreclr.dll, version: 4.700.21.41101, time stamp: 0x61140c57
Exception code: 0xc0000005
Fault offset: 0x0000000000191ee7
Faulting process id: 0x2b48
Faulting application start time: 0x01d7d03ef7bba43c
Faulting application path: C:\Program Files\NextPVR\NextPVRServer.exe
Faulting module path: C:\Program Files\NextPVR\coreclr.dll
Report Id: 000a9265-c989-4ebc-9d38-3f43db5be86c
Faulting package full name:
Faulting package-relative application ID:
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Application Error" />
    <EventID Qualifiers="0">1000</EventID>
    <Version>0</Version>
    <Level>2</Level>
    <Task>100</Task>
    <Opcode>0</Opcode>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2021-11-09T02:54:04.0191348Z" />
    <EventRecordID>1959</EventRecordID>
    <Correlation />
    <Execution ProcessID="0" ThreadID="0" />
    <Channel>Application</Channel>
    <Computer>WALLY</Computer>
    <Security />
  </System>
  <EventData>
    <Data>NextPVRServer.exe</Data>
    <Data>1.0.0.0</Data>
    <Data>611b1159</Data>
    <Data>coreclr.dll</Data>
    <Data>4.700.21.41101</Data>
    <Data>61140c57</Data>
    <Data>c0000005</Data>
    <Data>0000000000191ee7</Data>
    <Data>2b48</Data>
    <Data>01d7d03ef7bba43c</Data>
    <Data>C:\Program Files\NextPVR\NextPVRServer.exe</Data>
    <Data>C:\Program Files\NextPVR\coreclr.dll</Data>
    <Data>000a9265-c989-4ebc-9d38-3f43db5be86c</Data>
    <Data>
    </Data>
    <Data>
    </Data>
  </EventData>
</Event>

[mvallevand]

Ok, hopefully sub will find something in the dump. I suspect it is replaced to the UI client crash reported here https://forums.nextpvr.com/showthread.php?tid=63201 but without sub's input that is truly a guess.

Martin

[sub]

Unfortunately there didn't seem to be anything obvious in the dump. It looked like very little was happening. Does the web server still respond when it's in this state?

[mvallevand]

EventViewer showed NextPVRServer.exe crashing with the access error 0xc0000005

Martin

[sub]

That’s a pretty generic error, so doesn’t really tell us much.

Oddly though, that type of error shouldn’t really be possible in managed .net code, like NextPVRServer.exe. I could imagine it being possible with devicehostwindows.exe, which uses unmanaged code.

[mvallevand]

I was pointing out the crash so the server couldn't be responding. I still tend to think it is a graphic error rendering a "bad" image since we have seen this from more than one user typically on Linux in System.Drawing and now libSkia. Here this is likely in the transition from png to jpg which is where @baj1 was also having trouble.

Martin

[sub]

I still tend to think it is a graphic error rendering a "bad" image since we have seen this from more than one user typically on Linux in System.Drawing and now libSkia.  Here this is likely in the transition from png to jpg which is where @baj1 was also having trouble.

Yep, totally could be something like that, and it is a call to native unmanaged code, so could generate that type of error.

Hopefully someone will be able to find a set of steps to reproduce it at will.

If you guys do get other dumps produced, supply them and I'll take a look. We might get lucky with a better example.

[BrettB]

Unfortunately there didn't seem to be anything obvious in the dump. It looked like very little was happening. Does the web server still respond when it's in this state?

No. The web server wasn't available either. Hence, I couldn't get log files from there. Presumably that's why that last log message isn't a complete line. (i.e. the log buffer hadn't been flushed)

But the service manager shows the service as still "running". And to fix I just do a restart in the service manager.