<solved> [NPVR 2.3.6] resource-exhaustion / memory leak

[whurlston]

I'm curious. Have you tried disabling the TV listings plugin?

[ShiningDragon]

Yeah, yesterday nearly the whole day. But it didn't changed anything.

[sub]

I wanted to make sure it wasnt some difference between the 'debug build' I use on my machine, and the 'release build' included in the installer, so I've setup a clean machine and installed 2.3.6. I've got perfmon monitoring the virtual bytes, private bytes and working set, and have kicked off playback of a folder containing about 12 hours of mkv and avi video files.

Its only been running about half an hour, but I can see the memory usage hasnt really moved since soon after the video playback was started.

I'll leave it running for the rest of the day.

Also,

It is normal to see the memory jump when you start playback, and maybe not go all the way back to where it started when playback stops. There is several reasons for this. For example, when you kick off playback, the process loads the directshow dlls and decoder dll/ax files into the process. These stay loaded in the process after the first run, and a reused next time you use directshow. (I'm mainly referring to code space here, but also any static memory allocated by objects that were created and destroyed during the use of the objects)

Memory can also be held onto by the .NET runtime, even if its freed by the application. It holds onto memory unless the OS is running low and requests it. It does this caching of memory on the assumption you'll probably want more memory again at some stage.

Also garbage collection can take some time.

A slow steadily increasing memory usage during is none of these things though, and almost certainly indicates something is leaking, but we have no idea what.

Its been running for about 5 hours now, and the memory hasnt moved much at all. Slight movements when it switches files - some move to slightly more memory, some move to slightly less. All in all it's stayed about the same all day.

Is there anything special that is common between your machines? Plugins, or decoder packs?

[ShiningDragon]

Mhm, where's the difference ...

The systems are all familar with each other. Same splitter, same decoder, same chipset/bios/drivers, but nothing special. Nothing has changed but the change from NPVR 2.2.6 to NPVR 2.3.6.
I'll starting now both vms on the mainrig and take a look if something changes ... (the vms are an identical setup to my systems).

Phew, i think i've caught it. One last stress run on real hardware. I'll go to bed now (it's 3:15 am here). Good night.

[sub]

Phew, i think i've caught it.

You've found it? What was it?

[ShiningDragon]

I am not 100% sure at the moment, so i'll let the mediacenter run again at least 10 hours (hopefully it works). If i am right, then npvr has nothing, or at least not much to do with it (perhaps a combination of both). We'll see later this day.

You've found it? What was it?

Oh well ... i am so happy now, that i could dance naked on the streets!

First of: mvavelland was a bit right and a bit not right. Indeed it was a "local" problem; hidden in the process of nextpvr.exe.

Short explanation:
The cause for the HUGE memory leak was NOT nextpvr.exe and NOT nrecord.exe and even NT ntray.exe. The CAUSE for the huge memory leak was: ffdshow_rev4238_20120109_clsid

Longer explanation:
As i prepared the virtual machines on my host (one 32 and one 64 bit version of win 7) i did an install of the same filters, plugins and decoders. Even the Win7DSFilter was used here the same way as on the real hardware.
I started the playback on both machines (d3d available within x86_32 and not available in x86_64) and watched perfmon (paging file, private byte, paging site (perhaps working set?) and it was growing and growing and growing.
I could see that the x86_64 edition could better handle the memory usage, but it growth too.

So i thought about it. What could be the cause. Perhaps the decoders? No, how could this be, i disabled ffdshow already for testing purposes and THOUGHT that it was never loaded.
Also did i search via google for "ffdshow leakage" and found a solved bug from 2008. I wondered why i didn't found newer release informations, especially for the "new" ffdshow_rev4257_20120115_clsid.
So i did a google search again:

Code:

4247
Apply a patch by shikin to threadpool. fixes memory leak of ffdshow converters.

The ffdshow developers reintegrated an old memory leak bug, which they fixed with 4247. Just great, this is something no one thinks about it. -.-

After i upgrade the ffdshow within the vm i checked the perfmon again: Wow, what's that! The lines are so smooth, no growing here, so i updated the mediacenter with the new ffdshow release.
The mediacenter now ran 8 hours and the memoryconsumption of nextpvr.exe is around 320 MiB WITHOUT using a swapfile. Yay, back to old days.



I am so sorry that i've stolen so much time from you for this problem. But it wasn't to identify. Disabling ffdshow in nextpvr didn't changed anything and even there was no ffdshow icon in the systray. I think, it were running.
Windows couldn't detect the memory exhaustion from ffdshow too. It just pointed over and over and over again to nextpvr.exe.

But thanks to you, i've learned something new on "how to identify a leakage".

This topic is SOLVED. Anyone who has problems with the memory exhaustion: check your decoders (in this case ffdshow_rev4238_20120109_clsid), disabling them within nextpvr.exe is NOT enough.

[ShiningDragon]

Yeah, NextPVR as it's best without swapfile. The spikes are just the osd. All is working well now as it should.

[sub]

Great - glad you got to the bottom of it.