Directory Listings

[imilne]

Just wondering if anybody had any hints for retrieving lists of files and folders within directories.

I know about DirectoryInfo and GetFileSystemInfos() and FileSystemInfo and so on, but the performance with them - particularly over a network - seems pretty dire, apparently due to them making too many network requests for each file returned, rather than storing the information that the underlying win32 calls they make has already received. It's something that's supposed to be much better with .NET4, but I guess we have to stay with .NET 2 for now? (and I'm not too sure how it works if you have a .NET2 compiled program but .NET4 is actually installed; can it use the latter or will it insist on running it with .NET2 anyway).

I did find this (and a few like it), that sounded hopeful, but it only returns files within a folder, whereas I need lists of files *and* folders together.
http://www.codeproject.com/KB/files/File...rator.aspx

It's possible part of the problem is that I'm trying to be too smart when browsing a folder, as my plugin also goes looking into the subdirectories of the list returned (eg to determine if they might contain DVD rips). This means it's actually doing more than just a single GetFileSystemInfos() call, but I kind of need to know this information before building a UiList of possible videos to play. That way DVD folders and folders with just a single video in them can be presented at the same level as other video files found within a given directory.

Edit: This was the other one I found: http://www.codeproject.com/KB/files/Fast...sg=3177659

Iain

[mvallevand]

When I was doing the MB4 update to ML3 I tried a non-database mode, but the performance as you wrote wasn't up to it for real situations with large lists and networks. There is a one word solution. Database.

Martin

[imilne]

When I was doing the MB4 update to ML3 I tried a non-database mode, but the performance as you wrote wasn't up to it for real situations with large lists and networks.

How large was large in your case? I'm hitting problems with 200 folders, but that's 200 sets of file listings it has to do.

There is a one word solution. Database.

Yeah, I know. But I was *really* hoping to avoid going down that route. Shame it's not a one line of code solution too

There's still a few things I can try before giving in... perhaps some fancy background threading along with priority updates for items on screen. That way the initial delay would only be for a single file listing, although it would compromise the ability to do a few things, such as different sort orders for items based on anything read from the metadata, because you wouldn't have all the information available at the time a list was built.

Iain

[imilne]

Hmm, a folder with about 1000 files (or subfolders) accessed over Wireless-N seems to take about 300-400ms to return its listings. That's still pretty crap. Gigabit wired takes about 1/10 of that.

Iain

[imilne]

Some more observations:

If you get a listing with GetFileSystemInfos() (which returns both files and directories), then it takes 300-400ms to do that. But you still don't know which are files and which are directories until you look at the FileAttributes. That (stupidly) seems to make *another* network request for each item, taking the total time up to around 8 seconds!

However, if you get the list in two separate calls: one to GetFiles() and another to GetDirectories(), then the time drops back down to 300-400ms, I guess because the remote system is doing the type-determination before returning anything, rather than having to do it over the network item by item..

Iain

[imilne]

There's still a few things I can try before giving in... perhaps some fancy background threading along with priority updates for items on screen.

Ignoring the rest of my waffling for the moment, is what I've said here possible, ie, can a plugin determine which UiList items are the ones on screen (and therefore need their details fetched from disk *now*). A trick with image callbacks perhaps? Because the renderer only requests images for currently visible items does it not?

Or perhaps a method could be added to UiList.IUiListCallback, eg, ItemsVisible(UiList.ListObject[]).

Iain

[InVermont]

Some more observations:
But you still don't know which are files and which are directories until you look at the FileAttributes. That (stupidly) seems to make *another* network request
Iain

It's been a while since I've tried something like this, but what about PINVOKE-ing FindFirstFile/FindNextFile and creating a List<WIN32_FIND_DATA>. Once you have them in memory, you can step through them at your leisure and not have to worry about further network hits. The dwFileAttributes == FILE_ATTRIBUTE_DIRECTORY should be the key.
Let me state I'm thinking off hand at this point, and haven't tried any code to see if performance will be better.

[sub]

Ignoring the rest of my waffling for the moment, is what I've said here possible, ie, can a plugin determine which UiList items are the ones on screen (and therefore need their details fetched from disk *now*). A trick with image callbacks perhaps? Because the renderer only requests images for currently visible items does it not?

Yes, you can implement the SkinHelper.IUpdateOnRenderElement interface, and add a "@callbackOnRender" = this parameter to your element. It'll give you an opportunity to do something to a list item before it is displayed.

[imilne]

Yes, you can implement the SkinHelper.IUpdateOnRenderElement interface, and add a "@callbackOnRender" = this parameter to your element. It'll give you an opportunity to do something to a list item before it is displayed.

Oooh, sounds perfect. Thanks.

Iain

[psycik]

Oooh, sounds perfect. Thanks.

Iain

Did you get this to work? So does this mean if you select an element it gets the image?

Is there a way of getting a bunch of images slow time (another thread maybe) and update the UiList later. So the first images may not be there, but theyll turn up later.