Loop through folders with shell object.

[DataMiser]

One more test using the VB6 methods in VB2005 result about 7.5 seconds to parse out the same folder on the same machine.

VB.Net tests and VB6 tests all ran in debug mode

[DataMiser]

And yet another test this time with the VB project posted by WoF compiled to Native code optimized for fast code.

Dir Method pumping out steady in the 3,0xx MS range
API Method steady in the 3,2xx range

[dglienna]

What folder? That was my point...

[DataMiser]

If you read the messages I posted I indicated which folder in at least one of them. All tests were done on the same folder, same machine.

Anyway I fail to see how which folder makes a difference so long as it is the same folder in all tests.

[WoF]

I have to admit that I never compiled the code. I always ran it in the IDE.

I have tried to make all routines nearly equal, either using a recursive call or disposing from the collection within the procedure (no recursive calling).
The FSO procedure is a little different, but is out of range anyway.

I think Dir is the closest to the API call you can get. Also using Dir you don't have to do fancy string picking because it delivers a clean VB string.
An advantage of the API call is, it delivers the file attributes and time stamps in one go.
So there is one thing we could make API go a little faster by not making a GetAttributes() cal, but using the attributes long delivered by the WIN32_FIND_DATA structure.

I was afraid the VB.NET comparisons would be slower for all this wrapping and encapsulating.

[WoF]

To all who are still interested in that topic:
I have worked out another fancy directory scanner which seems to be even faster than API, at least under the testing environment as described before, which was a set of about 47000 files held in 6 different folders.
To use it you have to add a reference to the 'Microsoft Shell Controls and Automation'
This makes available several shell objects like NameSpace and FolderItem, which allow browsing also through Non-FileSystem Objects like Nethood, Network Computers, Desktop, etc.
I have broken this in two small subs.
The first simply finds the shell32.folder object for the given StartingPath, to call the second one.
The second calls itself recursively for every subfolder found during scanning.

Private Sub ListFilesShell(StartingPath As String)
  Dim SHL As New Shell
  Dim dsk As Shell32.Folder
  Dim fit As Shell32.FolderItem
  
  Set dsk = SHL.NameSpace(ssfDESKTOP)
  Set fit = dsk.ParseName(StartingPath)
  If Not fit Is Nothing Then
     If fit.IsFolder Then ListAllFilesShell fit.GetFolder
  End If
End Sub

Private Sub ListAllFilesShell(StartingFolder As Shell32.Folder)
  Dim fit As Shell32.FolderItem
  For Each fit In StartingFolder.Items
      lstFiles.AddItem fit.Name
      If fit.IsFolder Then ListAllFilesShell fit.GetFolder
  Next
End Sub

I was surprised to find it even faster than the API call. Under the given conditions the fastest API version was 4.7 seconds. This shell version is 3.2 seconds.
Could be this is along the mechanisms PowerShell will also make profit off. It is very close to the explorer's structures. You even can get access to the verbs (right-click-commands) associated with a certain item (file/folder)

[vb5prgrmr]

I'm not familiar with that object but it looks real good (code size wise) but...and there is always a but... I don't see where it can recurse/explore sub-sub folders. I see where it gets the given folder (GF) and from that it gets the first level of sub folders (SF1) but after that...

On another note, I bet that this could also be converted to a getobject/late binding method on certain OS's so one would not have to include another dependency...

Really nice work WoF

[HanneSThEGreaT]

Tried another .net test that I thought might be faster but turned out to be a couple of seconds slower.

This code steps through each directory in the tree and grabs the filenames from each of them adding the file name to the listbox.
Takes 12 seconds on my test machine

Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        ListBox1.Items.Clear()
        Dim x As DateTime = Now()
        ListBox1.Visible = False
        For Each strFolder As String In Directory.GetDirectories("c:\program files", "*.*", SearchOption.AllDirectories)
            For Each strFile As String In Directory.GetFiles(strFolder, "*.*", SearchOption.TopDirectoryOnly)
                ListBox1.Items.Add(strFile)
            Next
        Next
        ListBox1.Visible = True
        MsgBox(DateDiff(DateInterval.Second, x, Now()))
    End Sub

This test grabs all the directories first then grabs all the filenames.
Takes 14 seconds on the test machine.

Private Sub Button2_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button2.Click
        ListBox1.Items.Clear()
        Dim x As DateTime = Now()
        ListBox1.Visible = False
        For Each strFolder As String In Directory.GetDirectories("c:\program files", "*.*", SearchOption.AllDirectories)
            ListBox1.Items.Add(strFolder)
        Next
        For Each strFile As String In Directory.GetFiles("c:\program files", "*.*", SearchOption.AllDirectories)
            ListBox1.Items.Add(strFile)
        Next
        ListBox1.Visible = True
        MsgBox(DateDiff(DateInterval.Second, x, Now()))
    End Sub

Good observation!

I have found that instead of loading all dirs and then loading all files takes quite long indeed. what I have done ( if you look into the VB.NET FAQ's for a Windows Explorer app ), I have loaded the tree first, then, as soon as the person expands a node, then only load the files only for that folder.

that seems to work the fastest.

i hope you get what I'm trying to say

[DataMiser]

I have not looked into it but windows has the indexing service which allows files to be searched rapidly. There should be objects or apis that take advantage of this service that would search files very quickly assuming one has the service enabled on the system in question. I do not use this service as I seldom have the need to search for files and would rather the search take a little longer rather than use resources for indexing in the background while I am doing other things.

Might be something worth looking into.

[dglienna]

Two modules from the Scripting Guys (part of a Backup cmd)

  foreach($objDstFile In $colDstFiles)
    {
        $strDstFile = $objDstFile.Path
        $strSrcFile = $strSrcFolder + "\" + $objDstFile.Name

        if( -not $FSO.FileExists($strSrcFile))
        {
            $FSO.DeleteFile($strDstFile)
            Write-Host "Deleted file $strDstFile"
            $script:iDestDeletedCount++
        }
    }

    DoSubFolders $objSrcFolder $objDstFolder
}


function DoSubFolders
{
    param([object] $objSrcFolder, [object] $objDstFolder)

    $colSrcSubFolders = $objSrcFolder.SubFolders

    foreach($objSrcSubFolder in $colSrcSubFolders)
    {
        $strSrcSubFolder = $objSrcSubFolder.Path
        $strDstSubFolder = $strSrcSubFolder.Replace($strSourceFolder, $strDestinationFolder)
        if( -not $FSO.FolderExists($strDstSubFolder))
        {
            $FSO.CreateFolder($strDstSubFolder)
            Write-Host "Created folder $strDstSubFolder"
        }
        $objDstSubFolder = $FSO.GetFolder($strDstSubFolder)
        BackupFolder $objSrcSubFolder $objDstSubFolder
        DoSubFolders $objSrcSubFolder $objDstSubFolder
    }
}

http://gallery.technet.microsoft.com...a-89aaf4927bb9

[DataMiser]

Using fso uugh, earlier tests showed that was far slower than any other method tried. just over 6x slower than vbs dir or windows api functions

[dglienna]

Part of a bigger backup program, that works recursively, with one command.
Those modules are part of it.

[DataMiser]

Could be useful but my first step would be to get rid of the fso if possible. I use it in VBscript because there is no othe roption and I used it in Embedded VB again because there was no other option. Native VB file routines have much much less overhead and are much much faster.

[WoF]

I'm not familiar with that object but it looks real good (code size wise) but...and there is always a but... I don't see where it can recurse/explore sub-sub folders. I see where it gets the given folder (GF) and from that it gets the first level of sub folders (SF1) but after that...

On another note, I bet that this could also be converted to a getobject/late binding method on certain OS's so one would not have to include another dependency...

Really nice work WoF

@vb5prgrmr: You might not have noticed that the second function ListAllFilesShell calls itself with a subfolder item as soon as encounters one. So if the subfolder has other subfolders in it it will again recursively call itself with this subfolders and so on. It will walk through all nested subfolders if there are any, like a good recursion will do. That's what the recursive call is for.
The first part only retrieves the folder object from the starting path.

The shell32 objects Folder and FolderItem allow also to walk non-file-system namespaces.
They provide much more data than the FSO structures.
And what surprised me most is a) the simplicity of code reqired, and b) the execution speed which is even better than API call.

All objects are defined in shell32.dll, so I'm sure a late bound concept could be worked out, also it is better to have the reference in when programming, so as intellisense gives you all the names of the object's properties. This would possibly allow the use in VBScript, too.

[vb5prgrmr]

Okay, I see it now. Must not have had enough caffine

Still really nice work though