Anything faster than FileSystemObject?

[Nand]

I'm currently using the FileSystemObject to scan the contents of folders and sub-folders and to get the sizes and last modification dates & times of the files.

Unfortunately, FSO is a bit slow. What would be the fastest alternative?

[Shuja Ali]

How about using DIR$() function?

Remember if you are doing recursive search then Dir will not work properly.

[WoF]

Buy a faster computer. (Only joking)

But FSO is not so bad in speed. I use it also to recursively scan through directory trees, listing and scanning files in folders and so on. I have to deal with folders of large quantities of files, and I'm not too disappointed, considering there are sometimes 32000 files to list up.
Maybe you show a little of your code to find points for accelleration?

[Shuja Ali]

You can try this. It should be faster than FSO.

Private Sub SearchFolder(ByVal startingFolder As String)
    
    Dim sFileName As String
    Dim sCurrentFolder As String
    Dim cFolderList As New Collection

    If Right$(startingFolder, 1) <> "\" Then
        'add a back slash to the foldername
        startingFolder = startingFolder & "\"
    End If
    
    'add the folder to the collection
    cFolderList.Add startingFolder
    
    While cFolderList.Count
        'remove the current folder from the collection
        sCurrentFolder = cFolderList.Item(1)
        cFolderList.Remove 1
        
        'search for all files and sub folders
        sFileName = Dir$(sCurrentFolder, vbDirectory)
        While Len(sFileName)
            If (sFileName <> ".") And (sFileName <> "..") Then
                If GetAttr(sCurrentFolder & sFileName) = vbDirectory Then
                    'if it is a folder then add it to the collection
                    cFolderList.Add sCurrentFolder & sFileName & "\"
                Else
                    'print the file information to the immediate window
                    Debug.Print FileLen(sCurrentFolder & sFileName) 'filesize in bytes
                    Debug.Print FileDateTime(sCurrentFolder & sFileName)    'file modified date & time
                    Debug.Print sCurrentFolder & sFileName  'path & filename
                End If
            End If
            sFileName = Dir$
        Wend
    Wend
End Sub

[WoF]

Good thing. ShujaAli is right.
I compared both methods using this short and no-nonsens FSO code:

Private Sub ListCompleteFolder(fFolder As Folder)
  Dim fld As Folder
  Dim fil As File
  
  For Each fld In fFolder.SubFolders
      ListCompleteFolder fld
  Next
  For Each fil In fFolder.Files
      lstFiles.AddItem fil.Size & fil.DateCreated & fil.Path
  Next
End Sub

it is called ListCompleteFolder FSO.GetFolder(strPathOfFolder)
It will list 20000 files in 11 seconds.
Shuja's version does it in about 7 seconds. It's some 30% faster. Well done.

[Nand]

Thanks for the input.
I'll give it a try this weekend and let you know.

[Nand]

Well, I did a little test on 60936 files (on my C drive).

FSO spoiled 17.938 seconds (17 sec and 938 millisec) to find the top 50 biggest files.

WIN32 API (FindFirstFile, FindNextFile) did the same in 2.672 seconds.

No comments needed I guess.

[Shuja Ali]

Well, I did a little test on 60936 files (on my C drive).

FSO spoiled 17.938 seconds (17 sec and 938 millisec) to find the top 50 biggest files.

WIN32 API (FindFirstFile, FindNextFile) did the same in 2.672 seconds.

No comments needed I guess.

FindFirstFile and FindNextFile will definitely be faster than any other methods.

[WoF]

Well, good for me to know, too.
There's a chance to improve my explorer-window for the display of large amounts of files.

[jtelle01]

Hi guys, I'm new to this forum and I'm in a really need of help, trying to understand fso and other options to make the search faster, I was using fso for a few files (less than 100) and it will work perfectly fine. The problem started when I got a bigger directory and tried to use it with 5k pdf documents, some of them pretty large. FSO is taking about 5 minutes.

This is the code with after the form for query is sent:
<% Server.ScriptTimeout = 100000000 %>
<%
'Search Text
Dim strtextToSearch
strtextToSearch = Request("TextToSearch")

'Now, we want to search all of the files
Dim fso

'Constant to read
Const ForReading = 1
Set fso = Server.CreateObject("Scripting.FileSystemObject")

'Specify the folder path to search.
Dim FolderToSearch
FolderToSearch = "C:\inetpub\mysite\Files\pdfs"

'Proceed if folder exists
if fso.FolderExists(FolderToSearch) then

Dim objFolder
Set objFolder = fso.GetFolder(FolderToSearch)

Dim objFile, objTextStream, strFileContents, bolFileFound
bolFileFound = False

Dim FilesCounter
FilesCounter = 0 'Total files found

For Each objFile in objFolder.Files
Set objTextStream = fso.OpenTextFile(objFile.Path,ForReading)
'Read the content
strFileContents = objTextStream.ReadAll
If InStr(1,strFileContents,strtextToSearch,1) then
%>
<a href="http://mysite.com/files/pdfs/<%Response.Write objFile.Name%>" target="_blank">
<%
Response.Write objFile.Name & "</a><br>"
FilesCounter = FilesCounter + 1
End If
objTextStream.Close
Next

if FilesCounter = 0 then
Response.Write "Sorry, No matches found."
else
Response.Write "Total files found : " & FilesCounter
end if

'Destroy the objects
Set objTextStream = Nothing
Set objFolder = Nothing

else
Response.Write "Sorry, invalid folder name"
end if
Set fso = Nothing
%>


I see in this thread Shuja is recommending a different typoe of search. I see some of the files may just be pdf images and may be causing the issue on timing, any way anyone (with my code) can help me implement the previous example?

Any help is greatly appreciated!

[DataMiser]

It looks like you are dealing with ASP code and while it uses a VB syntax it is not VB so many options are not available to you.

It also looks like you are reading all of those files so that is the part that is really making it slow

[jtelle01]

Yeah, you are right, I tried Indexing the server itself thinking it will make it faster. Still taking quite a while.

[jtelle01]

While playing with the code I'm able to search by title and it does bring the search super fast for all files, so it's got to be maybe a way to tweak the code, I really will like to know more about DIR vs FSO but I'm not familiar with that method at all. Any helps is greatly appreciated.