Re: Reading chunks from large files
This method is just about the fastest. Using the API is noticebly faster that the Get #1, or Input #1.
using the 'Microsoft Scripting Runtime' is also a bit faster but i'm not too sure how it compares to the API..
But you may have to tweek it a little to read the full file..
Gremmy
Re: Reading chunks from large files
Ok, thanks a lot for all your help :)
Re: Reading chunks from large files
Quote:
Originally Posted by GremlinSA
This method is just about the fastest. Using the API is noticebly faster that the Get #1, or Input #1.
using the 'Microsoft Scripting Runtime' is also a bit faster but i'm not too sure how it compares to the API..
But you may have to tweek it a little to read the full file..
Gremmy
Using ONLY API Calls is the fastest. MSR only calls these functions, built into Windows, so it takes a bit longer to process the instruction.
Re: Reading chunks from large files
GremlinSA, I'm working on a program that copies 40GB backup files from one server to another on my network. The APIs look like they would get the job done, but I must admit that it is a bit above my skill level.
The standard VB6 file copy works, but gives no update to the user as to the status of the copy. It just appears to lock up the program until the operation is complete (approx. 70 minutes per file).
Could you show me how to use the API to copy a file in chunks?
Re: Reading chunks from large files
Check out Powershell 2.0 (it's FREE)
It should be in every network admin's toolbox. You can INVOKE a command on a CLIENT, or run a background task. (anything that the Framework can do)
It's built into most of the new Server Operating Systems and Programs
Re: Reading chunks from large files
Quote:
Originally Posted by
tasrock
GremlinSA, I'm working on a program that copies 40GB backup files from one server to another on my network. The APIs look like they would get the job done, but I must admit that it is a bit above my skill level.
The standard VB6 file copy works, but gives no update to the user as to the status of the copy. It just appears to lock up the program until the operation is complete (approx. 70 minutes per file).
Could you show me how to use the API to copy a file in chunks?
After the work done in this thread i did write a article covering the basic's on using the API's, and how the module works...
Breaking the 2gig file barrier is the article, and it also has details on how to use it in several other languages...
That should get you going nicely...
Re: Reading chunks from large files
GremlinSA,
I've looked over both the OPers code, your code, and your tutorial. But I'm still significantly confused on how to copy larger then 2 gig files.
I want to be able to simply do something similiar to:
CopyHugeFile(strSource, strDestination)
Then within the CopyHugeFile subroutine I would make it update a progress bar. I've created several different sets of code and can't seem to understand how to use your module properly.
Please help,
Knightofoldcode.
Re: Reading chunks from large files
You don't have to use his module. You can create a breakpoint on a line, and run the app up to that point. Then, pressing F8, you can execute a line at a time, and hold the mouse over a variable to see it's temp value.
(You can also use print Len(mystring) in the Debug Windows at any point that it's stopped.)
Doesn't quite work that easily in VS2008 (64bit) but that's a different story.
Re: Reading chunks from large files
Quote:
Originally Posted by
dglienna
You don't have to use his module. You can create a breakpoint on a line, and run the app up to that point. Then, pressing F8, you can execute a line at a time, and hold the mouse over a variable to see it's temp value.
(You can also use print Len(mystring) in the Debug Windows at any point that it's stopped.)
Doesn't quite work that easily in VS2008 (64bit) but that's a different story.
I appreciate your help, but I don't understand how the debug tools are going to help me understand what/how to use the module properly to copy files over 2 gb size.
I need to use his module in order to copy the files that are in excess of 2 gb.
Thanks,
Knight.
Re: Reading chunks from large files
You step thru the code, see that it picks up one chunk, adds to a counter, and then sends it along, until the counter is finished. If you see how he passes the file to his module, you'll know how to substitute your own.
He'll be here later.
Re: Reading chunks from large files
Quote:
Originally Posted by
dglienna
You step thru the code, see that it picks up one chunk, adds to a counter, and then sends it along, until the counter is finished. If you see how he passes the file to his module, you'll know how to substitute your own.
He'll be here later.
Okay, yeah, I see what your saying, but I don't see any working examples... though, it wouldn't surprise me if it's staring me in the eye. ;)
Knight.
Re: Reading chunks from large files
Well, there is an attachment of a .bas module at one of Gremmy's recent posts.
But I should like to point out, that this thread here is 3 years old, and not so recent at all. :)
Re: Reading chunks from large files
Quote:
Originally Posted by
WoF
Well, there is an attachment of a .bas module at one of Gremmy's recent posts.
But I should like to point out, that this thread here is 3 years old, and not so recent at all. :)
There is an attachment of the module, however that module only has some sub routines... this doesn't show me any examples of how to use the subroutines, and I'm seriously confused. And yes, I have read the tutorial presented by GremlinSA.
Duly noted that the original thread is 3 years old, however, the most recent post was March 5th, 2009, 05:42 AM, by GremlinSA.
I'm just trying to get more information on how to use the subroutines to copy a file. I'm seriously confused on how to do this.
Knight.
Re: Reading chunks from large files
Put a breakpoint at the START of the subroutine, and execution will stop THERE when you run the app. You can see what he passes (and the values)
Of course, things might have changed since then. It may NOT work anymore...
VB6 has been known to do that. Look up my old CAPICOM posts. Used to be the greatest...
Re: Reading chunks from large files
Quote:
Originally Posted by
dglienna
Put a breakpoint at the START of the subroutine, and execution will stop THERE when you run the app. You can see what he passes (and the values)
Of course, things might have changed since then. It may NOT work anymore...
VB6 has been known to do that. Look up my old CAPICOM posts. Used to be the greatest...
Alright... I have been trying easily for 2-3 days trying to get this right. I can not copy a file larger then 2 gb!
I have went back over the tutorial located at http://www.codeguru.com/vb/controls/...le.php/c12917/ I have tried to download GremmlinSA's demonstration program, Hex Editor, however with no modifications it errors (ByRef Argument type Mismatch) on the following line:
Code:
API_OpenFile Tmp_File, File_Num, File_Len
And highlights the "File_Len" portion of the above line. This is in the File -> Open menu subroutine.
If I can't see any code/examples, I'm left blindly trying... to that end, here is what I have, taken from the OPers post and modified to add copying the file:
Code:
Private Declare Function WriteFile Lib "kernel32" (ByVal hFile As Long, lpBuffer As Any, ByVal nNumberOfBytesToWrite As Long, lpNumberOfBytesWritten As Long, ByVal lpOverlapped As Any) As Long
Private Declare Function ReadFile Lib "kernel32" (ByVal hFile As Long, lpBuffer As Any, ByVal nNumberOfBytesToRead As Long, lpNumberOfBytesRead As Long, ByVal lpOverlapped As Any) As Long
Private Declare Function CreateFile Lib "kernel32" Alias "CreateFileA" (ByVal lpFileName As String, ByVal dwDesiredAccess As Long, ByVal dwShareMode As Long, ByVal lpSecurityAttributes As Any, ByVal dwCreationDisposition As Long, ByVal dwFlagsAndAttributes As Long, ByVal hTemplateFile As Long) As Long
Private Declare Function CloseHandle Lib "kernel32" (ByVal hObject As Long) As Long
Private Declare Function SetFilePointer Lib "kernel32" (ByVal hFile As Long, ByVal lDistanceToMove As Long, lpDistanceToMoveHigh As Long, ByVal dwMoveMethod As Long) As Long
Private Declare Function GetFileSize Lib "kernel32" (ByVal hFile As Long, lpFileSizeHigh As Long) As Long
Private Declare Function SetEndOfFile Lib "kernel32" (ByVal hFile As Long) As Long
Private Function CopyHugeFile(strSource As String, strDestination As String)
Const FILE_BEGIN = 0
Const FILE_SHARE_READ = &H1
Const FILE_SHARE_WRITE = &H2
Const OPEN_EXISTING = 3
Const GENERIC_READ = &H80000000
Dim i As Integer
Dim b() As Byte
Dim hFileSource As Long
Dim hFileDestination As Long
Dim FileLenght As Long
Const CHUNK_SIZE As Long = 9728000
hFileSource = CreateFile(strSource, GENERIC_READ, FILE_SHARE_READ Or FILE_SHARE_WRITE, ByVal 0&, OPEN_EXISTING, 0, 0)
hFileDestination = CreateFile(strDestination, GENERIC_READ, FILE_SHARE_READ Or FILE_SHARE_WRITE, ByVal 0&, OPEN_EXISTING, 0, 0)
FileLenght = GetFileSize(hFileSource, 0)
ProgressBar1.Max = FileLenght
SetFilePointer hFileSource, 0, 0, FILE_BEGIN
SetFilePointer hFileDestination, 0, 0, FILE_BEGIN
ReDim b(0 To CHUNK_SIZE - 1)
Do While Not EOF(hFileSource)
i = i + 1
If FileLenght < CHUNK_SIZE Then
ReDim b(0 To FileLenght - 1)
ReadFile hFileSource, b(1), UBound(b), Result, ByVal 0&
WriteFile hFileDestination, b(1), UBound(b), Result, ByVal 0&
Exit Do
Else
ReadFile hFileSource, b(1), UBound(b), Result, ByVal 0&
WriteFile hFileDestination, b(1), UBound(b), Result, ByVal 0&
End If
FileLenght = FileLenght - CHUNK_SIZE
ProgressBar1.Value = FileLenght
DoEvents
Loop
CloseHandle hFileSource
CloseHandle hFileDestination
End Function
However this code is flawed. I make sure to emphasize that to anyone who might try to use this code first, it's wrong. When I figure it out, I'll post a correct working version. The code doesn't work, when ran it gives me the following error: "Bad filename or number." And it hightlights the following line:
Code:
Do While Not EOF(hFileSource)
I'm not even sure this is the proper way to copy files larger then 2gb.
Please advise,
Knight.