Working with Random files

[trims30]

If I dothe following:

open Filename fro Random Access Read #1 len=4096

I want to bring in 4096 bytes
and then process each 16 byte portion of the 4096 bytes as a UDT

I an able to open using len=16 and read each record that way but it's way too slow over network.

Want to bring in 4096 bytes and process 256 - 16 byte chunks then get another 4096 and process that.

Need help on this.

[WoF]

You could use binary access instead.
Crucial is the definition of a data structure which can be read in one go and is 4096 bytes large.
I don't know the structure of your udt, so I assume it be an array of 16 byte.

Private Type My16
  My(15) As Byte 'can be any 16 byte structure you define
End Type

Private Chunk(255) as My16 'declare an array of 256 My16 structures, being 4096 bytes addressable in groups of 16 as structured

Open FileName For Binary as #1
While Not Eof(1)
  Get #1, , Chunk 'read 4096 bytes
  'now process
Wend
Close #1

The Get statement reads as many bytes as the given variable (her the array Chunk()) is in size.

[trims30]

Here's what I'm doing - must be better way though...

My original code reading 16 byte records...

dim tbtrindex() as Index_def

Open GPath$ & "TrimsIdx.ddf" For Random As #1 Len = 16
ReDim TbtrIndex(1)
I = 1
Do
ReDim Preserve TbtrIndex(UBound(TbtrIndex) + 1)
Get 1, I, TbtrIndex(I)
If EOF(1) Then Close #1: Exit Do
I = I + 1

Loop


Type Index_Def
numsegs As Integer 'Number of Segments
segpoint(6) As Integer 'Pointer to Field Table
End Type


'Modification to read one chunk and load array from the chunk

Function GetIndexDef(stValue As String) As Index_Def
'function converts 16 byte string to UDT
GetIndexDef.numsegs = CVI(Mid(stValue, 1, 2))
For I = 0 To 6
GetIndexDef.segpoint(I) = CVI(Mid(stValue, 3 + 2 * I, 2))
Next
End Function


'============== To load TbtrIndex Array from file with one Input
dim strdata as string
dim tbtrindex() as Index_def

Open GPath$ & "TrimsIdx.ddf" For Binary Access Read As #1
strdata = String$(LOF(1), " ")
Get #1, , strdata
Close #1

ReDim TbtrIndex(1)
I = 1
Do
ReDim Preserve TbtrIndex(UBound(TbtrIndex) + 1)
TbtrIndex(I) = GetIndexDef(Mid(strdata, 1 + 16 * (I - 1), 16))


I = I + 1
If (1 + 16 * (I - 1)) > Len(strdata) Then Exit Do

Loop

[WoF]

Well, yes, that's already a progress. You are reading all the file in one chunk.
Still you have the conversion loop do run.
It could be a little more adept to read the array in one chunk. You do not need any conversion loop:

Type Index_Def
    numsegs As Integer 'Number of Segments
    segpoint(6) As Integer 'Pointer to Field Table
End Type

Dim TbtrIndex() as Index_Def

Open GPath$ & "TrimsIdx.ddf" For Binary Access Read As #1
ReDim TbtrIndex(LOF(1) / 16 - 1)
Get #1, , TbtrIndex
Close #1

Understood? Instead of preparing a string to which you will perform the Get operation, you prepare the array as you want it.
Dim TbtrIndex() as Index_Def prepares a dynamic array of Index_Def structures
LOF(1)/16 returns the number of Index_Def structures in the file.
Redim TbtrIndex(LOF(1) / 16 - 1) therefore makes the array the size of the file
Get #1, , TbtrIndex will read as many bytes as the array is in size, in fact reads all the bytes in one go.
No more conversion necessary.
To acces the first structure's values you go:
debug.print TbtrIndex(0).numsegs
debug.print TbtrIndex(0).segpoint(0)

[trims30]

That's what I was looking for.

Now, last part of problem.

Another file with all 16 byte records
1st 100 records are of type File_Def and remainder are of type Field_Def

File_Def data goes into array FileDef()
Field_Def data goes into array FieldDef()

How do we handle that without my conversion stuff.

Lee

[dglienna]

Re-write the data?

[WoF]

No.
The handling is exactly the same, only that the previous sample took LOF() to get the length of the file and divided it by the length of the structure to redim the array.

Assumed you have the structures File_Def and Field_Def, no matter what they are, then you compute like this:

Dim FileDef() As File_Def
Dim FieldDef() As Field_Def

ReDim FileDef(99) 'holds 100 File_Defs
Dim Remainder As Long 'will reflect the remaining bytes in file after the File_Defs
Open YourFile For Binary Access Read As #1
Get #1, , FileDef 'read the 100 File_Defs
Remainder = LOF(1) - 100*Len(File_Def) 'remaining bytes in file
ReDim FieldDef(Remainder/Len(Field_Def) - 1) 'resize the second array to read n Field_Defs
Get #1, , FieldDef
Close #1

Note that Len(File_Def) returns the size of the File_Def structure in how many bytes it takes up in memory. Usually this is also the number of bytes it takes up in a file.

[trims30]

Thank you.
That's exactly what I was looking for.

[WoF]

I thought so.

[trims30]

WOF:
Been a pleasure to work with you - look forward to another session soon.
Just getting my feet wet in VB.Net/MySql and I'm sure I'm going to need a guiding hand along the way.

Regards,

Lee