Need Fast File Reads (binary data)

[akgalp]

Hi,

I have been searching this forum for fast file reads, but didn't find anything suited for me.

ok, here is the scenerio
we have around 500 files, each with around 6.7MB ( all files are raw image files, same size. the number of files can be even greater).

based on some calculation, i need to read a block of data from all files ( same block is read from all files) to a single memory buffer.

the (total)size of block to be read may vary ( 1MB to 50MB).

my current code looks like

        for( int nProj = 0; nProj < g_params->nTotal_Projections; ++ nProj)
        {
           // code to eliminate duplicte data fetch

            fseek(fileArray[nProj],lBeginOffset* sizeof(float), SEEK_SET );
            fread(ipp->fIPProjections_o + offset , sizeof(float),lElementstoRead, fileArray[nProj]);
        }

The problem is the througput is very low, when i time this loop. i have seen that some times it takes around >10s to read ~5MB of data . ( the data is read, processed and writen back to some other files, same drive. the reading, processing and writing are done from different threads ).

I tried it after disabling the virus scanners, but of no use.

Now my HDD is 250GB SATA ||, and my HDD diagnostic utility reports me an average speed of 60MBps.

I modified this code to make use of win api. but the results were not positive.

Then i made all the 500 files to a single file (~3GB), defragmented the drive, modifed the code to read it from that, and timed again. still the results were not as required.

Now this part became the projects bottleneck.

Hope some one can shed some light to this problem.

Thanks in advance