CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Results 1 to 8 of 8

Threaded View

  1. #1
    Join Date
    Jan 2003
    Location
    Sweden
    Posts
    115

    Question Improve data loading time, pre-caching?

    Hi there.
    I have a question on improving data loading times. The data sets that I am reading into my program consists of 300-500 files each of size 512 kB.

    I have noticed that loading times vary greatly, normal values range from 10s to 30s for loading the ca 200 MB data. However, when reading a data set that has recently been used (not necessarily the previous data set), loading time is around 1s.

    - How come?
    - Are these files in some kind of cache?
    - Since the hard drive cache is 16MB I would not expect several hundred MB of files to be in the disk cache.
    - Is there some other kind of cache around that helps improving the reading speed?
    - If so, would it be possible to order the OS to do some pre-caching if the next set of data files to be read is known in advance?

    And finally, a code question: now I am reading the files using
    Code:
    BinaryReader binReader = new BinaryReader(File.Open(fileName, FileMode.Open, FileAccess.Read));
    byte[] rawBytes = binReader.ReadBytes(512*1024)
    - is this the best way to do it or is there any other faster command to read lots of data?
    ____
    Edit: Running VS2005 on a machine with WinXP SP2
    Last edited by Cyanide; March 19th, 2008 at 09:34 AM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  





Click Here to Expand Forum to Full Width

Featured