Is there a way to read from a file, without delcaring a massive bufffer?

[voidflux]

Hello everyone..

currently i have the following code that seems to work fine from reading the lines of a text file and storing that information in an array named filebuffer. THe problem is, I made the length of the file buffer 1000 characters. Because if the user enters more data I don't want it to just stop working and get a bad stack area because of it.

So is there a way to figure out the length of the text file bofore hand and allocate that much memory into the file buffer rather than just making a guess?

Here is my code:

// Buffers

	char LocalPart[1000];
	char Domain[1000];
	char FileBuffer[10000];


fstream file_op("C:\\EmailTest.txt",ios::in);
	while(file_op.getline(FileBuffer,1000))
	{
		if(IsValidEmail(FileBuffer))
			Parse(FileBuffer,LocalPart,Domain);
		else
			cout <<"0: " << FileBuffer << endl;
	
		if(IsLocalValid(LocalPart) && IsDomainValid(Domain))
			cout <<"1: " << FileBuffer << endl;	
		else
			cout <<"0: " << FileBuffer << endl;

	}
		file_op.close();
}

Thanks!

[Paul McKenzie]

So is there a way to figure out the length of the text file bofore hand and allocate that much memory into the file buffer rather than just making a guess?

Just append onto a std::string for each line read.

#include <vector>
#include <string>
#include <iostream>
#include <fstream>

using namespace std;

void foo()
{
    // Buffers

    char LocalPart[1000];
    char Domain[1000];

    fstream file_op("C:\\EmailTest.txt",ios::in);
    std::string fbuf;
    std::string FileBuffer;
    FileBuffer.reserve(10000);

    while(getline(file_op, fbuf))
        FileBuffer += fbuf;

      if(IsValidEmail(fbufall.c_str()))
            Parse(FileBuffer.c_str(),LocalPart,Domain);
      else
	   cout <<"0: " << FileBuffer << endl;
    //....
     file_op.close();
}

Regards,

Paul McKenzie

[voidflux]

Thanks for the help Paul!

I was alittle confused on a few things though,

This line:

    while(getline(file_op, fbuf))
        FileBuffer += fbuf;

is the 2nd argument of getline fbuf just saying your using fbuf to store whatever is in file_op?

And FileBuffer.reserve(10000) is this saying, well we will reserve 10,000 character placeholders but if you don't need them we will terminate with a '\0'? and free the space but if we need 10000 we'll use it?

And

FileBuffer += fbuf

I'm assuming this is just taking whatever fbuf read from the file and storing it now into the FileBuffer string.


all my functions use char *x parameter data types, so is

FileBuffer.c_str()  converting the string into a c string?

Thanks alot for the help that looks much nicer!

oops one last question...
Why would you need to include <vector> btw?

[Paul McKenzie]

Thanks for the help Paul!

I was alittle confused on a few things though,

This line:

    while(getline(file_op, fbuf))
        FileBuffer += fbuf;

is the 2nd argument of getline fbuf just saying your using fbuf to store whatever is in file_op?

It gets a line of data from the file, just like the getline() function you were using before. Each line is then appended on the FileBuffer string.

And FileBuffer.reserve(10000) is this saying, well we will reserve 10,000 character placeholders but if you don't need them we will terminate with a '\0'?

No. What it does is reserve memory for the appending to FileBuffer. The 10,000 is the number of bytes to reserve before having to get memory from the heap again. It is used to speed up the processing of the append.

And

FileBuffer += fbuf

I'm assuming this is just taking whatever fbuf read from the file and storing it now into the FileBuffer string.

The operator is +=, not just =. So it appends onto the FileBuffer string, not just assign.
[quote]all my functions use char *x parameter data types, so is

FileBuffer.c_str()  converting the string into a c string?

The c_str() returns a const char * representation of the string that is null-terminated.

oops one last question...
Why would you need to include <vector> btw?

You don't need to include it, it's there by mistake.

Also, I would suggest getting a C++ book, since all of this is discussed in a good book. The code I have posted is nothing more than using the C++ standard library.

Regards,

Paul McKenzie

[Tannin]

As an alternative, you can get the file-size like this, after you opened the file:

file_op.seekg(0, ios_base::end);
int length = is.tellg();
is.seekg (0, ios::beg);

After that, you can "new" a buffer to hold the whole file or mmap it into memory for very easy and fast access.

[voidflux]

thanks for the help guys!