Write file of base objects into vector of derived objects

[cbpetro]

I have a binary file that I have written base objects. I want to read those base objects into a vector of objects that I have derived from base.

Can it be done and how.

#include <vector>

class CBase
{
public:
	CBase( ) : m_val1(0.0f),
			   m_val2(0.0f) { }

	virtual ~CBase( ) { }

	float	m_val1;
	float	m_val2;
};

class CDerived : public CBase
{
public:
	CDerived( ) : m_val3(0.0f) { }
	CDerived(const CBase& base ) : CBase(base),
		                           m_val3(0.0f) { }

	virtual ~CDerived( ) { }

	float	m_val3;
};

int main(int argc, char* argv[])
{
	// how to: read base objects into vector of derived objects
	std::vector<CDerived> myVector;
	myVector.resize(5);

	// let's say my binary file has 5 base objects in it
	FILE *fp = fopen("somefile", "rb");
	// can I write this line of code to write the base objects into vector
	fread(&myVector[0], 5, sizeof(CBase), fp);
	fclose( fp );
	
	return 0;
}

[monarch_dodra]

As a general rule, you can't memcpy (or fread/fwrite in this case) a class instance, as it is not POD.

This is because they usually have members like pointers, counters or god knows what, that can't be reduced to some arbitrary binary value. Furthermore, class instances require construction, which you by-pass with fwrite.

Having already answered that, in your case, saving a base and writing it to a derived class is guaranteed failure, because of the splicing problem: NEVER STORE A DERIVED CLASS IN AN ARRAY OF BASE!

For what you are trying to do, I recommend doing something called serialization. It is the process of storing an object into a persistent format.

The easiest way to do this is to save each member, one by one, in a defined structured, using standard reads and writes. Forget trying to copy an entire class at once.

[Paul McKenzie]

I have a binary file that I have written base objects. I want to read those base objects into a vector of objects that I have derived from base.

First, a vector<CDerived> can only hold CDerived objects safely -- it should not hold CBase objects, even if CDerived is a derived class of CBase. In general, a vector<T> can only hold T objects. It cannot hold any other type, even if the other type is derived from T. Trying to hod other types in a vector that is meant to hold a certain type leads to undefined behaviour, even if the program compiles. The call to fwrite() that you're using breaks this rule.

Second, your base class has a virtual destructor, making the class non-POD. In C++, you can't wipe out or populate non-POD objects using low-level functions such as fwrite(), memcpy(), memset(), etc. These functions rip your non-POD types to shreds, making them unusable and unstable. Instead, you create or set these object's members using proper construction of the objects, and calling the public member functions of the object, nothing else.

So as monarch_dodra suggested, you write each member of the object to a file in a way so that when it comes time to read the file, you can recreate the object from the contents of the file. The easiest way to do this is to write an operator << for your class to do output, and an operator >> for input.

Regards,

Paul McKenzie

[D_Drmmr]

class CDerived : public CBase
{
public:
	CDerived( ) : m_val3(0.0f) { }
	CDerived(const CBase& base ) : CBase(base),
		                           m_val3(0.0f) { }

	virtual ~CDerived( ) { }

	float	m_val3;
};

This is not related to your problem, but note that the second constructor allows for an implicit conversion from CBase to CDerived. If you have a function that takes e.g. a const CDerived& and you (accidentally) pass it a CBase the compiler will silently create a temporary CDerived and pass that to the function. You will not get a compiler error. To avoid this, in general, you can put the explicit keyword in front of the constructor. This will only allow explicit conversions, not implicit ones.

[cbpetro]

Thanks for the information, but I'm not sure it's clear what I want to do.

Would this work. Because I don't see any problems. Note the second constructor of CDerived.

int main(int argc, char* argv[])
{
	// how to: read base objects into vector of derived objects
	std::vector<CDerived> myVector;
//	myVector.resize(5);

        CBase base;

	// let's say my binary file has 5 base objects in it
	FILE *fp = fopen("somefile", "rb");
	// can I write this line of code to write the base objects into vector
        for(int i = 0; i < 5; ++i)
        {
	     fread(&base, 1, sizeof(CBase), fp);
             CDerived derived(base);
             myVector.push_back(derived);
        }
	fclose( fp );
	
	return 0;
}

[Skizmo]

fread(&base, 1, sizeof(CBase), fp);

sizeof (CLASS) is never a good idea, because it has nothing to do with the actual size. Also, a class is a object, not a POD-type struct. Therefor, reading a class from file isn't possible.

[Lindley]

Ignoring the whole base/derived discussion for a moment, to properly read and write you might do something like this (ignoring endianness concerns for now):

#include <vector>
#include <iostream>

class CBase
{
public:
	CBase( ) : m_val1(0.0f),
			   m_val2(0.0f) { }

	virtual ~CBase( ) { }

        friend std::istream& operator>>(std::istream &in, CBase &obj);
        friend std::ostream& operator<<(std::ostream &out, const CBase &obj);

	float	m_val1;
	float	m_val2;
};

std::istream& operator>>(std::istream &in, CBase &obj)
{
    in.read(&m_val1,sizeof(float));
    in.read(&m_val2,sizeof(float));
    return in;
}

std::ostream& operator<<(std::ostream &out, const CBase &obj)
{
    out.write(&m_val1,sizeof(float));
    out.write(&m_val2,sizeof(float));
    return out;
}

int main(int argc, char* argv[])
{
	std::vector<CBase> myVector;
	myVector.resize(5);

	// let's say my binary file has 5 base objects in it
        ifstream in("somefile", ifstream::binary);
        for (unsigned i = 0; i < 5; ++i)
            in >> myVector[i];
	
	return 0;
}

This works because, unlike the CBase object itself, the float objects are Plain-Old-Data (POD) and thus can be safely written and read byte-for-byte. The fact that you're storing the information byte-for-byte is hidden within the serialization functions; that doesn't matter to the outside world, which is as it should be.

[cbpetro]

Ignoring the whole base/derived discussion for a moment, to properly read and write you might do something like this (ignoring endianness concerns for now):

[CODE]
#include <vector>
#include <iostream>

class CBase
{
public:
CBase( ) : m_val1(0.0f),
m_val2(0.0f) { }

virtual ~CBase( ) { }

friend std::istream& operator>>(std::istream &in, CBase &obj);
friend std:stream& operator<<(std:stream &out, const CBase

...

This works because, unlike the CBase object itself, the float objects are Plain-Old-Data (POD) and thus can be safely written and read byte-for-byte. The fact that you're storing the information byte-for-byte is hidden within the serialization functions; that doesn't matter to the outside world, which is as it should be.

The method you provided is the prefered method and safest. I normally would use this.

But I am reading large files, hence my method. I did not want to get into the subject of speed over safety but speed is an issue for me. So I profiled each method.

#include <vector>
#include <iostream>
#include <fstream>
#include "precisiontimer.h"

using namespace std;

class CBase
{
public:
	CBase( ) : m_val1(0.0f),
			   m_val2(0.0f) { }

	virtual ~CBase( ) { }

	friend std::istream& operator>>(std::istream &in, CBase &obj);
	friend std::ostream& operator<<(std::ostream &out, const CBase &obj);

	float	m_val1;
	float	m_val2;
};

class CDerived : public CBase
{
public:
	CDerived( ) : m_val3(0.0f) { }
	CDerived(const CBase& base ) : CBase(base),
		                           m_val3(0.0f) { }

	virtual ~CDerived( ) { }

	float	m_val3;
};

std::istream& operator>>(std::istream &in, CBase &obj)
{
    in.read((char*)&obj.m_val1,sizeof(float));
    in.read((char*)&obj.m_val2,sizeof(float));
    return in;
}

std::ostream& operator<<(std::ostream &out, const CBase &obj)
{
    out.write((char*)&obj.m_val1,sizeof(float));
    out.write((char*)&obj.m_val2,sizeof(float));
    return out;
}


int main(int argc, char* argv[])
{
	// how to: read base objects into vector of derived objects
	std::vector<CDerived> myVector;
	myVector.resize(5);

	// let's say my binary file has 5 base objects in it
//	FILE *fp = fopen("somefile", "rb");
	// can I write this line of code to write the base objects into vector
//	fread(&myVector[0], 5, sizeof(CBase), fp);
//	fclose( fp );


	CPrecisionTimer pt;
	CBase base;
	int i = 0;

	FILE *fp = fopen("c:\\test1.bin", "wb");
	pt.Start();
	for(i = 0; i < 1000000; ++i)
	{
		fwrite(&base, 1, sizeof(CBase), fp);
	}
	fclose(fp);

	fp = fopen("c:\\test1.bin", "rb");
	for(i = 0; i < 1000000; ++i)
	{
		fread(&base, 1, sizeof(CBase), fp);
	}
	fclose( fp );
	double dResult1 = pt.Stop();
	cout << "Result 1: " << dResult1 << endl;


	ofstream out("c:\\test1.bin", ofstream::binary);
	pt.Start();
	for(i = 0; i < 1000000; ++i)
	{
		out << base;
	}

	ifstream in("c:\\test1.bin", ifstream::binary);
	for(i = 0; i < 1000000; ++i)
	{
		in >> base;
	}
	fclose( fp );
	double dResult2 = pt.Stop();
	cout << "Result 2: " << dResult2 << endl;
	
	return 0;
}

Result2 is almost 4x slower than Result1.

[Paul McKenzie]

Thanks for the information, but I'm not sure it's clear what I want to do.

You certainly do not do this:

	     fread(&base, 1, sizeof(CBase), fp);

As I stated, do not use functions that attempt to set an object like this. Again, low-level 'C' functions that just copies bytes to a non-POD type are no good and can't be used. Where is "base" going to get its v-table pointer from, since it has a virtual destructor? That isn't in the file, but (and I'll describe it again), you're tearing up your object to shreds by setting the v-table pointer to who-knows-what when you use fread() to set the value.

I can't emphasize how important this is -- non-POD types cannot be treated as if they're POD types. You cannot memset(), fwrite(), fread(), memcpy(), ZeroMemory(), or whatever 'C' based function you want to use on non-POD types.

The only way you set member values, get member values, or use a non-POD type is through the valid access specification that is available to you. Either it will be only the public interface, or public and protected interface if your inheriting.

Regards,

Paul McKenzie

[Paul McKenzie]

The method you provided is the prefered method and safest. I normally would use this.

But I am reading large files, hence my method. I did not want to get into the subject of speed over safety

But you fail to understand that code that is ill-formed cannot be tested for anything. You can only test valid programs, and your program is not a valid one. It may be valid syntactically, but it is invalid none-the-less. Just the fact that you're wiping out the v-table pointer is enough to render this code absolutely worthless if you happen to call functions polymorphically (assuming that the compiler uses v-tables when virtual functions are implemented, and most, if not all compilers today use this method).

If you want to read POD types, then specify a POD type to read/write to, not a non-POD type. That is the solution to your problem. A POD struct would be this:

// Read into this type
struct MyData
{
   float x_axis;
   float y_axis;
};


class Base
{
//...
   virtual ~Base();
    MyData m_data;
};

Then you can use fwrite() and fread() all you want if you use MyData as the type you're reading into.

Secondly, are you running an optimized build or debug build? If you're not sure, then you must test optimized builds, as you never mentioned anything about what options you used to compile the code, let alone what compiler and version you're using to run these tests.

Regards,

Paul McKenzie

[cbpetro]

You certainly do not do this:

	     fread(&base, 1, sizeof(CBase), fp);

As I stated, do not use functions that attempt to set an object like this. Again, low-level 'C' functions that just copies bytes to a non-POD type are no good and can't be used. Where is "base" going to get its v-table pointer from, since it has a virtual destructor? That isn't in the file, but (and I'll describe it again), you're tearing up your object to shreds by setting the v-table pointer to who-knows-what when you use fread() to set the value.

I can't emphasize how important this is -- non-POD types cannot be treated as if they're POD types. You cannot memset(), fwrite(), fread(), memcpy(), ZeroMemory(), or whatever 'C' based function you want to use on non-POD types.

The only way you set member values, get member values, or use a non-POD type is through the valid access specification that is available to you. Either it will be only the public interface, or public and protected interface if your inheriting.

Regards,

Paul McKenzie

ok, I think understand. My problem was that I was operating under the assumption that the fread, fwrite functions were C++ specific since they are listed on a C++ reference site that I use and I thought they were safe. I will rethink my approach.

So is the v-table used by C++ only?

[Paul McKenzie]

ok, I think understand. My problem was that I was operating under the assumption that the fread, fwrite functions were C++ specific since they are listed on a C++ reference site that I use and I thought they were safe.

When C++ was created, it was deemed advantageous to also inherit the 'C' standard library. The definitions of those 'C' functions remain the same, and that is where we get into problems.

The problem is that unlike 'C', C++ has two basic types, POD and non-POD types. 'C' only has POD types, so there is no issue using functions such as fwrite and fread for anything you want with little to no issue. However for C++, you need to know that what you can do with a 'C' or 'C'-compatible type (a POD type) cannot be done on a C++ type (one that contains virtual functions, for example).

So is the v-table used by C++ only?

Yes. There is no such thing as virtual functions in C. You can mimic what virtual functions do in 'C' (not trivial), but on a language level, there are no virtual functions in 'C', so no v-tables.

Regards,

Paul McKenzie

[CppCoder2010]

When C++ was created, it was deemed advantageous to also inherit the 'C' standard library. The definitions of those 'C' functions remain the same, and that is where we get into problems.

The problem is that unlike 'C', C++ has two basic types, POD and non-POD types. 'C' only has POD types, so there is no issue using functions such as fwrite and fread for anything you want with little to no issue. However for C++, you need to know that what you can do with a 'C' or 'C'-compatible type (a POD type) cannot be done on a C++ type (one that contains virtual functions, for example).

Yes. There is no such thing as virtual functions in C. You can mimic what virtual functions do in 'C' (not trivial), but on a language level, there are no virtual functions in 'C', so no v-tables.

Regards,

Paul McKenzie

Most of the time I see you to be right 100&#37; Paul, but you said that 'C' has nothing but POD types. I thought POD types also required that there be no pointers as member variables too?

So, 'C' has non-POD types as well. Consider.

typedef struct __POD_DATA
{
    unsigned long Val1;
    double Val2;
} POD_DATA;

typedef struct __NON_POD_DATA
{
    POD_DATA * Header;
    unsigned long Val1;
} NON_POD_DATA;

Although mentioned before about pointers being non-POD in another post, consider the following. You know very well what happens.

// Mixing C and C++ can still be dangerous if you don't know what you are doing.

NON_POD_DATA * pData = new NON_POD_DATA; // C++ new operator

pData->Header = new POD_DATA;

pData->Header->Val1 = 100;
pData->Header->Val2 = 100.0;
pData->Val1 = 200;

// here comes the mistake
fwrite(pData,1,sizeof(NON_POD_DATA),file_pointer);

// let's free the header value to show what happens upon read. If I didn't delete it
// it would be worse with a memory leak as well.
delete pData->Header;
pData->Header = NULL;

// consider the following code happens after closing file_pointer and reopening it as read only.

fread(pData,1,sizeof(NON_POD_DATA),file_pointer);

This does not fail in terms of reading from the file, but the problem is the pointer member variable (the Header variable) is pointing at invalid memory.

The real reason I deleted the header variable was for two reasons. It is correct to do so when you are done with it, and if this was the same program (and not a new instance) then it would appear to have worked since it reread in the same address that wasn't deleted. Didn't want to leave a hanging pointer issue in this as well.

[Paul McKenzie]

Most of the time I see you to be right 100&#37; Paul, but you said that 'C' has nothing but POD types. I thought POD types also required that there be no pointers as member variables too?

So, 'C' has non-POD types as well.

No. Pointers are POD types, since they are integral values. Anything that is considered a 'C' compatible structure is POD. If a struct contains only pointer members, it is a POD struct.

class x
{
   int *px;
   int *py;
};

class x is a POD type. Whether you want to munge up the pointers to do weird or bad things doesn't change the definition of what a POD type is.

Regards,

Paul McKenzie

[CppCoder2010]

Okay, so they are POD types, but they are just not safe to dump the contents of the structure to memory or file straight up.