Passing an STL array by reference - help

[jefe9]

how can I pass a matrix as a reference parameter?
I am using the following declarations:

typedef std::vector< std::vector<std::string> > ss_matrix_t;

I declare the matrix with the following statement, where nRows and nCols are integers

std::vector< std::vector<std::string> > vI2Matrix(nRows, std::vector<std::string>(nCols,""));

The function is called with:

int read_files(std::string fname, int nCols, int nRows,  ss_matrix_t &ssMat )

but I get a linker error:


error LNK2019: unresolved external symbol "int __cdecl read_splayed_files(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >,int,int,class std::vector<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >,class std::allocator<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > > >,class std::vector<class std::vector<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >,class std::allocator<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > > >,class std::allocator<class std::vector<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >,class std::allocator<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > > > > >)" (?read_files@@YAHV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@HHV?$vector@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@V?$allocator@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@2@@2@V?$vector@V?$vector@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@V?$allocator@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@2@@std@@V?$allocator@V?$vector@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@V?$allocator@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@2@@std@@@2@@2@@Z) referenced in function "int __cdecl readDatafile(void)" (?readDatafile@@YAHXZ)
1>C:\PROJECTS\cppVS2010\Harness\Debug\Harness.exe : fatal error LNK1120: 1 unresolved externals

I suspect the syntax of the declaration, but I am not sure what to do here?


If I change the call to the function, then the array ( matrix ) is passed by value, and it takes forever:

int read_files(std::string fname, int nCols, int nRows,  ss_matrix_t ssMat )

// this takes ages ( it does compile and link )

How can this be resolved?

Any advice appreciated

Jefe

[laserlight]

Perhaps you should check that the forward declaration of your read_files function matches the definition. If you can find no problem with that then post the smallest and simplest program that demonstrates this error.

By the way, you might as well make the fname parameter a const std::string&.

[jefe9]

Maestro, thanks, the forward reference was the culprit there!

as an aside, I erase that matrix when I have finished with it, and it is taking a real long time ... 600 seconds !

	JournalLog("Immediate before matrix.erase(begin, end)  ");
	std::vector<std::vector<std::string> >::iterator row1 = vI2Matrix.begin();
	std::vector<std::vector<std::string> >::iterator row2 = vI2Matrix.end();
	vI2Matrix.erase(row1, row2);		
	JournalLog("Immediate before return from readDatafile()  ");

is there a way to spped this erase operation?

the declaration of the matrix is, again:

  std::vector< std::vector<std::string> > vI2Matrix(nRows, std::vector<std::string>(nCols,""));

Thanks again

Jefe

[D_Drmmr]

as an aside, I erase that matrix when I have finished with it, and it is taking a real long time ... 600 seconds !

Did you time a fully optimized build (without iterator checking)?
What are the characteristics of your data? How many rows and columns, what type of strings are stored?

[Paul McKenzie]

Maestro, thanks, the forward reference was the culprit there!

as an aside, I erase that matrix when I have finished with it, and it is taking a real long time ... 600 seconds !

Unless your compiler implementation of std::string and std::vector is a bunch of crap, in no way should it take 600 seconds if you run an optimized build. Note the bold text.

If you're running an unoptimized, "debug" build, then you cannot use that as an indication of how fast or slow the erase will perform.

Also, exactly what compiler and version are you using?

is there a way to spped this erase operation?

There is nothing to do except to make sure you're running an optimized build. Again, no decent C++ compiler would have such a slow implementation, unless you're running an iterator-checked, unoptimized build, and not an optimized build.

Regards,

Paul McKenzie

[jefe9]

The compiler is the MS VS-2010 express version;

The number of rows is slightly over 1million = 1,118,135 rows
The number of columns is 5

The release non-debug optimised build does see a significant performance improvement ie 13 seconds for my code , ..., and 225 seconds for the erase() of that matrix.

I wrote this code for a performance improvement to some open source software, and the debug version was marginal, but the release version is excellent; except for this nasty sting in the tail ... 225 seconds to erase that matrix is still too unpleasant.

I have tried looking at making the strings fixed size; in the hope that would have an improved performance, but have found no ideas there, the syntax of C strings of a pointer to the first element of an array almost ignores the possibility of a fixed string usage?

Any suggestions very welcome.

Thanks

[Paul McKenzie]

The compiler is the MS VS-2010 express version;

The number of rows is slightly over 1million = 1,118,135 rows
The number of columns is 5

The release non-debug optimised build does see a significant performance improvement ie 13 seconds for my code , ..., and 225 seconds for the erase() of that matrix.

Well, erase() can't do magic. If you have that many rows, then what routine(s) do you think exist that can make the deallocation any faster?

You're accessing the heap, and any massive amount of allocation would cause this issue, STL or no STL. You are eventually allocating 5 * sizeof(std::string) for each row. Multiply that by sizeof(std::vector<std::string>) * 1,118,135.

So the total number of bytes allocated just to maintain this data is very large and that is only for the matrix alone, without even allocating the data needed for the vector or string's contents. When the vector is destroyed, you have to deallocate the memory used by over 5 million strings. Now do you think anything you can think of can make those deallocations fast?

Simply stated, you need to rethink your design if you have this many strings.

Regards,

Paul McKenzie

[laserlight]

I suppose you could try swapping with an empty vector, but it may or may not make a difference:

std::vector<std::vector<std::string> >().swap(vI2Matrix);