Getting data in to C++ from excel, any help would be greatly appreciated

**Surreall** · February 1st, 2013 07:03 AM

Originally Posted by VictorN

I wonder why you check the value of z to be zero after if has been used in

Code:

                float t = ( y / z );

Why not before it?

Debug it and you will see!

You are absolutely right, very silly of me

(and a little embarressed lol)

What is the best way to debug in code::blocks?

**VictorN** · February 1st, 2013 07:06 AM

Originally Posted by Surreall

What is the best way to debug in code::blocks?

Perhaps, you should read about it in the "code::blocks" documentation?
Or in Google!

**Surreall** · February 1st, 2013 07:30 AM

Originally Posted by VictorN

Perhaps, you should read about it in the "code::blocks" documentation?
Or in Google!

Yep got it now..i had to click on the project.

**2kaud** · February 1st, 2013 07:59 AM

You mention that it is taking about 6 minutes to run. Roughly, do you know how much of this time is spent reading the csv file and building the arrays?

**Surreall** · February 1st, 2013 08:01 AM

Originally Posted by 2kaud

You mention that it is taking about 6 minutes to run. Roughly, do you know how much of this time is spent reading the csv file and building the arrays?

Approx it takes, 0.45 seconds to read and load the intial arrays. Ie dataarray, directionarray & headerarray.

**Surreall** · February 1st, 2013 08:11 AM

I have ran the debugger, it is something to do with the stack. I get this error:

#0 76F315DE ntdll!LdrQueryProcessModuleInformation() (C:\Windows\system32\ntdll.dll:??)
#1 76F315DE ntdll!LdrQueryProcessModuleInformation() (C:\Windows\system32\ntdll.dll:??)
#2 76F2014E ntdll!LdrFindResource_U() (C:\Windows\system32\ntdll.dll:??)
#3 001AD80C ?? () (??:??)
#4 004010FD __mingw_CRTStartup () (??:??)
#5 ?? ?? () (??:??)

On this part of the code:

Code:

for (int l = 0; l < 200000; l++)
    {
                oneres.q = resultsarraystring[m];
                oneres.c = resultsarraystring[m+1];
                oneres.y = resultsarrayfloat[l];
                oneres.z = resultsarrayfloat[l+1];
                oneres.t = resultsarrayfloat[l+2];
                oneres.indicator1position = resultsarrayint[n];
                oneres.indicator2position = resultsarrayint[n+1];
                oneres.trade1position = resultsarrayint[n+2];
                oneres.trade2position = resultsarrayint[n+3];
                oneres.numberofcells = resultsarrayint[n+4];
                oneres.indicator1value = resultsarrayint[n+5];
                oneres.indicator2value = resultsarrayint[n+6];
                oneres.trade1value = resultsarrayint[n+7];
                oneres.trade2value = resultsarrayint[n+8];
                oneres.tradevalue = resultsarrayint[n+9];
                resultsarray.push_back(oneres);
                m = m + 2;
                l = l + 2;
                n = n + 10;
	}

I have noticed when i add watches to q & c, in the watch window they come up int, and description in the watch window says "<optimized out>" when they are in the code above.

Current full code;

Code:

#include <fstream>
#include <iostream>
#include <sstream>
#include <string>
#include <vector>

using namespace std;

// defines structure to contain waht is required for each line in the csv file
struct results {
    string q, c;
    double y, z, t;
    int indicator1value, indicator2value, trade1value, trade2value;
    int numberofcells;
    int indicator1position, indicator2position, trade1position, trade2position;
    int tradevalue;
};

int main()
{
/////This populats the dataarray with data1.csv crap
int dataarray[6077][37];

ifstream data2("Data1.csv");
string line2;
int rows1= 0;
while (getline(data2,line2,'\n'))
{
    stringstream bb(line2);
   string field1;
    int columns1 = 0;
    while (getline(bb,field1,','))
    {
        istringstream buffer(field1);
        int value = 2;
        buffer >> value;
        dataarray[rows1][columns1] = value;
        columns1++;
    }
    rows1++;
}

// do the same for directionarray
int directionarray[6077];
ifstream direction("direction.csv");
string line3;
int rows2 = 0;
while (getline(direction,line3, '\n'))
    {
    istringstream buffer1(line3);
    int value1 = 0;
    buffer1 >> value1;
    directionarray[rows2] = value1;
    rows2++;
    }
//cout << "Directionarray element, " << directionarray[0] << endl;

// do the same for headerarray
//int headerarray[columns];

string headerarray[37];
ifstream header("Header.csv");
string line4;
int columns2 = 0;
while (getline(header,line4, ','))
{
    headerarray[columns2] = line4;
    columns2++;
}

int binaryarray[32][5];

ifstream binary("Binary.csv");
string line5;
int rows4= 0;
while (getline(binary,line5,'\n'))
{
    stringstream cc(line5);
   string field2;
    int columns4 = 0;
    while (getline(cc,field2,','))
    {
        istringstream buffer2(field2);
        int value2 = 2;
        buffer2 >> value2;
        binaryarray[rows4][columns4] = value2;
        columns4++;
    }
    rows4++;
}

cout << "Binary element 0,0 =" << binaryarray[0][0] << endl;
cout << "Binary element 0,1 =" << binaryarray[0][1] << endl;
cout << "Binary element 0,2 =" << binaryarray[0][2] << endl;
cout << "Binary element 0,3 =" << binaryarray[0][3] << endl;
cout << "Binary element 0,4 =" << binaryarray[0][4] << endl;
cout << "Binary element 1,0 =" << binaryarray[1][0] << endl;
cout << "Binary element 1,1 =" << binaryarray[1][1] << endl;
cout << "Binary element 1,2 =" << binaryarray[1][2] << endl;
cout << "Binary element 1,3 =" << binaryarray[1][3] << endl;
cout << "Binary element 1,4 =" << binaryarray[1][4] << endl;


//i have now created arrays, dataarray, directionarray & headerarray with correct elements

int indicator1value = 1;
int indicator2value = 0;
int trade1value = 0;
int trade2value = 1;



int indicator1position = 4;
int indicator2position = 5;
int trade1position = 4;
int trade2position = 5;
int numberofcells = 4;

int tradevalue = 1;


int q = 0;
int c = 0;
float y = 0;
float z = 0;
float t = 0;
int jRows = 0;
int rows = 6077;
int columns = 37;
//int binary1 = 0;

//float resultsarraynumbers[5000][18];
vector<float> resultsarrayfloat;
resultsarrayfloat.reserve(5000);
vector<int> resultsarrayint;
resultsarrayint.reserve(5000);
vector<string> resultsarraystring;
resultsarraystring.reserve(5000);

//for (binary1 = 0;binary1 < 32; binary1++)
//{
//int indicator1value = binaryarray[binary1][0];
//int indicator2value = binaryarray[binary1][1];
//int trade1value = binaryarray[binary1][2];
//int trade2value = binaryarray[binary1][3];
//int tradevalue = binaryarray[binary1][4];

for (indicator1position = 4;indicator1position < 13; indicator1position++)
{
for (indicator2position = 1 + indicator1position;indicator2position < 9+ indicator1position; indicator2position++)
{
for (trade1position = 4;trade1position < 13; trade1position++)
{
for (trade2position = 1 + trade1position;trade2position < 9 + trade1position; trade2position++)
{
for (q = 0;q < columns - 1; q++)
    {
   for (c = 0;c < columns - 1; c++)
        {
        if (q==c) {} else
            {
            for (jRows = 0;jRows < rows - indicator2position - trade2position; jRows++)
                {
                if (dataarray[jRows][q]==2 || directionarray[jRows]==15 || directionarray[jRows]==45){}
                else if (numberofcells == 4)
                    {
                        if (dataarray[jRows + indicator1position][c]==indicator1value && \
                        dataarray[jRows + indicator2position][c]==indicator2value && \
                        dataarray[jRows + trade1position][q]==trade1value && \
                        dataarray[jRows + trade2position][q]==trade2value)
                        {
                        if (dataarray[jRows][q]==tradevalue)
                            {y++;} else {z++;}
                        }
                    }
                }
                //t = ( y / z );
                if (y==0){y = 1;}
                if (z==0){z = 1;}
                t = y / z;
                if ( t > 3 && y > 9 )
                {
                //cout << t << endl;
                resultsarrayfloat.push_back(t);
                resultsarrayfloat.push_back(y);
                resultsarrayfloat.push_back(z);

                resultsarrayint.push_back(indicator1position);
                resultsarrayint.push_back(indicator2position);
                resultsarrayint.push_back(trade1position);
                resultsarrayint.push_back(trade2position);
                resultsarrayint.push_back(numberofcells);
                resultsarrayint.push_back(indicator1value);
                resultsarrayint.push_back(indicator2value);
                resultsarrayint.push_back(trade1value);
                resultsarrayint.push_back(trade2value);
                resultsarrayint.push_back(tradevalue);

                resultsarraystring.push_back (headerarray[q]);
                resultsarraystring.push_back (headerarray[c]);
                }
               }
                y = 0;
                z = 0;
            }
        }
}
}
}
}
//}

//for (vector<int>::iterator it = v.begin(); it!=v.end(); ++it) {
//printout
//   cout << *it << ", " << endl;
//}

cout << "Results number, t=" << resultsarrayfloat[0] << endl;
cout << "Results number, y=" << resultsarrayfloat[1] << endl;
cout << "Results number, z=" << resultsarrayfloat[2] << endl;
cout << "Results trade, " << resultsarraystring[0] << endl;
cout << "Results indicator, " << resultsarraystring[1] << endl;
cout << "Results, y= " << y << endl;
cout << "Results, z= " << z << endl;


vector<results> resultsarray;

results oneres;

//Open output file for the csv
ofstream ofs("results.csv");

int m = 0;
int n = 0;

//Put some data into the result vector for test
	for (int l = 0; l < 200000; l++)
    {
                oneres.q = resultsarraystring[m];
                oneres.c = resultsarraystring[m+1];
                oneres.y = resultsarrayfloat[l];
                oneres.z = resultsarrayfloat[l+1];
                oneres.t = resultsarrayfloat[l+2];
                oneres.indicator1position = resultsarrayint[n];
                oneres.indicator2position = resultsarrayint[n+1];
                oneres.trade1position = resultsarrayint[n+2];
                oneres.trade2position = resultsarrayint[n+3];
                oneres.numberofcells = resultsarrayint[n+4];
                oneres.indicator1value = resultsarrayint[n+5];
                oneres.indicator2value = resultsarrayint[n+6];
                oneres.trade1value = resultsarrayint[n+7];
                oneres.trade2value = resultsarrayint[n+8];
                oneres.tradevalue = resultsarrayint[n+9];
                resultsarray.push_back(oneres);
                m = m + 2;
                l = l + 2;
                n = n + 10;
	}

//Write result vector to output csv file
	for (vector<results>::iterator it = resultsarray.begin(); it != resultsarray.end(); ++it) {
		ofs << it->q << "," << it->c << "," << it->y << "," << it->z << "," << it->t << "," << "," << \
		       it->indicator1position << "," << it->indicator2position << "," << it->trade1position << "," << it->trade2position << "," << \
		       "," << it->numberofcells << "," << "," << \
		       it->indicator1value << "," << it->indicator2value << "," << it->trade1value << "," << it->trade2value << "," << "," << \
		       it->tradevalue << endl;
	}
	ofs.close();

cout << "DONE!" << endl;

return 0;
}

**Surreall** · February 1st, 2013 08:26 AM

Just as a point of interest. All those for loops iterate 36889042944 times.

**2kaud** · February 1st, 2013 08:40 AM

Rather than having three different results vectors - one each for float, int and string - just have one vector and put the data direct into this rather than into the three different and later on copying to yet another vector.

Replace these

Code:

vector<float> resultsarrayfloat;
resultsarrayfloat.reserve(5000);
vector<int> resultsarrayint;
resultsarrayint.reserve(5000);
vector<string> resultsarraystring;
resultsarraystring.reserve(5000);

with

Code:

vector<results> resultsarray;
resultsarray.reserve(5000);
results oneres;

ofstream ofs("results.csv");

Replace

Code:

 
                resultsarrayfloat.push_back(t);
                resultsarrayfloat.push_back(y);
                resultsarrayfloat.push_back(z);

                resultsarrayint.push_back(indicator1position);
                resultsarrayint.push_back(indicator2position);
                resultsarrayint.push_back(trade1position);
                resultsarrayint.push_back(trade2position);
                resultsarrayint.push_back(numberofcells);
                resultsarrayint.push_back(indicator1value);
                resultsarrayint.push_back(indicator2value);
                resultsarrayint.push_back(trade1value);
                resultsarrayint.push_back(trade2value);
                resultsarrayint.push_back(tradevalue);

                resultsarraystring.push_back (headerarray[q]);
                resultsarraystring.push_back (headerarray[c]);

with

Code:

     oneres.t = t;
     oneres.y = y;
     oneres.z = z;
     oneres.indicator1position = indicator1position;
     .... etc

     resultsarray.push_back(oneres);

Then remove

Code:

vector<results> resultsarray;

results oneres;

//Open output file for the csv
ofstream ofs("results.csv");

int m = 0;
int n = 0;

//Put some data into the result vector for test
	for (int l = 0; l < 200000; l++)
    {
                oneres.q = resultsarraystring[m];
                oneres.c = resultsarraystring[m+1];
                oneres.y = resultsarrayfloat[l];
                oneres.z = resultsarrayfloat[l+1];
                oneres.t = resultsarrayfloat[l+2];
                oneres.indicator1position = resultsarrayint[n];
                oneres.indicator2position = resultsarrayint[n+1];
                oneres.trade1position = resultsarrayint[n+2];
                oneres.trade2position = resultsarrayint[n+3];
                oneres.numberofcells = resultsarrayint[n+4];
                oneres.indicator1value = resultsarrayint[n+5];
                oneres.indicator2value = resultsarrayint[n+6];
                oneres.trade1value = resultsarrayint[n+7];
                oneres.trade2value = resultsarrayint[n+8];
                oneres.tradevalue = resultsarrayint[n+9];
                resultsarray.push_back(oneres);
                m = m + 2;
                l = l + 2;
                n = n + 10;
	}

**Surreall** · February 1st, 2013 08:58 AM

Would these two lines be right?

Previously:

Code:

                resultsarraystring.push_back (headerarray[q]);
                resultsarraystring.push_back (headerarray[c]);

New?

Code:

                oneres.headerarray[q] = q;
                oneres.headerarray[c] = c;

ignore that, i was being thick

oneres.q = q;

**2kaud** · February 1st, 2013 09:16 AM

You would have

oneres.q = headerarray[q];
oneres.c = headerarray[c];

**Surreall** · February 1st, 2013 09:27 AM

Yep, struts definitely confuse me.

Working wonderfully thank you very much for your time and effort

**Surreall** · February 1st, 2013 09:37 AM

I am having one slight problem, with the headerarray[q] & c
Getting this warning,
C:\Users\Mike\Desktop\C++\Final\main.cpp||In function 'int main()':|
C:\Users\Mike\Desktop\C++\Final\main.cpp|10|warning: inlining failed in call to 'results::~results()': call is unlikely and code size would grow [-Winline]|
C:\Users\Mike\Desktop\C++\Final\main.cpp|238|warning: called from here [-Winline]|
C:\Users\Mike\Desktop\C++\Final\main.cpp|10|warning: inlining failed in call to 'results::~results()': call is unlikely and code size would grow [-Winline]|
C:\Users\Mike\Desktop\C++\Final\main.cpp|238|warning: called from here [-Winline]|
||=== Build finished: 0 errors, 4 warnings (0 minutes, 3 seconds) ===|

And after exporting, the q's and c's are empty

**2kaud** · February 1st, 2013 09:49 AM

Can you post your code again please after all the changes. I'll crank it through my VC compiler and see what comes out.

**Surreall** · February 1st, 2013 10:04 AM

I think it was me (user error), maybe i hadnt compiled before one of changes. But getting all info that was requested on the export now: Still getting those warnings though

Code:

#include <fstream>
#include <iostream>
#include <sstream>
#include <string>
#include <vector>

using namespace std;

// defines structure to contain waht is required for each line in the csv file
struct results {
    string q, c;
    double y, z, t;
    int indicator1value, indicator2value, trade1value, trade2value;
    int numberofcells;
    int indicator1position, indicator2position, trade1position, trade2position;
    int tradevalue;
};

int main()
{
/////This populats the dataarray with data1.csv crap
int dataarray[6077][37];

ifstream data2("Data1.csv");
string line2;
int rows1= 0;
while (getline(data2,line2,'\n'))
{
    stringstream bb(line2);
   string field1;
    int columns1 = 0;
    while (getline(bb,field1,','))
    {
        istringstream buffer(field1);
        int value = 2;
        buffer >> value;
        dataarray[rows1][columns1] = value;
        columns1++;
    }
    rows1++;
}

// do the same for directionarray
int directionarray[6077];
ifstream direction("direction.csv");
string line3;
int rows2 = 0;
while (getline(direction,line3, '\n'))
    {
    istringstream buffer1(line3);
    int value1 = 0;
    buffer1 >> value1;
    directionarray[rows2] = value1;
    rows2++;
    }
//cout << "Directionarray element, " << directionarray[0] << endl;

// do the same for headerarray
//int headerarray[columns];

string headerarray[37];
ifstream header("Header.csv");
string line4;
int columns2 = 0;
while (getline(header,line4, ','))
{
    headerarray[columns2] = line4;
    columns2++;
}

int binaryarray[32][5];

ifstream binary("Binary.csv");
string line5;
int rows4= 0;
while (getline(binary,line5,'\n'))
{
    stringstream cc(line5);
   string field2;
    int columns4 = 0;
    while (getline(cc,field2,','))
    {
        istringstream buffer2(field2);
        int value2 = 2;
        buffer2 >> value2;
        binaryarray[rows4][columns4] = value2;
        columns4++;
    }
    rows4++;
}

cout << "Binary element 0,0 =" << binaryarray[0][0] << endl;
cout << "Binary element 0,1 =" << binaryarray[0][1] << endl;
cout << "Binary element 0,2 =" << binaryarray[0][2] << endl;
cout << "Binary element 0,3 =" << binaryarray[0][3] << endl;
cout << "Binary element 0,4 =" << binaryarray[0][4] << endl;
cout << "Binary element 1,0 =" << binaryarray[1][0] << endl;
cout << "Binary element 1,1 =" << binaryarray[1][1] << endl;
cout << "Binary element 1,2 =" << binaryarray[1][2] << endl;
cout << "Binary element 1,3 =" << binaryarray[1][3] << endl;
cout << "Binary element 1,4 =" << binaryarray[1][4] << endl;
cout << "Binary element 1,4 =" << headerarray[0] << endl;
cout << "Binary element 1,4 =" << headerarray[1] << endl;


//i have now created arrays, dataarray, directionarray & headerarray with correct elements

int indicator1value = 1;
int indicator2value = 0;
int trade1value = 0;
int trade2value = 1;



int indicator1position = 4;
int indicator2position = 5;
int trade1position = 4;
int trade2position = 5;
int numberofcells = 4;

int tradevalue = 1;


int q = 0;
int c = 0;
float y = 0;
float z = 0;
float t = 0;
int jRows = 0;
int rows = 6077;
int columns = 37;
//int binary1 = 0;

//float resultsarraynumbers[5000][18];
vector<results> resultsarray;
resultsarray.reserve(10000);
results oneres;

ofstream ofs("results.csv");

//for (binary1 = 0;binary1 < 32; binary1++)
//{
//int indicator1value = binaryarray[binary1][0];
//int indicator2value = binaryarray[binary1][1];
//int trade1value = binaryarray[binary1][2];
//int trade2value = binaryarray[binary1][3];
//int tradevalue = binaryarray[binary1][4];

for (indicator1position = 4;indicator1position < 13; indicator1position++)
{
for (indicator2position = 1 + indicator1position;indicator2position < 9+ indicator1position; indicator2position++)
{
for (trade1position = 4;trade1position < 13; trade1position++)
{
for (trade2position = 1 + trade1position;trade2position < 9 + trade1position; trade2position++)
{
for (q = 0;q < columns - 1; q++)
    {
   for (c = 0;c < columns - 1; c++)
        {
        if (q==c) {} else
            {
            for (jRows = 0;jRows < rows - indicator2position - trade2position; jRows++)
                {
                if (dataarray[jRows][q]==2 || directionarray[jRows]==15 || directionarray[jRows]==45){}
                else if (numberofcells == 4)
                    {
                        if (dataarray[jRows + indicator1position][c]==indicator1value && \
                        dataarray[jRows + indicator2position][c]==indicator2value && \
                        dataarray[jRows + trade1position][q]==trade1value && \
                        dataarray[jRows + trade2position][q]==trade2value)
                        {
                        if (dataarray[jRows][q]==tradevalue)
                            {y++;} else {z++;}
                        }
                    }
                }
                //t = ( y / z );
                if (y==0){y = 1;}
                if (z==0){z = 1;}
                t = y / z;
                if ( t > 3 && y > 9 )
                {
                oneres.t = t;
                oneres.y = y;
                oneres.z = z;
                oneres.indicator1position = indicator1position;
                oneres.indicator2position = indicator2position;
                oneres.trade1position = trade1position;
                oneres.trade2position = trade2position;
                oneres.numberofcells = numberofcells;
                oneres.indicator1value = indicator1value;
                oneres.indicator2value = indicator2value;
                oneres.trade1value = trade1value;
                oneres.trade2value = trade2value;
                oneres.tradevalue = tradevalue;
                oneres.q = headerarray[q];
                oneres.c = headerarray[c];
                resultsarray.push_back(oneres);
                }
               }
                y = 0;
                z = 0;
            }
        }
}
}
}
}
//}

//for (vector<int>::iterator it = v.begin(); it!=v.end(); ++it) {
//printout
//   cout << *it << ", " << endl;
//}

//cout << "Results number, t=" << results[0] << endl;
//cout << "Results number, y=" << results[1] << endl;
//cout << "Results number, z=" << results[2] << endl;
//cout << "Results trade, " << results[3] << endl;
//cout << "Results indicator, " << results[4] << endl;
//cout << "Results, y= " << y << endl;
//cout << "Results, z= " << z << endl;


//Write result vector to output csv file
	for (vector<results>::iterator it = resultsarray.begin(); it != resultsarray.end(); ++it) {
		ofs << it->q << "," << it->c << "," << it->y << "," << it->z << "," << it->t << "," << "," << \
		       it->indicator1position << "," << it->indicator2position << "," << it->trade1position << "," << it->trade2position << "," << \
		       "," << it->numberofcells << "," << "," << \
		       it->indicator1value << "," << it->indicator2value << "," << it->trade1value << "," << it->trade2value << "," << "," << \
		       it->tradevalue << endl;
	}
	ofs.close();

cout << "DONE!" << endl;

return 0;
}

**2kaud** · February 1st, 2013 10:18 AM

It compiles without any warnings with my VS. Is the info generated correct and what you expected? How long is the program now taking to run?

Thread: Getting data in to C++ from excel, any help would be greatly appreciated

Thread Tools

Display

Re: Getting data in to C++ from excel, any help would be greatly appreciated

Re: Getting data in to C++ from excel, any help would be greatly appreciated

Re: Getting data in to C++ from excel, any help would be greatly appreciated

Re: Getting data in to C++ from excel, any help would be greatly appreciated

Re: Getting data in to C++ from excel, any help would be greatly appreciated

Re: Getting data in to C++ from excel, any help would be greatly appreciated

Re: Getting data in to C++ from excel, any help would be greatly appreciated

Re: Getting data in to C++ from excel, any help would be greatly appreciated

Re: Getting data in to C++ from excel, any help would be greatly appreciated

Re: Getting data in to C++ from excel, any help would be greatly appreciated

Re: Getting data in to C++ from excel, any help would be greatly appreciated

Re: Getting data in to C++ from excel, any help would be greatly appreciated

Re: Getting data in to C++ from excel, any help would be greatly appreciated

Re: Getting data in to C++ from excel, any help would be greatly appreciated

Re: Getting data in to C++ from excel, any help would be greatly appreciated

Bookmarks

Bookmarks

Posting Permissions