CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Results 1 to 11 of 11
  1. #1
    Join Date
    May 2009
    Location
    Boston
    Posts
    364

    segfault at declaration of 2D array

    Hello,

    I have a little program that does some statistical processing on text files. The program has thrown a segmentation fault with an input file that is larger that I have used before. I didn't write this tool, so I went searching for the problem. Data from the input file is dumped into a 2D array and the program fails at the declaration of that array.

    The code is,
    Code:
    double A[cont][Nr_col_split];
    The vales for cont and Nr_col_split and determined from the input file and in this case, cont=807 and Nr_col_split=350. I assume that these values are just too large and not enough memory can be allocated, or something like that. Do I need to use reserve or similar to set aside enough memory? I will have input files that are quite a bit bigger than this one, so perhaps there needs to be a different solution for storing the input file data.

    LMHmedchem

  2. #2
    2kaud's Avatar
    2kaud is offline Super Moderator Power Poster
    Join Date
    Dec 2012
    Location
    England
    Posts
    7,822

    Re: segfault at declaration of 2D array

    What version of c or c++ are you using?
    All advice is offered in good faith only. All my code is tested (unless stated explicitly otherwise) with the latest version of Microsoft Visual Studio (using the supported features of the latest standard) and is offered as examples only - not as production quality. I cannot offer advice regarding any other c/c++ compiler/IDE or incompatibilities with VS. You are ultimately responsible for the effects of your programs and the integrity of the machines they run on. Anything I post, code snippets, advice, etc is licensed as Public Domain https://creativecommons.org/publicdomain/zero/1.0/ and can be used without reference or acknowledgement. Also note that I only provide advice and guidance via the forums - and not via private messages!

    C++23 Compiler: Microsoft VS2022 (17.6.5)

  3. #3
    Join Date
    May 2009
    Location
    Boston
    Posts
    364

    Re: segfault at declaration of 2D array

    Quote Originally Posted by 2kaud View Post
    What version of c or c++ are you using?
    gcc version 4.8.2 (GCC)

    I am compiling with g++, but it looks as if this is written in c. Would it make any difference to compile with gcc?

    If I convert to static and hard code for this input file, it works fine.
    Code:
    static double A[807][350];
    How do I maintain the dynamic allocation without overloading the stack? I don't see how I can declare as a global without knowing the size. I could make it arbitrarily large, but I don't think that's the best way to do things in general.


    LMHmedchem
    Last edited by LMHmedchem; January 19th, 2014 at 04:01 PM.

  4. #4
    2kaud's Avatar
    2kaud is offline Super Moderator Power Poster
    Join Date
    Dec 2012
    Location
    England
    Posts
    7,822

    Re: segfault at declaration of 2D array

    With standard c++ you are not allowed to declare an array like this using variables as the array size is required to be known at compile time. IMO for c/c++ I would be using dynamically allocated heap memory storage for arrays of that size. If it was c++ I would use a vector container.
    All advice is offered in good faith only. All my code is tested (unless stated explicitly otherwise) with the latest version of Microsoft Visual Studio (using the supported features of the latest standard) and is offered as examples only - not as production quality. I cannot offer advice regarding any other c/c++ compiler/IDE or incompatibilities with VS. You are ultimately responsible for the effects of your programs and the integrity of the machines they run on. Anything I post, code snippets, advice, etc is licensed as Public Domain https://creativecommons.org/publicdomain/zero/1.0/ and can be used without reference or acknowledgement. Also note that I only provide advice and guidance via the forums - and not via private messages!

    C++23 Compiler: Microsoft VS2022 (17.6.5)

  5. #5
    Join Date
    May 2009
    Location
    Boston
    Posts
    364

    Re: segfault at declaration of 2D array

    This program has worked for a long time as written. I will process a smaller input file with no issues. It was also my understanding that the compiler needed to know the size of an array to know how much memory to allocate, so I'm not sure why this worked, or even compiled. I compile it in c++, but I don't know if that makes a difference.

    LMHmedchem

  6. #6
    Join Date
    Apr 1999
    Posts
    27,449

    Re: segfault at declaration of 2D array

    Quote Originally Posted by LMHmedchem View Post
    This program has worked for a long time as written.
    That means very little in the world of C++. There are programs that run for years that have hidden bugs, and those bugs get finally exposed when the program changes, it gets run on another computer, a hardware change, or for any other reason.
    It was also my understanding that the compiler needed to know the size of an array to know how much memory to allocate,
    You are confusing syntax with what actually happens.

    In C++, it is illegal to declare an array with a variable or an expression that uses a variable. The array size must be a compile time constant or expression. More than likely, you used gcc with the ANSI C++ flag turned off (it is off by default). Turn the flag on, and you will get the compiler errors concerning the array indices not being constant expressions. That means that all along, you were using an extension of the gcc compiler, and not 100% ANSI C++.

    Whatever this does in the world of gcc:
    Code:
    int x = 1000;
    int array[x][x];
    you need to figure out what it really does, the limitations of doing this, the minimum and maximum array sizes, etc. Again, C++ knows nothing about this construct.

    Regards,

    Paul McKenzie
    Last edited by Paul McKenzie; January 19th, 2014 at 04:52 PM.

  7. #7
    2kaud's Avatar
    2kaud is offline Super Moderator Power Poster
    Join Date
    Dec 2012
    Location
    England
    Posts
    7,822

    Re: segfault at declaration of 2D array

    Is this being compiled as c or c++ because gcc c supports c99 variable length arrays?
    All advice is offered in good faith only. All my code is tested (unless stated explicitly otherwise) with the latest version of Microsoft Visual Studio (using the supported features of the latest standard) and is offered as examples only - not as production quality. I cannot offer advice regarding any other c/c++ compiler/IDE or incompatibilities with VS. You are ultimately responsible for the effects of your programs and the integrity of the machines they run on. Anything I post, code snippets, advice, etc is licensed as Public Domain https://creativecommons.org/publicdomain/zero/1.0/ and can be used without reference or acknowledgement. Also note that I only provide advice and guidance via the forums - and not via private messages!

    C++23 Compiler: Microsoft VS2022 (17.6.5)

  8. #8
    Join Date
    May 2009
    Location
    Boston
    Posts
    364

    Re: segfault at declaration of 2D array

    Quote Originally Posted by Paul McKenzie View Post
    Whatever this does in the world of gcc:
    Code:
    int x = 1000;
    int array[x][x];
    you need to figure out what it really does, the limitations of doing this, the minimum and maximum array sizes, etc. Again, C++ knows nothing about this construct.
    Well, as I said, I didn't write this code and had never really looked at it before it failed today. I can hard code the array size using static and process the files I am looking at, but I would have to re-compile for each new data set.

    I need to figure out how to write this properly so that it will run with any size input file. If I was writing this fresh, I would try to make it more of a streaming app and process one column at a time so I didn't need some huge structure to hold the entire input file. I'm sure there are some nice things that could be done with vectors and templates like this example I found,

    Code:
    include <vector>
    using namespace std;
    namespace {
      template <size_t N>
      struct Y : public vector<double> { Y() : vector<double>(N) {} };
      template <size_t N1, size_t N2>
      struct XY : public vector< Y<N2> > { XY() : vector< Y<N2> > (N1) {} } ;
    }
    int main(int, char **) {
      XY<5000,500> x, y, z;
      XY<500,50> mini_x, mini_y, mini_z;
      // ...
      return 0;
    }
    I guessing that if I converted to something like this, I would need to change the code where the A array is accessed. As it is, I need to get this working and don't have time to extensively re-write it.

    Suggestions other than my hard coded hack would be greatly appreciated.

    LMHmedchem

  9. #9
    2kaud's Avatar
    2kaud is offline Super Moderator Power Poster
    Join Date
    Dec 2012
    Location
    England
    Posts
    7,822

    Re: segfault at declaration of 2D array

    I guessing that if I converted to something like this, I would need to change the code where the A array is accessed.
    No. You would still access the vector in the same way. Have a look at
    http://forum.codecall.net/topic/5495...sional-vector/
    http://www.yolinux.com/TUTORIALS/Lin...ialC++STL.html
    All advice is offered in good faith only. All my code is tested (unless stated explicitly otherwise) with the latest version of Microsoft Visual Studio (using the supported features of the latest standard) and is offered as examples only - not as production quality. I cannot offer advice regarding any other c/c++ compiler/IDE or incompatibilities with VS. You are ultimately responsible for the effects of your programs and the integrity of the machines they run on. Anything I post, code snippets, advice, etc is licensed as Public Domain https://creativecommons.org/publicdomain/zero/1.0/ and can be used without reference or acknowledgement. Also note that I only provide advice and guidance via the forums - and not via private messages!

    C++23 Compiler: Microsoft VS2022 (17.6.5)

  10. #10
    Join Date
    May 2009
    Location
    Boston
    Posts
    364

    Re: segfault at declaration of 2D array

    I am looking at something like this,

    Code:
    #include <vector>
    #include <iterator>
    
    // resize 2D storage structure
    void resizeVec( std::vector<std::vector<double> > &vec , int ROWS , int COLUMNS ) {
        vec.resize( ROWS );
        for( std::vector<std::vector<double> >::iterator it = vec.begin(); it != vec.end(); ++it )
        //for( auto &it : vec )
        {
            it->resize( COLUMNS );
        }
    }
    
    // declare 2D vector
    std::vector< std::vector<double> > A;
    
    // resize 2D vector to cont x Nr_col_split
    resizeVec(A , cont , Nr_col_split);
    
    ...
    I think that all the references to A such as, SAFE_FSCANF(2,file,"%s%lf",tmp,&A[j][i]);, will still work.

    Does this look like a reasonable approach?

    This does compile and run without error, but then again so do the last version most of the time. Performance it not super critical here, but I would like to know if I am doing something that is super slow or inefficient. I see allot of this kind if thing done with a template, when does it make sense to go that route and what are the advantages?

    LMHmedchem
    Last edited by LMHmedchem; January 19th, 2014 at 08:01 PM.

  11. #11
    Join Date
    Apr 1999
    Posts
    27,449

    Re: segfault at declaration of 2D array

    Quote Originally Posted by LMHmedchem View Post
    I am looking at something like this,
    Yes, that will work.

    1) Use typedefs:
    Code:
    #include <vector> 
    typedef std::vector<double> Double1D;
    typedef std::vector<Double1D> Double2D;
    Then instead of always typing in "std::vector<std::vector<double>>", you just say "Double2D".

    2) You can write the initialization using for_each and bind.
    Code:
    #include <vector>
    #include <functional>
    #include <algorithm>
    
    typedef std::vector<double> Double1D;
    typedef std::vector<Double1D> Double2D;
    
    // resize 2D storage structure
    void resizeVec( Double2D &vec , int ROWS , int COLUMNS ) 
    {
        vec.resize( ROWS );
        std::for_each(vec.begin(), vec.end(),  
                            std::bind2nd(std::mem_fun_ref(&Double1D::resize), COLUMNS));
    }
    Since you seem to be using C++ 11 specification, there is a "nicer" form of bind. However the code above should work for C++ 98 and above.

    The idea is to minimize writing hand-coded loops when you have for_each and other looping algorithm functions available. The reason is that you could make a silly mistake writing the loop (like forgetting to update the iterator, or not optimally updating the loop counter). And yes -- I have seen code where this mistake has been made, so don't think it doesn't happen in the field.

    Using the for_each(), most, if not all mistakes are compiler errors, and you would rather catch errors at compile time than run time. In other words, typos, silly mistakes, etc. that are still valid syntax when writing the for-loop could practically never happen when writing the equivalent for_each().

    Also, you can swap in and out functions (for example, what if you wanted to do some other function over all of the rows?).
    Does this look like a reasonable approach?
    As long as you're passing the address of either the cell itself (as you're doing now), or a pointer to the beginning of a row (for example)
    Code:
    void SomeFunction(double *d, int num)
    {
       // Old C-like function to do something with the row data.
    }
    
    Double2D A;
    SomeFunction(&A[1][0], A[1].size());  // pass the beginning of the second row, along with the number of elements in the row.
    then you should be ok. What you can't do is pass the entire 2d vector (or a pointer to it) to a function that expects a 2d array to be created from a double**. That will not work.

    Regards,

    Paul McKenzie
    Last edited by Paul McKenzie; January 19th, 2014 at 10:28 PM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  





Click Here to Expand Forum to Full Width

Featured