Variables overwriting each other in memory

[Shaner]

Hello . I'm experiencing a really weird bug that's been plaguing me throughout the development of my program - some of my variables (even ones that are not referenced anywhere except my debug segment) are having their values overwritten when I declare other variables.

Here's a very clear and simple example excerpt from my code.

int newvariable = 5;
cout<<newvariable; //prints 5
int anothernewvariable = 3;
cout<<newvariable; //prints 3

Its current manifestation is wreaking havoc on one of my arrays, setting what should be default 0 values to huge integers and negative numbers at random. Furthermore, I can't write 0 over the new integers without my program crashing. Does anyone have any idea what could be causing this? If you need I'll be glad to post the whole project up, but I'm starting off with this tentative excerpt post to see if anyone knows the culprit right off the bat. Thanks in advance, and I hope this is interesting for you all and worth posting .

[Lindley]

If it's happening with individual variables like that, then I have no idea; I can only suggest you completely clean and rebuild your project.

If it's happening with arrays, it may be caused by out-of-bounds writes.

[Shaner]

I cut out the majority of my code and isolated the event, here you go.

#include <cstdlib>
#include "search.cpp"

using namespace std;

int main(int argc, char *argv[])
{
Search emt;
emt.parse();
}

-----------------------------------------------------------

#include <iostream>
#include <fstream>
using namespace std;

class Search {
public:
int map[25][25][10][2];
Search() {
}
bool parse()

cout<<"test";
map[20][25][0][0] = 10; //this works
cout<<"test"; //gets to here just fine
map[25][25][0][0] = 10; //this breaks.
cout<<"test";
return true;
}
};


It's in bounds of the array, but... out of bounds of memory?

[Shaner]

Sorry, must have chopped off the opening curly brace of bool parse() { ... }. That was a typo and the code works just fine until after printing test a second time.

[Shaner]

I nested four for loops to iterate through the array and print out the integer at each index. Every integer was 0 until I got up into the high x's (22 and some distance in y), then I ran into a block of skewed integers. After that there were a few random radicals, then around x=23 the floodgates opened and every index had an arbitrary giant integer. It looks very much to me like my program is running over into in-use memory from other applications. I'm still not sure why this happened with single variables earlier, but it's likely the same problem induced by old variables I had declared before I took out most of the code.

[Lindley]

             int map[25][25][10][2];
             ....
                        map[20][25][0][0] = 10; //this works

25 is not a valid index for the 2nd dimension (valid indexes are 0 to 24).

[Shaner]

Interesting, I did not know that, thank you. Still though, I already went back and tried it with dimensions map[20][20][10][2], and again the integers toward the end got all corrupted. Even stranger, with this same set of dimensions, it let me examine all the way up to map[27][0][0][0] without error, when it clearly should have broken :S. For this same reason no errors were reported with the y range being 25.

When the x dimension was 25, errors did not pop up until x=22. When the x dimension was 20, errors started popping up around 17. I devised a temporary workaround by keeping the size of the array [25][25][10][2], but only operating in the sub-20 range for both x and y. I'd still really like to know what causes this bizarre series of phenomena.

[Paul McKenzie]

I'd still really like to know what causes this bizarre series of phenomena.

You need to realize that when you write beyond the bounds of an array in C++, the program's behaviour is undefined.

The program could work, could fail, could work only on your machine and fail on another machine, could work on 1,000 different machines and fail on machine 1,001, etc.

Regards,

Paul McKenzie

[Lindley]

Interesting, I did not know that, thank you. Still though, I already went back and tried it with dimensions map[20][20][10][2], and again the integers toward the end got all corrupted. Even stranger, with this same set of dimensions, it let me examine all the way up to map[27][0][0][0] without error, when it clearly should have broken :S. For this same reason no errors were reported with the y range being 25.

When the x dimension was 25, errors did not pop up until x=22. When the x dimension was 20, errors started popping up around 17. I devised a temporary workaround by keeping the size of the array [25][25][10][2], but only operating in the sub-20 range for both x and y. I'd still really like to know what causes this bizarre series of phenomena.

You'll need to show an example of code which is failing even when you do not access the array out of bounds. Otherwise we would just be guessing.

[jnmacd]

Interesting, I did not know that, thank you. Still though, I already went back and tried it with dimensions map[20][20][10][2], and again the integers toward the end got all corrupted. Even stranger, with this same set of dimensions, it let me examine all the way up to map[27][0][0][0] without error, when it clearly should have broken :S. For this same reason no errors were reported with the y range being 25.

When the x dimension was 25, errors did not pop up until x=22. When the x dimension was 20, errors started popping up around 17. I devised a temporary workaround by keeping the size of the array [25][25][10][2], but only operating in the sub-20 range for both x and y. I'd still really like to know what causes this bizarre series of phenomena.

You are not understanding arrays. You are still not accessing within the boundaries.

[Shaner]

You are not understanding arrays. You are still not accessing within the boundaries.

Could you explain a little bit more? I've posted the simplest incarnation of the bug already. values at map[25][25][0][0] *are* within the bounds of an array defined as map[25][25][10][2], aren't they?

Just to be clear I have tested this with map being defined as int map[20][20][10][2], and the same error came up when I tried to access a value at map[18][1][0][0]... it's not a problem with running over the max secondary array limit...

[Russco]

Could you explain a little bit more? I've posted the simplest incarnation of the bug already. values at map[25][25][0][0] *are* within the bounds of an array defined as map[25][25][10][2], aren't they?

Just to be clear I have tested this with map being defined as int map[20][20][10][2], and the same error came up when I tried to access a value at map[18][1][0][0]... it's not a problem with running over the max secondary array limit...

No they are not. Arrays are zero indexed in C++ therefore your arrays bounds are map[0][0][0][0] to map[24][24][9][1]. If you use any number higher than those shown in the placement shown you are out of logical bounds but not necessarily physical bounds due to the way the array is laid out in memory. I suspect you are writing to a lot of [x][y][z][2]'s and so by the 22nd outer array element you are starting to write outside the physical bounds but you have been exceeding logical bounds for a lot longer.

[Shaner]

oh, wow... silly mistake :/. The reason I didn't really consider it an array bounding problem was because it was happening to individual variables too, but that must have been caused by a separate bounding error elsewhere in the program reaching onto my variables' memory block. Fascinating, and thank you very much! I learned a lot from this.

[Paul McKenzie]

oh, wow... silly mistake :/. The reason I didn't really consider it an array bounding problem was because it was happening to individual variables too, but that must have been caused by a separate bounding error elsewhere in the program reaching onto my variables' memory block. Fascinating, and thank you very much! I learned a lot from this.

When you declare variables, most compilers place the variables in memory contiguously. This means that if you go and write beyond the boundaries of an array, you can potentially change other variable values.

Not only that, array boundary violations can be brought to the surface if you change the program in any way and rebuild and rerun the app. Then you'll scratch your head wondering why adding that trivial function, member variable, or that seemingly innocent line of code causes your app to crash. The reason is that a change in the binary executable "moves" code around, and that code includes the bug. That's why if you ever get this, always revert the code back to the failing state and fix the issue. Never believe you're fixing the bug by adding or removing code that really doesn't do anything buggy. We have had many "I've fixed my bug by adding this line of code..." topics here, and we have the fateful task of throwing cold water on the poster by letting them know that they haven't fixed anything.

Unlike other languages, there is no warning, dump, or anything like that occurring when you make mistakes like this in C/C++. If you did the same array access violation in a language such as Java or C#, then you get a stack trace and the program halts.

This is why you don't see products such as BoundsChecker, valgrind, Purify, etc. for Java, C#, Visual Basic, etc. These languages have part of their rules to do array boundary checking, so there is no need for tools such as the ones I mentioned. For C++, you are flying blind, and the language is trusting that you are not doing anything illegal. The problem is that when C++ programs become large and non-trivial, you need tools to figure out what went wrong when something crashes.

But the bottom line is this -- for C++, the rule is that if you access outside the bounds of an array, anything can happen, even having the program seemingly "work" flawlessly (until that fateful day when you need to demo the app at a customer site, and bingo, it crashes). I have seen programs that have run for years that on further inspection, have boundary access violations that were never fixed.

Regards,

Paul McKenzie