dynamic memory allocation for multidimensional arrays

[dinesh999lama]

How can i create a multidimensional array for more than 3 dimension like below:

int dim1, dim2, dim3;
dim1=10000; dim2=99; dim3=9;
#define s(i,j,k) (array[dim2*dim3*i + dim3*j + k])
double * array = (double *)malloc(dim1*dim2*dim3*sizeof(double));

If someone knows please tell me, i really need this thing.

[2kaud]

When posting code, please use code tags. Go Advanced, select the formatted code and click '#'.

Is this c or c++?

Basically, to have more dimensions you just extend the concept that you are using here

int dim1, dim2, dim3, dim4, dim5;
dim1=10000; dim2=99; dim3=9; dim4 = 100; dim5 = 20;
#define s(i, j, k, l, m) (array[dim2 * dim3 * dim4 * dim5 * i + dim3 * dim4 * dim5 * j + dim4 * dim5 * k + dim 5 * l + m])
double * array = (double *)malloc(dim1 * dim2 * dim3 * dim4 * dim5 * sizeof(double));

[dinesh999lama]

When i tried this i'm getting segmentation error. How can i occupy large memory for this, without getting segmentation error.
int dim1, dim2, dim3,dim4,dim5,dim6,dim7;
dim1=1000; dim2=50; dim3=50, dim4=50,dim5=9,dim6=9,dim7=9;
#define s(i,j,k,l,m,n) (array[dim6*i + dim5*j + dim4*k+dim3*l+dim2*m+dim1*n])
double * array = (double *)malloc(dim1*dim2*dim3*dim4*dim5*dim6*sizeof(double));
s(23,23,23,4,4,4)=1;
cout<<s(23,23,23,4,4,4)<<endl;

[superbonzo]

so, you really want to allocate a 1000*50*50*50*9*9*9 dimensional array of doubles ~ 679 GBytes !?
if yes, you 'd better look for so called sparse tensors data structures ( assuming most of entries are zero or some constant/lazily computable value )
what are you trying to do ?

[dinesh999lama]

actually it can be also used as type int. since d1,d2,.. all r in int format. i'll use any method to do this task and thank u for telling me about sparse tensors data structures, i'll see that once. and this is for a computer vision project. it recognises a 3d object and for every point in the object it saves all the possible d1,d2,d1d,... values. than in next cpp file it analyses a 2d colour image and if the recognised object is present in the image than it will detect it, along with its oriantation, distance, even if the object is partially seen.

[2kaud]

actually it can be also used as type int

You're still looking at potentially ~350Gb - which is way more than memory which is usually available! Even using short int (2 bytes) you're looking at ~ 175Gb (oh for a computer with 256GB of memory!). If you need this much data (??) and sparse tensors data structures aren't appropriate then you're possibly looking at implementing your own memory management system backed by disk storage using a paging mechanism.

PS How many of these elements will actually contain data? If the actual data to be stored can fit in memory, then a simplistic way of storing the data is to use a map - with the key value being a struct with an element for each dimension and the mapped value being the element data value. So something like

struct key {
  int dim1;
  int dim2;
  int dim3;
  int dim4;
  int dim5;
};

map<key, int> array;

You'll need to implement the operator< for type key which is fairly straightforward. Then by setting the various elements of key to the required element you can read/write the array. This won't be the fastest as map internally uses a tree but it will be quite easy to do - if the amount actual data to be stored can fit in memory.

[2kaud]

For c++, one simple way using a map as per post #6 could be

#include <map>
#include <iostream>
using namespace std;

struct dim {
	int dim1;
	int dim2;
	int dim3;
	int dim4;
	int dim5;

	dim(int d1, int d2, int d3, int d4, int d5) : dim1(d1), dim2(d2), dim3(d3), dim4(d4), dim5(d5) {}
};

class bigarray
{
public:
	int& operator()(int d1, int d2, int d3, int d4, int d5)
	{
		return myarray[dim(d1, d2, d3, d4, d5)];
	}

	bool exists(int d1, int d2, int d3, int d4, int d5) const
	{
		return myarray.count(dim(d1, d2, d3, d4, d5));
	}

private:
	map<dim, int> myarray;
};

bool operator<(const dim& ld, const dim& rd)
{
	if (ld.dim1 < rd.dim1)
		return true;

	if (ld.dim1 == rd.dim1) {
		if (ld.dim2 < rd.dim2)
			return true;

		if (ld.dim2 == rd.dim2) {
			if (ld.dim3 < rd.dim3)
				return true;

			if (ld.dim3 == rd.dim3) {
				if (ld.dim4 < rd.dim4)
					return true;

				if (ld.dim4 == rd.dim4)
					if (ld.dim5 < rd.dim5)
						return true;
			}
		}
	}
	return false;
}

int main()
{
	bigarray arr;

	arr(2, 3, 4, 5, 2) = 5;
	arr(1, 1, 1, 1, 1) = 1;

	cout << "arr(2, 3, 4, 5, 2): " << arr(2, 3, 4, 5, 2) << endl;
	cout << "arr(1, 1, 1, 1, 1): " << arr(1, 1, 1, 1, 1) << endl;
		
	cout << "arr(1, 1, 1, 1, 1): " << (arr.exists(1, 1, 1, 1, 1) ? "Exists" : "Not exists") << endl;
	cout << "arr(2, 2, 2, 2, 2): " << (arr.exists(2, 2, 2, 2, 2) ? "Exists" : "Not exists") << endl;
}

Note that using a map with [] creates an entry if it's not present even if used just to see if present. That's why .exists() is used.

[dinesh999lama]

O my god, that map method provided by 2kaud is perfect. wow. I just tried it, and it works like charm before this i was trying other methodes. thank u very much for telling me about this, although i am a cse student, i think i was sleeping when teacher was teaching me about this map, or i may not haves studied it till now. but seems like my problem for large dimension array is solved, cas of ur help. so thank u very much.

[2kaud]

You're welcome. For further info about map see http://www.cplusplus.com/reference/map/map/.

Note my comment in post #6 though. Although this is probably the easiest way to implement a sparse multi-dimensional array, it certainly isn't the fastest! If program speed is a major factor then other more complicated sparse array methods may still be needed.