C++ load data
CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Results 1 to 5 of 5

Thread: C++ load data

  1. #1
    Join Date
    Jul 2010
    Posts
    5

    C++ load data

    Dear All,

    I need to load data (24 X 28 X 40) from 24 X 28 files into a datastructure so that my program can use
    them to do operations.

    Each file has 40 data items, which are floating numbers.

    I hope that the data structure can be indexed by item name not only by integer.

    For example, for 24 states in USA, each state's power consumption is POWC in year Y.

    I hope that the data structure indexing is like this

    datastructure[newYork][year][powerconsumption]

    My question is :

    (1) how to load the data from 24 x 28 files by C++?

    It is boring to type file's name one by one in program.

    Is there an alternative to load the data by indexing the file names ?

    for example,

    for (filename = name1; filename < filenum ; filename++)
    {
    load data from filename;
    store loaded data in the datastructure.
    }

    (2) how to index the item of the datastructure by C++ ?


    Any help is appreciated.

    Jack

    July 11 2010

  2. #2
    Join Date
    Jul 2005
    Location
    Netherlands
    Posts
    2,014

    Re: C++ load data

    Quote Originally Posted by dtustudy68 View Post
    I hope that the data structure can be indexed by item name not only by integer.

    For example, for 24 states in USA, each state's power consumption is POWC in year Y.

    I hope that the data structure indexing is like this

    datastructure[newYork][year][powerconsumption]
    You can use enums to bind an identifier to an integer value. For the year that won't work, however, unless you prefix the year with a letter. Instead, you could just subtract the first year (stored in some variable).
    Alternatively, you can use maps for this, but that will make your program a lot slower.
    (1) how to load the data from 24 x 28 files by C++?

    It is boring to type file's name one by one in program.

    Is there an alternative to load the data by indexing the file names ?
    You can use a stringstream to generate the file name.
    Code:
    for (int i = 1; i <= filenum; ++i)
    {
        std::ostringstream oss;
        oss << "file_" << i;
        std::ifstream ifs(oss.str());
        // read from ifs
    }
    (2) how to index the item of the datastructure by C++ ?
    I don't understand your question.
    Cheers, D Drmmr

    Please put [code][/code] tags around your code to preserve indentation and make it more readable.

    As long as man ascribes to himself what is merely a posibility, he will not work for the attainment of it. - P. D. Ouspensky

  3. #3
    Join Date
    Jul 2010
    Posts
    5

    Re: C++ load data

    thanks

    I am working on a multi-computer Open MPI cluster system.

    If I put some data files in /home/mypath/folder, is it possible that all non-head nodes can access the files in the folder ?

    I need to load some data to some nodes, if all nodes can access the data, I do not need to load them to each node one by one.

    If multiple nodes access the same file to get data, is there conflict ?

    For example,

    fopen(myFile) by node 1, at the same time fopen(myFile) by node 2.

    Is it allowed to do that on MPI cluster without conflict ?

    Any help is appreciated.

    Jinxu Ding

    July 12 2010

  4. #4
    Join Date
    Aug 2000
    Location
    West Virginia
    Posts
    7,612

    Re: C++ load data

    How are the calculations distributed among the nodes ?
    Normally, each node would work on a sub-section of the
    data, but the amount of data is so small that I don't know
    how much improvement you would get using MPI unless
    there are a lot of calculations that need to be done.

    non-head nodes can access the files if they can see the folder
    (for example, it is NFS mounted). I imagine the conflicts would
    be the same as running separate codes on a single processor.

    IO on MPI systems is often done in one of the following ways:

    1) the head node does all the IO and sends/receives the data
    from the appropriate nodes.

    2) Each node does IO on its local system.

    3) a combination of above where you have N reader/writer nodes
    (where N is considerably less than the total number of nodes).

    4) MPI IO

  5. #5
    Join Date
    Jul 2010
    Posts
    5

    Re: C++ load data

    thanks

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  


Windows Mobile Development Center


Click Here to Expand Forum to Full Width

This is a CodeGuru survey question.


Featured


HTML5 Development Center