CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Page 1 of 2 12 LastLast
Results 1 to 15 of 29
  1. #1
    Join Date
    May 2003
    Location
    Pakistan
    Posts
    223

    Finding out how many memory my application consuming while running

    What is the maximum memory a window application , can take .
    actually i am developing a server which is going to consume huge memory , so i want to know what max i , can get in windows.

    and how can i judge how much memory , my server is consuming at a particular time .

    so if
    Current Consumed >=Total Possible Memory
    i will simply stop accepting new clients .

    What api might help me.

  2. #2
    Join Date
    May 2000
    Location
    Scotland, Livingston.
    Posts
    728
    See GlobalMemoryStatusEx in MSDN.

    Also look at Address Windowing Extensions (AWE) in MSDN. These give access to up to 256GB of memory depending on your OS and hardware.
    Dave Mclelland.

  3. #3
    Join Date
    Jun 2001
    Location
    Switzerland
    Posts
    4,443
    Normally, you should design the process of acceptind a client in a commit-or-rollback fasion. Then, if during the process you exhaust the memory, you can rollback the process.

    Because most modern operating systems are multitasking, you cannot assume that a memory snapshot you take at the time T will still have any value at the time T+t. Thus taking a snapshot and assuming that you will have enough memory is risky.

    My 2 cts.
    Gabriel, CodeGuru moderator

    Forever trusting who we are
    And nothing else matters
    - Metallica

    Learn about the advantages of std::vector.

  4. #4
    Join Date
    May 2003
    Location
    Pakistan
    Posts
    223
    Thanks for ure reply ,
    according to AWE from msdn my application program can take upto the 2GB of physical memory(If Vailbale).

    To Experiment it i made a small program and in the loop executed 5000 threads.i have 256 of phsical memory .
    What happened after 2008 threads , AfxBeginThread Started to return NULL, At this point my application was consuming alomost 23 MB.Because other program was running as well , so at that point my physical memory was on 100% usage.

    But i am imazed what about virtual memory , if my physical memory was finished , windows should have given me , slices from virual memory.
    can some one explain this.


    One more point , if we assume i have 2GB of physical memory , and i experiment this , in that condition , what might have been the result.

  5. #5
    Join Date
    Sep 2002
    Location
    Belarus - Tirol, Austria
    Posts
    647
    Well, 32-bit type processors can address 4Gb of virual memory. So every application gets 4Gb of address space for virtual memory. Ussualy, this memory devides: 2 Gb for user address space + 2 Gb for kernel address space. (3:1 - another possible case).
    2 Gb, U've told about, is an amount of virual memory which is available for your application for user address space. So that is what concerns to the question about virtual memory.
    Another question is why U can create a limited number of threads. One of the possible reasons is a limit for virtual memory for user/kernel-level stack for threads or for some internal sytem table size per process.
    In default configuration, every Win32 thread has 1Mb stack in user address space and some amount (possible that 1Mb too) in kernel address space. So 2000 threads has for about 2Gb in user address space and + ~2Gb in kernel = ~4Gb of virtual address space. So there is no any available resources for new threads.
    Possible solution is decrease the default stack size for new thread (I donot remember if that is possible for Win from user space Win32 API).
    Of course, as I've mentioned, system can has some limitations for internal system tables, as result for the number of available resources per process (threads, file handles,...).
    "UNIX is simple; it just takes a genius to understand its simplicity!"

  6. #6
    Join Date
    May 2000
    Location
    Scotland, Livingston.
    Posts
    728
    I suspect the what has happened is that you have filled your "virtual address space". Your program has access to 2GB of addresses. These addresses may be backed by physical (RAM), virtual (disk pagefile) memory, or be uncommited (no ram or VM mapped yet). Each thread also consumes some kernel resources (which may be limited).

    Its probably very inneficient to create a thread for each user connection. Threads have an overhead, the more threads you have running the more CPU time is spent context switching rather than doing real work.

    Maybe think of using a thread pool and work queues.
    Dave Mclelland.

  7. #7
    Join Date
    Sep 2002
    Location
    Belarus - Tirol, Austria
    Posts
    647
    Yep, U need to choose the model of scalability , which will be appropriate for your project goals.
    E.g. some servers use model when a main process runs some copies of working processes, which share the same TCP/IP socket handle and all of them are listening on it (at least in *nix world, there are enough servers doing that way). Then every process has a pool of working threads which can be resizing, depending on the level of client connections, etc.
    "UNIX is simple; it just takes a genius to understand its simplicity!"

  8. #8
    Join Date
    May 2003
    Location
    Pakistan
    Posts
    223
    Thanks for all u ,
    First of all as Dave McLelland said it is inefficient to make thread for each individual clients , actually i am developing a messenger service , for my company .In a messenger service it is very much necessry that ure response time is very fast. so i thought it to be the best approach.



    As dimm_coder said that each thread takes upto 1 MB of stack size . ok then why Task Manger still show 23 MB of Physical and 23 MB of Virtual Memory.
    I also decreased the Stack Size for each thread(Optional Parameter in AfxBeginThread) But the result was same.

    Another point raised by dimm_coder about server

    {
    "some servers use model when a main process runs some copies of working processes, which share the same TCP/IP socket handle and all of them are listening on it (at least in *nix world, there are enough servers doing that way). Then every process has a pool of working threads which can be resizing, depending on the level of client connections, etc.
    "}
    What i understand from it that at a prticual time , many process which share the same socket , r in the listening mode using same
    port.(If it is posible).But client will connect to which server .
    can u expain it further cauz if it is possible it is really powerfull.


    One last point what if my server runs on 64 bit OS(Windows 2003 server)
    Last edited by atif_ilm; October 21st, 2003 at 01:46 AM.

  9. #9
    Join Date
    Sep 2002
    Location
    Belarus - Tirol, Austria
    Posts
    647
    Originally posted by atif_ilm

    As dimm_coder said that each thread takes upto 1 MB of stack size . ok then why Task Manger still show 23 MB of Physical and 23 MB of Virtual Memory.
    I also decreased the Stack Size for each thread(Optional Parameter in AfxBeginThread) But the result was same.
    Hum... Something strange to hear for me... 23Mb of virtual memory for all 2000 threads? Hum... seems that it'll be better to write some small test and look at it byself...

    What i understand from it that at a prticual time , many process which share the same socket , r in the listening mode using same
    port.(If it is posible).But client will connect to which server .
    can u expain it further cauz if it is possible it is really powerfull.
    Yes, I can.
    First, I'll try to explain from Unix systems side .
    Processes in *nix systems have a possibility to fork themselves. That means that after the system call fork(), U'll have 2 equal copies of the same process. (a parent process (wich called fork()) and a new child process). Child processes inherit all opened handles of the parent process (socket, file descriptors,...). So if before fork(), the parent process opens some socket and set it to LISTEN state, then all its child processes will inherit it and will have a possibility to listen on it.
    Second, there is nothing strange or abnormal from system behavior on that side.
    Say, every system object (e.g. socket ) has an internal queue of threads/processes, who wait for its (socket's in this case) signaled state.
    When some connection is estableshed (by OS's TCP module), TCP module gives it to the first thread/process in that queue. So U cannot know wich process will accept next connection (but U don't have to warry about it, if U have the right design). By that way, server application can increase a reaction time per estabishing connection. If U have the only one copy of the listen server, then in a time when a new connection is establishing (before calls to select()), no any other connection can be accepted and must wait in the system queue.
    So, many post/web servers works by this way.

    For Win world, as I remember, DuplicateHandle() can provide the sharing of the same descriptor among some processes.
    "UNIX is simple; it just takes a genius to understand its simplicity!"

  10. #10
    Join Date
    May 2003
    Location
    Pakistan
    Posts
    223
    As far as windows concersns , microsoft says , if u want to share a socket among different threads , what u do first detach socket from the context of one thread and then Attach it in the context of other thread.So a socket at a prticual time is valid in a context of only one thread.
    It is what i do when in the OnAccept event handler , first i accept socket in the context of main thread , then i detach it and pass it to my thread object.
    then i start my thread, in the Initinstance new thread attatch this handle to local variable.
    So this is the situation in windows , so how can i implement this scenario in windows.


    that is why linux is powerfull.
    Last edited by atif_ilm; October 21st, 2003 at 02:48 AM.

  11. #11
    Join Date
    Sep 2002
    Location
    Maryland - Fear The Turtle!
    Posts
    7,537
    Originally posted by atif_ilm
    that is why linux is powerfull.
    Try not to insert MFC into the sockets picture...

    Try to understand the 'E' in economics...and you'll find linux isn't powerful.....nor is UN*X these days...

  12. #12
    Join Date
    Sep 2002
    Location
    Belarus - Tirol, Austria
    Posts
    647
    Originally posted by Mick
    Try not to insert MFC into the sockets picture...
    Yep, forget about it... I 'll be so confused if M$ has no such a possibility...

    Try to understand the 'E' in economics...and you'll find linux isn't powerful.....nor is UN*X these days...
    Oh.. come on, Mickey... and just don't miss the wind of changes flying over your head
    "UNIX is simple; it just takes a genius to understand its simplicity!"

  13. #13
    Join Date
    Sep 2002
    Location
    Maryland - Fear The Turtle!
    Posts
    7,537
    Originally posted by dimm_coder
    Yep, forget about it... I 'll be so confused if M$ has no such a possibility...
    Too many people don't know the difference between CSocket and just plain socket...that's the sad part...

    Oh.. come on, Mickey... and just don't miss the wind of changes flying over your head
    Yep in 1995 I saw the winds of change flying over my head, and move from UN*X to Windows...that was such a silly mistake..

    shall I quote the UN*X systems I worked on??? mind you I worked for a company that ported a certain protocol over to UN*X so it can be a long (and obscure) list

  14. #14
    Join Date
    Sep 2002
    Location
    Belarus - Tirol, Austria
    Posts
    647
    Originally posted by Mick
    Too many people don't know the difference between CSocket and just plain socket...that's the sad part...
    Yes, that's why I always say it's important to know and learn about conceptual concepts and idioms but not small details like methods of wrapper classes or even API functions names...

    Yep in 1995 I saw the winds of change flying over my head, and move from UN*X to Windows...that was such a silly mistake..

    shall I quote the UN*X systems I worked on??? mind you I worked for a company that ported a certain protocol over to UN*X so it can be a long (and obscure) list
    I remember that, if it'll be something like showed by the next link, then nooo thanks :
    http://www.levenez.com/unix/
    "UNIX is simple; it just takes a genius to understand its simplicity!"

  15. #15
    Join Date
    Sep 2002
    Location
    Maryland - Fear The Turtle!
    Posts
    7,537
    Originally posted by dimm_coder
    Yes, that's why I always say it's important to know and learn about conceptual concepts and idioms but not small details like methods of wrapper classes or even API functions names...
    Ohh dimm you didn't just say that did you??? I kinda agree, to make commerical software, who cares...but for you knowledge...I care...are you a better coder because you know the internals of windoze??? yes or no???


    I remember that, if it'll be something like showed by the next link, then nooo thanks :
    http://www.levenez.com/unix/
    Nope...good try..think the early 90's, think what they were using to move files back and fro...now think I'm gonna buy this shiny new UN*X system, but I need to talk to those old what is that word??? systems

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  





Click Here to Expand Forum to Full Width

Featured