Win32 "Special Debug Heap"
CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Results 1 to 12 of 12

Thread: Win32 "Special Debug Heap"

  1. #1
    Join Date
    Sep 2003
    Location
    Forever Gone... For Now...
    Posts
    1,515

    Win32 "Special Debug Heap"

    Hi all,

    Wondering if anyone out there can shed some additional light on something I've just recently stumbled across and can't find much documentation on.

    Apparently, in Windows NT/2K/XP/Server2003, a different "special debug heap" is used when a process is started by a debugger. I am NOT talking about the different CRT heaps used by different project builds (RELEASE vs. DEBUG) etc.

    For example, the WinDbg documentation states:
    Processes created by the debugger behave slightly differently than they would under normal conditions.

    Instead of using the standard heap API, processes created by the debugger use a special debug heap. On Windows XP and Windows Server 2003, you can force a spawned process to use the standard heap instead of the debug heap by using the _NO_DEBUG_HEAP environment variable or the -hd command-line option.
    One thing I have noticed is that this special debug heap uses a fill pattern of 0xbaadf00d. To witness this, open VC++, and compile a RELEASE build of the following program:
    Code:
    #include <stdio.h>
    
    int main( void )
    {
        int* pn = new int;
        printf( "0x%08x (%d)\n", *pn, *pn );
        getchar();
        return 0;
    }
    When run outside of the debugger, I get the following (probably whatever random value is in memory):
    0x00320518 (3278104)

    When run from within the debugger (even though there is no debugging information), (or DependencyWalker), I get the following:
    0xbaadf00d (-1163005939)

    For additional reference info, I might point to the following thread on Google:
    http://groups.google.com/groups?hl=e...3DN%26tab%3Dwg

    Can anyone comment on this, or provide additional details (or where one might go for additional details) pertaining to this "special debug heap" and what _else_ it may be doing? I'm also interested in how common knowledge of this "special debug heap" is, and how others came to know about it...
    Last edited by vicodin451; October 10th, 2003 at 12:41 PM.
    Thought for the day/week/month/year:
    Windows System Error 4006:
    Replication with a nonconfigured partner is not allowed.

  2. #2
    Join Date
    Sep 2003
    Location
    Forever Gone... For Now...
    Posts
    1,515
    I'll toss in the following additional "references":
    http://groups.google.com/groups?hl=e...oogle%2BSearch

    http://groups.google.com/groups?hl=e...oogle%2BSearch

    http://groups.google.com/groups?hl=e...oogle%2BSearch

    Additionally (though it only makes sense, I'll mention it anyway), starting a program and attaching to it with a debugger does _NOT_ cause it to use the "special debug heap" to be used.
    Thought for the day/week/month/year:
    Windows System Error 4006:
    Replication with a nonconfigured partner is not allowed.

  3. #3
    Join Date
    Sep 2002
    Location
    Maryland - Fear The Turtle!
    Posts
    7,537
    The actual flag is:

    #define DEBUG_CREATE_PROCESS_NO_DEBUG_HEAP

    which is part of the dbgeng.h

    Not having XP/2003 NET I can't really look into this. But I think there is just a weebit of a mix up here. The comment in windbg, is in my opinion referencing the Debug CRT, so I assume it is saying don't use the debug CRT use the release CRT as there is no special debug heap other than the CRT heap as I know, the code you posted uses the CRT heap from the static link libC*D.lib.

    Also only windbg supports this for xp/2003 as far as I know, but I don't have .NET so maybe it supports the usage of the flag/env setting. Though MSDN doesn't reference it at all.

    Now on to your code....what version of VC are you running, and what OS are you running on.

    You should be getting 0xCD as your fill, which I get when run under vc6.0 and w2k.

    badf00d is used explicitly in a lot of places of course, or deadbeef, depends on the coder. But the CRT libs use the above fill 0xCD as you know for new in the debug CRT calls.

    Try stepping into it in the debugger, the new that is, step all the way into the CRT code, watch the memory and vars, for when you switch pop to badf00d.

    BTW: if you exit the IDE come back in and run this code, you still get the same results?

    Should be somewhere round here...

    Code:
    void * __cdecl _nh_malloc_dbg (
            size_t nSize,
            int nhFlag,
            int nBlockUse,
            const char * szFileName,
            int nLine
            )
    {
    
    ....
    pvBlk = _heap_alloc_dbg(nSize, nBlockUse, szFileName, nLine);
    I mean this kinda spells it out to me:

    Behavior of Spawned Processes
    [This is preliminary documentation and subject to change.]

    Processes created by the debugger behave slightly differently than they would under normal conditions.

    Instead of using the standard heap API, processes created by the debugger use a special debug heap. On Windows XP and Windows .NET Server 2003, you can force a spawned process to use the standard heap instead of the debug heap by using the _NO_DEBUG_HEAP environment variable or the -hd command-line option.

    Also, since the target application is itself a child process of the debugger, it inherits the debugger's permissions. This may allow the target application to perform certain actions that would otherwise be forbidden. For example, the target application might be able to affect protected processes.
    Insert CRT debug heap for specail debug heap.

  4. #4
    Join Date
    Sep 2003
    Location
    Forever Gone... For Now...
    Posts
    1,515
    Hi Mick_2002,

    Thanks very much for taking the time to reply...

    Originally posted by Mick_2002
    The actual flag is:

    #define DEBUG_CREATE_PROCESS_NO_DEBUG_HEAP

    which is part of the dbgeng.h
    I wish to look into this a bit more...

    Not having XP/2003 NET I can't really look into this. But I think there is just a weebit of a mix up here. The comment in windbg, is in my opinion referencing the Debug CRT, so I assume it is saying don't use the debug CRT use the release CRT as there is no special debug heap other than the CRT heap as I know, the code you posted uses the CRT heap from the static link libC*D.lib.
    Taking this out of the context of XP and Windows Server 2003 and .NET... My initial impression was that the comment in the WinDbg help was referencing the debug CRT as well, but the tests I ran seem to support that it is something else. Mainly because I was NOT using the DEBUG CRT:
    From my original post:
    To witness this, open VC++, and compile a RELEASE build of the following program
    [QUUTE]
    Also only windbg supports this for xp/2003 as far as I know, but I don't have .NET so maybe it supports the usage of the flag/env setting. Though MSDN doesn't reference it at all.
    [/QUOTE]
    The env var and the cmd line option do only work for XP and 2003 (and they do indeed work).

    Now on to your code....what version of VC are you running, and what OS are you running on.
    This was in VC++ 6.0 SP5, on XPSP1. It was a RELEASE BUILD of the code.

    You should be getting 0xCD as your fill, which I get when run under vc6.0 and w2k.
    Nope. I got different things - random junk when run from outside of the debugger (or when I attached the debugger AFTER having started the process from CMD.EXE) and 0xbaadf00d when run from INSIDE the debugger (DependencyWalker and MSDEV). Again, this was a RELEASE build of the program.

    badf00d is used explicitly in a lot of places of course, or deadbeef, depends on the coder. But the CRT libs use the above fill 0xCD as you know for new in the debug CRT calls.
    Of course, the problem here is that I wasn't using the DEBUG CRT.

    Try stepping into it in the debugger, the new that is, step all the way into the CRT code, watch the memory and vars, for when you switch pop to badf00d.
    I'll give this a shot later today or on Monday...

    BTW: if you exit the IDE come back in and run this code, you still get the same results?
    Yes. Always 0xbaadf00d when the release build is run under any debugger. Always random CRAP when run outside of the debugger.


    I mean this kinda spells it out to me:
    Insert CRT debug heap for specail debug heap.
    Again, I wasn't using the CRT debug heap.

    The additional links I provided (4 of them to threads one can search via GoogleGroups) also make reference to this "special debug heap"...
    Thought for the day/week/month/year:
    Windows System Error 4006:
    Replication with a nonconfigured partner is not allowed.

  5. #5
    Join Date
    Sep 2002
    Location
    Maryland - Fear The Turtle!
    Posts
    7,537
    Interesting...run it in a while(1) don't delete the pointer....what values do you get?

  6. #6
    Join Date
    Sep 2002
    Location
    Maryland - Fear The Turtle!
    Posts
    7,537
    Processor specific?

    My amd:
    0x00000000
    my intel:
    0xbaadf00d

  7. #7
    Join Date
    Sep 2003
    Location
    Forever Gone... For Now...
    Posts
    1,515
    Originally posted by Mick_2002
    Interesting...run it in a while(1) don't delete the pointer....what values do you get?
    When the release build of the program is spawned by the debugger, I get 0xbaadf00d every time. When run from CMD.EXE, I get frequently different (?) random values (but 0x00320178 occurs most often):
    Code:
    0x003202f0 (3277552)
    0x003202d8 (3277528)
    0x003202c0 (3277504)
    0x003202a8 (3277480)
    0x00320290 (3277456)
    0x00320278 (3277432)
    0x00320260 (3277408)
    0x00320248 (3277384)
    0x00320230 (3277360)
    0x00320218 (3277336)
    0x00320200 (3277312)
    0x003201e8 (3277288)
    0x003201d0 (3277264)
    0x00320178 (3277176)
    0x00320178 (3277176)
    0x00320178 (3277176)
    0x00320178 (3277176)
    0x00320178 (3277176)
    0x00320178 (3277176)
    0x00320178 (3277176)
    0x00320178 (3277176)
    0x00320178 (3277176)
    0x00320178 (3277176)
    0x00320178 (3277176)
    0x00320178 (3277176)
    Processor specific?

    My amd:
    0x00000000
    my intel:
    0xbaadf00d
    Hmm... now that is interesting. I've tried it on Intel CPUs only - a Celeron & a P3. That's all I have access to, unfortunately. Wonder if anyone else has an AMD and would care to run it... Or a TransMeta even...
    Thought for the day/week/month/year:
    Windows System Error 4006:
    Replication with a nonconfigured partner is not allowed.

  8. #8
    Join Date
    Sep 2003
    Location
    Forever Gone... For Now...
    Posts
    1,515
    OK... I wrote a cheesy little debugger that just continued the debug events. When I launched the process with CreateProcess, specifying the DEBUG_PROCESS creation flag, the RELEASE build of the test app always spit out 0xbaadf00d (instead of a while(1) I did it in a for loop 1000 times). When I specified 0 for the creation flags, I got random values.

    The debug CRT was not being used, and the debugger was doing nothing fancy (nothing plain, either - it was just a zombie debugger)... yet there is a definite fill pattern used when the process is created with the DEBUG_PROCESS flag.

    I wonder what else...
    Thought for the day/week/month/year:
    Windows System Error 4006:
    Replication with a nonconfigured partner is not allowed.

  9. #9
    Join Date
    Sep 2002
    Location
    Maryland - Fear The Turtle!
    Posts
    7,537
    I assume your familiar with windbg and are OS symbol happy? I'd put the checked build on and see if I can find more but here's what I got running a free build.

    Run a release mode in windbg, stepping into the Rtl calls, you'll see it uses the RtlDebugAllocateHeap.

    Eg:
    ntdll!RtlDebugAllocateHeap
    ntdll!RtlAllocateHeapSlowly
    ntdll!RtlAllocateHeap

    Now run it outside of the debugger, and attach. Break, and switch to the primary thread.

    Step into, into, into, into and so forth.

    You'll see it never calls into the RtlDebugAllocateHeap.

    So internally when ZwCreateProcess is called for the child proc being debugged, and the debug flag is set, it is obviously checking the flag, and denoting if a call to RtlDebugAllocateHeap is required.

    Now the difference between Intel and AMD, not a CPU hardware wizard but I guess AMD zero fills, while Intel likes to be funny.

  10. #10
    Join Date
    Sep 2003
    Location
    Forever Gone... For Now...
    Posts
    1,515
    I was able to duplicate your results, Mick_2002. So it indeed seems there is a special heap used when an app is started by a debugger. What are your thoughts on this "revelation"? No big deal? Notable, but not anything to worry or concern one's self about? Does it raise further questions about what the implications of creating a process with DEBUG_PROCESS are? Why isn't it documented more?

    Thanks again for taking the time to look into this!
    Thought for the day/week/month/year:
    Windows System Error 4006:
    Replication with a nonconfigured partner is not allowed.

  11. #11
    Join Date
    Sep 2002
    Location
    Maryland - Fear The Turtle!
    Posts
    7,537
    No big deal, it's been done that way for awhile, I just never noticed that it doesn't call RtlDebugAllocateHeap outside of the debugger.

    Though I'd like to know, if you pass that switch (I assume you have XP, one day I'll install it ) and see if under the debugger you now don't call into RtlDebugAllocateHeap...that would explain the comment and the switch eg: don't call RtlDebugAllocateHeap while in the debugger.

  12. #12
    Join Date
    Sep 2003
    Location
    Forever Gone... For Now...
    Posts
    1,515
    I am running XP; I did a "set _NO_DEBUG_HEAP=1" and invoked WinDbg; the env var does indeed work (RtlDebugAllocateHeap was not called). However, I am unable to get the -hd command-line option to work (windbg -hd does not have the same affect as setting _NO_DEBUG_HEAP and invoking WinDbg).

    Also, in the interest of being as complete as possible, freeing memory in a process created by a debugger results in calls to:
    ntdll!RtlDebugFreeHeap
    ntdll!RtlFreeHeapSlowly+0x36
    ntdll!RtlFreeHeap+0xf9
    Thought for the day/week/month/year:
    Windows System Error 4006:
    Replication with a nonconfigured partner is not allowed.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  


Azure Activities Information Page

Windows Mobile Development Center


Click Here to Expand Forum to Full Width

This is a CodeGuru survey question.


Featured


HTML5 Development Center