dcsimg
CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Results 1 to 14 of 14

Thread: problem allocating memory in dll and freeing in application

  1. #1
    Join Date
    Aug 2006
    Posts
    515

    problem allocating memory in dll and freeing in application

    I have a function in a dll which returns some data. I allocate this data using 'new' in dll and work with it fine in calling application. The problem comes when I free this data after use, my application crashes with an error (corrupt heap).

    I have narrowed down the problem to the fact that I am allocating memory in dll and freeing up in application (it was not easy to figure this out since it works in some cases but not others). Apparently similar problem can occur if dll and exe are using different verions of CRT libraries but in my case I made sure they are the same but I still get this error so I am sure I can rule this one out.

    This is the function in DLL
    Code:
    void CMyClass::GetData( MY_STRUCT ** data)
    {
        *data = new MY_STRUCT[2];
    
        (*data)[0].num = 1;
        (*data)[0].msg = "first itme";
    
        (*data)[1].num = 2;
        (*data)[1].msg = "2nd item";
    }
    Here is the calling code in EXE:
    Code:
        MY_STRUCT * item = 0;
        
        file->GetData( &item);
    
        delete [] item; // bang!
    To make sure it is dll boundary related issue I wrote a simple function (below) in dll which deletes the memory. Now if I use the deleteMyStruct instead of 'delete' in exe, it works fine since the deallocation now occurs inside the dll.
    Code:
    MYDLL_API void deleteMyStruct(MY_STRUCT * p)
    {
        delete [] p;
    }
    My question now is how can I come around this in better way..should I override the delete constructor for the troubled class/structs? My goal is also for the dll to be independent of which ever CRT libraries the client application maybe using (debug/release/MTd etc). How can I return dynamic size data from the dll that a client application can free up after using?

    Any insight as to why this is happening in the first place would be greatly appreciated as well as there are no CRT objects in this simple structure and data types very simple. (std::string is there but the problem still exist if I remove it).
    Code:
    typedef struct tagMY_STRUCT
    {
        int num;
        std::string msg;
    } MY_STRUCT;

  2. #2
    Join Date
    May 2002
    Location
    Lindenhurst, NY
    Posts
    867

    Re: problem allocating memory in dll and freeing in application

    Rather than messing with low level memory allocation, why don't you just have your dll export functions that will wrap the new/delete calls?

  3. #3
    Join Date
    Aug 2006
    Posts
    515

    Re: problem allocating memory in dll and freeing in application

    Quote Originally Posted by Martin O View Post
    Rather than messing with low level memory allocation, why don't you just have your dll export functions that will wrap the new/delete calls?
    Does that mean overload new and delete operators?

  4. #4
    Join Date
    Apr 1999
    Posts
    27,449

    Re: problem allocating memory in dll and freeing in application

    Quote Originally Posted by zspirit View Post
    I have a function in a dll which returns some data. I allocate this data using 'new' in dll and work with it fine in calling application. The problem comes when I free this data after use, my application crashes with an error (corrupt heap).
    You should not try to allocate using "new" in one module and delete in another. To even be more blunt, you shouldn't try to allocate any memory in one module, and free it in another module. The one exception would be if you're allocating with a Windows API function such as GlobalAlloc(), and you have the app handle this memory and eventually free it using GlobalFree().

    Having allocation and deallocation done in separate modules leads to hard to maintain programs. The module that allocates should be the one that deletes the memory.
    Apparently similar problem can occur if dll and exe are using different verions of CRT libraries but in my case I made sure they are the same but I still get this error so I am sure I can rule this one out.
    You are sure you can rule it out, but we don't have your project to verify this ourselves. One thing that happens here often is that a poster will tell us one thing, but under further investigation they didn't do that thing they stated they did. This leads the ones that are trying to help down the wrong track from the beginning.
    This is the function in DLL
    Code:
    void CMyClass::GetData( MY_STRUCT ** data)
    {
        *data = new MY_STRUCT[2];
    
        (*data)[0].num = 1;
        (*data)[0].msg = "first itme";
    
        (*data)[1].num = 2;
        (*data)[1].msg = "2nd item";
    }
    Code:
    typedef struct tagMY_STRUCT
    {
        int num;
        std::string msg;
    } MY_STRUCT;
    The version of std::string must be exactly the same between application and DLL. This is another issue you have -- you're passing STL objects around from DLL to app. What if the version of std::string in the app is slightly different than the one the DLL uses?
    My goal is also for the dll to be independent of which ever CRT libraries the client application maybe using (debug/release/MTd etc). How can I return dynamic size data from the dll that a client application can free up after using?
    Your problem is not just memory allocation -- it's passing object where you have no control whether the application is using the same exact object. The app could have been built with different optimizations, settings, a different service pack, or a totally different version of Visual Studio.

    You should not have the application delete the memory, and in your case, also not directly manipulate the std::string. You need to create exported functions that manipulate this MY_STRUCT structure, and not have the application explicitly make changes to it. The exported functions are the ones that change or alter MY_STRUCT, not the application itself.

    Secondly, what if you change the way you allocate data? The application now has to change also just because you've changed the allocation. What if you now change to another string type instead of std::string? The application now has to change its code to handle your changes. Therefore your design is flawed in that everyone has to scramble changing their app, just because your DLL changes things.

    Using exported functions, the app never needs to make changes as your DLL controls everything.

    Regards,

    Paul McKenzie

  5. #5
    Join Date
    Apr 1999
    Posts
    27,449

    Re: problem allocating memory in dll and freeing in application

    Quote Originally Posted by zspirit View Post
    Does that mean overload new and delete operators?
    No, it means the app should not be in the "delete" business at all. The application should be calling an exported DLL function that is responsible for deleting the memory.

    Overloading operator new and delete should have nothing to do with solving the problem you're having now.

    Regards,

    Paul McKenzie

  6. #6
    Join Date
    Aug 2006
    Posts
    515

    Re: problem allocating memory in dll and freeing in application

    Quote Originally Posted by Paul McKenzie View Post
    Using exported functions, the app never needs to make changes as your DLL controls everything.
    I agree with you. Originally I was exporting a class from the dll, more precisely an interface class. The client therefore doens't know about the implementation and this makes for object oriented interface (this is why you see file->GetData(...)).

    I have to reevaluate this approach now because I return pointer to interface class which would eventually need to be deleted by client but based on my experience and this discussion, it would require another dll export function. I wanted the interface to be object oriented though so why can't I just do this?:
    Code:
     file->deleteMyself();// pardon the bad function name
    which is somewhat similar to the class' delete operator in that both are called on the class object.

    I actually had to create a simple demo application/dll to figure this bug out and test my design. It is attached to this post but do note the delete[] bug is NOT producible here. That's why it was not easy to figure out, the behavior is not predictable, it happens in some cases but doesn't in other.

    The demo project is in VC2010.
    Attached Files Attached Files
    Last edited by zspirit; May 20th, 2011 at 01:57 PM.

  7. #7
    Join Date
    Apr 1999
    Posts
    27,449

    Re: problem allocating memory in dll and freeing in application

    Quote Originally Posted by zspirit View Post
    I have to reevaluate this approach now because I return pointer to interface class which would eventually need to be deleted by client but based on my experience and this discussion, it would require another dll export function. I wanted the interface to be object oriented though so why can't I just do this?:
    Code:
     file->deleteMyself();// pardon the bad function name
    which is somewhat similar to the class' delete operator in that both are called on the class object.
    I have never exported classes, so I can't answer your question.

    But the probable reason why delete didn't work is your std::string member. How would you have enforced that std::string is exactly the same between the application and DLL?

    Externally they are the same, but the internals of the string classes could be different. So having the app just manipulate the string explicitly can't be done, even if you ensured that the DLL and app use the same heap.

    Regards,

    Paul McKenzie

  8. #8
    Join Date
    Aug 2006
    Posts
    515

    Re: problem allocating memory in dll and freeing in application

    My first thought was that since dll loads into the application's address space, anything it allocates will be from application heap which can therefore be freed by the application itself. I was able to produce the delete bug without std::string member in the structure so its a little confusing.

    Martin suggest wrapper APIs but even that would not solve the problem of dynamically allocating memory (return data of dynamic size).

    Assume we don't have any CRT objects in the class or structure, is it safe to allocate it in dll and released in the application? We need to reestablish that. MSDN documentation seems to imply that would be okay since this article discuss potential problem with CRT objects only.

  9. #9
    Join Date
    Apr 1999
    Posts
    27,449

    Re: problem allocating memory in dll and freeing in application

    Quote Originally Posted by zspirit View Post
    My first thought was that since dll loads into the application's address space, anything it allocates will be from application heap which can therefore be freed by the application itself. I was able to produce the delete bug without std::string member in the structure so its a little confusing.
    Again, it isn't just the heap that is the problem. Both application and DLL must be built using the same compiler, same options, same run-time heap, everything must be the same.
    Martin suggest wrapper APIs but even that would not solve the problem of dynamically allocating memory (return data of dynamic size).
    Why not? How does the Windows API and OS work? The OS dynamically allocates memory internally for window handles, GDI objects, etc. when you call, say CreateBrush, CreateWindowEx, etc. Why can't you have a "CreateMyStruct", "DestroyMyStruct", "UpdateMyStruct", etc. exported functions? Each one of these functions would have a pointer to MyStruct as a parameter.
    Assume we don't have any CRT objects in the class or structure, is it safe to allocate it in dll and released in the application?
    If the sizeof() that object is different, different internals, it doesn't matter what heap you use, and it doesn't matter if it's the CRT or your own objects. The allocation and deallocation in two different modules leads to undefined behaviour. Add to that, the object may dynamically allocate memory internally (i.e. std::string, or even your own classes), you have compounded the issue.
    Code:
    struct foo
    {
       #ifdef MY_SYMBOL_DEFINED
              int x[10];
       #else
             int x[20];
        #endif
    };
    Now, compile that with different compiler settings, in particular, the MY_SYMBOL_DEFINED is specified in a DLL, but is not specified in the application. What do you think will happen when you dynamically allocate in the DLL, and attempt to deallocate in the application? I just created an issue, as the two sizes are different.

    As an example of what can happen, the debugging CRT places guard bytes around dynamically allocated memory, and when you deallocate, it checks if the guard bytes have changed. But in the example above, the guard bytes are placed after the x array in the DLL, which has a size of sizeof(int) * 10. In the application, you call delete, but the checking for the guard bytes are done in a different location, since the size of the struct is different in the app. This change in size renders the debugging CRT as useless, since the guard bytes are in different locations.

    Regards,

    Paul McKenzie
    Last edited by Paul McKenzie; May 21st, 2011 at 01:06 PM.

  10. #10
    Join Date
    Jul 2005
    Location
    Netherlands
    Posts
    2,042

    Re: problem allocating memory in dll and freeing in application

    Quote Originally Posted by zspirit View Post
    I have to reevaluate this approach now because I return pointer to interface class which would eventually need to be deleted by client but based on my experience and this discussion, it would require another dll export function. I wanted the interface to be object oriented though so why can't I just do this?:
    Code:
     file->deleteMyself();// pardon the bad function name
    which is somewhat similar to the class' delete operator in that both are called on the class object.
    This is possible. It is used e.g. in the Xerces library, where the function is called 'release'.
    Another way would be to return a std::shared_ptr (or boost::shared_ptr) instead of a raw pointer. Since the call to delete is wrapped in the shared_ptr object, it will be called from the dll, always.
    Cheers, D Drmmr

    Please put [code][/code] tags around your code to preserve indentation and make it more readable.

    As long as man ascribes to himself what is merely a posibility, he will not work for the attainment of it. - P. D. Ouspensky

  11. #11
    Join Date
    Aug 2006
    Posts
    515

    Re: problem allocating memory in dll and freeing in application

    Quote Originally Posted by Paul McKenzie View Post
    Why not? How does the Windows API and OS work? The OS dynamically allocates memory internally for window handles, GDI objects, etc. when you call, say CreateBrush, CreateWindowEx, etc. Why can't you have a "CreateMyStruct", "DestroyMyStruct", "UpdateMyStruct", etc. exported functions? Each one of these functions would have a pointer to MyStruct as a parameter.
    I see a difference, in my case I am only returning data. The calling application has no idea about the size of data it will receive, the dll knows that and that's why it is allocating memory. One way to come around it would be have a dll function just return the size of data, the application than allocates the buffer and pass on to dill to fill in. It's two step process, seems a little untidy and I haven't seen this approach used anywhere...that's why the created std::string class to come around similar problems with return char strings. I am not clear on how the API can solve this problem.

    I in fact did modeled after a Windows/MFC API which I though closely resemble my case. In this case GetProfileBinary() reads data of unknown size from registry and it accomplishes it in one call. The calling application than frees the memory. From MSDN:
    Code:
        complex* pData;
        UINT n;
        BOOL ret = GetProfileBinary(pszKey, "ComplexData", (LPBYTE*)&pData, &n);
    
        delete [] pData; // free the buffer
    My original dll function is very similar, the demo was simple to catch delete bug so I omitted 2nd parameter.
    Code:
    void GetLocalData( MY_STRUCT ** data, int & size)
    {
        size = 2; // dll determines it, hard coded to 2 here
        *data = new MY_STRUCT[size];
    
        (*data)[0].num = 1; // hard coded for this demo
        (*data)[0].msg = "test";
    
        (*data)[1].num = 2;
        (*data)[1].msg = "niton";
    }
    Quote Originally Posted by Paul McKenzie View Post
    The allocation and deallocation in two different modules leads to undefined behaviour. Add to that, the object may dynamically allocate memory internally (i.e. std::string, or even your own classes), you have compounded the issue.
    I agree and I would think the dll function will have to allocate the memory once and keep it transparent. std::string or CRT objects shall not be used because dll function doesn't directly manage its memory (in this case by not deallocating it).

    So if the structures are plain and there is no hidden allocations. In theory, if the dll allocates memory, application should be able to release it (I think)? Is it any better for dll function to do business in terms of basic BYTE characters and type cast appropriately in the application (somewhat similar to the GetProfileBinary() function?. Thanks!

  12. #12
    Join Date
    Apr 1999
    Posts
    27,449

    Re: problem allocating memory in dll and freeing in application

    Quote Originally Posted by zspirit View Post
    to come around it would be have a dll function just return the size of data, the application than allocates the buffer and pass on to dill to fill in. It's two step process, seems a little untidy and I haven't seen this approach used anywhere...
    You haven't been looking very hard. Many Windows API functions and many other API's use the two step process. For these functions, if a parameter is a certain value, then the function returns the size of the data. This allows applications written in languages such as 'C' to use the API to get all the data instead of truncated data. Then the usual next step is that a C++ programmer creates a C++ wrapper for this interface, so that these steps are hidden.
    that's why the created std::string class to come around similar problems with return char strings.
    Not when you're talking about straddling DLL and module boundaries. If you build your DLL and it uses std::string, std::vector, std::list, or even your own classes as parameters to the functions being called by the application, the only applications that can use that DLL are ones built with the same compiler, version, options, etc. as the DLL.

    The only safe way to copy strings is to use regular char buffers. Again, look at the Windows API -- how do they send and return strings? The application has to give them a buffer, and then the API fills the buffer, or the app sends a pointer to the beginning of the buffer along with its size.
    I am not clear on how the API can solve this problem.
    See above. The DLL is the only module that should be doing anything directly to the class. When you have your application explicitly changing the struct or class, then you are going to get issues.
    I in fact did modeled after a Windows/MFC API which I though closely resemble my case. In this case GetProfileBinary() reads data of unknown size from registry and it accomplishes it in one call. The calling application than frees the memory. From MSDN:
    Code:
        complex* pData;
        UINT n;
        BOOL ret = GetProfileBinary(pszKey, "ComplexData", (LPBYTE*)&pData, &n);
    
        delete [] pData; // free the buffer
    This is not the same thing as what you're describing. This function allocates data within the same module as the caller, therefore it is safe to call delete[].

    What you are talking about is something completely different. You're talking about memory management between application and DLL.

    The only memory allocation functions that are guaranteed to work between app and DLL, regardless of the version of compilers used, the brand of compiler used, or even the computer language used are the Windows API allocation functions such as GlobalAlloc(), HeapAlloc(), etc. C++ memory management functions cannot be guaranteed to work consistently between DLL and application, unless the DLL and application are built with the same compiler, compiler version, and compiler options (and the heaps are the same).

    The bottom line is this -- if everything was OK, why are you getting an error? Could it be that all of the things you thought were correct aren't really correct?

    Regards,

    Paul McKenzie
    Last edited by Paul McKenzie; May 22nd, 2011 at 12:59 PM.

  13. #13
    Join Date
    Aug 2006
    Posts
    515

    Re: problem allocating memory in dll and freeing in application

    Quote Originally Posted by Paul McKenzie View Post
    The bottom line is this -- if everything was OK, why are you getting an error? Could it be that all of the things you thought were correct aren't really correct?
    I do think there is something incorrect that's why I got the error as well..it took me sometime time to figure out it was related to dll's boundary. I am just trying to better my solution. The question of how and where the dll can be used is always relevant and seems to have its challenges in most cases.

    My dll is reading data from a file. I am using virtual class for interface so all details are hidden from client..an important feature. If I use simple C style functions, I will have to expose some sort of file handle (as to which file the function is referring to). It has its own issues. I think C++ interface is the best way to go in my case and since we are C++ now I am trying to be as objected oriented as possible.

    I would like the structures that hold data to be plain old type though so other languages could possibly write wrapper code to use them as well. It think I should be able to delegate clean up to the dll and that could take care of this issue.

    I have bit of concern now passing STL objects like std::string between dll and exe. One one hand STL is powerful, make thing simpler and is 'Standard' library after all. On the other hand we don't have control over it and to make it part of dll interface might not be a wise thing to do.
    Quote Originally Posted by Paul McKenzie View Post
    This is not the same thing as what you're describing. This function allocates data within the same module as the caller, therefore it is safe to call delete[].
    I was looking at this more from user perspective. I don't know the internal implementation on this but an application is calling an API or function which allocates memory (just like the dll) and the application is freeing it. I don't know if GetProfileBinary() is build into the application, I was just thinking that MFC can be build as shared dll as well, so it could be similar. That was my thought. Thanks.

  14. #14
    Join Date
    Apr 1999
    Posts
    27,449

    Re: problem allocating memory in dll and freeing in application

    Quote Originally Posted by zspirit View Post
    I have bit of concern now passing STL objects like std::string between dll and exe. One one hand STL is powerful, make thing simpler and is 'Standard' library after all.
    That is only in the context of a single application or binary.

    The issue is not only STL -- anything you have no control over, you shouldn't pass to other modules as parameters or use as return values. For example, the FILE construct from <stdio.h> -- that is a 'C' struct, but in no way can you pass FILE* to an external module for the same reasons you can't pass a std::string -- incompatibilities between versions or build settings between app and DLL.

    If you want an MFC example, how about CString? There are many threads that talk about using an app built with Visual C++ 6.0 trying to pass (or return) a CString to/from a DLL built with Visual Studio 200x. Disastrous results can only happen in that scenario.
    I was looking at this more from user perspective. I don't know the internal implementation on this but an application is calling an API or function which allocates memory (just like the dll) and the application is freeing it. I don't know if GetProfileBinary() is build into the application, I was just thinking that MFC can be build as shared dll as well, so it could be similar. That was my thought. Thanks.
    Yes, but for every version of Visual Studio, you have to have the proper version of the MFC DLL's accompany you. If you want to break that coupling, then you can't use the shared DLL. So as sophisticated as MFC may seem, it also has the very same issues that you're encountering.

    Regards,

    Paul McKenzie

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  


Windows Mobile Development Center


Click Here to Expand Forum to Full Width




On-Demand Webinars (sponsored)