problem in DLL and calling conventions

Hello...

I have a strange problem with calling conventions. I am building a MFC dll in VS 2005. I exported following function with the help of .def file.

Code:

---

bool GenerateSerialNumber(string strActivationCode, string strVID, string strPID, string& strSerialNumber);

---

I havent changed the calling convention for the project so the default is now __cdecl. Thus functions in the DLL file must be decorated. But when I use dependency walker to check, this function is not decorated. This is the first problem.

I build a test client to use this library as follow..

Code:

---

typedef bool (*GenerateSerialNumber)(string strActivationCode, string strVID, string strPID, string& strSerialNumber);
....
...
void CTestActivatorShimDlg::OnBnClickedOk()
{
        HMODULE hModule = LoadLibrary("c:\\ActivatorShimDll.dll");
       
        GenerateSerialNumber ptrGenerate;
        ptrGenerate = (GenerateSerialNumber)GetProcAddress(hModule,"GenerateSerialNumber");
       

        bool bRet = false;
        string str1("122345");
        string str2("");
        bRet = ((*ptrGenerate)(str2,str2,str2,str1));
        FreeLibrary(hModule);
}

---

This clients loads the library and calls the function properly. But at the end of the function i get following error..

Code:

---

Run-Time Check Failure #0 - The value of ESP was not properly saved across a function call.  This is usually a result of calling a function declared with one calling convention with a function pointer declared with a different calling convention.

---

which tells that there is something wrong about calling convention or function pointer . Is there something I am missing...

plz help...

That error occurs also if you have a memory overwrite. It doesn't appear only if there is a calling convention problem.

Quote:

---

Originally Posted by sachin871

I havent changed the calling convention for the project so the default is now __cdecl.

---

How is the function really declared when you compile the DLL? We can't be certain what you're showing us is the actual declaration.

Also, regardless of what you set the default calling convention to, it is easily overriden by using various modifiers on the function declaration (i.e. __stdcall or any of the other macros that have __stdcall in the definition).

Another thing, your DLL is passing strings by value. The only programs that can use your DLL and guarantee that there are no issues are programs compiled for Visual C++ 2005 and for that particular version/service pack of VC 2005. If you want to use the DLL for Visual Studio 2008, you will have to create another DLL for Visual Studio 2008.

The reason is that std::string is a class that may change how things are done internally, depending on version of the compiler. The only function parameter types that are guaranteed to work for all versions of the compiler are integral types (LONG, char, etc.), and pointers to these types, and user-defined POD structs.

To add to this, you must make sure your application is using the DLL version of the runtime. The reason why is again, std::string. You must make sure that the memory manager used for your app is the same one used by the DLL.

There are a lot of problems when passing std::string that go far beyond getting the calling convention correct. What is usually done is that the DLL takes a char buffer, and either fills it in with the information, or reads from it.

Regards,

Paul McKenzie

Hi paul...

Thanks a lot for the reply..

I was unknown to the fact of using std::string in a DLL and now I must used char* for this DLL. Thanks u alomost saved me..

I will go ahead and will change those definations to using char*.

Thanks again...

Hi Paul...

As you said, I changed the impementation for the dll functions to use char* instead of std::string. But I am still getting the same error...

I am not using any macro for calling conventions and also not specifying any calling convetion while declaring/defing the function.

I am posting the code for the dll as well as client below...

Code:

---

//ActivatorShimWrapper.h
//declaration
bool GenerateSerialNumber(const char* strActivationCode, const char* strVID, const char* strPID, char* strSerialNumber);

---

Code:

---

//CActivatorShimWrapper.cpp
//
bool CActivatorShimWrapper::GenerateSerialNumber(const char* szActivationCode, const char* szVID, const char* szPID, char* szSerialNumber)
{
        AFX_MANAGE_STATE(AfxGetStaticModuleState());
       
        string strActivationCode(szActivationCode);
        string strVID(szVID);
        string strPID(szPID);
       
        //validate parameters
        if(        0 == strActivationCode.length() ||
                0 == strVID.length() ||
                0 == strPID.length()
                )
        {
                return false;
        }
}

---

Client :

Code:

---

typedef bool (*GenerateSerialNumber)(const char* strActivationCode, const char* strVID, const char* strPID, char* strSerialNumber);
...
...

void CTestActivatorShimDlg::OnBnClickedOk()
{
        HMODULE hModule = LoadLibrary("C:\\Portal_VC8\\KeyPoint\\ActivatorShimDll\\debug\\ActivatorShimDll.dll");
               
        GenerateSerialNumber ptrGenerate;
        ptrGenerate = (GenerateSerialNumber)GetProcAddress(hModule,"GenerateSerialNumber");
       
        bool bRet = true;
        char* szRes = new char[10];
        memcpy(szRes,"",10);
        string str2("");
       
        bRet = ((*ptrGenerate)(str2.c_str(),str2.c_str(),str2.c_str(),szRes));
               
        delete []szRes;
        FreeLibrary(hModule);

}

---

Thanks...

OK, that looks a little better.

You should have a single header file that declares the exported functions. Do you have this? The header is to be used both for the DLL and for any application that calls the function.

Second, the easiest thing is to create a 'C' exported interface to the function:

Code:

---

#ifndef DLL_HEADER
#define DLL_HEADER

#ifdef BUILDING_DLL
    DLL_DEF __declspec( dllexport ) __cdecl
#else
    DLL_DEF  __cdecl
#endif
 
#ifdef __cplusplus
extern "C" {
#endif
LONG DLL_DEF GenerateSerialNumber(const char* strActivationCode, const char* strVID, const char* strPID, char* strSerialNumber);
#ifdef __cplusplus
}
#endif

#endif

---

This is an example of how to define your functions for a 'C' exported interface. If you're including this header when building your DLL, you define BUILDING_DLL in your preprocessor. If you're using it in an app that will use your DLL, you don't define BUILDING_DLL.

By making one common header, you don't make a mistake.

I also changed the return type to LONG. The "bool" is a C++ thing, and there is no guarantee that a "bool" on one version of Visual C++ will be exactly the same as a "bool" on another version of VC++. Yes, a "bool" is an integral type, but there is no standard as to the sizeof(bool). You should return 1 if true, and 0 if false, instead of "true" or "false".

The tricks to getting DLL's to behave properly across compilers, apps, etc. is to use the most generic interface that is viable with respect to parameters and return types.

Regards,

Paul McKenzie

Hi paul...

thanks for the reply...

I am loading the library dynamically using LoadLibrary(). Also I am using .def file, so i guess I dont need to use dllexport and dllimport explicitly.
Actually this is a MFC library and will be used by another MFC library. So we dont need to worry about diff compilers.
The actual problem of calling conventions still exist. I have build a sample Dll and sample client so that ppl can help me. I am attaching the zip with the same.

[note : keep the SampleDll.dll in your C: or change the path in the code]

Thanks..

Guru's...

Please help me solve my problem.

Why did you not mention that you are exporting class member functions? All the time, it looked as if you were exporting non-class functions.

A non-static member function is *not* the same thing as a non-class member function or a static member function, therefore this is totally incorrect:

Code:

---

typedef bool (*CallGetSet)(const char* szGet,char* szSet);

---

This is not the prototype for calling a class member function, but your code tries to cast to this type and call it through this type.

Code:

---

typedef bool (CSampleDLLApp::*CallGetSet)(const char* szGet,char* szSet);

---

That is the correct prototype.

The point is this -- pointers to non-static class members, and pointers to functions are not compatible. The reason for the crash is that you are trying to call functions using a type that is not correct.

The compiler generates code for a non-class member on the app side, and on the DLL side, the function is really a non-static member function. What ends up happening is that the stack is screwed up at the end of the DLL function call, due to problems with this pointers and other things.

Regards,

Paul McKenzie

Thats gr8..

Thanks a lot for solving my problem..

Yes I was exporting the class member and thats what is wrong. I need to declare static functions or export the whole class.

Thanks again...