Return non-unicode string from dll

[ill_comms]

Hi,

I've been trying to make an exportable function return a string value. With not much success so far.

It is exporting just fine but what I'm getting back is a string full of rubbish. The program I'm calling the dll from is MapBasic basically a cutdown version of VB.

It says that the string to be returned must be as follows: You should store your string (not unicode) in a zero-terminated buffer and return a pointer to the buffer to MB.

As far as the function goes this is what I have:

 
LPTSTR __declspec(dllexport) __stdcall SayHello(BSTR BSTR_str)
{
 AFX_MANAGE_STATE(AfxGetStaticModuleState());

 CString CString_str = _T("");
 if (BSTR_str != NULL)
 {
  CString s;
  LPSTR p = s.GetBuffer(::SysStringLen(BSTR_str) + 1);
  BOOL UsedDefaultChar;
  ::WideCharToMultiByte(CP_ACP, 0, BSTR_str, -1, p, ::SysStringLen(BSTR_str)+1, NULL, &UsedDefaultChar);
 
  if (UsedDefaultChar)
  {
   CString_str = (LPCTSTR)BSTR_str;
  }
  else
  {
   CString_str = (LPCWSTR)BSTR_str;
  }
 }
 AfxMessageBox(CString_str);
 CString_str = CString_str + " plus some text";
 LPTSTR returnValue;
 returnValue = CString_str.GetBuffer(0);
 return returnValue;
}

At the moment I'm not sure what is non-unicode and how to send back the pointer.

Any suggestions will be greatly appreciated!
Regards Hayden

[D_Drmmr]

You are returning a pointer to a temporary object. Once you exit the function, CString_str's destructor will be called. That why you get rubbish.

If you want to return a pointer like this, you'll have to dynamically allocate memory for the string. You'll also need a mechanism to release that memory later on then.

[ill_comms]

Hi D_Drmmr

Thanks, ok this is where I get confused. Hope you don't mind I'm going to be talking about C++ and MapBasic (which is basically VB).

Would I allocate this memory back in my MapBasic/VB program by declaring the string then pass that string through ByRef to the dll add whatever text to that string, then, end the dll. If so back in the original program it will have updated that variable as well. Which also means that that actually function could be of 'void' return type.

Do you see this as logical? Or have some more advice on how to proceed?

Regards Hayden

[D_Drmmr]

Would I allocate this memory back in my MapBasic/VB program by declaring the string then pass that string through ByRef to the dll add whatever text to that string, then, end the dll. If so back in the original program it will have updated that variable as well. Which also means that that actually function could be of 'void' return type.

I think this is possible if you make sure that your C++ function doesn't write to any memory it shouldn't write to. Since that can be hard, I wouldn't consider it a very good option.

What I had in mind in my previous post, is to create (and export) two function in your C++ dll. In one you allocate the memory for the string you want to return and in the second you release that memory. In your code calling the dll functions, you'll have to make sure you call the second function for every call to the first function.

LPTSTR __declspec(dllexport) __stdcall SayHello(BSTR BSTR_str)
{
	//...
	char* ret = new char[size + 1];
	return strncpy(ret, str, size);
}

void __declspec(dllexport) __stdcall Release(char* str)
{
	delete [] str;
}

This is just to show the idea, don't count on this code to work.

[0xC0000005]

This is not directly related to your question, but I must point out as I have done before that using 'T' types and functions in a dll is not a good idea. 'T' is a compile-time macro that essentially goes away once the dll is built. After that, it's either char or wchar_t - 'T' is no longer relevant. With 'T' types, users of your dll get the wrong idea that it will work with either a UNICODE or ANSI build.

[ill_comms]

Hi 0xC0000005, D_Drmmr

thanks, what you say makes sense I'll use your path instead of the way I was going down!

0xC0000005, As I am a beginner please elaborate, to what part of D_Drmmr's example are you referring to as being 'T' type. Also what would you use in place of?

Regards Hayden

[0xC0000005]

I was actually referring to your original code, but the only thing that I would note about D_Drmmr's example is that it returns LPTSTR where I would suggest LPSTR.

If you use LPSTR then your dll is ANSI and cannot be used by UNICODE applications unless they do the adjustment first.

If you use LPWSTR and wchar_t then your dll is UNICODE and cannot be used by applications unless they do the adjustment first.

But if you use LPTSTR the problem is that your dll is compiled either as ANSI or UNICODE and the 'T' turns into either char or wchar_t. However, users of your dll who include the header file don't know that - they think that the 'T' will resolve to whatever their build specifies - but it won't if the application is different from the dll. In typical cases, the application will compile and even possibly link to the dll but you will have unexplained runtime errors due to the difference in character type. You may even get a message that the application failed to find the function in the dll which won't make sense unless you know the reason.

There are many reasons for selecting one method of building a dll over another but I don't want to offer advice here - just give you the facts.

If your application is the only one that is going to use the dll and both the app and the dll use the same character type (char or wchar_t) then the best solution may be to just to ignore what I have said and continue to use 'T' types.

The next option would be to remove the 'T' types and use specific types: i.e. LPSTR rather than LPTSTR and char rather than TCHAR. That makes it very clear what types of applications can use your dll.

My personal preference is to follow Microsoft's example by including and ANSI and UNICODE function in every dll and make them accessible via 'T' macros. For example:

LPWSTR __declspec(dllexport) __stdcall SayHelloW(BSTR BSTR_str)
{
	//...
	wchar_t* ret = new wchar_t[size + 1];
	return wcscpy(ret, str, size);
}

LPSTR __declspec(dllexport) __stdcall SayHelloA(BSTR BSTR_str)
{
	//...
	char* ret = new char[size + 1];
        // Need to concvert here from BSTR to char...
	return ret;
}

#if defined(_UNICODE)
#define SayHello SayHelloW
#else
#define SayHello SayHelloA
#endif

This is the most flexible option since any type of application can link to your dll, but obviously it requires more work and complexity.

Once again, I don't mean to offer advice but just present the facts so that you can make an educated decision.

[ill_comms]

Thanks for the advice the return is working perfectly. And I have implemented both functions just in case, however I believe that I'll only ever use ANSI, but assumptions are the mother of all mistakes.

One issue remains and that is the clearing of the memory. I understand that this is important and why, though how to do this I'm not sure.

0xC0000005 do you have any ideas that would expand on D_Drmmr's Release function?

[Paul McKenzie]

Thanks for the advice the return is working perfectly. And I have implemented both functions just in case, however I believe that I'll only ever use ANSI, but assumptions are the mother of all mistakes.

One issue remains and that is the clearing of the memory. I understand that this is important and why, though how to do this I'm not sure.

0xC0000005 do you have any ideas that would expand on D_Drmmr's Release function?

Please note that practically every Windows API function that handles strings relies on the caller to get the memory. The API function merely fills in the memory with the data. Therefore the API function doesn't care where that memory comes from, and doesn't worry about deleting the memory.

I would rather prefer that method than having to micromanage who allocates and deletes memory -- let the caller get their own buffer and pass it to your DLL function.

Regards,

Paul McKenzie

[ill_comms]

Hi,
Thanks paul understand what you're saying however in this case I would prefer to keep things as they are, and would like to keep memory allocation fairly exact.

With your way either I'd be guessing the longest string length and crossing fingers it doesn't overload or I'd be calling the dll twice anyway first to get the 'to be' string length returned and then creating that buffer with the returned size and passing it through to be loaded. If two calls is micro-managing then the second option which is what I'd use is still doing it.

I'd still like to find out how to clear out the memory too!

[Paul McKenzie]

With your way either I'd be guessing the longest string length and crossing fingers

No.

If you look at the API functions, the caller passes the buffer and the maximum length of the buffer. There is no guessing involved. You fill in the buffer with the minimum of the actual string length or the length specified by the user.

Why not take a look at functions such as GetWindowText() and other API functions. A buffer and length parameter are required by the user.

If two calls is micro-managing then the second option which is what I'd use is still doing it.

And what if the user uses your function in complex ways? How are you going to keep track of all of those new's and delete's? What if the user passes you a bogus pointer to delete by mistake? How are you going to check it is the one you originally created? Do you have a factory pattern in place, where you have some sort of bookeeping to know what you allocated? These are all of the issues you bring upon yourself by trying to do too much hand-holding with the user of your DLL. Assuming that your users are not novices, I feel they should be handling any buffer management, not the DLL.

As far as two calls, it takes just two calls to allocate and free any memory in any application (new/delete, malloc()/ free(), etc.) -- that is not the point. It isn't the number of calls, it's the different ways in which those calls are used. If it were so easy, why are there so many error debugging tools for C++ programmers?

What I'm giving you is the way that most API programs and exported DLL functions handle strings in non-COM, professionally written DLL's. Either you can take that advice, or keep going against what has been proven to work. The only effort comes from the caller of the function -- let him/her worry about the buffers, that really shouldn't be your concern.

The only exception I know of where a DLL allocates memory is one where the DLL doesn't intend to clean it up (image data, for example, where the user wants the image to stay valid, even if the DLL is unloaded).

Regards,

Paul McKenzie

[ill_comms]

Hi Paul,

maybe I came over wrong, I'll back up a little. I have limited knowledge as yet and was trying to relay my understanding.

I have used a buffer with API before as below

Dim nRc As Integer,
 szBuffer As String,
 nBuffSize As Integer
 
 OnError GoTo ErrorOccured
 
'MAX_COMPUTERNAME_LENGTH is a const define as 31 chars
 nBuffSize = MAX_COMPUTERNAME_LENGTH + 1
 szBuffer = String$(nBuffSize, Chr$(32))
 
 nRc = GetComputerName(szBuffer, nBuffSize)
 APIGetComputerName = RTrim$(Left$(szBuffer, nBuffSize))

To give you a bit of background into what I'm doing I've got a class I've defined within my dll that reads an XML file. The values that it is going to be returning could be any length.

With your example I'll amend what I said before, it won't over-populate the buffer, it might however truncate it?

[Paul McKenzie]

To give you a bit of background into what I'm doing I've got a class I've defined within my dll that reads an XML file. The values that it is going to be returning could be any length.

With your example I'll amend what I said before, it won't over-populate the buffer, it might however truncate it?

Here are your options:

1) Allocate the memory using a Windows allocation function (not the C++/C heap allocation functions), and document that the user themselves must call the Windows deallocation function to clean up. One such function is GlobalAlloc()/GlobalFree() and their equivalents. Many imaging libraries work this way, and I guess you want the user's XML buffer to remain as long as the user wants it, even if they unload your DLL (they could have used LoadLibrary to access your DLL, a case you must consider).

The reason why you can't really use C++ functions such as new[] or C functions such as malloc() is that the user may not be using the same allocator as the DLL is using. Therefore they cannot free the memory themselves -- you have to do it, and that defeats the goal of having the data stick around as long as they want.

2) You allocate the memory, but you keep track of the pointers you've allocated. Then when the user wants to call your "XMLDeallocate" function or whatever you want to call it, you check the pointer passed to you, and you delete the memory if the pointer is a valid one. This is also used, and is the classic "I give you a handle, you use the handle, when you're done, you give it back to me for disposal" type of interface.

You are attempting 2), but the approach that you're using is unsafe, as again, you have no check that the user is giving you back something valid to delete.

3) You have the function work in two modes -- when you call the function with a NULL or some other argument to indicate "just return the length of the buffer that will be needed". The user will call this version of the function to get the length. Then when they have the length, they call your function again, but this time the argument is the buffer and length. Of course, you have to be a little smart and make sure you cache the XML file somewhere so that you don't have to rebuild the XML data on the second call.

For example:

int length = GetXMLData(NULL, 0);  // just get the length of data (NULL indicates that the user just wants the length
// now I know the length
std::vector<char> MyBuffer(length);
GetXMLData(&MtBuffer[0], length);  // get the data

Assuming the GetXMLData() is your exported DLL function, a lot of Windows API functions work this way. This gives the user a chance to query how big the data will be, create a buffer of that size, then call the function again with the buffer, sized appropriately.

So those are 3 typical ways to handle this that work universally for string handling.

Method 1) requires you to allocate memory that the user is totally responsible for.

Method 2) differs from 1) in that you want to take full responsibility for the memory, so you use a "handle" approach, and you check for validity when the handle is used/returned to you for deletion,

3) is similar to 1), but gives the user a chance to properly size their buffer before giving you the buffer to them.

So in a nutshell, your interface needs a little more sophistication to be safe to use with a reduction of memory leak.

I've written DLL's that must communicate strings to other clients that may not be C++ or C clients. and usage of one or more of the 3 techniques above is the way to make string handling and DLL's universally adaptable to any environment using the DLL.

Regards,

Paul McKenzie

[rxbagain]

If MapBasic behaves exactly like VB, you can return a BSTR containing ANSI string and VB (or MapBasic) will automatically release the pointer using SysFreeString.

The DLL code would be like this

BSTR WINAPI TestFunc()
{
   LPCSTR pszText = "This is my string";
   return SysAllocStringByteLen(pszText, strlen(pszText));
}

And in VB

Private Declare Function TestFunc Lib "TESTDLL.dll" () As String

...

MsgBox TestFunc

Note that I used SysAllocStringByteLen to create the "non-unicode" BSTR and VB will take care of freeing the memory.

[ill_comms]

Hi Paul,

brilliant thanks I'm going to go with option 3.

This is something else I haven't tried yet. Passing a buffer through by reference.
I know this hasn't worked maybe you could give me some pointers on what I'm doing wrong, as far as declaring the buffer on the dll side and populating it.

 
int __declspec(dllexport) __stdcall ReadXML(CString &MtBuffer, int length, BSTR xmlFilePath, BSTR xmlNode)
{
 AFX_MANAGE_STATE(AfxGetStaticModuleState());
 CString str_xmlFilePath = BSTR2CString(xmlFilePath);
 CString str_xmlNode = BSTR2CString(xmlNode);
 
 CString m_xml_value;
 m_xml_value = _T("");
 std::string _label;
 
 ParamIO inXml;
 inXml.readFile(str_xmlFilePath);
 inXml.read(str_xmlNode, _label, std::string(""));
 m_xml_value = _label.c_str();
 
 MtBuffer = m_xml_value;
 return 0;
}