Problem in strtok()

Hello All,

I am using strtok() function to get the next token available from a string.

I am reding a file using memory mapping and storing the data in a char* buffer. from this buffer I am getting line by line using strtok() function by putting the token delimiters as "\r\n". I am having an error file that is not ended with the above delimeters as it is error file. strtok is crashing with this file at the last statement.

Is there any problem with strtok()? or with my code? Can any one have any idea? If so what is the solution for this?

I am sending the file as an attachment. If I try to change the contents and save it is working fine. So use this file without altering.

My code:

HANDLE hFile;
HANDLE hMap;
char* chFileData;

hFile = CreateFile("C:\\Temp\\temp.001",GENERIC_READ,0,NULL,OPEN_EXISTING,FILE_ATTRIBUTE_NORMAL,0);
DWORD fSize = GetFileSize(hFile,NULL);
hMap = CreateFileMapping(hFile,NULL,PAGE_READONLY,0,fSize,NULL);
int nError = 0;
nError = GetLastError();
if ((hMap != NULL) && (nError == ERROR_ALREADY_EXISTS))
{
CloseHandle(hMap);
hMap = NULL;
return FALSE;
}
else if (hMap == NULL) //ERROR_FILE_NOT_FOUND or ERROR_PATH_NOT_FOUND
{
CloseHandle(hFile);
hFile = NULL;
hMap = NULL;
return FALSE;
}
chFileData = (char*)MapViewOfFile(hMap,FILE_MAP_COPY,0,0,0);
char* chBuffer = strtok(chFileData,"\r\n");

while (chBuffer != NULL)
{
//AfxMessageBox(chBuffer);
chBuffer = strtok(NULL,"\r\n");
}

if(hFile!=NULL)
{
CloseHandle(hFile);
hFile = NULL;
}
if(hMap!=NULL)
{
CloseHandle(hMap);
hMap = NULL;
}
if((chFileData != "") || (chFileData != NULL))
{
UnmapViewOfFile(chFileData);
chFileData = NULL;
}

Nothing wrong with strtok except that it expects the end of string to be marked with '\0'. When you use MapViewOfFile, it copies the whole file into memory (mmap for the Unix users who are reading this) but it is a byte by byte copy. If the file does not end with '\0', strtok does not know where the string ends. As a result, it tries to go beyond the end of the string resulting in a crash. The simplest solution is to put in a '\0' at the end of file.

Hi,

Thanks for the prompt reply but that didn't work. From my yesterday post, I have modified the code as per your suggestion like:

chFileData = (char*)MapViewOfFile(hMap,FILE_MAP_COPY,0,0,0);
chFileData[fSize] = '\0';//Added this line but it is crashing here.

then I modified the the line from yesterday post:
hMap = CreateFileMapping(hFile,NULL,PAGE_READONLY,0,fSize,NULL);

to

hMap = CreateFileMapping(hFile,NULL,PAGE_READONLY,0,fSize+1,NULL);

if ((hMap != NULL) && (nError == ERROR_ALREADY_EXISTS))
{
CloseHandle(hMap);
hMap = NULL;
return FALSE;
}
else if (hMap == NULL) //ERROR_FILE_NOT_FOUND or ERROR_PATH_NOT_FOUND
{
CloseHandle(hFile);
hFile = NULL;
hMap = NULL;
return FALSE;
}

Now hMap is getting NULL and above else if condition is satified inturn I am not able to execute the below lines:

chFileData = (char*)MapViewOfFile(hMap,FILE_MAP_COPY,0,0,0);
chFileData[fSize] = '\0';

Is there any solution for the above crashes. Can you give details for the solution?

Thanks,
Narayana Murty.

This is what I call extreme bloatware. What is in it that you need memory mapped file for? Just open the **** file, read the whole file into a chunk of memory, and do parsing from there. It should NOT take more than half a dozen code lines to do it.

Get rid of your memory mapped file, use the regular way of opening/reading a file, and your problem will disappear for good.

If it is readonly, then you cannot write to it, therefore chFileData[fSize] = '\0'; is illegal: hence the crash. You have to add the '\0' in the file: not in the code.

hMap is probably coming up as NULL because the size of the map requested is bigger than that of the file.

A couple of comments on this thread:

1) The OP may very well be using file mapping for a specific
reason (other processes may already have this file mapped).
Telling the OP to just use the basic file read/write functions is
not the advice the OP is asking for.

2) If, down the road you want to allow PAGE_READWRITE. you
shouldn't use strtok to parse the shared memory. It is a
destructive routine by nature.

regards, willchop

Hi Cup,

The file is not readonly. How I have to add '\0' to the file. Do I need to open it and go to end and add '\0' and then close that file again file map it? Is this the way? or any thing else we can do easily?

Thanks,
Narayana Murty.

Well, that is the only way I can think of. Some text editors will allow insertion of control characters but I don't know if the keyboard sequence for \0 works for all of them. When I last used it on aedit in RMX, it was ctrl @.