-
May 8th, 2010, 03:21 PM
#1
How to convert string with double high/wide characters to normal string [VC++6]
My application typically recieves a string in the following format:
" Item $5.69 "
Some contants I always expect:
- the LENGHT always 20 characters
- the start index of the text always [5]
- and most importantly the index of the DECIMAL for the price always [14]
In order to identify this string correctly I validate all the expected contants listed above ....
Some of my clients have now started sending the string with Doube-High / Double-Wide values (pair of characters which represent a single readable character) similar to the following:
" Item $x80x90.x81x91x82x92 "
For testing I simply scan the string character-by-character, compare char[i] and char[i+1] and replace these pairs with their corresponding single character when a match is found (works fine) as follows:
Code:
for (int i=0; i < sData.length(); i++)
{
char ch = sData[i] & 0xFF;
char ch2 = sData[i+1] & 0xFF;
if (ch == '\x80' && ch2 == '\x90')
zData.replace("\x80\x90", "0");
else if (ch == '\x81' && ch2 == '\x91')
zData.replace("\x81\x91", "1");
else if (ch == '\x82' && ch2 == '\x92')
zData.replace("\x82\x92", "2");
...
...
...
}
But the result is something like this:
" Item $5.69 "
Notice how this no longer matches my expectation: the lenght is now 17 (instead of 20) due to the 3 conversions and the decimal is now at index 13 (instead of 14) due to the conversion of the "5" before the decimal point.
Ideally I would like to convert the string to a normal readable format keeping the constants (length, index of text, index of decimal) at the same place (so the rest of my application is re-usable) ... or any other suggestion (I'm pretty much stuck with this)... Is there a STANDARD way of dealing with these type of characters?
Any help would be greatly appreciated, I've been stuck on this for a while now ...
Thanks,
-
May 8th, 2010, 03:27 PM
#2
Re: How to convert string with double high/wide characters to normal string [VC++6]
Why are your clients sending the information in a different format in the first place?
Also, double-high, double-wide aren't terms I've ever heard before. Not exactly sure what this is.
-
May 8th, 2010, 03:32 PM
#3
Re: How to convert string with double high/wide characters to normal string [VC++6]
Customer decided to change things up (for display purposes on their own application) and I have to live with the changes.
I think it might be Wide Char (from what I've been reading this morning) - customer refers to it as double high/double wide because on screen when displayed they are 2x the size of a normal character.
-
May 8th, 2010, 03:35 PM
#4
Re: How to convert string with double high/wide characters to normal string [VC++6]
Code:
for (int i=0; i < sData.length(); i++)
{
char ch = sData[i] & 0xFF;
char ch2 = sData[i+1] & 0xFF;
if (ch == '\x80' && ch2 == '\x90')
zData.replace("\x80\x90", "0");
else if (ch == '\x81' && ch2 == '\x91')
zData.replace("\x81\x91", "1");
else if (ch == '\x82' && ch2 == '\x92')
zData.replace("\x82\x92", "2");
...
...
...
}
Programmers would use lookup tables instead of an endless bunch of if() statements. What if all of the characters are at the last "else()"? Then this becomes inefficient, since you're going through all of the previous else's to get to the one that matches.
Secondly, how do you know what those characters represent? You have code pages to deal with, and you can't do it by writing ad-hoc if() statements.
Ideally I would like to convert the string to a normal readable format
WideCharToMultibyte
http://msdn.microsoft.com/en-us/libr...8VS.85%29.aspx
Any help would be greatly appreciated, I've been stuck on this for a while now ...
You should have let your customers be aware that multibyte characters were not handled correctly. This is what every single-byte character based application I know of state clearly up front.
There could be other places where the characters may be wide characters, and your app will not handle that area of code correctly.
Regards,
Paul McKenzie
-
May 8th, 2010, 03:53 PM
#5
Re: How to convert string with double high/wide characters to normal string [VC++6]
Won't using WideCharToMultiByte still change the constants (like the overall string size, location of the decimal, etc...)? Also, not all characters are represented in Wide format, some are still normal like the "item" part. In reality it is only the price I need to worry about.
Customer was made aware, didn't really care ... and we can't really stop supporting them because of it... One of those customer-is-always-right situations.
-
May 8th, 2010, 04:05 PM
#6
Re: How to convert string with double high/wide characters to normal string [VC++6]
Originally Posted by Shaitan00
Won't using WideCharToMultiByte still change the constants (like the overall string size, location of the decimal, etc...)?
Also, not all characters are represented in Wide format, some are still normal like the "item" part. In reality it is only the price I need to worry about.
Take that part of the string that isn't single-byte, store it in a temporary, convert it, and just build the new string from it.
I really don't quite understand your concern about decimal positions, string sizes, etc. Fix that later -- the goal right now is to clearly and safely convert a wide set of characters to non-wide characters, and the only way to do that safely is to use API's to do it. You don't do it by coding if() statements that reflect what you believe is the right translation. If your code is reviewed by a someone else, coding stuff like that would get rejected immediately if the platform you're working with has API's to do this work. Again, code pages makes it virtually impossible for you to write correct if() statements for this type of work.
Customer was made aware, didn't really care ... and we can't really stop supporting them because of it... One of those customer-is-always-right situations.
Then that is either not communicating clearly that single byte and wide characters are not the same thing, or the customer risking a hack that could break other parts of the program. Then when the customer starts to use other multi-byte characters in the app (they now want Chinese to be supported for some reason), then what? Sometimes you have to be forceful, if not, you could be starting a slippery slope -- I've seen it all before.
Regards,
Paul McKenzie
Last edited by Paul McKenzie; May 8th, 2010 at 04:11 PM.
-
May 8th, 2010, 04:16 PM
#7
Re: How to convert string with double high/wide characters to normal string [VC++6]
Curious to ask (opening up a can on worms but) are you sure this format is even Unicode? I always assumed it was WCHAR but that was simply because I wasn't sure what else this stuff could be...
I was talking with a friend of mine and he mentioned that this didn't look like unicode at all but more like Print Data - which is also very possible seeing as the data I receiving is receipt data that ultimatly MAY be sent to a printer ...
Sadly at this point most of my ideas are assumption (customer really has no clue, they enabled a feature that gave them the double high/double wide functionality and the rest is magic).
Just wanted to throw that out there incase...
Thanks,
-
May 8th, 2010, 04:22 PM
#8
Re: How to convert string with double high/wide characters to normal string [VC++6]
Originally Posted by Shaitan00
Curious to ask (opening up a can on worms but) are you sure this format is even Unicode?
If it isn't UNICODE, then it must be some other proprietary format. Your customer has to tell you what it is, so that you can at least build a lookup table and do the translation's easily (instead of an endless set of if() statements).
I was talking with a friend of mine and he mentioned that this didn't look like unicode at all but more like Print Data - which is also very possible seeing as the data I receiving is receipt data that ultimatly MAY be sent to a printer ...
You shouldn't be trying to figure this out -- the customer in all their wisdom should be telling you what it is and where to get the information for where and how those characters were generated.
Sadly at this point most of my ideas are assumption (customer really has no clue, they enabled a feature that gave them the double high/double wide functionality and the rest is magic).
Ask them what the name of the application they're using, and do a web search to get more information. They may not have a clue, but the app they're using has all the answers, and all you need to know is what application they're using to create this stream.
Regards,
Paul McKenzie
-
May 8th, 2010, 04:27 PM
#9
Re: How to convert string with double high/wide characters to normal string [VC++6]
First step was to ask the customer, they gave us the map of what pairs convert to what and were told to use it that way - the customer is dealing with another software company (which I've also spoken to) and told us it is the format "they always use". No follow-up (but I just sent an email to see if I can get more details).
However I have the map of all pairs->chars that they use, I placed them in a if/else format just to get the code to work for now - making it more efficient comes later (using a map, etc...). So I can convert the strings no problem - my issue is with the formatting afterwards.
Good point about looking up their software online, I did that a while back (got no where) but then I got the map and life was good for a while ... I'll go dig more.
Thanks for the help ...
Last edited by Shaitan00; May 8th, 2010 at 04:33 PM.
-
May 8th, 2010, 04:56 PM
#10
Re: How to convert string with double high/wide characters to normal string [VC++6]
From what I can see it is double-high / double-wide VDisplay characters provided by QVS TSNT application ... can't really find anything more then that ...
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|