How to convert string with double high/wide characters to normal string [VC++6]
CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Results 1 to 10 of 10

Thread: How to convert string with double high/wide characters to normal string [VC++6]

  1. #1
    Join Date
    Oct 2004
    Posts
    429

    Question How to convert string with double high/wide characters to normal string [VC++6]

    My application typically recieves a string in the following format:
    " Item $5.69 "

    Some contants I always expect:
    - the LENGHT always 20 characters
    - the start index of the text always [5]
    - and most importantly the index of the DECIMAL for the price always [14]
    In order to identify this string correctly I validate all the expected contants listed above ....

    Some of my clients have now started sending the string with Doube-High / Double-Wide values (pair of characters which represent a single readable character) similar to the following:
    " Item $x80x90.x81x91x82x92 "

    For testing I simply scan the string character-by-character, compare char[i] and char[i+1] and replace these pairs with their corresponding single character when a match is found (works fine) as follows:

    Code:
    for (int i=0; i < sData.length(); i++)
    {
       char ch = sData[i] & 0xFF;
       char ch2 = sData[i+1] & 0xFF;
    
       if (ch == '\x80' && ch2 == '\x90')
          zData.replace("\x80\x90", "0");
       else if (ch == '\x81' && ch2 == '\x91')
          zData.replace("\x81\x91", "1");
       else if (ch == '\x82' && ch2 == '\x92')
          zData.replace("\x82\x92", "2");
       ...
       ...
       ...
    }
    But the result is something like this:
    " Item $5.69 "
    Notice how this no longer matches my expectation: the lenght is now 17 (instead of 20) due to the 3 conversions and the decimal is now at index 13 (instead of 14) due to the conversion of the "5" before the decimal point.


    Ideally I would like to convert the string to a normal readable format keeping the constants (length, index of text, index of decimal) at the same place (so the rest of my application is re-usable) ... or any other suggestion (I'm pretty much stuck with this)... Is there a STANDARD way of dealing with these type of characters?

    Any help would be greatly appreciated, I've been stuck on this for a while now ...
    Thanks,

  2. #2
    Join Date
    Aug 2008
    Posts
    902

    Re: How to convert string with double high/wide characters to normal string [VC++6]

    Why are your clients sending the information in a different format in the first place?

    Also, double-high, double-wide aren't terms I've ever heard before. Not exactly sure what this is.

  3. #3
    Join Date
    Oct 2004
    Posts
    429

    Re: How to convert string with double high/wide characters to normal string [VC++6]

    Customer decided to change things up (for display purposes on their own application) and I have to live with the changes.

    I think it might be Wide Char (from what I've been reading this morning) - customer refers to it as double high/double wide because on screen when displayed they are 2x the size of a normal character.

  4. #4
    Join Date
    Apr 1999
    Posts
    27,427

    Re: How to convert string with double high/wide characters to normal string [VC++6]

    Code:
    for (int i=0; i < sData.length(); i++)
    {
       char ch = sData[i] & 0xFF;
       char ch2 = sData[i+1] & 0xFF;
    
       if (ch == '\x80' && ch2 == '\x90')
          zData.replace("\x80\x90", "0");
       else if (ch == '\x81' && ch2 == '\x91')
          zData.replace("\x81\x91", "1");
       else if (ch == '\x82' && ch2 == '\x92')
          zData.replace("\x82\x92", "2");
       ...
       ...
       ...
    }
    Programmers would use lookup tables instead of an endless bunch of if() statements. What if all of the characters are at the last "else()"? Then this becomes inefficient, since you're going through all of the previous else's to get to the one that matches.

    Secondly, how do you know what those characters represent? You have code pages to deal with, and you can't do it by writing ad-hoc if() statements.
    Ideally I would like to convert the string to a normal readable format
    WideCharToMultibyte

    http://msdn.microsoft.com/en-us/libr...8VS.85%29.aspx
    Any help would be greatly appreciated, I've been stuck on this for a while now ...
    You should have let your customers be aware that multibyte characters were not handled correctly. This is what every single-byte character based application I know of state clearly up front.

    There could be other places where the characters may be wide characters, and your app will not handle that area of code correctly.

    Regards,

    Paul McKenzie

  5. #5
    Join Date
    Oct 2004
    Posts
    429

    Re: How to convert string with double high/wide characters to normal string [VC++6]

    Won't using WideCharToMultiByte still change the constants (like the overall string size, location of the decimal, etc...)? Also, not all characters are represented in Wide format, some are still normal like the "item" part. In reality it is only the price I need to worry about.

    Customer was made aware, didn't really care ... and we can't really stop supporting them because of it... One of those customer-is-always-right situations.

  6. #6
    Join Date
    Apr 1999
    Posts
    27,427

    Re: How to convert string with double high/wide characters to normal string [VC++6]

    Quote Originally Posted by Shaitan00 View Post
    Won't using WideCharToMultiByte still change the constants (like the overall string size, location of the decimal, etc...)?
    Also, not all characters are represented in Wide format, some are still normal like the "item" part. In reality it is only the price I need to worry about.
    Take that part of the string that isn't single-byte, store it in a temporary, convert it, and just build the new string from it.

    I really don't quite understand your concern about decimal positions, string sizes, etc. Fix that later -- the goal right now is to clearly and safely convert a wide set of characters to non-wide characters, and the only way to do that safely is to use API's to do it. You don't do it by coding if() statements that reflect what you believe is the right translation. If your code is reviewed by a someone else, coding stuff like that would get rejected immediately if the platform you're working with has API's to do this work. Again, code pages makes it virtually impossible for you to write correct if() statements for this type of work.
    Customer was made aware, didn't really care ... and we can't really stop supporting them because of it... One of those customer-is-always-right situations.
    Then that is either not communicating clearly that single byte and wide characters are not the same thing, or the customer risking a hack that could break other parts of the program. Then when the customer starts to use other multi-byte characters in the app (they now want Chinese to be supported for some reason), then what? Sometimes you have to be forceful, if not, you could be starting a slippery slope -- I've seen it all before.

    Regards,

    Paul McKenzie
    Last edited by Paul McKenzie; May 8th, 2010 at 04:11 PM.

  7. #7
    Join Date
    Oct 2004
    Posts
    429

    Re: How to convert string with double high/wide characters to normal string [VC++6]

    Curious to ask (opening up a can on worms but) are you sure this format is even Unicode? I always assumed it was WCHAR but that was simply because I wasn't sure what else this stuff could be...

    I was talking with a friend of mine and he mentioned that this didn't look like unicode at all but more like Print Data - which is also very possible seeing as the data I receiving is receipt data that ultimatly MAY be sent to a printer ...

    Sadly at this point most of my ideas are assumption (customer really has no clue, they enabled a feature that gave them the double high/double wide functionality and the rest is magic).

    Just wanted to throw that out there incase...
    Thanks,

  8. #8
    Join Date
    Apr 1999
    Posts
    27,427

    Re: How to convert string with double high/wide characters to normal string [VC++6]

    Quote Originally Posted by Shaitan00 View Post
    Curious to ask (opening up a can on worms but) are you sure this format is even Unicode?
    If it isn't UNICODE, then it must be some other proprietary format. Your customer has to tell you what it is, so that you can at least build a lookup table and do the translation's easily (instead of an endless set of if() statements).
    I was talking with a friend of mine and he mentioned that this didn't look like unicode at all but more like Print Data - which is also very possible seeing as the data I receiving is receipt data that ultimatly MAY be sent to a printer ...
    You shouldn't be trying to figure this out -- the customer in all their wisdom should be telling you what it is and where to get the information for where and how those characters were generated.
    Sadly at this point most of my ideas are assumption (customer really has no clue, they enabled a feature that gave them the double high/double wide functionality and the rest is magic).
    Ask them what the name of the application they're using, and do a web search to get more information. They may not have a clue, but the app they're using has all the answers, and all you need to know is what application they're using to create this stream.

    Regards,

    Paul McKenzie

  9. #9
    Join Date
    Oct 2004
    Posts
    429

    Re: How to convert string with double high/wide characters to normal string [VC++6]

    First step was to ask the customer, they gave us the map of what pairs convert to what and were told to use it that way - the customer is dealing with another software company (which I've also spoken to) and told us it is the format "they always use". No follow-up (but I just sent an email to see if I can get more details).

    However I have the map of all pairs->chars that they use, I placed them in a if/else format just to get the code to work for now - making it more efficient comes later (using a map, etc...). So I can convert the strings no problem - my issue is with the formatting afterwards.

    Good point about looking up their software online, I did that a while back (got no where) but then I got the map and life was good for a while ... I'll go dig more.
    Thanks for the help ...
    Last edited by Shaitan00; May 8th, 2010 at 04:33 PM.

  10. #10
    Join Date
    Oct 2004
    Posts
    429

    Question Re: How to convert string with double high/wide characters to normal string [VC++6]

    From what I can see it is double-high / double-wide VDisplay characters provided by QVS TSNT application ... can't really find anything more then that ...

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  


Windows Mobile Development Center


Click Here to Expand Forum to Full Width

This is a CodeGuru survey question.


Featured


HTML5 Development Center