Turkish character problem,wstring

I couldn't solve Turkish character problem in C++.If I use wstring for a Turkish

character,writing:

wstring TurkishCharacters;

and then :

TurkishCharacter=L"çı"; // I don't know if you are able to see the Turkish characters
// "ç",it is a character similar to "c" and the other character
// is similar to "i" but a bit different.

It gives this error:
"... error: converting to execution character set: Invalid argument"

If I don't use "L" it gives another error.

But for example,if I use:

getline(wcin,TurkishCharacter);
wcout << TurkishCharacter;

It is able to print what I entered from keyboard.But I am not able to assign a text to a string without
requesting user input...



If I use string instead of wstring,then I am able to assign Turkish characters to a string

string TurkishCharacter;
Turkishcharacter="&#231;ı";

Although it doesn't give any error,it doesn't work as expected,cout prints abnormal
characters.

You shouldn't put extended character literals in your source code - because how those characters are treated by the compiler is implementation defined. Read more about it here: http://www.codeguru.com/forum/showpo...8&postcount=14

For your source code, use either L"\x00C7" or L"\u00C7".

For console output, you need to imbue the user's locale into wcout. That can be done with "wcout.imbue(locale(""));".

Under linux, you can only hope that the character encoding associated with the user's locale understands that character.

Under Windows, this still isn't enough. With VS2008 and higher, you can add "_setmode(_fileno(stdout), _O_U16TEXT);". Then you just have to use a console font (like Lucida Console) that has glyphs for the characters you're trying to output.

gg

Thanks for your answer.So is there no solution for wcout under Windows in pure C++(without using Visual C++) ?

Quote:

---

Originally Posted by AwArEnEsS

TurkishCharacter=L"çı"; // I don't know if you are able to see the Turkish characters

---

And that is the problem right there.

You asked us "if you are able to see the Turkish characters". Why did you ask us this question? If everyone wasn't able to see those characters, then what makes you think that the compiler and/or editor used will see the same thing you're seeing? What if we're using a browser that doesn't display what you typed correctly? See all of the problems now?

Use hex codes if you know the exact hex value for the characters. Once you do that, then whatever the platform's environment is set for when you run the program will display something -- it may be Turkish character, a Chinese character, an Arabic character, who knows. Once you get to that stage, then you figure out how your OS works in terms of displaying the correct characters.

Regards,

Paul McKenzie

Code:

---

#include <iostream>
#include <locale>
using namespace std;

#ifdef _MSC_VER
#include <fcntl.h>
#include <io.h>
#include <stdio.h>
#endif

int main()
{
    locale::global(locale(""));

#ifdef _MSC_VER
    _setmode(_fileno(stdout), _O_U16TEXT);
#endif
   
    wcout << L"\u00c7" << endl;
    return 0;
}//main

---

gg

Thanks for your answers.

Paul Mckenzie,you said I can use hex code.I've read that 0xDE is equal to a Turkish character,but when I wrote :

wstring acharacter;
acharacter="L\0xDE"
wcout << acharacter;


It didn't print anything on the screen.Did you mean hex characters like Codeplug said?If yes,I tried "converting" =0xDE into u00DE and it printed a very similar character to a Turkish character but not exactly.


Codeplug,before I wrote your code,I tried with just:

wstring acharacter;
acharacter =L"\u00c7";
wcout << acharacter;


It printed an interesting character which looks like "A" but a bit different,it is not a Turkish character.Then I wrote your code exactly,and the result was same,it printed the same character.

So,I think if I can find which hex code equal to Turkish character which I am trying to use,it will solve my problem,like you said.Now the problem is,how I can find hex code list:)

Quote:

---

Originally Posted by AwArEnEsS

Thanks for your answers.

Paul Mckenzie,you said I can use hex code.I've read that 0xDE is equal to a Turkish character,but when I wrote :

wstring acharacter;
acharacter="L\0xDE"
wcout << acharacter;

---

Is that your exact code? If it is, then this is wrong:

Code:

---

acharacter="L\0xDE"

---

It should be this:

Code:

---

acharacter=L"\0x00DE";

---

Do you see where the "L" is? It is on the outside of the quotes.

Regards,

Paul McKenzie

Quote:

---

Originally Posted by AwArEnEsS

So,I think if I can find which hex code equal to Turkish character which I am trying to use,it will solve my problem,like you said.Now the problem is,how I can find hex code list:)

---

http://unicode.org/Public/MAPPINGS/V...LE/TURKISH.TXT

Second, instead of trying to find this one character, write a loop that displays all the characters, so that you see right away what hex code to use for any character you're interested in. Trying to hunt down one single character by trial and error is not the way to solve this problem.

Code:

---

#include <string>
#include <iostream>

unsigned int start, last;

using namespace std;

int main()
{
        cout << "Enter starting value: ";
        cin >> start;
        cout << "\nEnter ending value: ";
        cin >> last;

        for (unsigned int i = start; i <= last; ++i )
                wcout << i << " - " << wstring(1, i) << endl;
}

---

Enter two decimal values denoting the start and ending character value. Then you will see all of the characters associated with those numbers. You just need to translate the decimal to hex yourself (or change the program to do that for you). In any event, this is how you figure out characters and their associated representation.

Regards,

Paul McKenzie

I still couldn't solve it.I look at the list in the webpage you gave,for example I select 0x00FA it is
"Latin Small Letter u With Acute".I write;

string character1
acharacter1="\u00FA";
cout << acharacter1;


wstring acharacter2;
acharacter2=L"u00FA";
wcout << acharacter2;


They print different characters but none of them is the character equal to the character in the list.

Also writing

cout << "\0x00FA";

or any other hex code like this prints nothing on the screen...only L"\u00FA" or L"\x00FA" style is working...Did you mean L"\x00FA"(no 0 before x) like Codeplug said?

Also I tried the codes you gave,in the list the codes generated,for example "152" was equal to a character which I want to use.Then I converted it to hex with an online converter,the result was "98".Now what should I do with "98"?

>> Did you mean L"\x00FA"(no 0 before x) like Codeplug said?
I'm sure he did :)

>> for example "152" was equal to a character which I want to use.
What character is that? Just type/paste it in the post.

Here is a Windows reference for Turkish characters: http://msdn.microsoft.com/en-us/goglobal/cc305147.aspx

gg

Thanks for your answer.

The characters don't match with those in the list you gave,interesting.Although some characters match(especially Enghlish characters) most of the others don't match...

In the webpage you gave,I have found a few other lists (like Turkish Windows Oem) and ISO but none of them matches exactly...

Turkish characters I am trying to write in code are "&#231;&#199;ğĞıİ&#246;&#214;şŞ&#252;&#220;".


">> for example "152" was equal to a character which I want to use.
What character is that? Just type/paste it in the post."

It is equal to "İ" character I think(it looks very similar but I am not sure if it's dot is a dot really or is a dash)

>> Turkish characters I am trying to write in code are "çÇğĞıİöÖşŞüÜ".
All of those characters are in the "Windows 1254" table...

Code:

---

ç = 0x00e7
Ç = 0x00c7
ğ = 0x011f
Ğ = 0x011e
ı = 0x0131
İ = 0x0130
ö = 0x00f6
Ö = 0x00d6
ş = 0x015f
Ş = 0x015e
ü = 0x00fc
Ü = 0x00dc

---

In the table, the Unicode value is right under the glyph - which is the same as the "U+XXXX" value in the listing below the table.

Code:

---

#include <iostream>
#include <locale>
using namespace std;

#ifdef _MSC_VER
#include <fcntl.h>
#include <io.h>
#include <stdio.h>
#endif

int main()
{
    locale::global(locale(""));

#ifdef _MSC_VER
    _setmode(_fileno(stdout), _O_U16TEXT);
#endif

    const wchar_t *turkish_chars =
        L"\u00e7\u00c7\u011f\u011e\u0131\u0130"
        L"\u00f6\u00d6\u015f\u015e\u00fc\u00dc";

    wcout << turkish_chars << endl;
    return 0;
}//main

---

gg

"
#include <iostream>
#include <locale>
using namespace std;

#ifdef _MSC_VER
#include <fcntl.h>
#include <io.h>
#include <stdio.h>
#endif

int main()
{
locale::global(locale(""));

#ifdef _MSC_VER
_setmode(_fileno(stdout), _O_U16TEXT);
#endif

const wchar_t *turkish_chars =
L"\u00e7\u00c7\u011f\u011e\u0131\u0130"
L"\u00f6\u00d6\u015f\u015e\u00fc\u00dc";

wcout << turkish_chars << endl;
return 0;
}//main

"


There is a problem.I copied and pasted your complete code,and the result is in the thumbnail I attached:

Set you console font to "Lucida Console" by right-clicking on the cmd.exe title bar and select properties.

Let us know what compiler you are using if that doesn't work.

gg

Thanks for your answer.

The same problem continued with "Lucida Console" font.I am using Qt Creator without using Qt related code and it is using gcc compiler if I know right.

Ah, yes - MinGW (gcc on Windows) does not support "_O_U16TEXT" with std::wcout. Only the Microsoft C++ standard library supports that trick.

The problem is that you have to use WriteConsoleW() in order to get Unicode output on the console. So to make it work for any standard C++ Windows compiler we have to do it ourself.

Code:

---

#include <iostream>
#include <iomanip>
#include <fstream>
#include <locale>
#include <string>

#include <windows.h>

//------------------------------------------------------------------------------

class wconbuf : public std::wstreambuf
{
public:
    typedef wchar_t char_type;
    typedef std::char_traits<char_type> traits_type;
    typedef traits_type::int_type int_type;
    typedef traits_type::pos_type pos_type;
    typedef traits_type::off_type off_type;

private:
    char_type *m_buffer;

    enum {BUFFER_SIZE = 4096 / sizeof(char_type)};

public:
    wconbuf() : m_buffer(new char_type[BUFFER_SIZE])
    {
        setp(m_buffer, m_buffer + BUFFER_SIZE);
    }//constructor

    ~wconbuf()
    {
        pubsync();
        delete[] m_buffer;
    }//destructor

protected:
    virtual int_type overflow(int_type c = traits_type::eof())
    {
        // send our buffer through WriteConsoleW() if we can get a handle
        HANDLE hout = ::GetStdHandle(STD_OUTPUT_HANDLE);
        if (hout && (hout != INVALID_HANDLE_VALUE))
        {
            std::streamsize n = static_cast<std::streamsize>(pptr() - pbase());
            DWORD written;
            ::WriteConsoleW(hout, pbase(), static_cast<DWORD>(n), &written, 0);
        }//if

        // reset our buffer
        setp(m_buffer, m_buffer + BUFFER_SIZE);

        // write the passed character if necessary (into our buffer)
        if (!traits_type::eq_int_type(c, traits_type::eof()))
        {
            traits_type::assign(*pptr(), traits_type::to_char_type(c));
            pbump(1);
        }//if

        return traits_type::not_eof(c);
    }//overflow

    virtual int sync()
    {
        // flush our buffer
        int_type c = overflow(traits_type::eof());

        // checking return for eof.
        if (traits_type::eq_int_type(c, traits_type::eof()))
            return -1;

        return 0;
    }//sync
};//wconbuf

//------------------------------------------------------------------------------

template <class charT, class traits = std::char_traits<charT> >
struct scoped_basic_streambuf_assignment
{
    typedef std::basic_ios<charT, traits> stream_type;
    typedef std::basic_streambuf<charT, traits> streambuf_type;

    stream_type &m_s;
    streambuf_type *m_orig_sb;

    scoped_basic_streambuf_assignment(stream_type &s, streambuf_type *new_sb)
        : m_s(s)
    {
        m_orig_sb = m_s.rdbuf(new_sb);
    }//constructor

    ~scoped_basic_streambuf_assignment()
    {
        m_s.rdbuf(m_orig_sb);
    }//destructor
};//scoped_streambuf_assignment

typedef scoped_basic_streambuf_assignment<char>    scoped_streambuf_assignment;
typedef scoped_basic_streambuf_assignment<wchar_t> scoped_wstreambuf_assignment;

//------------------------------------------------------------------------------

int main()
{
    std::locale::global(std::locale(""));

    wconbuf wcon;
    scoped_wstreambuf_assignment swsba(std::wcout, &wcon);
   
    const wchar_t *turkish_chars =
        L"\u00e7\u00c7\u011f\u011e\u0131\u0130"
        L"\u00f6\u00d6\u015f\u015e\u00fc\u00dc";
       
    std::wcout << turkish_chars << std::endl;
   
    // do it again with just the Win32 API
    DWORD written;
    ::WriteConsoleW(::GetStdHandle(STD_OUTPUT_HANDLE),
                    turkish_chars,
                    lstrlenW(turkish_chars),
                    &written, 0);
                   
    std::wcout << std::endl;
    return 0;
}//main

---

wconbuf is a custom wide-stream buffer that pumps everything through WriteConsoleW().

gg

Thanks for your answer.

Could you write a simple full application which uses Turkish characters?I didn't understand what should I write after writing your last code.

>> Could you write a simple full application which uses Turkish characters?
The code in post #16 is as simple as it gets under MinGW.
The code in post #12 is as simple as it gets under VC++

If there is something you don't understand, ask a specific question.

gg

Although I couldn't understand what wconbuf does,just using writeconsoleW() which you mentioned worked for Turkish output.Thanks

What should I write after writing wconbuf code,what is it for?

std::wcout << turkish_chars << std::endl;
The only reason that works is because std::wcout's stream buffer has been replaced by an instance of 'wconbuf' - which sends everything through WriteConsoleW.

I first learned all about C++ streams from a book - http://www.angelikalanger.com/iostreams.html

There is some online material as well:
http://gcc.gnu.org/onlinedocs/libstd...treambufs.html
http://stdcxx.apache.org/doc/stdlibug/39-2.html

gg

Thanks