Open utf8 coded file name in C ++ Windows

Question

Consider the following code:

#include <iostream> #include <boost\locale.hpp> #include <Windows.h> #include <fstream> std::string ToUtf8(std::wstring str) { std::string ret; int len = WideCharToMultiByte(CP_UTF8, 0, str.c_str(), str.length(), NULL, 0, NULL, NULL); if (len > 0) { ret.resize(len); WideCharToMultiByte(CP_UTF8, 0, str.c_str(), str.length(), &ret[0], len, NULL, NULL); } return ret; } int main() { std::wstring wfilename = L"D://Private//Test//एउटा फोल्दर//भित्रको फाईल.txt"; std::string utf8path = ToUtf8(wfilename ); std::ifstream iFileStream(utf8path , std::ifstream::in | std::ifstream::binary); if(iFileStream.is_open()) { std::cout << "Opened the File\n"; //Do the work here. } else { std::cout << "Cannot Opened the file\n"; } return 0; } 

If I run the file, I cannot open the file by entering it in the else block. Even using boost::locale::conv::from_utf(utf8path ,"utf_8") instead of utf8path does not work. The code works if I consider using wifstream and using wfilename as its parameter, but I do not want to use wifstream . Is there a way to open a file with its name utf8 encoded? I am using Visual Studio 2010 .

Answer 1

On Windows, you MUST use 8-bit ANSI (and it must match the user's language) or UTF16 for file names, there is no other option available. You can continue to use string and UTF8 in your main code, but you will have to convert the UTF8 file names to UTF16 when opening the files. Less effective, but this is what you need to do.

Fortunately, the VC ++ implementation std::ifstream and std::ofstream has non-standard overloads of its constructors and open() methods to accept wchar_t* strings for UTF16 file names.

 explicit basic_ifstream( const wchar_t *_Filename, ios_base::openmode _Mode = ios_base::in, int _Prot = (int)ios_base::_Openprot ); void open( const wchar_t *_Filename, ios_base::openmode _Mode = ios_base::in, int _Prot = (int)ios_base::_Openprot ); void open( const wchar_t *_Filename, ios_base::openmode _Mode ); 

 explicit basic_ofstream( const wchar_t *_Filename, ios_base::openmode _Mode = ios_base::out, int _Prot = (int)ios_base::_Openprot ); void open( const wchar_t *_Filename, ios_base::openmode _Mode = ios_base::out, int _Prot = (int)ios_base::_Openprot ); void open( const wchar_t *_Filename, ios_base::openmode _Mode ); 

You will have to use #ifdef to detect Windows compilation (unfortunately, different C ++ compilers identify this differently) and temporarily convert the UTF8 string to UTF16 when opening the file.

 #ifdef _MSC_VER std::wstring ToUtf16(std::string str) { std::wstring ret; int len = MultiByteToWideChar(CP_UTF8, 0, str.c_str(), str.length(), NULL, 0); if (len > 0) { ret.resize(len); MultiByteToWideChar(CP_UTF8, 0, str.c_str(), str.length(), &ret[0], len); } return ret; } #endif int main() { std::string utf8path = ...; std::ifstream iFileStream( #ifdef _MSC_VER ToUtf16(utf8path).c_str() #else utf8path.c_str() #endif , std::ifstream::in | std::ifstream::binary); ... return 0; } 

Please note that this is guaranteed to work only in VC ++. Other C ++ compilers for Windows do not guarantee similar extensions.