Send utf-8 encoded strings in HTTP HTTP request

Question

I made an HTTP message inside my Android app. Values ​​are sent as strings from my application to my web server. The problem is that the values ​​are not in UTF-8 as I want them to be. My web server has UTF-8 encoding, so I know that there is code in my application that I need to change. See my snippet below:

private void sendPostRequest(String facebookId, String name, String email) { class SendPostReqAsyncTask extends AsyncTask<String, Void, String>{ @Override protected String doInBackground(String... bcs) { String bcFacebookId = bcs[0]; String bcName = bcs[1]; String bcEmail = bcs[2]; HttpClient httpClient = new DefaultHttpClient(); HttpPost httpPost = new HttpPost("URL"); BasicNameValuePair facebookIdBasicNameValuePair = new BasicNameValuePair("bcFacebookId", bcFacebookId); BasicNameValuePair nameBasicNameValuePair = new BasicNameValuePair("bcName", bcName); BasicNameValuePair emailBasicNameValiePair = new BasicNameValuePair("bcEmail", bcEmail); List<NameValuePair> nameValuePairList = new ArrayList<NameValuePair>(); nameValuePairList.add(facebookIdBasicNameValuePair); nameValuePairList.add(nameBasicNameValuePair); nameValuePairList.add(emailBasicNameValiePair); try { UrlEncodedFormEntity urlEncodedFormEntity = new UrlEncodedFormEntity(nameValuePairList); httpPost.setEntity(urlEncodedFormEntity); try { HttpResponse httpResponse = httpClient.execute(httpPost); InputStream inputStream = httpResponse.getEntity().getContent(); InputStreamReader inputStreamReader = new InputStreamReader(inputStream); BufferedReader bufferedReader = new BufferedReader(inputStreamReader); StringBuilder stringBuilder = new StringBuilder(); String bufferedStrChunk = null; while((bufferedStrChunk = bufferedReader.readLine()) != null){ stringBuilder.append(bufferedStrChunk); } return stringBuilder.toString(); } catch (ClientProtocolException cpe) { cpe.printStackTrace(); } catch (IOException ioe) { System.out.println("Second Exception caz of HttpResponse :" + ioe); ioe.printStackTrace(); } } catch (UnsupportedEncodingException uee) { System.out.println("An Exception given because of UrlEncodedFormEntity argument :" + uee); uee.printStackTrace(); } return null; } 

For example, the letter "ö" becomes "?". How to fix it? Hooray!

Answer 1

The biggest reason that characters are converted to question marks is the conversion of characters to bytes and then back to characters, rather than a match.

The code you supplied has the following line:

 InputStreamReader inputStreamReader = new InputStreamReader(inputStream); 

This is problematic because you did not specify how to convert bytes to characters. Instead, you probably want to:

 InputStreamReader inputStreamReader = new InputStreamReader(inputStream, "UTF-8"); 

What you specify for character encoding will depend on the character encoding you specify elsewhere. Without specifying a character encoding, you will get a default character encoding, and this depends on the settings on both the client and server. Java uses Unicode, and UTF-8 is the only encoding that retains all the characters that Java allows.

For debugging, you can use an InputStream and extract bytes from it, and also print the byte values ​​to make sure that they are really UTF-8 encoded representations of the original characters. The correct encoding for "ö" (x00F6) is "ö" (x00C3 x00B6).

You also need to make sure that the original POST request is encoded correctly in UTF-8 encoding. The UrlEncodedFormEntity class also uses the default character encoding, which may not be UTF-8. Change this:

 UrlEncodedFormEntity uefe = new UrlEncodedFormEntity(nameValuePairList); 

to

 UrlEncodedFormEntity uefe = new UrlEncodedFormEntity(nameValuePairList, "UTF-8"); 

Answer 2

if the database encoding is set correctly + the table encoding is configured correctly + the column encoding is correct, then all data is stored correctly. This is the first part. Now the second important part - make sure you have this command after mysql connection: SET NAMES utf8

This was my case on the same issue. Hope this works for you too.