Android, UTF8 - How to ensure that UTF8 is used for general preferences

Question

How to ensure that UTF8 is used for a general preference menu? I have an Android settings menu that allows the user to specify their name, among other things.

I need to know how to convert data stored in general settings to UTF8 format

The preferences menu is laid out in xml using utf8 encoding in a file with parameters in the res / xml folder and looks like this:

<?xml version="1.0" encoding="utf-8"?> <PreferenceScreen xmlns:android="http://schemas.android.com/apk/res/android"> <EditTextPreference android:key="@string/name_key" android:title="@string/hof_name_title" android:summary="@string/hof_name_summary" android:defaultValue="Anonymous" /> <CheckBoxPreference android:key="@string/new_game_preference_key" android:title="@string/new_game_preference_title" android:summary="@string/new_game_preference_summary" android:defaultValue="false" /> </PreferenceScreen> 

The class that handles this is

 public class PrefsActivity extends PreferenceActivity implements OnSharedPreferenceChangeListener{ @Override protected void onCreate(Bundle savedInstanceState) { requestWindowFeature(Window.FEATURE_NO_TITLE); getWindow().setFlags(WindowManager.LayoutParams.FLAG_FULLSCREEN, WindowManager.LayoutParams.FLAG_FULLSCREEN); super.onCreate(savedInstanceState); addPreferencesFromResource(R.xml.settings); } @Override protected void onPause() { super.onPause(); // Unregister the listener whenever a key changes getPreferenceScreen().getSharedPreferences() .unregisterOnSharedPreferenceChangeListener(this); } @Override protected void onResume() { super.onResume(); // Set up a listener whenever a key changes getPreferenceScreen().getSharedPreferences() .registerOnSharedPreferenceChangeListener(this); } @Override public void onSharedPreferenceChanged(SharedPreferences sharedPreferences, String key) { // Util.log_debug_message("@@@@ Changed prefrence key = " + key); if (key.equalsIgnoreCase(getString(R.string.name_key))){ update_webserver(); }else if (key.equalsIgnoreCase(getString(R.string.new_game_preference_key))){ update_webserver(); } } protected void update_webserver(){ Intent webServerReg = new Intent(this, RegService.class); webServerReg.putExtra(Config.C2DM_REG_ID_EXTRA, C2DMessaging.getRegistrationId(this)); startService(webServerReg); } } 

I assumed that setting <?xml version="1.0" encoding="utf-8"?> Ensures that the text will be encoded in UTF8 format, but this is not always the case. I get values ​​in all formats. Some examples

"\ nFe \ xF1a", "L \ xFAcia", "$ '\ xE5 \ xEE'",

or

": - $ B-) O: -): - P = -ob -): - D: -Q: -X: -: - |! \ P: '(: - * XDo_O: -X: -C: -X: O: -X = -O; -): -): -); -): - D: OX- (o_O: -V: - @: - V: -X: -Do_O :: - \ XBF \ xa1 \ xa1 \ XAB \ Xbb \ XAE \ xA9 ^ \ xA5 \ xA5 [\ XA2} \ XA2 \ XA2 \ xA5 \ XA3 $ \ XBF \ xa1 \ XAE \ xA7> <??? [\ XA3 ~? = ~~ \xB0]\xA3?\xA7\xA1\\\xAB\xBB\xAE^``]]||||}{_| ] ||| \ XB0 \ XB0? "

Yes, this is one set of rubies in the last example.

Problems that I run when trying to send nmes as json HTTP POST or PUT request to my web server using the following code

 public void update(String regId) throws AuthenticationException { JSONObject mu_to_send = createJsonToPost(regId, false); HttpResponse response; try { response = HttpRequest.httpPutJSONObject(this, Config.UPDATE_USER_URL, mu_to_send); int responseCode = response.getStatusLine().getStatusCode(); String responseJson; try { responseJson = EntityUtils.toString(response.getEntity()); String msg = ""; if (responseCode == 201) { // mobile_user = new // JSONObject(responseJson).getJSONObject("mobile_user"); // Broadcast.sendBroadcast(this, // ResponseReceiver.ACTION_RESP, // Intent.CATEGORY_DEFAULT, Config.C2DM_MESSAGE_EXTRA, // Config.BROADCAST_WEBSERVER_USER_CREATED); } else { msg = "Failed to register with server! HTTP response code = " + responseCode; } } catch (ParseException e) { // TODO Auto-generated catch block e.printStackTrace(); } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); } } catch (ClientProtocolException e1) { // TODO Auto-generated catch block e1.printStackTrace(); } catch (IOException e1) { // TODO Auto-generated catch block e1.printStackTrace(); } } public JSONObject createJsonToPost(String regId, boolean set_password) { JSONObject holder = new JSONObject(); JSONObject data = new JSONObject(); try { data.put("auth", regId); data.put("name", C2DMessaging.getName(this)); data.put("receive_new_game_notification", C2DMessaging.getGameStartedNotificationPreference(this)); holder.put("mobile_user", data); } catch (JSONException e) { // TODO Auto-generated catch block e.printStackTrace(); } return holder; } public static HttpResponse httpPutJSONObject(Context context, String url, JSONObject data) throws ClientProtocolException, IOException, AuthenticationException { DefaultHttpClient httpclient = getHttpClient(context); HttpPut httpput = new HttpPut(url); httpput.addHeader("User-Agent", Config.USER_AGENT); httpput.addHeader("Accept", "application/json"); httpput.addHeader("Content-type", "application/json"); httpput.addHeader(new BasicScheme().authenticate( getCredentials(context), httpput)); StringEntity se = new StringEntity(data.toString()); httpput.setEntity(se); return httpclient.execute(httpput); } 

This causes all kinds of problems on my web server, which expects valid JSON (i.e. UTF8) since JSON should be sent to UTF8 first.

So,

1) Why <?xml version="1.0" encoding="utf-8"?> Not set the UTF8 encoding? when used in layout?

2) What is the best way to ensure that I always receive valid UTF8 format characters sent to my web server? Should this be handled by a put request or code that preserves shared access or either code that populates a json object, or perhaps a combination of the above? or something else?

This is the next question from RoR Rails 3 - How to handle an incomplete multibyte PG error character

Answer 1

The key is to understand the difference between UTF-8 and Unicode .

• Java processes characters and strings in memory using Unicode. Each character is stored in two bytes.
• When text is transferred between processes (for example, to a web server) or written to / read from disk, the internal representation is converted to a format other than wired. This is encoding or decoding. UTF-8 is the most popular, but in other formats:
  • Utf-16
  • ISO 8859-1

In your question, you mention that the XML files are encoded in utf-8: this is good, and you can put other people's characters in it, but this indicates the encoding only for that particular XML file.

These XML files will be compiled into Android resources and will contain the correct values ​​(you can check this if you wish in the debugger or save intermediate Java resource files from the assembly chain).

The problem almost certainly is that you are sending data and receiving data from an HTTP server, in particular where this data is converted between bytes on the network and Java String . You do not currently set it in the request - this can be done as described in the documentation for Apache HTTPClient .

Although the server may already require / suppose this, it is certainly a good thing to clearly indicate in the request.

You also need to make sure the server (one in Rails 3 - how to handle a PG error with an incomplete multibyte character ):

• Awaiting UTF-8
• Decodes a request using the UTF-8 decoder
• Encodes a response using UTF-8 encoding

(Sorry, I don’t know Ruby on Rails, so I don’t know how to specifically help there).

At the end of Android, you should also make sure your HTTP library decodes the response using the UTF-8 decoder. If you can handle this yourself, make sure you use this String constructor and the argument is "utf-8":

• public String (byte [] data, String charsetName)

Once the client and server use UTF-8, your problems will be resolved.

To help debugging here, I suggest:

• Several logging statements on the server and client that print the appropriate lines as close as possible to the HTTP code

• Launch with a client configured to talk through a debug proxy. Examine the request and response and make sure that they are truly UTF-8. Proxies include:

  • Charles
  • Webscarab
  • Fiddler