Android client parses multiple form data for non ascii

Question

Can you explain why the android client analyzes multi-factor data in chat without ascii. while file uploading works fine using a postman here is my app.js code

var multipart = require('connect-multiparty'); var apiRoutes = require('./routes/apiRoutes'); app.set('views', path.join(__dirname, 'views')); app.use(logger('dev')); app.use(bodyParser.json({limit: '50mb'})); app.use(bodyParser.urlencoded([{extended: false},{ uploadDir:path.join(__dirname, 'uploads') }, {parameterLimit:100000}, {limit: '50mb'}])); app.use(cookieParser()); app.use(express.static(path.join(__dirname, 'public'))); app.use(express.static(path.join(__dirname, 'uploads'))); app.use(multipart()); app.use(apiRoutes); 

and apiRoutes indicate that my load function contains a simple req param parameter. use postman is working fine

 console.log("mediaChat called", req.body, req.files); 

answer

 mediaChat called { apiKey: '123' } { media: { fieldName: 'media', originalFilename: 'default.png', path: '/tmp/KFnwsKGp-f4woTaBH6aPR-qa.png', headers: { 'content-disposition': 'form-data; name="media"; filename="default.png"', 'content-type': 'image/png' }, size: 716, name: 'default.png', type: 'image/png' } } 

here is my client code for Android (note that this working code works with php $ _FILE but does not work with expression)

 com.info.acruss.wave; import android.os.AsyncTask; import android.util.Log; import java.io.BufferedReader; import java.io.DataOutputStream; import java.io.File; import java.io.FileInputStream; import java.io.InputStreamReader; import java.net.HttpURLConnection; import java.net.MalformedURLException; import java.net.URL; /** * Created by Naitik on 9/20/2016. */ public class UploadImage extends AsyncTask<Void, Void, String> { String CallingURL; URL url; OnTaskCompleted myListener; private static final String TAG = "UploadImage"; int timeoutMilli = 60000; String sourceFileUri; String ApiKey,Type; public UploadImage(String sourceFileUri, String URL,String apiKey, String type, OnTaskCompleted listener) { Log.e("Uploading", "API:" + URL); this.sourceFileUri = sourceFileUri; this.CallingURL = URL; this.myListener = listener; this.ApiKey=apiKey; this.Type=type; try { url = new URL(CallingURL); Log.e(TAG, "Url : " + CallingURL); } catch (MalformedURLException e) { e.printStackTrace(); } } @Override protected String doInBackground(Void... params) { String fileName = sourceFileUri; HttpURLConnection conn = null; DataOutputStream dos = null; String lineEnd = "\r\n"; String twoHyphens = "--"; String boundary = "*****"; int bytesRead, bytesAvailable, bufferSize; byte[] buffer; int maxBufferSize = 10 * 1024 * 1024; File sourceFile = new File(sourceFileUri); if (!sourceFile.isFile()) { Log.e("UploadImage", "Source File Does not exist"; return null; } String serverResponseMessage = ""; try { // open a URL connection to the Servlet FileInputStream fileInputStream = new FileInputStream(sourceFile); URL url = new URL(CallingURL); // Open a HTTP connection to the URL conn = (HttpURLConnection) url.openConnection(); conn.setDoInput(true); // Allow Inputs conn.setDoOutput(true); // Allow Outputs conn.setUseCaches(false); // Don't use a Cached Copy conn.setRequestMethod("POST"; conn.setReadTimeout(timeoutMilli); conn.setConnectTimeout(timeoutMilli); conn.setRequestProperty("Connection", "Keep-Alive"; conn.setRequestProperty("ENCTYPE", "multipart/form-data"; conn.setRequestProperty("Content-Type", "multipart/form-data;boundary=" + boundary); conn.setRequestProperty("media", fileName); conn.setRequestProperty("apiKey",ApiKey); conn.setRequestProperty("media_type",Type); conn.setRequestProperty("media", fileName); dos = new DataOutputStream(conn.getOutputStream()); dos.writeBytes(twoHyphens + boundary + lineEnd); dos.writeBytes("Content-Disposition: form-data; name='media';filename='" + fileName + "'" + lineEnd); dos.writeBytes(lineEnd); // create a buffer of maximum size bytesAvailable = fileInputStream.available(); bufferSize = Math.min(bytesAvailable, maxBufferSize); buffer = new byte[bufferSize]; // read file and write it into form... bytesRead = fileInputStream.read(buffer, 0, bufferSize); while (bytesRead > 0) { dos.write(buffer, 0, bufferSize); bytesAvailable = fileInputStream.available(); bufferSize = Math.min(bytesAvailable, maxBufferSize); bytesRead = fileInputStream.read(buffer, 0, bufferSize); } // send multipart form data necesssary after file data... dos.writeBytes(lineEnd); dos.writeBytes(twoHyphens + boundary + twoHyphens + lineEnd); // Responses from the server (code and message) int serverResponseCode = conn.getResponseCode(); BufferedReader br = new BufferedReader(new InputStreamReader(conn.getInputStream())); String line = ""; StringBuilder responseOutput = new StringBuilder(); while ((line = br.readLine()) != null) { responseOutput.append(line); } br.close(); serverResponseMessage = responseOutput.toString();//output.toString(); Log.e("uploadFile", "HTTP Response is : " + serverResponseMessage); if (serverResponseCode == 200) { //status code 200 //status ok } //close the streams // fileInputStream.close(); dos.flush(); dos.close(); } catch (MalformedURLException ex) { ex.printStackTrace(); Log.e("Upload file to server", "error: " + ex.getMessage(), ex); } catch (Exception e) { Log.e("Upload file to server", "error: " + e.getMessage(), e); e.printStackTrace(); } return serverResponseMessage; } @Override protected void onPostExecute(String result) { super.onPostExecute(result); Log.e("Result", "" + result); if (myListener != null) if (result != null) { myListener.onFileUploadComplete(result); } else { myListener.onFileUploadComplete(""; } } public interface OnTaskCompleted { void onFileUploadComplete(String result); } } 

using android, this wired answer shows below:

 mediaChat called {null: '     \ u0000 \ u0010JFIF \ u0000 \ u0001 \ u0001 \ u0000 \ u0000 \ u0001 \ u0000 \ u0001 \ u0000 \ u0000   \ u0000C \ u0000 \ u0010 \ u000b \ f \ u000e \ f \ n \ u0010 \ u000e \ r \ u000e \ u0012 \ u0011 \ u0010 \ u0013 \ u0018 (\ u001a \ u0018 \ u0016 \ u0016 \ u00181 #% \ u001d (: 3 = Mqypdx \\ egc   \ u0000C \ u0001 \ u0011 \ u0012 \ u0012 \ u0018 \ u0015 \ u0018 / \ u001a \ u001a / cB8Bccccccccccccc ....
   \ u001f. [   _  \ u0014) M   XIjX  7 ` =   /  8`  ïDʚ \ u0018 D    #  V # q ~ m q10L  ' }

I also tried multer and another multiprocessor handler, but noted the work. please help me out of this hell.

Answer 1

It seems that the server response is encoded as UTF-8. To decode and read correctly, you can try

 BufferedReader br = new BufferedReader(new InputStreamReader(conn.getInputStream(), "UTF-8")); 

Setting the Encoding Part InputStreamReader will decode and read the stream using the specified encoding scheme, which in this case is UTF-8. See javadocs for more details.

Answer 2

You need to re-allocate the buffer based on the new bufferSize , I revised the following code:

 // create a buffer of maximum size bytesAvailable = fileInputStream.available(); bufferSize = Math.min(bytesAvailable, maxBufferSize); buffer = new byte[bufferSize]; // read file and write it into form... bytesRead = fileInputStream.read(buffer, 0, bufferSize); while (bytesRead > 0) { dos.write(buffer, 0, bufferSize); bytesAvailable = fileInputStream.available(); bufferSize = Math.min(bytesAvailable, maxBufferSize); buffer = null; buffer = new byte[bufferSize]; bytesRead = fileInputStream.read(buffer, 0, bufferSize); } 

Answer 3

Submitting a multi-page form takes away a lot of the uncertainty that had percent encoding: the server can now explicitly request certain encodings, and the client can explicitly tell the server when the form is submitted in which encoding the fields are.

There are two ways to use this function: leave it inactive and send the browser to the same encoding as the page, or install it in UTF-8, and then perform the other side of the conversion server. Each method has drawbacks, especially the first ones.

If you tell the browser to submit the form in the same encoding as the page, you still have a problem with what to do with characters outside the range of the character encoding. The behavior, again, is changing: Firefox 2.0 converts them into links to character entities, while Internet Explorer 7.0 manages them without discrimination. For serious internationalization purposes this is not an option.

Another possibility is to install Accept-Encoding on UTF-8, which asks the question: why aren't you using UTF-8 for all this? This route is more enjoyable, but there is a noticeable caveat: your data will be presented as UTF-8, so you will have to explicitly convert it to your preferred local character encoding.

Unicode-based encoding, such as UTF-8, can support many languages ​​and can host pages and forms in any mixture of these languages. Its use also eliminates the need for server-side logic to individually determine the character encoding for each page served or for each incoming form. This greatly reduces the complexity of working with a multilingual website or application.

Unicode encoding also allows you to mix several more languages ​​on one page than any other encoding option.

see Why UTF-X