JSON.stringify is very slow for large objects

Question

I have a very large object in JavaScript (about 10 MB).

And when I convert it to a string, it takes a lot of time, so I send it to the backend and parse it for an object (actually nested objects with arrays), and this also takes a lot of time, but this is not our problem in this matter.

This problem:

How can I do it JSON.stringifyfaster, any ideas or alternatives, I need a javaScript solution, libraries that I can use, or ideas here.

What i tried

I am a lot JSON.stringifyand there seems to be no better performance than JSON.stringifyor my skills JSON.stringify!

Result

I accept any offer that can solve my long-term savings (sending to the server) in the request (I know its big request).

Sample problem code (details about the problem)

Request URL:http://localhost:8081/systemName/controllerA/update.html;jsessionid=FB3848B6C0F4AD9873EA12DBE61E6008
Request Method:POST
Status Code:200 OK

I send POST to the backend, and then to JAVA

request.getParameter ("BigPostParameter")

and I read it to convert to an object using

 public boolean fromJSON(String string) {
        if (string != null && !string.isEmpty()) {
            ObjectMapper json = new ObjectMapper();
            DateFormat dateFormat = new SimpleDateFormat(YYYY_MM_DD_T_HH_MM_SS_SSS_Z);
            dateFormat.setTimeZone(TimeZone.getDefault());
            json.setDateFormat(dateFormat);
            json.configure(DeserializationFeature.ACCEPT_SINGLE_VALUE_AS_ARRAY, true);
            WebObject object;
//            Logger.getLogger("JSON Tracker").log(Level.SEVERE, "Start");
            try {
                object = json.readValue(string, this.getClass());
            } catch (IOException ex) {
                Logger.getLogger(JSON_ERROR).log(Level.SEVERE, "JSON Error: {0}", ex.getMessage());
                return false;
            }
//            Logger.getLogger("JSON Tracker").log(Level.SEVERE, "END");
            return this.setThis(object);
        }
        return false;
    }

Like this

BigObject someObj = new BigObject();
someObj.fromJSON(request.getParameter("BigPostParameter"))

PS : FYI this line is object = json.readValue(string, this.getClass());also very, very, very slow.

Summarize again

• Problem in publication time (stringify) JavaScript bottle nickname.

• Another problem when parsing a row into an object (using Jackson), and basically I have the contents of the svg tags in that object as a style column, and the other columns are strings, mostly int

Answer 1

As commentators said, there is no way to make the analysis faster.

, /, , .

, .

• ...
• ...
• ...
• ...
• , JSON
• ( , )
• ...
• JSON ,

json-diff https://github.com/andreyvit/json-diff, , .

Answer 2

. POST- 10 , , . /blob/buffer, formdata application/json application/x-www-form-urlencoded.

Express/Multer

Answer 3

, "" , async!

, JS , ... ... Service-Workers, , .

mainPage.js

//= Functions / Classes =============================================================|
// To tell JSON stringify that this is already processed, don't touch
class SerializedChunk {
  constructor(data){this.data = data}
  toJSON() {return this.data}
}

// Attach all events and props we need on workers to handle this use case
const mapCommonBindings = w => {
  w.addEventListener('message', e => w._res(e.data), false)
  w.addEventListener('error', e => w._rej(e.data), false)
  w.solve = obj => {
    w._state && await w._state.catch(_=>_) // Wait for any older tasks to complete if there is another queued
    w._state = new Promise((_res, _rej) => {
      // Give this object promise bindings that can be handled by the event bindings
      // (just make sure not to fire 2 errors or 2 messages at the same time)
      Object.assign(w, {_res, _rej})
    })
    w.postMessage(obj)
    return await w._state // Return the final output, when we get the 'message' event
  }
}

//= Initialization ===================================================================|
// Let make our 10 workers
const workers = Array(10).fill(0).map(_ => new Worker('worker.js'))
workers.forEach(mapCommonBindings)

// A helper function that schedules workers in a round-robin
workers.schedule = async task => {
  workers._c = ((workers._c || -1) + 1) % workers.length
  const worker = workers[workers._c]
  return await worker.solve(task)
}
// A helper used below that takes an object key, value pair and uses a worker to solve it
const _asyncHandleValuePair = async ([key, value]) => [key, new SerializedChunk(
  await workers.schedule(value)
)]

//= Final Function ===================================================================|
// The new function (You could improve the runtime by changing how this function schedules tasks)
// Note! This is async now, obviously
const jsonStringifyThreaded = async o => {
  const f_pairs = await Promise.all(Object.entries(o).map(_asyncHandleValuePair))

  // Take all final processed pairs, create a new object, JSON stringify top level
  final = f_pairs.reduce((o, ([key, chunk]) => (
    o[key] = chunk,  // Add current key / chunk to object
    o                // Return the object to next reduce
  ), {})             // Seed empty object that will contain all the data
  return JSON.stringify(final)
}

/* lot of other code, till the function that actually uses this code */

async function submitter() {
  // other stuff
  const payload = await jsonStringifyThreaded(input.value)
  await server.send(payload)
  console.log('Done!')
}

worker.js

self.addEventListener('message', function(e) {
  const obj = e.data
  self.postMessage(JSON.stringify(obj))
}, false)

:

:

• 10 .
  • async .solve(Object): String, , , .
• : async jsonStringifyThreaded(Object): String, JSON.stringify
  • ( , :))
  • SerializedChunk, JSON.stringify , ( .toJSON())
  • , , (, )

You might want to consider a few more things to improve performance:

• Use of portable facilities that will significantly reduce the cost of transferring facilities to service workers
• Redesign jsonStringifyThreaded()to plan more objects at deeper levels.