Marklogic mlcp custom transform splits an aggregated document into multiple files

Question

I have a “cumulative” JSON file that I want to split and swallow as multiple documents in MarkLogic using mlcp.

I want to convert the contents while swallowing using javascript .

My JSON file looks something like this:

{ "type": "FeatureCollection", "features": [ {blobA}, {blobB}, {blobC} ...... ] } 

... and I want to run this file through MLCP so that each document contains an element in an array.

i.e. One document will contain {blobA}, another will contain {blobB}, and the other will contain {blobC} .... etc.

How to write your own .sjs conversion module?

Answer 1

See an example here: http://docs.marklogic.com/guide/mlcp/import#id_26044

The original input document is expected to take the following form:

 { uri: string, value: node } 

This is also the expected output form for each document. You will also want your return to be of type document-node, since you want mlcp to split it and swallow it as JSON documents.

So your custom .sjs conversion module will look something like this.

 function splitFeatures(doc) { const features = doc.value.toObject().features; return xdmp.arrayValues( features.map(function(feature) { return { uri: '/path/itemhere-' + xdmp.random() + '.json', value: xdmp.toJSON(feature) } }) ); } exports.transform = splitFeatures; 

Aside, this is a useful resource when working with JSON in MarkLogic .