Insert 1000+ nodes and attributes using XMLStarlet - Run slowly

Question

This is a matter of efficiency, not troubleshooting. I have the following code snippet:

# The -R flag restores malformed XML
xmlstarlet -q fo -R <<<"$xml_content" | \
    # Delete xml_data
    xmlstarlet ed -d "$xml_data" | \
    # Delete index
    xmlstarlet ed -d "$xml_index" | \
    # Delete specific objects
    xmlstarlet ed -d "$xml_nodes/objects" | \
    # Append new node
    xmlstarlet ed -s "$xml_nodes" -t elem -n subnode -v "Hello World" | \
        # Add x attribute to node
        xmlstarlet ed -i "($xml_nodes)[last()]" -t attr -n x -v "0" | \
        # Add y attribute to node
        xmlstarlet ed -i "($xml_nodes)[last()]" -t attr -n y -v "0" | \
        # Add z attribute to node
        xmlstarlet ed -i "($xml_nodes)[last()]" -t attr -n z -v "1" \
            > "$output_file"

• The variable $xml_contentcontains the xml content tree and the
  nodes are analyzed from a 472.6 MB file using the command cat.

• The variable $output_file, as its name indicates, contains the path to the output file.

• The rest of the variables simply contain the corresponding XPaths that I want to change.

According to this brief article that helped come up with this code, this means that:

This is a bit inefficient as the XML file is parsed and written twice.

( loop 1000 ).

, script, 4 7 .

, , , , - , , , /, .

, , , , , , xmlstarlet XML-.

---

UPDATE

@Cyrus :

xmlstarlets :

xmlstarlet -q fo -R <<<"$xml_content" |\
  xmlstarlet ed \
    -d "$xml_data" \
    -d "$xml_index" \
    -d "$xml_nodes/objects" \
    -s "$xml_nodes" -t elem -n subnode -v "Hello World" \
    -i "($xml_nodes)[last()]" -t attr -n x -v "0" \
    -i "($xml_nodes)[last()]" -t attr -n y -v "0" \
    -i "($xml_nodes)[last()]" -t attr -n z -v "1" > "$output_file"

:

• -:691.84: Attribute x redefined
• -:691.84: Attribute z redefined
• -:495981.9: xmlSAX2Characters: huge text node: out of memory
• -:495981.9: Extra content at the end of the document

, , , , , :

xmlstarlet ed --omit-decl -L \
    -d "$xml_data" \
    -d "$xml_index" \
    -d "$xml_nodes/objects" \
    -s "$xml_nodes" -t elem -n subnode -v "Hello World" \
    "$temp_xml_file"

xmlstarlet ed --omit-decl -L \
    -i "($xml_nodes)[last()]" -t attr -n x -v "0" \
    -i "($xml_nodes)[last()]" -t attr -n y -v "0" \
    -i "($xml_nodes)[last()]" -t attr -n z -v "1" \
    "$temp_xml_file"

data, , , :

...
<node>
    <subnode>A</subnode>
    <subnode>B</subnode>
    <objects>1</objects>
    <objects>2</objects>
    <objects>3</objects>
    ...
</node>
...

(split) , :

...
<node>
    <subnode>A</subnode>
    <subnode>B</subnode>
    <subnode x="0" y="0" z="1">Hello World</subnode>
</node>
...

, xmlstarlet attributes node, last() Xpath , --subnode . - , , .

xmlstarlet ed --omit-decl -L \
    -d "$xml_data" \
    -d "$xml_index" \
    -d "$xml_nodes/objects" \
    -s "$xml_nodes" -t elem -n subnode -v "Hello World" \
    -i "($xml_nodes)[last()]" -t attr -n x -v "0" \
    -i "($xml_nodes)[last()]" -t attr -n y -v "0" \
    -i "($xml_nodes)[last()]" -t attr -n z -v "1" \
    "$temp_xml_file"

, :

...
<node>
    <subnode>A</subnode>
    <subnode x="0" y="0" z="1">B</subnode>
    <subnode>Hello World</subnode>
</node>
...

xmlstarlets , , @Cyrus, - attributes, --subnode innerText - Hello World.

• - , ?

, , " "

, , xmlstarlet ed \. :

• ($xml_nodes)[last()] $xml_nodes[text() = 'Hello World']
• $prev ( $xstar:prev) -i, answer. []
• -r temp node attr

--subnode, attributes.

. XMLStarlet 1.6.1 OS X El Capitan v 10.11.3

---

BONUS

, loop - :

list="$(tr -d '\r' < $names)"

for name in $list; do
    xmlstarlet ed --omit-decl -L \
    -d "$xml_data" \
    -d "$xml_index" \
    -d "$xml_nodes/objects" \
    -s "$xml_nodes" -t elem -n subnode -v "$name" \
    -i "($xml_nodes)[last()]" -t attr -n x -v "0" \
    -i "($xml_nodes)[last()]" -t attr -n y -v "0" \
    -i "($xml_nodes)[last()]" -t attr -n z -v "1" \
    "$temp_xml_file"
done

$list , attributes. --value loop. :

• loop, node?

• txt , XML- ( txt) XML. , ? , sed grep?

, - . node, xml txt, , . XPath , .

: . loop 26 --subnodes loop 3 4 attr --subnode. xmlstarlet attr , - . .

Answer 1

( sem), , ? , , , , , .

for array in "${listofarrays[@]}"; do
    local var1;local var2
    IFS=, read var1 var2 <<< $array
    sem -j +0
    <code goes here>
done
sem --wait