How can I use XmlReader in PowerShell to stream large / large XML files?

Question

I have XML from a couple of gigabytes. There are no spaces in XML.

So, I wrote a little C # code for splitting in single files (which has some additional code to do some things, like randomization during testing)

using (XmlReader MyReader = XmlReader.Create(@"d:\xml\test.xml")) { while (MyReader.Read()) { switch (MyReader.NodeType) { case XmlNodeType.Element: if (MyReader.Name == "Customer") { XElement el = XElement.ReadFrom(MyReader) as XElement; if (el != null) { custNumber = (string)el.Element("CustNumber"); output = @"d:\xml\output\" + custNumber; File.WriteAllText(output, el.ToString()); } } break; } } } 

Then I parse the resulting files using PowerShell, mainly because it’s easier for me to work with the server, while the specifications can change, and I can change the script on the fly.

So ... what is the easiest way to convert the above to PowerShell and put [.Net here] first? should I read byte for byte only if it has "<cust" on one line and "omer>" on the next?

Answer 1

This should be very close to what you wanted to do in Powershell:

 $f = [System.Xml.XmlReader]::create("d:\xml\test.xml") while ($f.read()) { switch ($f.NodeType) { ([System.Xml.XmlNodeType]::Element) # Make sure to put this between brackets { if ($f.Name -eq "Customer") { $e = [System.Xml.Linq.XElement]::ReadFrom($f) if ($e -ne $null) { $custNumber = [string] $e.Element("CustNumber") $e.ToString() | Out-File -Append -FilePath ("d:\xml\output\"+$e.ToString()) } } break } } }