How to parse a JSON-XML hybrid file in Python

Question

I would like to parse a file with the following syntax (but with an indefinite number of nesting) with Python:

<XProtocol> { <str1."fds"> "str2" <str3> 123.0 <str4> { 1 2 3 4 5 6 6 "str" "str" 43 "str" 4543 } <weird1."str5"> { <weird2."str6"> { "str" } <also."weird3"> 1 <againweird> { 1 "fds" } { } <even> <more."weird4"> { } { } { } { "a" } } } 

The highlighted output will look something like this:

 'XProtocol': { 'str1."fds"': 'str2', 'str3': 123.0, 'str4': (1, 2, 3, 4, 5, 6, 6, 'str', 'str', 43, 'str', 4543), 'weird1."str5"': { 'weird2."str6"': ( 'str' ), 'also."weird3"': 1, 'againweird': ((1, 'fds'), None), 'even': { 'more."weird4"': (None, None, None, 'a') }, } } 

I tried unsuccessfully to use the following code:

 import pyparsing as pp def parse_x_prot(text): lbra = pp.Literal('{').suppress() rbra = pp.Literal('}').suppress() lang = pp.Literal('<').suppress() rang = pp.Literal('>').suppress() dot = pp.Literal('.').suppress() cstr = pp.quotedString.addParseAction(pp.removeQuotes) tag = pp.Group( lang + pp.Word(pp.alphanums) + pp.Optional(pp.Group(dot + cstr)) + rang) val = pp.OneOrMore( cstr | pp.Word(pp.nums + '.') ) exp = pp.Forward() exp << pp.OneOrMore( pp.Group( tag + pp.OneOrMore( (lbra + (val | exp) + rbra) | (val + exp) ) ) ) return exp.parseString(text) 

I have to do something wrong, but have not yet figured out what exactly ... more precisely: the following code tells me that it expects "}" instead of a new "tag".

Answer 1

A few things:

In your tag definition, you wrap it in Group , but I think you really want to use Combine .

Secondly, your investment in exp mixes repetition with recursion.

This works for me (also, take .suppress() on dot ):

 tag = pp.Combine( lang + pp.Word(pp.alphas, pp.alphanums) + pp.Optional(dot + cstr) + rang).setName("tag") exp = pp.Forward() key_value = pp.Group(tag + exp) number = pp.Regex(r'[+-]?\d+(\.\d*)?').setName("number") exp <<= (number | cstr | key_value | pp.Group(lbra + pp.ZeroOrMore(exp) + rbra)) 

Donation:

 ['XProtocol', [['str1.fds', 'str2'], ['str3', '123.0'], ... [0]: XProtocol [1]: [['str1.fds', 'str2'], ['str3', '123.0'], ['str4', ['1', '2', '3',... [0]: ['str1.fds', 'str2'] [1]: ['str3', '123.0'] [2]: ['str4', ['1', '2', '3', '4', '5', '6', '6', 'str', 'str', '43', ... [0]: str4 [1]: ['1', '2', '3', '4', '5', '6', '6', 'str', 'str', '43', ... [3]: ['weird1.str5', [['weird2.str6', ['str']], ['also.weird3', ... [0]: weird1.str5 [1]: [['weird2.str6', ['str']], ['also.weird3', '1'], ['againweird', ... [0]: ['weird2.str6', ['str']] [0]: weird2.str6 [1]: ['str'] [1]: ['also.weird3', '1'] [2]: ['againweird', ['1', 'fds']] [0]: againweird [1]: ['1', 'fds'] [3]: [] [4]: ['even', ['more.weird4', []]] [0]: even [1]: ['more.weird4', []] [0]: more.weird4 [1]: [] [5]: [] [6]: [] [7]: ['a'] 

Answer 2

I changed a few things in your code to make it work correctly, I used comments to indicate what went wrong.

 def parse_x_prot(text): # Set up some shortcuts lbra = pp.Literal('{').suppress() rbra = pp.Literal('}').suppress() lang = pp.Literal('<').suppress() rang = pp.Literal('>').suppress() dot = pp.Literal('.') cstr = pp.quotedString.addParseAction(pp.removeQuotes) # Define how a correct tag looks like (we use combine here to get the full tag in the output) tag = pp.Combine( lang + pp.Word(pp.alphanums) + pp.Optional(pp.Group(dot + pp.quotedString)) + rang) # Define legal value (first combine is for decimal values) val = pp.Combine(pp.Word(pp.nums) + dot + pp.Word(pp.nums)) | cstr | pp.Word(pp.nums) # Define the array with statement as recursion element statement = pp.Forward() array = pp.Group(pp.OneOrMore(tag) + pp.OneOrMore( ( # Note the one or more function here as we have # a kind of list here which will have elements (lbra + pp.OneOrMore(val | statement) + rbra) | val | (lbra + rbra) ) ) ) statement << array return statement.parseString(text) 

Answer 3

This may not be the answer you want, but I think Flex can help you do this. There could even be a python shell for it