Is it possible to maintain the structure of the YAML block when discarding the analyzed document?

Question

We use PyYAML for ready-made configuration files for different environments. But our YAML blocks lose integrity.

Give input.yml ...

 pubkey: | -----BEGIN PUBLIC KEY----- MIGfMA0GCSq7OPxRrQEBAQUAA4GNADCBiQKBgQCvRVUKp6pr4qBEnE9lviuyfiNq QtG/OCyBDXL4Bh3FmUzfNI+Z4Bh3FmUx+z2n0FCv/4BpgHTDl8D95NPopWVo1RH2 UfhyMd6dQ/x9T5m+y38JMzmSVAk+Fqu8ya18+yQVOEyEIx3Gxpsgegow33gcxfjK EsUgJHXcpw7OPxRrCQIDAQAB -----END PUBLIC KEY----- 

... running this program with python3 ...

 import yaml with open('input.yml', mode='r') as f: parsed = yaml.safe_load(f) with open('output.yml', mode='w') as f: yaml.dump(parsed, f) 

... produces this output.yml ...

 pubkey: '-----BEGIN PUBLIC KEY----- MIGfMA0GCSq7OPxRrQEBAQUAA4GNADCBiQKBgQCvRVUKp6pr4qBEnE9lviuyfiNq QtG/OCyBDXL4Bh3FmUzfNI+Z4Bh3FmUx+z2n0FCv/4BpgHTDl8D95NPopWVo1RH2 UfhyMd6dQ/x9T5m+y38JMzmSVAk+Fqu8ya18+yQVOEyEIx3Gxpsgegow33gcxfjK EsUgJHXcpw7OPxRrCQIDAQAB -----END PUBLIC KEY----- ' 

Is it possible to save the structure of my block using PyYAML?

Answer 1

Yes, this is possible with pyyaml , but you must provide your own extended versions, at least Scanner, Parser and Constructor, which are used by safe_load, Emitter, Serializer and Presenter, used by dump, and by providing a specialized string version of a class that stores information about this original formatting.

This is part of what was added to ruamel.yaml (disclaimer: I am the author of this package) since it was obtained from PyYAML. Using ruamel.yaml, the preferred way to do this is:

 import sys import ruamel.yaml yaml_str = """\ pubkey: | -----BEGIN PUBLIC KEY----- MIGfMA0GCSq7OPxRrQEBAQUAA4GNADCBiQKBgQCvRVUKp6pr4qBEnE9lviuyfiNq QtG/OCyBDXL4Bh3FmUzfNI+Z4Bh3FmUx+z2n0FCv/4BpgHTDl8D95NPopWVo1RH2 UfhyMd6dQ/x9T5m+y38JMzmSVAk+Fqu8ya18+yQVOEyEIx3Gxpsgegow33gcxfjK EsUgJHXcpw7OPxRrCQIDAQAB -----END PUBLIC KEY----- """ yaml = ruamel.yaml.YAML() # defaults to round-trip yaml.indent(mapping=4) data = yaml.load(yaml_str) yaml.dump(data, sys.stdout) 

Or an older, more PyYAML-like style (which has some limitations in the options you can set)

 import sys import ruamel.yaml as yaml yaml_str = """\ pubkey: | -----BEGIN PUBLIC KEY----- MIGfMA0GCSq7OPxRrQEBAQUAA4GNADCBiQKBgQCvRVUKp6pr4qBEnE9lviuyfiNq QtG/OCyBDXL4Bh3FmUzfNI+Z4Bh3FmUx+z2n0FCv/4BpgHTDl8D95NPopWVo1RH2 UfhyMd6dQ/x9T5m+y38JMzmSVAk+Fqu8ya18+yQVOEyEIx3Gxpsgegow33gcxfjK EsUgJHXcpw7OPxRrCQIDAQAB -----END PUBLIC KEY----- """ data = yaml.load(yaml_str, Loader=yaml.RoundTripLoader) yaml.dump(data, sys.stdout, Dumper=yaml.RoundTripDumper, indent=4) 

Both of which give you:

 pubkey: | -----BEGIN PUBLIC KEY----- MIGfMA0GCSq7OPxRrQEBAQUAA4GNADCBiQKBgQCvRVUKp6pr4qBEnE9lviuyfiNq QtG/OCyBDXL4Bh3FmUzfNI+Z4Bh3FmUx+z2n0FCv/4BpgHTDl8D95NPopWVo1RH2 UfhyMd6dQ/x9T5m+y38JMzmSVAk+Fqu8ya18+yQVOEyEIx3Gxpsgegow33gcxfjK EsUgJHXcpw7OPxRrCQIDAQAB -----END PUBLIC KEY----- 

at least with Python 2.7 and 3.5+.

indent=4 necessary because the default for RoundTripDumper is two spaces, and the original indentation of the file is not saved (this does not make it easy to re-set the indentation for the YAML file).

If you cannot switch to ruamel.yaml you can use its source to extract all the necessary changes, but if you can, you can also use its other functions, such as saving comments and merging keys.