Ordered words

Question

Can I extend the syntax in python to understand dict for other dicts such as OrderedDict in the collections module or my own types that inherit from dict ?

Just reordering the dict name obviously does not work, the {key: value} syntax still gives you a simple old dict for understanding and literals.

 >>> from collections import OrderedDict >>> olddict, dict = dict, OrderedDict >>> {i: i*i for i in range(3)}.__class__ <type 'dict'> 

So, if possible, how would I do this? This is normal if it only works in CPython. For syntax, I suppose I'll try it with the prefix O{k: v} , as we have on r'various' u'string' b'objects' .

note: Of course, we can use the generator expression instead, but I'm more interested in learning how hacker python is in terms of grammar.

Answer 1

There is no direct way to change Python syntax from a language. Understanding a dictionary (or a simple mapping) will always create a dict , and there is nothing you can do about it. If you use CPython, it uses special bytecodes that directly generate dict, which ultimately call the PyDict API PyDict and / or the same basic functions used by this API. If you use PyPy, these bytecodes are instead implemented on top of the RPython dict object, which in turn is implemented on top of the compiled and optimized Python dict . And so on.

There is an indirect way to do this, but you will not like it. If you read the documents in the import system , you will see that the importer who is looking for cached compiled code or calls the compiler, and the compiler that calls the parser, and so on. In Python 3.3+, almost everything in this chain is either written in pure Python or has an alternative pure Python implementation, that is, you can develop the code and do your thing. Which includes parsing the source code with your own PyParsing code, which builds an AST or compiles an understanding of AD ASTM node into its own bytecode instead of standard or post-processing bytecode or ...

In many cases, import hook is enough; if not, you can always write a custom finder and loader.

If you are not using Python 3.3 or later, I highly recommend migrating before playing with this material. In earlier versions, this is more complex and less well documented, and you will end up trying 10 times to find out something that will be deprecated when you migrate.

In any case, if this approach seems interesting to you, you can take a look at MacroPy . You can borrow some code from him - and, more importantly, find out how some of these functions are used (which do not have good examples in the documents).

Or, if you're willing to settle for something less cool, you can simply use MacroPy to create a “macro for understanding the expression” and use it. (Note that MacroPy currently only works in Python 2.7, not 3.x.) You cannot get o{…} , but you can get, say, od[{…}] , which is not so bad . Download od.py , realmain.py and main.py and run python main.py to see how it works. The key is this code, which the DictionaryComp AST accepts, converts it to the equivalent GeneratorExpr by the value of the Tuple s key and transfers it to Call in collections.OrderedDict :

 def od(tree, **kw): pair = ast.Tuple(elts=[tree.key, tree.value]) gx = ast.GeneratorExp(elt=pair, generators=tree.generators) odict = ast.Attribute(value=ast.Name(id='collections'), attr='OrderedDict') call = ast.Call(func=odict, args=[gx], keywords=[]) return call 

---

Another alternative is, of course, modifying the Python interpreter.

I would advise you to abandon the idea of o{…} syntax for your first exit, and just make compilation of normal dictons compilation in odicts. The good news is that you really don't need to change the grammar (which is higher than the hairy ...), just any of:

• bytecodes that compile dictcomps,
• the way the interpreter runs these bytecodes, or
• PyDict type PyDict

The bad news, while all this is much simpler than changing the grammar, none of them can be done from the extension module. (Well, you can do the first by doing basically the same thing as with pure Python ... and you can do any of them by connecting .so / .dll / .dylib to fix your own functions, but that's for sure same work as hacking in Python plus the extra work of connecting at runtime.)

If you want to hack the source of CPython , the code you need is in Python/compile.c , Python/ceval.c and Objects/dictobject.c , and the dev guide tells you how to find everything you need. But instead, you might consider hacking the PyPy source , as it is mainly written in (a subset) of Python, not C.

---

As a side note, your attempt would not work, even if everything was done at the Python language level. olddict, dict = dict, OrderedDict creates a binding with the name dict in your global global module, which obscures the name in the built-in functions, but does not replace it. You can replace things in the built-in (well, Python does not guarantee this, but there are things related to the implementation / version that would happen for every implementation / version I tried ...), but you did it. This is the way to do this is.

Answer 2

Sorry, not possible. Dict's literals and understanding of the recorder correspond to the built-in type of dict, so that it is hardcoded at level C. This cannot be redefined.

You can use this as an alternative though:

 OrderedDict((i, i * i) for i in range(3)) 

Answer 3

By slightly modifying @Max Noel's answer, you can use the list view instead of the generator to create an OrderedDict in an orderly way (which, of course, is impossible using a dict understanding).

 >>> OrderedDict([(i, i * i) for i in range(5)]) OrderedDict([(0, 0), (1, 1), (2, 4), (3, 9), (4, 16)])