Python Templates for Huge HTML / XML

Question

Recently, I needed to create a huge HTML page containing a report with several thousand rows of a table. And, obviously, I did not want to build all the HTML (or the base tree) in memory. As a result, I created a page with good old string interpolation, but I don't like the solution.

So, I wonder if there are Python template engines that can cause page content to appear in parts.

UPD 1 . I am not interested in listing all the available frameworks and templates. I'm interested in template solutions that I can use separately from any frameworks and which can give content in parts, and not build the entire result in memory.

I understand the usability improvements from partially downloading content using client scripts, but this is beyond the scope of my current question. Let's say I want to create a huge HTML / XML and pass it to a local file.

Answer 1

Most popular template engines have a way to generate or write the result to store objects with pieces. For instance:

• Template.generate() in Jinja2
• Template.render_context() in Mako
• Stream.serialize() in Genshi

Answer 2

It would be more convenient (provided that they have javascript) to build a table through javascript, using, for example, the jQuery plugin, which allows you to automatically load content as soon as you scroll down. Then, only a few lines are initially loaded, and when the user scrolls down, more lines are loaded on demand.

If this is not a solution, you can use three patterns: one for everything before the lines, one for everything after the lines, and the third for the lines. Then you first send the before-rows pattern, then generate the strings and send them right away, and then the pattern after the strings. Then you will only have one block / row in memory instead of the entire table.

Answer 3

There is no problem creating something like that in memory. A few thounsand lines are by no means large.

For your template needs, you can use any of:

• rst http://docutils.sourceforge.net/rst.html (just need the docutils core)
• markdown http://daringfireball.net/projects/markdown/ ( python lib )
• sphinx http://sphinx.pocoo.org/ (python based)
• json http://www.json.org/ ( python lib )

There are some tools that allow you to generate HTML from these markup languages.

Answer 4

You do not need a template tracing mechanism - I do this all the time, and long before you run into something with an indefinitely heavy server, the browser will begin to choke. Rendering a row table of 10,000 will tie the processor in seconds in almost any browser; scrolling will be too intermittent in chrome, and mem browser usage will increase regardless of browser.

What you can do (and I previously implemented it, although this was not necessary in retrospect) is the client side xslt. Printing xslt processing instructions and opening and closing tags using strings is easy and fairly safe; then you can pass each individual line as a separate xml element using any xml writer technique you prefer.

However - you really don't need this and probably never will - if your html generator is too slow, the browser will be an order of magnitude more problematic.

So, if you have not compared this and determined that you really have a problem, do not waste your time. If you have a problem, you can solve it without fundamentally changing the method - memory generation may work fine.

Answer 5

Do you use a web framework for this? http://www.pylonshq.com includes compatibility with multiple template templates. http://www.djangoproject.com/ Django has its own template language.

I think the answer, which included lazy loading lines with javascript, would work for web browsing, but I suppose the report would need to be printed out, in which case you would need to build it all at some point, right?