How does the LuaJIT compiler work?

Question

I read mostly the JIT and LuaJIT compiler, and I had some questions.

From what I understand, LuaJIT JIT does not compile hot methods like Java HotSpot, it compiles hot paths coming from loops. Does this mean that if something does not come from the loop (let's say I call Lua functions from C-api), then the code will never be skewed? And what happens when you hit another loop? Will there be a path to the second JIT'ed loop, and then a new path from this loop, and will the second loop also be part of the same path?

How does the interpreter choose the most optimal hot way? Let's say I have a hash table of strings ints ->. Now imagine that I called table [x] with x equal to 3 and 5 enough to become hot tracks and jitted, as the interpreter decided which jitted code to call for table [x], where x is 4?

Another thing that was breaking my brain. Since paths are compiled, not functions, will the trace compiler require more memory? Since you cannot really reuse the compiled code of another path, I mean, and since in the general case the paths will have more single functions ...

Answer 1

Mike Pall responded in some detail to the LuaJIT mailing list. http://www.freelists.org/post/luajit/How-does-LuaJITs-trace-compiler-work,1

Answer 2

The first part you have to stand on is LuaJIT IR and Bytecode , which you can check in the wiki, is that the LuaJIT interpreter starts and optimizes, and therefore tracks the trace that needs to be compiled, as well as various as well additional optimizations such as cyclic deployment for hot contours in the trace path.

The second place to check is the LJ FAQ , which states the following:

Q: Where can I find out more about the compiler technology used by LuaJIT?

I plan to write additional documentation on the internals of LuaJIT. At the same time, please use the following Google Scholar search for suitable documents:

Search: Trace Compiler

Search: JIT compiler

Search: Dynamic language optimizations

Search: SSA Form

Search: Linear distribution of scan register

Here is a list of innovative features in LuaJIT. And, you know, reading the source is, of course, the only way to enlightenment. :-)

Abet is very tongue on the cheek (mainly because Mike focuses on development rather than documentation), the most important part is the last sentence, the source is very clean and the only real way to find out how LJ does its magic. In addition, a link to innovative features also provides another key to the search.

Wikipedia has a more descriptive JIT trace page , however the papers below are what you find most useful to help understand the concepts used in the LJ source.

Some source files (in C) to get started

• lj_record.c : main recorder, converts bytecode to IR
• lj_trace.c : trace control
• lj_snap.c : processes / creates trace snapshots
• lj_ffrecord.c : writes data for fast functions
• lj_crecord.c : writes C data statements