How to analyze and translate DSL using Red or Rebol

Question

I'm trying to figure out if I can use Red (or Rebol) to implement a simple DSL. I want to compile my DSL source code for another language, perhaps Red or C # or both, instead of directly interpreting and executing it.

DSL has only a few simple statements, plus an if / else statement. Statements can be grouped into rules. The rule will be translated into a function definition, with each operator representing an equivalent operator in the target language.

The parsing capability in Red / Rebol is great and allows me to implement the parser very easily - in fact it is just a definition of the grammar itself.

However, I could not find examples of how to take the following steps, especially by processing the if statement and translating it to another source code. Translating an if statement seems like a good example of something minimal, but still a bit complicated - because in Red, having a different one, means that you need to change the if value to another, and not just an optional optional other.

Traditionally, during parsing, I created an abstract syntax tree, and then had functions for working in AST and generating new source code. Should I follow the same approach or is there some other more idiomatic way in red?

I experimented using the collect / keep command in my analysis rules to return a block of nested blocks, which actually forms an AST. Another approach would be to store data in specific objects representing different instructions, etc.

I am still going to collect / store when a new block is created and what will be saved. I would also like my parsing rules to be as clean as possible, since few other code was interwoven in it. Therefore, I still do not know how best to add the Red Code in parentheses in the parsing rules. Adding code too early can cause the red code to execute even if the rule ultimately fails. Adding code too late means that the code may not execute in the expected order, especially when working with multilevel instructions such as if, which may contain other statements.

So, in particular, any help on how to translate my DSL example to red source code will be greatly appreciated. Also, any links to implementing DSL like this in Red or Rebol would be great! :)

Here are my parsing rules: -

Red [ Purpose: example rules for parsing a simple language ] SimpleLanguageParser: make object! [ Expr: [string! | integer! | block!] Data: ['Person.AGE | 'Person.INCOME] WriteMessageToLog: ['write 'message 'to 'log Expr] SetData: ['set 'data Data '= Expr] IfStatement: ['if Expr [any Statement] opt ['else [any Statement]] 'endif] Statement: [WriteMessageToLog | SetData | IfStatement] Rule: [ 'rule word! [any Statement] 'endrule ] AnySimpLeLanguage: [Rule | [any Statement]] ] SL: function [slInput] [ parse slInput SimpleLanguageParser/AnySimpleLanguage ] 

An example of some source in DSL: -

 RULE TooYoung IF [Person.Age < 15] WRITE MESSAGE TO LOG "too young to earn an income" SET DATA Person.Income = 0 ELSE WRITE MESSAGE TO LOG "old enough" ENDIF ENDRULE 

Translation into red source code: -

 TooYoung: function [] [ either Person.Age < 15 [ WriteMessageToLog "too young to earn an income" Person.Income: 0 ] [ WriteMessageToLog "old enough" ] ] 

Data i.e. Person.Age, Person.Income, and the WriteMessageToLog function are all that were previously defined. Please note: for simplicity, I left Expr as a block! etc., and not define Expr in more detail in the DSL itself. In addition, setting Person.Income in a function does not work the way it is encoded, since it sets local, but now it's fine :)

Answer 1

It's always nice to see someone digging language programming, support it and welcome to Red !;)

---

Setting the correct grammar rules is the hardest part of the task, and you've already nailed it. What remains is to cross your PEG (syntax collapsible grammar of an expression) with the expressions set , copy , collect/keep combo and paren! in the right places, and then either create an AST from this, or in simpler cases emit code.

Example

Here's a quick baked (and certainly buggy!) Example of how I solved your problem. Basically, it reworked your code a bit, where the matching patterns are set ted, copy ed or collect ed, and then limited to certain words, which are then simply inserted into the "template" ( compose function inside the emit-rule ) to create a red code.

This is not the only way, I think. @rebolek may come up with a more industrial solution, because he has experience with complex parsers that I miss: P

Followup

As for the if/else dilemma, I followed the approach suggested above - instead of using opt I wrapped the else-branch rule in a block and added an alternative match that simply sets false-block to none .

What to use for AST is all that allows you to express a hierarchical structure, which is either block! (although you can use hash! or map! ) or object! . The advantage of object! lies in the fact that it provides a link to the context, but here we come closer to the sphere of the so-called Bindology (the rules for determining the scope in red language), which is another animal :)

Emitting C # code will be harder, but doable - you will need to collect the string instead of Red code. I think, however, that as a beginner, you should stick to the parsing directly at the block level (as you did in your example), because it is much simpler and much expressive.

Another interesting (but very hairy) approach would be to redefine all the words used in your DSL block to work the way you want.

Resources

We have a wiki entry on Red / Rebol dialects on github that you can find if not useful, but interesting to read.

Besides the two articles ( this and this ) on the red blog, I think you are viewing the first one first (if not, you need to!).

Finally, but not least, an exhaustive overview of Parse principles and keywords (there are a couple of wrong parts in this, so let the buyer be vigilant). This is written for Rebol, but you should easily adapt the examples to Red.

As a relative newbie to this language, I agree that there are no examples and guidelines for developing DSL, but we are working on it (at least in our heads) :)

Answer 2

Taking the 9214 answer as a starting point, I encoded one possible solution. My approach: -

• try to keep the rules of the analysis as clean as possible.
• use collect and save to return the block as a result, instead of trying to build a more complex AST
• perform minimal translation in , keeping
• The resulting block must be valid. Red code
• which uses predefined functions where more complex processing should be performed.

Most simple statements are easily translated into functions, for example, WRITE MESSAGE TO LOG becomes SL_WriteMessageToLog, which can then do whatever it needs.

More complex structure statements, such as If / Else, become functions that accept block parameters, which can then process blocks as needed.

To complicate If / Else, I did this in two separate functions: SL_If and SL_Else. SL_If stores the result of the condition in a sequence, and SL_Else checks the last result and deletes it. This allows you to insert If / Elses.

The presence of the final endrule can be checked to ensure that the input has been correctly disassembled. Once this is removed, we must have the correct function definition.

Here's the code: -

 Red [ Purpose: example rules for parsing and translating a simple language ] ; some data Person.AGE: 0 Person.INCOME: 0 ; functions to implement some simple SL statements SL_WriteMessageToLog: function [value] [ print value ] SL_SetData: function [parmblock] [ field: parmblock/1 value: parmblock/2 if type? value = word! [ value: do value ] print ["old value" field "=" do field] set field value print ["new value" field "=" do field] ] ; hold the If condition results, to be used to determine whether or not to do Else IfConditionResults: [] SL_If: function [cond stats] [ cond_result: do cond head insert IfConditionResults cond_result if cond_result stats ] SL_Else: function [stats] [ cond_result: first IfConditionResults remove IfConditionResults if not cond_result stats ] ; parsing rules SimpleLanguageParser: make object! [ Expr: [logic! | string! | integer! | block!] Data: ['Person.AGE | 'Person.INCOME] WriteMessageToLog: ['write 'message 'to 'log set x Expr keep ('SL_WriteMessageToLog) keep (x)] SetData: ['set 'data set d Data '= set x Expr keep ('SL_SetData) keep (reduce [dx])] IfStatement: ['if keep ('SL_If) keep Expr collect [any Statement] opt ['else keep ('SL_Else) collect [any Statement]] 'endif] Statement: [WriteMessageToLog | SetData | IfStatement] Rule: [collect [ 'rule set fname word! keep (to set-word! fname) keep ('does) collect [any Statement] keep 'endrule ] ] AnySimpLeLanguage: [Rule | [any Statement]] ] SL: function [slInput] [ parse slInput SimpleLanguageParser/Rule ] 

For example, in the original message, the output is: -

 TooYoung: does [ SL_If [Person.Age < 15] [ SL_WriteMessageToLog "too young to earn an income" SL_SetData [Person.Income 0] ] SL_Else [ SL_WriteMessageToLog "old enough" ] ] ENDRULE 

Thanks for your help to get this far. Feedback on this approach and solution will be appreciated :)