Chapter 11: Syntax Macros
Where science turns into magic once more.
You've now learned how to create your own macros to make your own custom syntax, and the features involved.
I would advice you to take a look at the many macros in the Lux Standard Library for inspiration as to what can be accomplished.
In the meantime, let's find out how to take our macro chops to the next level.
The lux/macro/syntax module houses some powerful tools.
For starters, it's the home of the Syntax type:
(type: (Syntax a)
(-> (lux;List lux;AST) (lux/data/error;Error [(lux;List lux;AST) a])))
Note: This is also a functorial/monadic type.
Syntax is the type of syntax-parsers: parsers which analyze AST nodes to extract arbitrary information.
The Syntax type works with streams of inputs instead of single elements, and it often consumes some of those inputs, which is why the output involves an updated list of ASTs.
There are many such syntax-parsers (and combinators) in the lux/macro/syntax module, and you should definitely take a look at what's available in the documentation.
But in that module there is also another mechanism for defining macros: the syntax: macro.
## A more advanced way to define macros than macro:.
## The inputs to the macro can be parsed in complex ways through the use of syntax parsers.
## The macro body is also (implicitly) run in the Lux/Monad, to save some typing.
## Also, the compiler state can be accessed through the *state* binding.
(syntax: #export (object [#let [imports (class-imports *state*)]]
[#let [class-vars (list)]]
[super (s;opt (super-class-decl^ imports class-vars))]
[interfaces (s;tuple (s;some (super-class-decl^ imports class-vars)))]
[constructor-args (constructor-args^ imports class-vars)]
[methods (s;some (overriden-method-def^ imports))])
(let [def-code ($_ Text/append-class:"
(spaced (list (super-class-decl$ (default object-super-class super))
(with-brackets (spaced (map super-class-decl$ interfaces)))
(with-brackets (spaced (map constructor-arg$ constructor-args)))
(with-brackets (spaced (map (method-def$ id) methods))))))]
(wrap (list (` (;_lux_proc ["jvm" (~ (ast;text def-code))] []))))))
This sample is a macro for making anonymous classes that lives in
lux/host.
The difference between macro: and syntax: is that syntax: allows you to parse, in a structured manner, the inputs to your macro, thereby reducing considerably the complexity necessary for making "big" macros.
Also, because you're using syntax-parsers for the hard work, you can write reusable parsers that you can share throughout your macros, if you want to have common syntax. You can compose your parsers, or use parsers from someone else's library.
There is already a small module called
lux/macro/syntax/commonwhich houses a few reusable parsers and generators.It will grow over time, as more syntax becomes standardized in the Lux Standard Library and it makes more sense to re-use resources.
Additionally, syntax: binds the Compiler value on a variable called *state*, so you can use it during your parsing.
What do those syntax-parsers look like?
Here are some samples:
(def: (type-param^ imports)
(-> ClassImports (Syntax TypeParam))
(s;either (do Monad<Syntax>
[param-name s;local-symbol]
(wrap [param-name (list)]))
(tuple^ (do Monad<Syntax>
[param-name s;local-symbol
_ (s;symbol! ["" "<"])
bounds (s;many (generic-type^ imports (list)))]
(wrap [param-name bounds])))))
(def: body^
(Syntax (List AST))
(s;tuple (s;many s;any)))
(type: #rec Infix
(#Const AST)
(#Call (List AST))
(#Infix Infix AST Infix))
(def: (infix^ _)
(-> Unit (Syntax Infix))
($_ s;alt
($_ s;either
(Syntax/map ast;bool s;bool)
(Syntax/map ast;int s;int)
(Syntax/map ast;real s;real)
(Syntax/map ast;char s;char)
(Syntax/map ast;text s;text)
(Syntax/map ast;symbol s;symbol)
(Syntax/map ast;tag s;tag))
(s;form (s;many s;any))
(s;tuple (s;either (do Monad<Syntax>
[_ (s;tag! ["" "and"])
init-subject (infix^ [])
init-op s;any
init-param (infix^ [])
steps (s;some (s;seq s;any (infix^ [])))]
(wrap (product;right (fold (lambda [[op param] [subject [_subject _op _param]]]
[param [(#Infix _subject _op _param)
(` and)
(#Infix subject op param)]])
[init-param [init-subject init-op init-param]]
steps))))
(do Monad<Syntax>
[_ (wrap [])
init-subject (infix^ [])
init-op s;any
init-param (infix^ [])
steps (s;some (s;seq s;any (infix^ [])))]
(wrap (fold (lambda [[op param] [_subject _op _param]]
[(#Infix _subject _op _param) op param])
[init-subject init-op init-param]
steps)))
))
))
In all of these cases,
sis being used as an alias forlux/macro/syntax.
And here are some samples of syntax macros:
(syntax: #export (^stream& [patterns (s;form (s;many s;any))] body [branches (s;some s;any)])
{#;doc (doc "Allows destructuring of streams in pattern-matching expressions."
"Caveat emptor: Only use it for destructuring, and not for testing values within the streams."
(let [(^stream& x y z _tail) (some-stream-func 1 2 3)]
(func x y z)))}
(with-gensyms [g!s]
(let [body+ (` (let [(~@ (List/join (List/map (lambda [pattern]
(list (` [(~ pattern) (~ g!s)])
(` (cont;run (~ g!s)))))
patterns)))]
(~ body)))]
(wrap (list& g!s body+ branches)))))
(syntax: #export (?> [branches (s;many (s;seq body^ body^))]
[?else (s;opt body^)]
prev)
{#;doc (doc "Branching for pipes."
"Both the tests and the bodies are piped-code, and must be given inside a tuple."
"If a last else-pipe isn't given, the piped-argument will be used instead."
(|> 5
(?> [even?] [(* 2)]
[odd?] [(* 3)]
[(_> -1)])))}
(with-gensyms [g!temp]
(wrap (list (` (let% [(~ g!temp) (~ prev)]
(cond (~@ (do Monad<List>
[[test then] branches]
(list (` (|> (~ g!temp) (~@ test)))
(` (|> (~ g!temp) (~@ then))))))
(~ (case ?else
(#;Some else)
(` (|> (~ g!temp) (~@ else)))
_
g!temp)))))))))
(syntax: (@runnable expr)
(wrap (list (`' (object [java.lang.Runnable]
[]
(java.lang.Runnable run [] void
(exec (~ expr)
[])))))))
By the way, the body of
syntax:runs inside a(do Monad<Lux> [] ...)expression, so you have immediate access towrapfor simple macros, like the last one.
This may be a short chapter, but not because it's subject is small.
The opportunities that syntax-parsers open are fantastic, as it puts within your reach macros which would otherwise be much harder to implement correctly.
Don't worry about complex inputs: your macros can implement entire new embedded programming languages if you want them to. Syntax-parsers can generate any data-type you want, so you can easily translate the information in the input syntax to whatever data-model you need.
But, now that we've spent 3 chapters about metaprogramming in Lux, I think it's fair that we clear our minds a little by looking at other subjects.
You're going to learn how to go beyond Lux and interact with everything and everyone.
See you in the next chapter!