RakuAST Grant Report

Summary

The RakuAST project was a rewrite and redesign of the compiler frontend, i.e. the part that parses the original source code, figures out what the different parts are, does checks and optimizations and creates a low level representation which will then be turned into bytecode by the backend. When I applied for the grant a lot of basic infrastructure was already in place. Many simple programs would already run, e.g. you could define variables, classes, functions, create objects, call methods and a lot more. However Raku is a large language. E.g. there's not just methods. There are also private methods, meta methods, methods with fully qualified names (i.e. when you want a method of a specific base class), method calls where the name is determined at compile time (e.g. $foo."$bar"()), hypered method calls and even more obscure ways of calling code. These were all still left to do. The same was true for all other areas of the compiler. My method for attacking this was simple: one spec test at a time. I went through the list of failing spec test files in lexicographical order. The spec tests are grouped by synopses which themselves are numbered vaguely in order of how fundamental they are. E.g. at the beginning there are tests for names and handling of types whereas latter spectests deal with specifics of the standard library. This order helped keeping my focus on a specific area. At least up to a point. As the tests are themselves Raku code, even tests for something very fundamental may use advanced syntax features simply because they are a convenient way to express the test. Thus I often first had to support those before I could tackle the more basic features. If I had to name the single hardest part of the project, it's certainly timing or sequencing. Raku is not an easy language to compile. A lot of code is already run during compilation. E.g. in class Foo does Bar { }, we call the Bar role's body which is really just a sub in disguise and we do so during compilation...