diff --git a/app/assets/images/1600_tests.png b/app/assets/images/1600_tests.png new file mode 100644 index 0000000..9cc329e Binary files /dev/null and b/app/assets/images/1600_tests.png differ diff --git a/app/views/posts/_2019-08-20-parsing-parfait-and-other-goodies.haml b/app/views/posts/_2019-08-20-parsing-parfait-and-other-goodies.haml new file mode 100644 index 0000000..8c06777 --- /dev/null +++ b/app/views/posts/_2019-08-20-parsing-parfait-and-other-goodies.haml @@ -0,0 +1,204 @@ +%p + As we rubyx will eventually need to parse and compile itself, i am very happy to + report success on the first steps on that journey. Also benchmarks, better design + and another conference are on the list. + +%h2 Compiling parfait +%p + As a recap, Parfait is that part of the core library that we need already during + compilation. Ie the compiler creates Parfait objects during compilation and uses + Parfait code to do this. This off course is a conundrum, which is solved by using the + Parfait Code as is in the compiler (and some ruby module magic to avoid name clashes) +%p + Naturally any meaningful program that the compiler generates will use Parfait + and so Parfait must be available at run-time, ie parsed and compiled. Since i have been + busy doing the basics, this has been on the ToDo for a long while. +%p + Now, finally, most the basics are inplace and i have started what feels like a tremendous + task. In fact i have succefully compiled + %em three files. + Object, DataObject and Integer, to be precise. +%p + The significance of this is actually much greater (especially since there are no tests) + yet. Parfait, as part of rubyx, is what one may call ideomatic ruby, ie real world ruby. + Off course i have to smoothen out a few bugs before compiling actually worked, but + surprisingly little. In other words the compiler is functional enough to + compile larger more feature ritch ruby programs, but more on that below. + +%h2 Design improvements +%p + The overall design has been like in the picture below for a while already. + Alas, the implementation oof this architecture was slightly lacking. + To be precise, when mom code was generated, it was immediately converted to risc. + In other words the layer only existed conceptually, or in transit. +%p.center.three_width + = image_tag "architecture.png" , alt: "Architectural layers" +%p + Now the code works + %em exactly + as advertised. Ruby comes in from the top and binary code out at the bottom. + But more than that, every layer is a distinct step, in fact there are methods on the + topmost compiler object to create every level down from ruby. This is obviously + very handy for testing. + +%h2 Automated binary tests +%p + Speaking of testing, we are at over 1600 tests, which is more than 200 up from before + the design rewrite. At over 15000 assertions this is still 95%, in other words everything + apart from a few fails. And with parallel execution still fast. + +%p.center.full_width + = image_tag "1600_tests.png" , alt: "Lots of test, never boring" +%p + But the main achievement a couple of weeks ago was the integration of binary testing + int the automated test flow. Specifically on + %em Travis. +%p + This uses a feature of Qemu that i had not know before, namely that one can get qemu + to run binaries from a different target on a machine, by simply calling it with + qemu-arm. +%p + I had done previous testing of binaries via ssh, usually to an qemu emulated pi on my + machine. This setup is vastly more complicated, as described + =ext_link "here" , "/arm/qemu.html" + and i had shied away from that. Meaning they would happen irregularily and all that. + My only consolation was that the test would run on the interpreter, but off course that + does not test the arm and elf genertion. +%p + The actual tests that i am talking about are a growing number of "mains" tests, found in + the tets/mains directory. + These are actal programs that calculate or output stuff. They are complete system + tests in the sense that we only test their output (system output). +%p + As we usually link to "a.out" files (thus overwriting and avoiding cleanup), the actual + invocation of qemu for a binary is really simple: + %pre + %code + qemnu-arm ./a.out + but that still leaves you to generate that binary. This can be done by using the + rubyxc compiler and linking the resulting object file (see bug #13). But sine i too am a + lazy programmer i have automated these steps into the rubyxc compiler, and so one + can just compile/link/execute a source file like this: + %pre + %code + :preserve + ./bin/rubyxc execute test/mains/source/fibo__8.rb + This will compile, link and execute this specific fibonachi test. This output + of this test will be 8, as encoded in the file name. + So this and 20 others will be tested as binaries now every time travid does its thing. +%p + BTW, i have also created arubyxc command to execute a file via the interpreter. + This can sometimes yield better errors when things go wrong. + %pre + %code + :preserve + ./bin/rubyxc interpret test/mains/source/puts_Hello-there_11.rb + And as a second btw, i also added an option to the compiler, so one can control the + Parfait factory size with the option --parfait. + + +%h2 Misc other news +%h3 Microbenchmarks +%p + At the last conference in Hamburg, someone asked the fair question: So how fast is it? + It's been so long that i did tests, that i could only mumble. + Now i finished updating the tests, but it will be a while before i can answer the + question more fully. +%p + So for starters, because the functionality of the compiler is limited, i did very small + benchmarks. Very small means 20 lines or less, loops, string output, fibonacchi, both + linear and recursive. I realized too late, that that will tell most about integer + performance. +%p + Now because of the early days, i will not go into detail here. In general speed was not + as fast as i had hoped from by 4 year old benchmarks, about the same as mri. I + will have to do some work on the calling convention and probably some on integer + handling too. I think i can quite easily shave 30-50% off, and that alone should + verify the saying that all benchmarks are lies. Like the one where rubyx is doing + "hello world" faster than C. Yes, C, not mri. But only because i switched the buffering + off, because also rubyx does not buffer (apples and oranges ...) +%p + So i will take this round as inspiration to do some optimisation and performance + measuring. And come back to it later. + +%h3 Implicit returns +%p + As part of parsing Parfait, i implemented a first version of implicit returns. + Low hanging fruits, and in fact most common use cases, included constants and calls. + So when a method ends in a simple variable, constant, or a call, a return will be added. + More complex rules like returns for if's or while will ave to wait, but i found that i + personally don't tend to use them anyway. +%p + Since class methods are basically methods (of the meta class), adding the unified + return handling to them was easy too. + +%h3 Improved Block handling +%p + Block handling, at least the simple implicit kind, has worked for a while, but was in + several ways too complicated. The block was unneccessarily assigned to a local, and + compiling was handled by picking them out. +%p + This all stemmed from a misunderstanding, or lack of understanding: Blocks, or should + i say Lambdas, are constants. Just like a string or integer. They are created once at + compile time and can not change identity. In fact Methods and Classes are also contants, + and i reflected this in the Vool level by calling them Expressions, instead of + before Statements. +%p + So now the Lambda Expression is created and just added as an argument to the send. + Compiling thee Lambda is triggered by the constant creation, ie the step down from + vool to mom, and the block compiler added to method compiler automatically. + +%h3 Vool coming into focus +%p + I've been saying ruby without the fluff, to descibe vool. And while that is true, + it is quite vague. Two major things have become clear about vool through the work above. +%p + Firstly, Vool has no complex or recursive send statements. Arguments must be variables or + constants. Calls are executed before and assigned to a temporary variable. In effect + recursive calls are flattened into a list, and as such the calling does not rely on a + stack as in ruby. +%p + Secondly, Vool distinguishes between expressions and statements. Like other lower level, + but not ruby. As a rule of thumb, Statements do things, Expression are things. In other + words, only expressions have value, statements (lke if or while) do not. + +%h2 Plans +%h4 GrillRB conference +%p + I will speak in + =ext_link "Wrazlaw" , "https://grillrb.com/" + in about a week. The plan is to make a comparison with rails and focus on the + possibilities, rather than technical detail. + +%h4 Calling +%p + The Calling can do with work and i noticed two mistakes i did. One is that creating + a new message for every call is unneccessarily complicated. Its is only in the + special case that a Proc is created that the return sequence (a mom instruciton) needs + to keep the message alive. +%h4 Integers +%p + I still want to hang on to Integers being objects, though creation is clealy costly. + In the future a full escape analysis will help off course, but for now it should be easy + enough to figure out wether an int is passed down. If not loops can be + destructively change the int. A simple special case is a the times method. + +%h4 Mom instruction invocation +%p + I have this idea of being able to code more stuff higher up. To make that more + efficient i am thinking of macros or instruction invocation at the vool level. + Only inside Parfait off course. The basic idea would be to save the call/return + code, and have eg X.return_jump map to the Mom::ReturnJump Instruction. "Just" have + to figure out the passing semantics, or how that integrates intot the vools code. +%h4 Better Builtin +%p + The generation of the current builtin methods has always bothered me a bit. + It is true that some things just can not be expressed as ruby and so some + alternative mechanism is needed (even in c one can embed assembler). +%p + The main problem i have is that those methods don't check their arguments and as such + may cause core dumps. So they are to high level and hopefully all we really need is + that previous idea of being able to integrate Mom code into vool. As Mom is extensible + that should take care of any possible need. And we could code the methods normally as + part of Parfait, make them safe, and just use the lower level inside them. Lets see!. +