First version of new post
Compiling Parfait, better mom, builtin etc
This commit is contained in:
parent
6e8864b238
commit
90977d1a7c
BIN
app/assets/images/1600_tests.png
Normal file
BIN
app/assets/images/1600_tests.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 17 KiB |
@ -0,0 +1,204 @@
|
||||
%p
|
||||
As we rubyx will eventually need to parse and compile itself, i am very happy to
|
||||
report success on the first steps on that journey. Also benchmarks, better design
|
||||
and another conference are on the list.
|
||||
|
||||
%h2 Compiling parfait
|
||||
%p
|
||||
As a recap, Parfait is that part of the core library that we need already during
|
||||
compilation. Ie the compiler creates Parfait objects during compilation and uses
|
||||
Parfait code to do this. This off course is a conundrum, which is solved by using the
|
||||
Parfait Code as is in the compiler (and some ruby module magic to avoid name clashes)
|
||||
%p
|
||||
Naturally any meaningful program that the compiler generates will use Parfait
|
||||
and so Parfait must be available at run-time, ie parsed and compiled. Since i have been
|
||||
busy doing the basics, this has been on the ToDo for a long while.
|
||||
%p
|
||||
Now, finally, most the basics are inplace and i have started what feels like a tremendous
|
||||
task. In fact i have succefully compiled
|
||||
%em three files.
|
||||
Object, DataObject and Integer, to be precise.
|
||||
%p
|
||||
The significance of this is actually much greater (especially since there are no tests)
|
||||
yet. Parfait, as part of rubyx, is what one may call ideomatic ruby, ie real world ruby.
|
||||
Off course i have to smoothen out a few bugs before compiling actually worked, but
|
||||
surprisingly little. In other words the compiler is functional enough to
|
||||
compile larger more feature ritch ruby programs, but more on that below.
|
||||
|
||||
%h2 Design improvements
|
||||
%p
|
||||
The overall design has been like in the picture below for a while already.
|
||||
Alas, the implementation oof this architecture was slightly lacking.
|
||||
To be precise, when mom code was generated, it was immediately converted to risc.
|
||||
In other words the layer only existed conceptually, or in transit.
|
||||
%p.center.three_width
|
||||
= image_tag "architecture.png" , alt: "Architectural layers"
|
||||
%p
|
||||
Now the code works
|
||||
%em exactly
|
||||
as advertised. Ruby comes in from the top and binary code out at the bottom.
|
||||
But more than that, every layer is a distinct step, in fact there are methods on the
|
||||
topmost compiler object to create every level down from ruby. This is obviously
|
||||
very handy for testing.
|
||||
|
||||
%h2 Automated binary tests
|
||||
%p
|
||||
Speaking of testing, we are at over 1600 tests, which is more than 200 up from before
|
||||
the design rewrite. At over 15000 assertions this is still 95%, in other words everything
|
||||
apart from a few fails. And with parallel execution still fast.
|
||||
|
||||
%p.center.full_width
|
||||
= image_tag "1600_tests.png" , alt: "Lots of test, never boring"
|
||||
%p
|
||||
But the main achievement a couple of weeks ago was the integration of binary testing
|
||||
int the automated test flow. Specifically on
|
||||
%em Travis.
|
||||
%p
|
||||
This uses a feature of Qemu that i had not know before, namely that one can get qemu
|
||||
to run binaries from a different target on a machine, by simply calling it with
|
||||
qemu-arm.
|
||||
%p
|
||||
I had done previous testing of binaries via ssh, usually to an qemu emulated pi on my
|
||||
machine. This setup is vastly more complicated, as described
|
||||
=ext_link "here" , "/arm/qemu.html"
|
||||
and i had shied away from that. Meaning they would happen irregularily and all that.
|
||||
My only consolation was that the test would run on the interpreter, but off course that
|
||||
does not test the arm and elf genertion.
|
||||
%p
|
||||
The actual tests that i am talking about are a growing number of "mains" tests, found in
|
||||
the tets/mains directory.
|
||||
These are actal programs that calculate or output stuff. They are complete system
|
||||
tests in the sense that we only test their output (system output).
|
||||
%p
|
||||
As we usually link to "a.out" files (thus overwriting and avoiding cleanup), the actual
|
||||
invocation of qemu for a binary is really simple:
|
||||
%pre
|
||||
%code
|
||||
qemnu-arm ./a.out
|
||||
but that still leaves you to generate that binary. This can be done by using the
|
||||
rubyxc compiler and linking the resulting object file (see bug #13). But sine i too am a
|
||||
lazy programmer i have automated these steps into the rubyxc compiler, and so one
|
||||
can just compile/link/execute a source file like this:
|
||||
%pre
|
||||
%code
|
||||
:preserve
|
||||
./bin/rubyxc execute test/mains/source/fibo__8.rb
|
||||
This will compile, link and execute this specific fibonachi test. This output
|
||||
of this test will be 8, as encoded in the file name.
|
||||
So this and 20 others will be tested as binaries now every time travid does its thing.
|
||||
%p
|
||||
BTW, i have also created arubyxc command to execute a file via the interpreter.
|
||||
This can sometimes yield better errors when things go wrong.
|
||||
%pre
|
||||
%code
|
||||
:preserve
|
||||
./bin/rubyxc interpret test/mains/source/puts_Hello-there_11.rb
|
||||
And as a second btw, i also added an option to the compiler, so one can control the
|
||||
Parfait factory size with the option --parfait.
|
||||
|
||||
|
||||
%h2 Misc other news
|
||||
%h3 Microbenchmarks
|
||||
%p
|
||||
At the last conference in Hamburg, someone asked the fair question: So how fast is it?
|
||||
It's been so long that i did tests, that i could only mumble.
|
||||
Now i finished updating the tests, but it will be a while before i can answer the
|
||||
question more fully.
|
||||
%p
|
||||
So for starters, because the functionality of the compiler is limited, i did very small
|
||||
benchmarks. Very small means 20 lines or less, loops, string output, fibonacchi, both
|
||||
linear and recursive. I realized too late, that that will tell most about integer
|
||||
performance.
|
||||
%p
|
||||
Now because of the early days, i will not go into detail here. In general speed was not
|
||||
as fast as i had hoped from by 4 year old benchmarks, about the same as mri. I
|
||||
will have to do some work on the calling convention and probably some on integer
|
||||
handling too. I think i can quite easily shave 30-50% off, and that alone should
|
||||
verify the saying that all benchmarks are lies. Like the one where rubyx is doing
|
||||
"hello world" faster than C. Yes, C, not mri. But only because i switched the buffering
|
||||
off, because also rubyx does not buffer (apples and oranges ...)
|
||||
%p
|
||||
So i will take this round as inspiration to do some optimisation and performance
|
||||
measuring. And come back to it later.
|
||||
|
||||
%h3 Implicit returns
|
||||
%p
|
||||
As part of parsing Parfait, i implemented a first version of implicit returns.
|
||||
Low hanging fruits, and in fact most common use cases, included constants and calls.
|
||||
So when a method ends in a simple variable, constant, or a call, a return will be added.
|
||||
More complex rules like returns for if's or while will ave to wait, but i found that i
|
||||
personally don't tend to use them anyway.
|
||||
%p
|
||||
Since class methods are basically methods (of the meta class), adding the unified
|
||||
return handling to them was easy too.
|
||||
|
||||
%h3 Improved Block handling
|
||||
%p
|
||||
Block handling, at least the simple implicit kind, has worked for a while, but was in
|
||||
several ways too complicated. The block was unneccessarily assigned to a local, and
|
||||
compiling was handled by picking them out.
|
||||
%p
|
||||
This all stemmed from a misunderstanding, or lack of understanding: Blocks, or should
|
||||
i say Lambdas, are constants. Just like a string or integer. They are created once at
|
||||
compile time and can not change identity. In fact Methods and Classes are also contants,
|
||||
and i reflected this in the Vool level by calling them Expressions, instead of
|
||||
before Statements.
|
||||
%p
|
||||
So now the Lambda Expression is created and just added as an argument to the send.
|
||||
Compiling thee Lambda is triggered by the constant creation, ie the step down from
|
||||
vool to mom, and the block compiler added to method compiler automatically.
|
||||
|
||||
%h3 Vool coming into focus
|
||||
%p
|
||||
I've been saying ruby without the fluff, to descibe vool. And while that is true,
|
||||
it is quite vague. Two major things have become clear about vool through the work above.
|
||||
%p
|
||||
Firstly, Vool has no complex or recursive send statements. Arguments must be variables or
|
||||
constants. Calls are executed before and assigned to a temporary variable. In effect
|
||||
recursive calls are flattened into a list, and as such the calling does not rely on a
|
||||
stack as in ruby.
|
||||
%p
|
||||
Secondly, Vool distinguishes between expressions and statements. Like other lower level,
|
||||
but not ruby. As a rule of thumb, Statements do things, Expression are things. In other
|
||||
words, only expressions have value, statements (lke if or while) do not.
|
||||
|
||||
%h2 Plans
|
||||
%h4 GrillRB conference
|
||||
%p
|
||||
I will speak in
|
||||
=ext_link "Wrazlaw" , "https://grillrb.com/"
|
||||
in about a week. The plan is to make a comparison with rails and focus on the
|
||||
possibilities, rather than technical detail.
|
||||
|
||||
%h4 Calling
|
||||
%p
|
||||
The Calling can do with work and i noticed two mistakes i did. One is that creating
|
||||
a new message for every call is unneccessarily complicated. Its is only in the
|
||||
special case that a Proc is created that the return sequence (a mom instruciton) needs
|
||||
to keep the message alive.
|
||||
%h4 Integers
|
||||
%p
|
||||
I still want to hang on to Integers being objects, though creation is clealy costly.
|
||||
In the future a full escape analysis will help off course, but for now it should be easy
|
||||
enough to figure out wether an int is passed down. If not loops can be
|
||||
destructively change the int. A simple special case is a the times method.
|
||||
|
||||
%h4 Mom instruction invocation
|
||||
%p
|
||||
I have this idea of being able to code more stuff higher up. To make that more
|
||||
efficient i am thinking of macros or instruction invocation at the vool level.
|
||||
Only inside Parfait off course. The basic idea would be to save the call/return
|
||||
code, and have eg X.return_jump map to the Mom::ReturnJump Instruction. "Just" have
|
||||
to figure out the passing semantics, or how that integrates intot the vools code.
|
||||
%h4 Better Builtin
|
||||
%p
|
||||
The generation of the current builtin methods has always bothered me a bit.
|
||||
It is true that some things just can not be expressed as ruby and so some
|
||||
alternative mechanism is needed (even in c one can embed assembler).
|
||||
%p
|
||||
The main problem i have is that those methods don't check their arguments and as such
|
||||
may cause core dumps. So they are to high level and hopefully all we really need is
|
||||
that previous idea of being able to integrate Mom code into vool. As Mom is extensible
|
||||
that should take care of any possible need. And we could code the methods normally as
|
||||
part of Parfait, make them safe, and just use the lower level inside them. Lets see!.
|
||||
|
||||
Loading…
Reference in New Issue
Block a user