ruby-x/ruby-x.github.io
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

move posts in directory by year

Browse Source
This commit is contained in:
Torsten Rüger
2019-12-07 11:30:52 +02:00
parent 5ce2d1b625
commit 285c6531e4
38 changed files with 8 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files

128
app/views/posts/2019/_02-28-class-methods-and-a-conference.haml Normal file
View File

@ -0,0 +1,128 @@
%p
In short, class methods are working. Plus is some meta stuff that and gotchas that
i will explain below. Also i was at presenting rubyx at the first international
conference, and there are a couple of misc improvements.
%h2 Classes are Singleton objects
%p
Class methods came up when parsing Parfait. To create objects we usually use new, a
class method. After briefly contemplating to hack that and use the global space
object, i decided to at least check it out. And since it wasn't so bad, i continued.
%p
At some point i thought class methods would be very simple, as they are just methods on
another object. After all the way we write code in classes is just a way to define methods
on a group of objects. And defining class methods "just" changes which objects we
define the methods on. And for the most part, this view is true. The only main difference
is that class objects are singletons.
%h2 Singletons means singleton classes
%p
Since we want to define methods on the single object, (and we are not programming
javascript) we need a class like object that allows us to define these methods and
manages them. There are several ways to do this, and i opted for a rather explicit
one for now.
%p
So i created a class called MetaClass, that is more or less like a Class. It does
currently not derive, just implements the same protocol. In the compiler it can
be used as a replacement for a class object, and that is exactly what we do.
A metaclass instance is created for every class instance and there is a one to one
relation with the class object.
%p
Ruby does this in a more general way, and that off course means that at some point
rubyx will have to move there too. In ruby every object has a singleton (meta) class.
This must be implemented with lasy creation, otherwise it seems very wasteful.
In Ruby the Metaclass is just a class, which means it has a metaclass, and
at the moment i have other problems than keeping my brain from melting by thinking
in circles like that.
%h2 Compiling class methods
%p
Actual compilation now becomes very simple. Before we were passing the class in
when compiling a method. Now we have methods and class methods, and depending on
which kind it is, we use a different object. The class object for method, and the classes
metaclass object for class methods.
%p
Sort of surprisingly, none of the code further down had to be changed. But there are
no tests yet for all the possible features, so there may be surprises. Also interpreting
a class method call worked straight away, no changes needed there.
%p
Class instance variables also have no tests, and so i don't know if they work or how
much work that would be. I sort of hope not much, but there clearly is an issue with
the types changing that will have to be addressed.
%h2 Change, specifically changing types
%p
As Types never change, we always create new instances. This works for normal objects, as
we can just use the new type for new objects. Also while compiling a binary (ie not on
the fly yet), we can just use the latest type for all existing objects of a class
when we create the binary.
%p
For singletons this hack does not work. Especially when adding class instance variables,
the actual type of the existing class object changes. This means we have to change
the type reference in the class object. This in turn means the new type has to be
binary compatible with the old, which basically means it can't grow beyond the
allocated size.
%p
This has been on the horizon for objects since the whole type idea was implemented.
Just the horizon was always far away, and now i have to see how much of this can
be done easily, how much later. We'll see, tests will tell.
%h2 Misc other
%p
Other things have happened that are maybe worth mentioning.
%h3 Ruby integer upgrade
%p
Since i started before 2.4 there were still uses of Fixnum. In 2.4 Integer and Fixnum
were united and later versions created a whole ton of warning. So i had stuck with 2.3
until now.
%p
Now all Fixnums are gone and i have upgraded to use 2.6.
%h3 Object creation
%p
When doing the Factories i hardcoded some page sizes. Hardcoding always bites you, as
they should teach everyone in kindergarten, and so it did. It massively increased
the test times, as for every Parfait boot (almost every test) several pages worth
of objects were created.
%p
I have now made the page size configurable and for good measure am passing some
config into the compiler. The compiler then passes a parfait key down, a mechanism
that could easily be extended to other subsystems.
%p
The result was that test times have more than halved. Which means my new machine
(the one i am just choosing) does not have to be quite as expensive :-)
%h3 Command Line Interface
%p
Since i am hoping that people will start playing with ruby-x at some point i thought
to make that easier. This has been out there for a while as well, but since the set
of features was so small it just didn't get done before.
%p
So now we have a thor based cli in the bin directory. Not that it does so terribly much
yet, but at least you can invoke it, get some help, and compile a ruby to binary.
Platforms are not supported (as we only have arm) and those Parfait options neither.
But a start.
%h3 Rubyconf talk
.container
%p.full_width
=image_tag "rubyconf.jpg"
%p
Last but not least, i had the
=ext_link "first talk" , "https://twitter.com/_swanand/status/1087279289672785920"
at a conference. At the RubyConf in India, Goa
to be specific. Many people were really interested, and Sherif and the organisers we very
friendly. Off course we had a nice holiday too. No video yet, but i put the slides up on
the homepage.
%h2 Work continues
%p
When starting two big projects at work last year i didn't have so much time, and there
were months when i had no juice to code. Now i am on part time (physical) work again
and am back chipping away at it.
%p
I am still looking for people to join and will increase the effort a bit now that
at least a reasonable subset is working.
%p
Welcome!

223
app/views/posts/2019/_08-20-parsing-parfait-and-other-goodies.haml Normal file
View File

@ -0,0 +1,223 @@
%p
As rubyx will eventually need to parse and compile itself, i am very happy to
report success on the first steps towards that goal. Also better design, benchmarks,
and another conference are on the list.
%h2 Compiling parfait
%p
As a recap, Parfait is that part of the core library that we need already during
compilation. Ie the compiler creates Parfait objects during compilation and uses
Parfait code to do this. This off course is a conundrum, which is solved by using the
Parfait Code as is in the compiler (and some ruby module magic to avoid name clashes)
%p
Naturally any meaningful program that the compiler generates will use Parfait
and so Parfait must be available at run-time, ie parsed and compiled. Since i have been
busy doing the basics, this has been on the ToDo for a long while.
%p
Now, finally, most the basics are in place and i have started what feels like a tremendous
task. In fact i have successfully compiled
%em three files.
Object, DataObject and Integer, to be precise.
%p
The significance of this is actually much greater (especially since there are no tests
yet). Parfait, as part of rubyx, is what one may call ideomatic ruby, ie real world ruby.
Off course i had to smoothen out a few bugs before compiling actually worked, but
surprisingly little. In other words the compiler is functional enough to
compile larger, more feature rich ruby programs, but more on that below.
%h2 Design improvements
%p
The overall design has been like in the picture below for a while already.
Alas, the implementation of this architecture was slightly lacking.
To be precise, when mom code was generated, it was immediately converted to risc.
In other words the layer existed only conceptually, or in transit.
%p.center.three_width
= image_tag "architecture.png" , alt: "Architectural layers"
%p
Now the code works
%em exactly
as advertised. Ruby comes in from the top and binary code out at the bottom.
But more than that, every layer is a distinct step, in fact there are methods on the
topmost compiler object to create every level down from ruby. This is obviously
very handy for testing, and by iself sped up testing by 30%, as risc is not renerated
before really needed.
%h2 Automated binary tests
%p
Speaking of testing, we are at over 1600 tests, which is more than 200 up from before
the design rewrite. At over 15000 assertions this is still 95% of the code, in other
words everything apart from a few fails. And with parallel execution still fast.
%p.center.full_width
= image_tag "1600_tests.png" , alt: "Lots of test, never boring"
%p
But the main achievement a couple of weeks ago was the integration of binary testing
into the automated test flow. Specifically on
%em Travis.
%p
This uses a feature of Qemu that i had not know before, namely that one can get qemu
to run binaries from a different target on a machine, by simply calling it with
qemu-arm.
%p
Previously i had done testing of binaries via ssh, usually to an qemu emulated pi on my
machine. This setup is vastly more complicated, as described
=ext_link "here" , "/arm/qemu.html"
and i had shied away from automating that. Meaning they would happen irregularily and
all that. My only consolation was that the test would run on the interpreter, but off
course that does not test the arm and elf genertion.
%p
The actual tests that i am talking about are a growing number of "mains" tests, found in
the tests/mains directory.
These are actual programs that calculate or output stuff. They are complete system
tests in the sense that we only test their output (system output and exit code).
%p
As we usually link to "a.out" files (thus overwriting and avoiding cleanup), the actual
invocation of qemu for a binary is really simple:
%pre
%code
qemu-arm ./a.out
but that still leaves you to generate that binary. This can be done by using the
rubyxc compiler and linking the resulting object file (see bug #13). But sine i too am a
lazy programmer i have automated these steps into the rubyxc compiler, and so one
can just compile/link/execute a source file like this:
%pre
%code
:preserve
./bin/rubyxc execute test/mains/source/fibo__8.rb
This will compile, link and execute this specific fibonacci test. The exit code
of this test will be 8, as encoded in the file name.
Now this test, and 20 others, will be run as binaries every time travis does its thing.
%p
BTW, i have also created a rubyxc command to execute a file via the interpreter.
This can sometimes yield better errors when things go wrong.
%pre
%code
:preserve
./bin/rubyxc interpret test/mains/source/puts_Hello-there_11.rb
And as a second btw, i also added an option to the compiler, so one can control the
Parfait factory size with the option --parfait.
%h2 Misc other news
%h3 Microbenchmarks
%p
At the last conference in Hamburg, someone asked the fair question: So how fast is it?
It's been so long that i did
=ext_link "tests," , "/misc/soml_benchmarks.html"
that i could only mumble.
Now i finished updating the tests, but it will be a while before i can answer the
question more fully.
%p
So for starters, because the functionality of the compiler is limited, i did very small
benchmarks. Very small means 20 lines or less, loops, string output, fibonacchi, both
linear and recursive. I realized too late, that all that will tell about is integer
performance.
%p
Now because it's early days, i will not go into detail here. In general speed was not
as fast as i had hoped for, about the same as mri. I
will have to do some work on the calling convention and probably some on integer
handling too. I think i can quite easily shave 30-50% off, and that alone should
verify the saying that all benchmarks are lies. Like the one where rubyx is doing
"hello world" faster than C. Yes, C, not mri. But only because i switched the buffering
off, because also rubyx does not buffer (apples and oranges ...)
%p
So i will take this round as inspiration to do some optimisation and performance
measuring. And come back to it later.
%h3 Implicit returns
%p
As part of parsing Parfait, i implemented a first version of implicit returns.
Low hanging fruits, and in fact most common use cases, included constants and calls.
So when a method ends in a simple variable, constant, or a call, a return will be added.
More complex rules like returns for if's or while will have to wait, but i found that i
personally don't tend to use them anyway.
%p
Since class methods are basically methods (of the meta class), adding the unified
return handling to them was easy too.
%h3 Improved Block handling
%p
Block handling, at least the simple implicit kind, has worked for a while, but was in
several ways too complicated. The block was unnecessarily assigned to a local, and
compiling was handled by looking for blocks statements, not during the normal flow.
%p
This all stemmed from a misunderstanding, or lack of understanding: Blocks, or should
i say Lambdas, are constants. Just like a string or integer. They are created once at
compile time and can not change identity. In fact Methods and Classes are also contants,
and i reflected this in the Vool level by calling them Expressions, instead of
before Statements.
%p
So now the Lambda Expression is created and just added as an argument to the send.
Compiling the Lambda is triggered by the constant creation, ie the step down from
vool to mom, and the block compiler added to method compiler automatically.
%h3 Vool coming into focus
%p
I've been saying ruby without the fluff, to descibe vool. And while that is true,
it is quite vague. Two major things have become clear about vool through the work above.
%p
Firstly, Vool has no complex or recursive send statements. Arguments must be variables or
constants. Calls are executed before and assigned to a temporary variable. In effect
recursive calls are flattened into a list, and as such the calling does not rely on a
stack as in ruby.
%p
Secondly, Vool distinguishes between expressions and statements. Like other lower level
languages, but not ruby. As a rule of thumb, Statements do things, Expression are things.
In other words, only expressions have value, statements (like if or while) do not.
%h2 Plans
%h4 Calling
%p
The Calling can do with work and i noticed two mistakes i did. One is that creating
a new message for every call is unneccessarily complicated. It is only in the
special case that a Proc is created that the return sequence (a mom instruction) needs
to keep the message alive.
%p
The other is that having arguments and local variables as seperate arrays may be handy
and easy to code. But it does add an extra indirection for every access _and_ store.
Since Mom is memory based, and Mom translates to risc, that does amount to a lot of
instructions.
%h4 Integers
%p
I still want to hang on to Integers being objects, though creation is clealy costly.
In the future a full escape analysis will help off course, but for now it should be easy
enough to figure out wether an int is passed down. If not loops can be made to
destructively change the int.
%h4 Mom instruction invocation
%p
I have this idea of being able to code more stuff higher up. To make that more
efficient i am thinking of macros or instruction invocation at the vool level.
Only inside Parfait off course. The basic idea would be to save the call/return
code, and have the compiler map eg X.return_jump to the Mom::ReturnJump Instruction. "Just" have
to figure out the passing semantics, or how that integrates into the vool code.
%h4 Better Builtin
%p
The generation of the current builtin methods has always bothered me a bit.
It is true that some things just can not be expressed as ruby and so some
alternative mechanism is needed (even in c one can embed assembler).
%p
The main problem i have is that those methods don't check their arguments and as such
may cause core dumps. So they are too high level and hopefully all we really need is
that previous idea of being able to integrate Mom code into vool. As Mom is extensible
that should take care of any possible need. And we could code the methods normally as
part of Parfait, make them safe, and just use the lower level inside them. Lets see!
%h4 Compiling Parfait tests
%p
Since Parfait is part of rubyx, we have off course unit tests for it. The plan is to
parse the tests too, and run them as a test for both Prfait and the compiler.
Off course this will involve writing some mini version of minitest that the compiler
can actually handle (Why do i have the feeling that the real minitest involves too much
mmagic).
%h4 GrillRB conference
%p
Last, but not least, i will speak in
=ext_link "Wrocław" , "https://grillrb.com/"
in about a week. The plan is to make a comparison with rails and focus on the
possibilities, rather than technical detail. See you there :-)