move posts

This commit is contained in:
Torsten Ruger 2018-04-10 19:11:06 +03:00
parent 5ee1c6d461
commit 5950fac3ce
30 changed files with 43 additions and 194 deletions

View File

@ -1,8 +1,3 @@
---
layout: news
author: Torsten
---
Well, it has been a good holiday, two months in Indonesia, Bali and diving Komodo. It brought
clarity, and so i have to start a daunting task.

View File

@ -1,8 +1,3 @@
---
layout: news
author: Torsten
---
The c machine
-------------

View File

@ -1,8 +1,3 @@
---
layout: news
author: Torsten
---
Part of the reason why i even thought this was possible was because i had bumped into Metasm.
Metasm creates native code in 100% ruby. Either from Assembler or even C (partially). And for many cpu's too.

View File

@ -1,10 +1,6 @@
---
layout: news
author: Torsten
---
Both "ends", parsing and machine code, were relatively clear cut. Now it is into unknown territory.
I had ported the Kaleidescope llvm tutorial language to ruby-llvm last year, so thee were some ideas floating.
I had ported the Kaleidoscope llvm tutorial language to ruby-llvm last year, so thee were some ideas floating.
The idea of basic blocks, as the smallest unit of code without branches was pretty clear. Using those as jump
targets was also straight forward. But how to get from the AST to arm Intructions was not, and took some trying out.

View File

@ -1,8 +1,3 @@
---
layout: news
author: Torsten
---
Parsing is a difficult, the theory incomprehensible and older tools cryptic. At least for me.
And then i heard recursive is easy and used by even llvm. Formalised as peg parsing libraries exists, and in ruby

View File

@ -1,9 +1,3 @@
---
layout: news
author: Torsten
title: More clarity
---
It's such a nice name, crystal. My first association is clarity, and that is exactly what i am trying to achieve.
But i've been struggling a bit to achieve any clarity on the topic of system boundary: where does OO stop. I mean i can't very well define method lookup in ruby syntax, as that involves method lookups. But tail recursion is so booring, it just never stops!

View File

@ -1,8 +1,3 @@
---
layout: news
author: Torsten
---
I was just reading my ruby book, wondering about functions and blocks and the like, as one does when implementing
a vm. Actually the topic i was struggling with was receivers, the pesty self, when i got the exception.

View File

@ -1,8 +1,3 @@
---
layout: news
author: Torsten
---
I am not stuck. I know i'm not. Just because there is little visible progress doesn't mean i'm stuck. It may just feel like it though.
But like little cogwheels in the clock, i can hear the background process ticking away and sometimes there is a gong.

View File

@ -1,8 +1,3 @@
---
layout: news
author: Torsten
---
In a picture, or when taking a picture, the frame is very important. It sets whatever is in the picture into context.
So it is a bit strange that having a **frame** had the same sort of effect for me in programming.

View File

@ -1,13 +1,8 @@
---
layout: news
author: Torsten
---
It has been a bit of a journey, but now we have arrived: Salama is officially named.
### Salama
Salama is a **real word**, not made up or an acronym (plus).
Salama is a **real word**, not made up or an acronym (plus).
It is a word of my **home-country** Finland, a finnish word (double plus)
@ -23,18 +18,18 @@ Also **my wife** suggested it, so it always reminds me of her.
I started with crystal, which i liked. It speaks of clarity. It is related to ruby. All was good.
But I was not the first to have this thought: The name is taken, as i found out by
chance. Ary Borenszweig started the [project](http://crystal-lang.org/) already two
chance. Ary Borenszweig started the [project](http://crystal-lang.org/) already two
years ago and they not only have a working system, but even compile themselves.
Alas, Ary started out with the idea of ruby on rockets (ie fast), but when the
dynamic aspects came (as they have for me a month ago), he went for speed, to be
precise for a static system, not for ruby.
precise for a static system, not for ruby.
So his crystal is now it's own language with ruby-ish style, but not semantics.
That is why i had not found it. But when i did we talked, all was friendly, and we
agreed i would look for a new name.
That is why i had not found it. But when i did we talked, all was friendly, and we
agreed i would look for a new name.
And so i did and many were taken. Kide (crystal in finish) was a step on the way,
And so i did and many were taken. Kide (crystal in finish) was a step on the way,
as was ruby in ruby. And many candidates were explored and discarded, like broom
(basic ruby object oriented machine), or som (simple object machine), even ahimsa.
@ -43,7 +38,7 @@ as was ruby in ruby. And many candidates were explored and discarded, like broom
But then i found it, or rather we did, as it was a suggestion from my wife: Salama.
After i found the name i made sure to claim it: I published first versions of gems
for salama and sub-modules. They don't work off course, but at least the name is
for salama and sub-modules. They don't work off course, but at least the name is
taken in rubygems too. Off course the github name is too.
So now i can get on with things at lightning speed :-)

View File

@ -1,10 +1,5 @@
---
layout: news
author: Torsten
---
While trying to figure out what i am coding i had to attack this storage format before i wanted to. The
immediate need is for code dumps, that are concise but readable. I started with yaml but that just takes
immediate need is for code dumps, that are concise but readable. I started with yaml but that just takes
too many lines, so it's too difficult to see what is going on.
I just finished it, it's a sort of condensed yaml i call sof (salama object file), but i want to take the
@ -13,40 +8,40 @@ moment to reflect why i did this, what the bigger picture is, where sof may go.
### Program lifecycle
Let's take a step back to mother smalltalk: there was the image. The image was/is the state of all the
objects in the system. Even threads, everything. Absolute object thinking taken to the ultimate.
objects in the system. Even threads, everything. Absolute object thinking taken to the ultimate.
A great idea off course, but doomed to ultimately fail because no man is an island (so no vm is either).
#### Development
Software development is a team sport, a social activity at it's core. This is not always realised,
Software development is a team sport, a social activity at it's core. This is not always realised,
when the focus is too much on the outcome, but when you look at it, everything is done in teams.
The other thing not really taken into account in the standard developemnt model is that it is a process in
The other thing not really taken into account in the standard developemnt model is that it is a process in
time that really only gets jucy with a first customer released version. Then you get into branches for bugs
and features, versions with major and minor and before long you'r in a jungle of code.
#### Code centered
But all that effort is concentrated on code. Ok nowadays schema evlolution is part of the game, so the
existance of data is acknowledged, but only as an external thing. Nowhere near that smalltalk model.
But off course a truely object oriented program is not just code. It's data too. Maybe currently "just"
configuration and enums/constants and locales, but that is exactly my point.
The lack of defined data/object storage is holding us back, making all our programs fruit-flies.
The lack of defined data/object storage is holding us back, making all our programs fruit-flies.
I mean it lives a short time and dies. A program has no way of "learning", of accumulating data/knowledge
to use in a next invocation.
#### Optimisation example
Let's take optimisation as an example. So a developer runs tests (rubyprof/valgrind or something)
Let's take optimisation as an example. So a developer runs tests (rubyprof/valgrind or something)
with some output and makes program changes accordingly. But there are two obvious problems.
Firstly the data is collected in development not production. Secondly, and more importantly, a person is
needed.
Of course a program could quite easily monitor itself, possibly over a long time, possibly only when
not at epak load. And surely some optimisations could be automated, a bit like the O1 .. On compiler
switches, more and more effort could be exerted on critical regions. Possibly all the way to
switches, more and more effort could be exerted on critical regions. Possibly all the way to
super-optimisation.
But even if we did this, and a program would improve/jit itself, the fruits of this work are only usable
@ -56,15 +51,15 @@ benefit. And thus start again, just like in Groundhog day.
### Storage
So to make that optimisation example work, we would need a storage: Theoretically we could make the program
change it's own executable/object files, in ruby even it's source. Theoretically, as we have no
change it's own executable/object files, in ruby even it's source. Theoretically, as we have no
representation of the code to work on.
In salama we do have an internal representation, both at the code level (ast) and the compiled code
In salama we do have an internal representation, both at the code level (ast) and the compiled code
(CompiledMethod, Intructions and friends).
#### Storage Format
Going back to the Image we can ask why was it doomed to fail: because of the binary,
Going back to the Image we can ask why was it doomed to fail: because of the binary,
proprietary implementation. Not because of the idea as such.
Binary data needs either a rigourous specification and/or software to work on it. Work, what work?
@ -72,7 +67,7 @@ We need to merge the data between installations, maintain versions and branches.
version control, because it basically is. Off course this "could" have been solved by the smalltalk
people, but wasn't. I think it's fair to say that git was the first system to solve that problem.
And git off course works with diff, and so for a 3-way merge to be successful we need a text format.
And git off course works with diff, and so for a 3-way merge to be successful we need a text format.
Which is why i started with yaml, and which is why also sof is text-based.
The other benefit is off course human readability.
@ -86,9 +81,8 @@ Ok, i'll sketch it a little: Salama would use sof as it's object file format, an
stored in git. For developers to work, tools would create source and when that is edited compile it to sof.
A program would be a repository of sof and resource files. Some convention for load order would be helpful
and some "area" where programs may collect data or changes to the program. Some may off course alter the
and some "area" where programs may collect data or changes to the program. Some may off course alter the
sof's directly.
How, when and how automatically changes are merged (via git) is up to developer policy . But it is
How, when and how automatically changes are merged (via git) is up to developer policy . But it is
easily imaginable that data in program designated areas get merged back into the "mainstream" automatically.

View File

@ -1,13 +1,8 @@
---
layout: news
author: Torsten
---
The time of introspection is coming to an end and i am finally producing executables again. (hurrah)
### Block and exception
Even neither ruby blocks or exceptions are implemented i have figured out how to do it, which is sort of good news.
Even neither ruby blocks or exceptions are implemented i have figured out how to do it, which is sort of good news.
I'll see off course when the day comes, but a plan is made and it is this:
No information lives on the machine stack.
@ -28,33 +23,33 @@ And, as stated above, all these objects live in memory.
### Single Set Instruction
Self and frame are duplicated information, because then it is easier to transfer. After inital trying, i settle on a
Self and frame are duplicated information, because then it is easier to transfer. After inital trying, i settle on a
single Instruction to move data around in the vm, Set. It can move instance variables from any of the objects to any
other of the 4 objects.
The implementation of Set ensures that any move to the self slot in Message gets duplicated into the Self register. Same
for the frame, but both are once per method occurances, and both are read only afterwards, so don't need updating later.
Set, like other instructions may use any other variables at any time. Those registers (r4 and up) are scratch.
Set, like other instructions may use any other variables at any time. Those registers (r4 and up) are scratch.
### Simple call
This makes calling relatively simple and thus easy to understand. To make a call we must be in a method, ie Message,
Self and Frame have been set up.
This makes calling relatively simple and thus easy to understand. To make a call we must be in a method, ie Message,
Self and Frame have been set up.
The method then produces values for the call. This involves operations and the result of that is stored in a variable
The method then produces values for the call. This involves operations and the result of that is stored in a variable
(tmp/local/arg). When all values have been calculated a NewMessage is created and all data moved there (see Set)
A Call is then quite simple: because of the duplication of Self and Frame, we only need to push the Message to the
A Call is then quite simple: because of the duplication of Self and Frame, we only need to push the Message to the
machine stack. Then we move the NewMessage to Message, unroll (copy) the Self into it's register and assign a new
Frame.
Returning is also not overly complicated: Remembering that the return value is an instance variable in the
Returning is also not overly complicated: Remembering that the return value is an instance variable in the
Message object. So when the method is done, the value is there, not for example in a dedicated register.
So we need to undo the above: move the current Message to NewMessage, pop the previously pushed message from the
So we need to undo the above: move the current Message to NewMessage, pop the previously pushed message from the
machine stack and unroll the Self and Frame copies.
The caller then continues and can pick up the return from it's NewMessage if it is used for further calculation.
The caller then continues and can pick up the return from it's NewMessage if it is used for further calculation.
It's like it did everything to built the (New)Message and immediately the return value was filled in.
As I said, often we need to calculate the values for the call, so we need to make calls. This happens in exacly the same
@ -72,7 +67,7 @@ And unfortunately we can't really make a call to get/create these objects as tha
We need a very fast way to create and reuse these objects: a bit like a stack. So let's just use a Stack :-)
Off course not the machine stack, but a Stack object. An array to which we append and take from.
Off course not the machine stack, but a Stack object. An array to which we append and take from.
It must be global off course, or rather accessible from compiling code. And fast may be that we use assembler, or
if things work out well, we can use the same code as what makes builtin arrays tick.

View File

@ -1,8 +1,3 @@
---
layout: news
author: Torsten
---
The register machine abstraction has been somewhat thin, and it is time to change that
### Current affairs
@ -14,7 +9,7 @@ Not having **any** experience at this level i felt that arm was pretty sensible.
so i abtracted the basic instruction classes a little and had the arm instructions implement them pretty much one
to one.
Then i tried to implement any ruby logic in that abstraction and failed. Thus was born the virtual machine
Then i tried to implement any ruby logic in that abstraction and failed. Thus was born the virtual machine
abstraction of having Message, Frame and Self objects. This in turn mapped nicely to registers with indexed
addressing.
@ -24,17 +19,17 @@ I just have to sidestep here a little about addressing: the basic problem is off
compile-time at what address the executable will end up.
The problem first emerged with calling functions. Mostly because that was the only objects i had, and so i was
very happy to find out about pc relative addressing, in which you jump or call relative to your current position
very happy to find out about pc relative addressing, in which you jump or call relative to your current position
(**p**rogram **c**ounter). Since the relation is not changed by relocation all is well.
Then came the first strings and the aproach can be extended: instead of grabbing some memory location, ie loading
and address and dereferencing, we calculate the address in relation to pc and then dereference. This is great and
and address and dereferencing, we calculate the address in relation to pc and then dereference. This is great and
works fine.
But the smug smile is wiped off the face when one tries to store references. This came with the whole object
aproach, the bootspace holding references to **all** objects in the system. I even devised a plan to always store
relative addresses. Not relative to pc, but relative to the self that is storing. This i'm sure would have
worked fine too, but it does mean that the running program also has to store those relative addresses (or have
worked fine too, but it does mean that the running program also has to store those relative addresses (or have
different address types, shudder). That was a runtime burden i was not willing to accept.
So there are two choices as far as i see: use elf relocation, or relocate in init code. And yet again i find myself
@ -65,10 +60,10 @@ conclusion that a load is meant. And since this is a fresh effort i would rather
it easier for others to learn sensible stuff than me get used to cryptics only to have everyone after me do the same.
So i will have instructions like RegisterMove, ConstantLoad, Branch, which will translate to mov, ldr and b in arm. I still like to keep the arm level with the traditional names, so people who actually know arm feel right at home.
But the extra register layer will make it easier for everyone who has not programmed assembler (and me!),
But the extra register layer will make it easier for everyone who has not programmed assembler (and me!),
which i am guessing is quite a lot in the *ruby* community.
In implementation terms it is a relatively small step from the vm layer to the register layer. And an even smaller
In implementation terms it is a relatively small step from the vm layer to the register layer. And an even smaller
one to the arm layer. But small steps are good, easy to take, easy to understand, no stumbling.
### Extra Benefits
@ -79,10 +74,10 @@ benefits clearly exist.
##### Clean interface for cpu specific implementation
That really says it all. That interface was a bit messy, as the RegisterMachine was used in Vm code, but was actually
an Arm implementation. So no seperation. Also as mentioned the instruction set was arm heavy, with the quirks
an Arm implementation. So no seperation. Also as mentioned the instruction set was arm heavy, with the quirks
even arm has.
So in the future any specific cpu implementation can be quite self sufficient. The classes it uses don't need to
So in the future any specific cpu implementation can be quite self sufficient. The classes it uses don't need to
derive from anything specific and need only implement the very small code interface (position/length/assemble).
And to hook in, all that is needed is to provide a translation from RegisterMachine instructions, which can be
done very nicely by providing a Pass for every instruction. So that layer of code is quite seperate from the actual

View File

@ -1,13 +1,8 @@
---
layout: news
author: Torsten
---
As before the original start of the project, i was 6 weeks on holiday. The distance and lack of computer really helps.
### Review
So i printed most of the code and the book and went over it. And apart from abismal spelling i found especially one mistake.
So i printed most of the code and the book and went over it. And apart from abismal spelling i found especially one mistake.
I had been going at the thing from the angle of producing binaries. Wrong aproach.
@ -42,6 +37,5 @@ And i noticed that what i have called Builtin up to now is actually part of the
### Some publicity
I have now given lightning talk on Frozen Rails 2014 and Ruby Bath 2015.
I have now given lightning talk on Frozen Rails 2014 and Ruby Bath 2015.
As 5 Minutes is clearly now enough i will work on a longer presentation.

View File

@ -1,8 +1,3 @@
---
layout: news
author: Torsten
---
Since i got the ideas of Slots and the associated instruction Set, i have been wondering how that
fits in with the code generation.

View File

@ -1,8 +1,3 @@
---
layout: news
author: Torsten
---
Quite long ago i [had already determined](/2014/06/27/an-exceptional-thought.html) that return
addresses and exceptional return addresses should be explicitly stored in the message.

View File

@ -1,8 +1,4 @@
---
layout: news
author: Torsten
---
After almost a year of rewrite: **Hello World** is back.
**Working executables again**

View File

@ -1,8 +1,3 @@
---
layout: news
author: Torsten
---
It really is like [Bret Victor](http://worrydream.com/#!/InventingOnPrinciple) says in his video:
good programmers are the ones who play computer in their head well.

View File

@ -1,8 +1,3 @@
---
layout: news
author: Torsten
---
It is the **one** thing i said i wasn't going to do: Write a language.
There are too many languages out there already, and just because i want to write a vm,
doesn't mean i want to add to the language jungle.

View File

@ -1,8 +1,3 @@
---
layout: news
author: Torsten
---
Ok, that was surprising: I just wrote a language in two months. Parser, compiler, working binaries
and all.

View File

@ -1,8 +1,3 @@
---
layout: news
author: Torsten
---
Writing Soml helped a lot to separate the levels, or phases of the ruby compilation process. Helped
me that is, to plan the ruby compiler.

View File

@ -1,8 +1,3 @@
---
layout: news
author: Torsten
---
So, the plan, in short:
1. I need to work a little more on docs. Reading them i notice they are still not up to date

View File

@ -1,8 +1,3 @@
---
layout: news
author: Torsten
---
## RubyX compiles ruby to binary
The previous name was from a time in ancient history, three years ago, in internet time over

View File

@ -1,8 +1,3 @@
---
layout: news
author: Torsten
---
I just read mri 2.4 "unifies" Fixnum and Integer. This, it turns out, is something quite
different from what i though, mostly about which class names are returned.
And that it is ok to have two implementations for the same class, Integer.

View File

@ -1,8 +1,3 @@
---
layout: news
author: Torsten
---
As i said in the last post, a step back and forward, possibly two, was taken and understanding
grows again. Especially when i think that some way is the way, it always changes and i turn out
to be at least partially wrong. The way of life, of imperfect intelligence, to strive for that

View File

@ -1,8 +1,3 @@
---
layout: news
author: Torsten
---
Going on holiday without a computer was great. Forcing me to recap and write things down on paper.
## Layers

View File

@ -1,8 +1,3 @@
---
layout: news
author: Torsten
---
Method caching can be done at language level. Wow. But first some boring news:
## Vool is ready, Mom is coming

View File

@ -1,8 +1,3 @@
---
layout: news
author: Torsten
---
While work on Mom (Minimal object machine) continues, i can see the futures a little clearer.
Alas, for now the shortest route is best, so the future will have to wait. But here is what i'm
thinking.

View File

@ -1,8 +1,3 @@
---
layout: news
author: Torsten
---
Since i currently have no time to do actual work, i've been doing some research.
Reading about other implementations, especially transpiling ones. Opal, ruby to

View File

@ -1,8 +1,3 @@
---
layout: news
author: Torsten
---
Now that i *have* had time to write some more code (250 commits last month), here is
the good news: