move posts in directory by year

This commit is contained in:
2019-12-07 11:30:52 +02:00
parent 5ce2d1b625
commit 285c6531e4
38 changed files with 8 additions and 11 deletions

View File

@ -0,0 +1,28 @@
%p As before the original start of the project, i was 6 weeks on holiday. The distance and lack of computer really helps.
%h3#review Review
%p So i printed most of the code and the book and went over it. And apart from abismal spelling i found especially one mistake.
%p I had been going at the thing from the angle of producing binaries. Wrong aproach.
%h4#ruby-is-dynamic Ruby is Dynamic
%p In fact ruby is so dynamic it is hard to think of anything that you need to do at compile time that you cant do at runtime.
%p
In other words,
%em all
functionality is available at run-time. Ie it needs to be available in ruby, and since it then is available in ruby, one should reuse it. I had just sort of tried to avoid this, as it seemed so big.
%p In fact it is quite easy to express what needs to happed for eg. a method call, in ruby. The hard thing is to use that code at compile time.
%h4#inlining Inlining
%p When i say hard, i mean hard to code. Actually it is quite easy to understand. One “just” needs to inline the code, easy actually. Off course i had known that inlining would be neccessary in the end, i had just thought later would be fine. Well, it isnt. Off course, is it ever!
%p Inlining is making the functionality happen, without initializing a method call and return. Off course this is only possible for known function calls, but thats enough. The objects/classes we use during method dispatch are well known, so everything can be resolved at compile time. Hunky dory. Just how?
%p As a first step we change the self, while saving the old self to a tmp. Then we have to deal with how the called function accesses variables (arguments or locals). We know it does this through the Message and Frame objects. But since those are different for an inlined function, we have to make them explicit arguments. So instead of the normal eg. Message, we can create an InlineMessage for inlined function. When resolving a variable name, this InlinedMessage will look up in the parents variables and arrange access to that.
%h4#changes Changes
%p So some of the concrete changes that will come once ive done all cosmetic fixes:
%ul
%li much more parfait classes / functionality
%li remove all duplication in vm (that is now parfait)
%li change of compile, using explicit message/frames
%li explicit logic type (alongside integer + reference)
%p I also decided it would be cleaner to use the visitor pattern for compiling the ast to vm. In fact the directory should be named compile.
%p And i noticed that what i have called Builtin up to now is actually part of the Register machine layer (not vm), so it needs to move there.
%h3#some-publicity Some publicity
%p
I have now given lightning talk on Frozen Rails 2014 and Ruby Bath 2015.
As 5 Minutes is clearly now enough i will work on a longer presentation.

View File

@ -0,0 +1,71 @@
%p
Since i got the ideas of Slots and the associated instruction Set, i have been wondering how that
fits in with the code generation.
%p
I moved the patched AST compiler methods to a Compiler, ok. But still what do all those compile
methods return.
%h2#expression Expression
%p
In ruby, everything is an expression. To recap “Expressions have a value, while statements do not”,
or statements represent actions while expressions represent values.
%p
So in ruby everything represents a value, also statements, or functions. There is no such thing
as the return void in C. Even loops and ifs result in a value, for a loop the last computed value
and for an if the value of the branch taken.
%p
Having had a vague grasp of this concept i tried to sort of haphazardly return the kind of value
that i though appropriate. Sometimes literals, sometimes slots. Sometimes “Return” , a slot
representing the return value of a function.
%h2#return-slot Return slot
%p Today i realized that the Slot representing the return value is special.
%p It does not hold the value that is returned, but rather the other way around.
%p A function returns what is in the Return slot, at the time of return.
%p
From there it is easy to see that it must be the Return that holds the last computed value.
A function can return at any time after all.
%p
The last computed value is the Expression that is currently evaluated. So the compile, which
initiates the evaluation, returns the Return slot. Always. Easy, simple, nice!
%h2#example Example
%p Constants: say the expression
%pre
%code
:preserve
true
%p would compile to a
%pre
%code
:preserve
ConstantLoad(ReturnSlot , TrueConstant)
%p While
%pre
%code
:preserve
2 + 4
%p would compile to
%pre
%code
:preserve
ConstantLoad(ReturnSlot , IntegerConstant(2))
Set(ReturnSlot , OtherSlot)
ConstantLoad(ReturnSlot , IntegerConstant(4))
Set(ReturnSlot , EvenOtherSlot)
MethodCall() # unspecified details here
%h2#optimisations Optimisations
%p
But but but i hear that is so totally inefficient. All the time we move data around, to and from
that one Return slot, just so that the return is simple. Yes but no.
%p
It is very easy to optimize the trivial extra away. Many times the expression moves a value to Return
just to move it away in the next Instruction. A sequence like in above example
%pre
%code
:preserve
ConstantLoad(ReturnSlot , IntegerConstant(2))
Set(ReturnSlot , OtherSlot)
%p can easily be optimized into
%pre
%code
:preserve
ConstantLoad(OtherSlot , IntegerConstant(2))
%p tbc

View File

@ -0,0 +1,57 @@
%p
Quite long ago i
%a{:href => "/2014/06/27/an-exceptional-thought.html"} had already determined
that return
addresses and exceptional return addresses should be explicitly stored in the message.
%p
It was also clear that Message would have to be a linked list. Just managing that list at run-time
in Register Instructions (ie almost assembly) proved hard. Not that i was creating Message objects
but i did shuffle their links about. I linked and unlinked messages by setting their next/prev fields
at runtime.
%h2#the-list-is-static The List is static
%p
Now i realized that touching the list structure in any way at runtime is not necessary.
The list is completely static, ie created at compile time and never changed.
%p
To be more precise: I created the Messages at compile time and set them up as a forward linked list.
Each Item had
%em caller
field (a backlink) which i then filled at run-time. I was keeping the next
message to be used as a variable in the Space, and because that is basically global it was
relatively easy to update when making a call.
But i noticed when debugging that when i updated the messages next field, it was already set to
the value i was setting it to. And that made me stumble and think. Off course!
%p
It is the data
%strong in
the Messages that changes. But not the Message, nor the call chain.
%p
As programmer one has the call graph in mind and as that is a graph, i was thinking that the
Message list changes. But no. When working on one message, it is always the same message one sends
next. Just as one always returns to the same one that called.
%p It is the addresses and Method arguments that change, not the message.
%p
The best analogy i can think of is when calling a friend. Whatever you say, it is alwas the same
number you call.
%p
Or in C terms, when using the stack (push/pop), it is not the stack memory that changes, only the
pointer to the top. A stack is an array, right, so the array stays the same,
even its size stays the same. Only the used part of it changes.
%h2#simplifies-call-model Simplifies call model
%p
Obviously this simplifies the way one thinks about calls. Just stick the data into the pre-existing
Message objects and go.
%p
When i first had the
%a{:href => "/2014/06/27/an-exceptional-thought.html"} return address as argument
idea,
i was thinking that in case of exception one would have to garbage collect Messages.
In the same way that i was thinking that they need to be dynamically managed.
%p
Wrong again. The message chain (double linked list to be precise) stays. One just needs to clear
the data out from them, so that garbage does get collected. Anyway, its all quite simple and thats
nice.
%p
As an upshot from this new simplicity we get
= succeed "." do
%strong speed

View File

@ -0,0 +1,60 @@
%hr/
%p
After almost a year of rewrite:
%strong Hello World
is back.
%p
%strong Working executables again
%p
So much has changed in the last year it is almost impossible to recap.
Still a little summary:
%h3#register-machine Register Machine
%p
The whole layer of the
%a{:href => "/2014/09/30/a-better-register-machine.html"} Register Machine
as an
abstraction was not there. Impossible is was to see what was happening.
%h3#passes Passes
%p
In the beginning i was trying to
= succeed "." do
%em just do it
%a{:href => "/2014/07/05/layers-vs-passes.html"} implemented Passes
to go between them.
%h3#the-virtual-machine-design The virtual machine design
%p
Thinking about what objects makes up a virtual machine has brought me to a clear understanding
of the
= succeed "." do
%a{:href => "/2014/09/12/register-allocation-reviewed.html"} objects needed
%a{:href => "/2014/06/27/an-exceptional-thought.html"} stopped using the machine stack
altogether and am using a linked list instead.
Recently is has occurred to me that that linked list
%a{:href => "/06/20/the-static-call-chain.html"}> doesnt even change
, so it is very simple indeed.
%h3#smaller-though-not-small-changes Smaller, though not small, changes
%ul
%li
The
%a{:href => "/2014/08/19/object-storage.html"} Salma Object File
format was created.
%li
The
%a{:href => "http://dancinglightning.gitbooks.io/the-object-machine/content/"} Book
was started
%li I gave lightning talks at Frozen Rails 2014, Helsinki and Bath Ruby 2015
%li I presented at Munich and Zurich user groups, lots to take home from all that
%h3#future Future
%p
The mountain is still oh so high, but at last there is hope again. The second dip into arm
(gdb) debugging has made it very clear that a debugger is needed. Preferably visual, possibly 3d,
definitely browser based. So either Opal or even Volt.
%p Already more clarity in upcoming fields has arrived:
%ul
%li inlining is high on the list, to code in higher language
%li
the difference between
%a{:href => "/2015/05/20/expression-is-slot.html"} statement and expression
helped
to structure code.
%li hopefully the debugger / interpreter will help to write better tests too.

View File

@ -0,0 +1,72 @@
%p
It really is like
%a{:href => "http://worrydream.com/#!/InventingOnPrinciple"} Bret Victor
says in his video:
good programmers are the ones who play computer in their head well.
%p Why? Because you have to, to program. And off course thats what im doing.
%p
But when it got to debugging, it got a bit much. Using gdb for non C code, i mean its bad enough
for c code.
%h2#the-debugger The debugger
%p
The process of getting my “hello world” to work was quite hairy, what with debugging with gdb
and checking registers and stuff. Brr.
%p
The idea for a “solution”, my own debugger, possibly graphical, came quite quickly. But the effort seemed a
little big. It took a little, but then i started.
%p
I fiddled a little with fancy 2 or even 3d representations but couldnt get things to work.
Also getting used to running ruby in the browser, with opal, took a while.
%p
But now there is a
%a{:href => "https://github.com/ruby-x/salama-debugger"} basic frame
up,
and i can see registers swishing around and ideas of what needs
to be visualized and partly even how, are gushing. Off course its happening in html,
but that ok for now.
%p
And the best thing: I found my first serious
%strong bug
visually. Very satisfying.
%p
I do so hope someone will pick this up and run with it. Ill put it on the site as soon as the first
program runs through.
%h2#interpreter Interpreter
%p
Off course to have a debugger i needed to start on an interpreter.
Now it wasnt just the technical challenge, but some resistance against interpreting, since the whole
idea of salama was to compile. But in the end it is a very different level that the interpreter
works at. I chose to put it at the register level (not the arm), so it would be useful for future
cpus, and because the register to arm mapping is mainly about naming, not functionality. Ie it is
pretty much one to one.
%p
But off course (he says after the fact), the interpreter solves a large part of the testing
issue. Because i wasnt really happy with tests, and that was because i didnt have a good
idea how to test. Sure unit tests, fine. But to write all the little unit tests and hope the
total will result in what you want never struck me as a good plan.
%p
Instead i tend to write system tests, and drop down to unit tests to find the bugs in system tests.
But i had no good system tests, other than running the executable. But
= succeed "." do
%strong now i do
%p
So two flies with one (oh i dont know how this goes, im not english), better test, and visual
feedback, both driving the process at double speed.
%p
Now i “just” need a good way to visualize a static and running program. (implement breakpoints,
build a class and object inpector, recompile on edit . . .)
%h2#debugger-rewritten-thrice Debugger rewritten, thrice
%p
Update: after trying around with a
%a{:href => "https://github.com/orbitalimpact/opal-pixi"} 2d graphics
implementation i have rewritten the ui in
%a{:href => "https://github.com/catprintlabs/react.rb"} react
,
%a{:href => "https://github.com/voltrb/volt"} Volt
and
= succeed "." do
%a{:href => "https://github.com/opal/opal-browser"} OpalBrowser
%p
The last is what got the easiest to understand code. Also has the least dependencies, namely
only opal and opal-browser. Opal-browser is a small opal wrapper around the browsers
javascript functionality.

View File

@ -0,0 +1,143 @@
%p
It is the
%strong one
thing i said i wasnt going to do: Write a language.
There are too many languages out there already, and just because i want to write a vm,
doesnt mean i want to add to the language jungle.
%strong But
%h2#the-gap The gap
%p
As it happens in life, which is why they say never to say never, it happens just like it
i didnt want. It turns out the semantic gap of what i have is too large.
%p
There is the
%strong register level
, which is approximately assembler, and there is the
%strong vm level
which is more or less the ruby level. So my head hurts from trying to implement ruby in assembler,
no wonder.
%p
Having run into this wall, which btw is the same wall that crystal ran into, one can see the sense
in what others have done more clearly: Why rubinus uses c++ underneath. Why crystal does not
implement ruby, but a statically typed language. And ultimately why there is no ruby compiler.
The gap is just too large to bridge.
%h2#the-need-for-a-language The need for a language
%p
As I have the architecture of passes, i was hoping to get by with just another layer in the
architecture. A tried an tested approach after all. And while i wont say that that isnt a
possibility, i just dont see it. I think it may be one of those where hindsight will be perfect.
%p
I can see as far as this: If i implement a language, that will mean a parser, ast and compiler.
The target will be my register layer. So a reasonable step up is a sort of object c, that has
basic integer maths and object access. Ill detail that more below, but the point is, if i have
that, i can start writing a vm implementation in that language.
%p
Off course the vm implementation involves a parser, an ast and a compiler, unless we go to the free
compilers (see below). And so implementing the vm in a new language is in essence swapping nodes of
the higher level tree with nodes of the lower level (c-ish) one. Ie parsing should not strictly
speaking be necessary. This node swapping is after all what the pass architecture was designed
to do. But, as i said, i just cant see that happening (yet?).
%h3#trees-vs-blocks Trees vs. Blocks
%p
Speaking of the Pass architecture: I flopped. Well, maybe not so much with the actual Passes, but
with the Method representation. Blocks holding Instructions, and being in essence a list.
Misinformed copying from llvm, misinformed by the final outcome. Off course the final binary
has a linear address space, but that is where the linearity ends. The natural structure of code
is a tree, not a list, as demonstrated by the parse
= succeed "." do
%em tree
%h2#soml---salama-object-machine-language Soml - Salama Object Machine Language
%h3#typed Typed
%p
Quite a while before crystallizing into the idea of a new language, i already saw the need for a type
system. Off course, and this dates back to the first memory layouts. But i mean the need for a
%em strong typing
system, or maybe its even clearer to call it compile time typing. The type that c
and c++ have. It is essential (mentally, this is off course all for the programmer, not the computer)
to be able to think in a static type system, and then extend that and make it dynamic.
Or possibly use it in a dynamic way.
%p
This is a good example of this too big gap, where one just steps on quicksand if everything is
all the time dynamic.
%p
The way i had the implementation figured was to have different versions of the same function. In
each function we would have compile time types, everything known. Ill probably still do that,
just written in Soml.
%h3#machine-language Machine language
%p
Soml is a machine language for the Salama machine. As i tried to implement without this layer, i was
essentially implementing in assembler. Too much.
%p
There are two main feature we need from the machine language, one is typed a typed oo memory model,
the other an oo call model.
%h3#object-c Object c
%p
The language needs to be object based, off course. Just because its typed and not dynamic
and closer to assembler, doesnt mean we need to give up objects. In fact we mustnt. Soml
should be a little bit like c++, ie compile time known variable arrangement and types,
objects. But no classes (or inheritance), more like structs, with full access to everything.
So a struct.variable syntax would mean grab that variable at that address, no functions, no possible
override, just get it. This is actually already implemented as i needed it for the slot access.
%p So objects without encapsulation or classes. A lower level object orientation.
%h3#whitequark Whitequark
%p
This new approach (and more experience) shed a new light on ruby parsing. The previous idea was to
start small, write the necessary stuff in the parsable subset and with time expand that set.
%p
Alas . . ruby is a beast to parse, and because of the
%strong semantic gap
writing the system,
even in a subset, is not viable. And it turns out the brave warriors of the ruby community have
already produced a pure, production ready,
= succeed "." do
%a{:href => "https://github.com/whitequark/parser"} ruby parser
%h3#interoperability Interoperability
%p
The system code needs to be callable from the higher level, and possibly the other way around.
This probably means the same or compatible calling mechanism and data model. The data model is
quite simple as the at the system level all is just machine words, but in object sized
packets. As for the calling it will probably mean that the same message object needs to be used
and what is now called calling at the machine level is supported. Sending off course wont be.
%h3#still-missing-a-piece Still missing a piece
%p
How the level below calling can be represented is still open. It is clear though that it does need
to be present, as otherwise any kind of concurrency is impossible to achieve. The question ties
in with the still open question of
= succeed "." do
%a{:href => "http://valerieaurora.org/synthesis/SynthesisOS/ch4.html"} Quajects
%h3#start-small Start small
%p The first next step is to wrap the functionality i have in the Passes as a language.
%p Then to expand that language, by writing increasingly more complex programs in it.
%p
And then to re-attack ruby using the whitequark parser, that probably means jumping on the
mspec train.
%p All in all, no biggie :-)
%h2#compilers-are-not-free Compilers are not free
%p
Oh and i re-read and re-watched Toms
%a{:href => "http://codon.com/compilers-for-free"} compilers for free
talk,
which did make quite an impression on me the first time. But when i really thought about actually
going down that road (who doest enjoy a free beer), i got into the small print.
%p
The second biggest of which is that writing a partial evaluator is just about as complicated
as writing a compiler.
%p
But the biggest problem is that the (free) compiler you could get, has the implementation language
of the evaluator, as its
= succeed "." do
%strong output
%em for
c, not for ruby.
%p
Ok, maybe it is not quite as bad as that makes it sound. As i do have the register layer ready
and will be writing a c-ish language, it may even be possible to write an interpreter
= succeed "," do
%strong in soml
%strong for soml
too.
%p
I will nevertheless go the straighter route for now, ie write a compiler, and maybe return to the
promised freebie later. It does feel like a lot of what the partial evaluator is, would be called
compiler optimization in another lingo. So may be road will lead there naturally.

View File

@ -0,0 +1,72 @@
%p
Ok, that was surprising: I just wrote a language in two months. Parser, compiler, working binaries
and all.
%p
Then i
%a{:href => "/typed/typed.html"} documented it
, detailed the
%a{:href => "/typed/syntax.html"} syntax
and even did
some
= succeed "." do
%a{:href => "/typed/benchmarks.html"} benchmarking
%p
So, the good news: it
%strong it works
%p
Working means: calling works, if, while, assignment, class and method definition. The benchmarks
were hello world and fibonacci, both recursive and by looping.
%p
I even updated the
%a{:href => "/book.html"}
%strong whole book
to be up to date. Added a Soml section, updated
parfait, rewrote the register level . . .
%h3#it-all-clicked-into-place It all clicked into place
%p
To be fair, i dont think anyone writes a language that isnt a toy in 2 months, and it was only
possible because a lot of the stuff was there already.
%ul
%li
%a{:href => "/typed/parfait.html"} Parfait
was pretty much there. Just consolidated it as it is all just adapter.
%li
The
%a{:href => "/typed/debugger.html"} Register abstraction
(bottom) was there.
%li Using the ast library made things easier.
%li
A lot of the
%a{:href => "https://github.com/ruby-x/salama-reader"} parser
could be reused.
%p And off course the second time around everything is easier (aka hindsight is perfect).
%p
One of the better movie lines comes to mind,
(
%a{:href => "http://www.imdb.com/title/tt1341188/quotes"}> paraphrased
) “We are all just one small
adjustment away from making our code work”. It was a step sideways in the head which brought a leap
forward in terms of direction. Not where i was going but where i wanted to go.
%h3#open-issues Open issues
%p
Clearly i had wobbled on the parfait front. Now its clear it will have to be recoded in soml,
and then re-translated into ruby. But it was good to have it there in ruby all the time for the
concepts to solidify.
%p
Typing is not completely done, and negative tests for types are non existant. Also exceptions and
the machinery for the returns.
%p
I did a nice framework for testing the binaries on a remote machine, would be nice to have it
on travis. But my image is over 2Gb.
%h3#and-onto-the-next-compiler And onto the next compiler
%p
The ideas about how to compile ruby into soml have been percolating and are waiting to be put to
action.
%a{:href => "http://book.salama-vm.org/object/dynamic_types.html"} The theory
looks good,but one has
to see it to believe it.
%p
The first steps are quite clear though. Get the
%a{:href => "https://github.com/whitequark/parser"} ruby parser
integrated, get the compiler up, start with small tests. Work the types at the same time.
%p And let the adventure continue.