move ast to compiler

new compile module does just that: compile
Dispatch with visitor pattern
no more patching into ast

lib/compiler/README.md

@@ -0,0 +1,84 @@
+### Compiling
+
+The Ast (abstract syntax tree) is created by salama-reader gem and the classes defined there
+
+The code in this directory compiles the AST to the virtual machine code.
+
+If this were an interpreter, we would just walk the tree and do what it says.
+Since it's not things are a little more difficult, especially in time.
+
+When compiling we deal with two times, compile-time and run-time.
+All the headache comes from mixing those two up.*
+
+Similarly, the result of compiling is two-fold: a static and a dynamic part.
+
+- the static part are objects like the constants, but also defined classes and their methods
+- the dynamic part is the code, which is stored as streams of instructions in the CompiledMethod
+
+Too make things a little simpler, we create a very high level instruction stream at first and then
+run transformation and optimization passes on the stream to improve it.
+
+Each ast class gets a compile method that does the compilation.
+
+#### Compiled Method and Instructions
+
+The first argument to the compile method is the CompiledMethod.
+All code is encoded as a stream of Instructions in the CompiledMethod.
+Instructions are stored as a list of Blocks, and Blocks are the smallest unit of code,
+which is always linear.
+
+Code is added to the method (using add_code), rather than working with the actual instructions.
+This is so each compiling method can just do it's bit and be unaware of the larger structure
+that is being created.
+The genearal structure of the instructions is a graph
+(with if's and whiles and breaks and what), but we build it to have one start and *one* end (return).
+
+
+#### Messages and frames
+
+Since the machine is virtual, we have to define it, and since it is oo we define it in objects.
+
+Also it is important to define how instructions operate, which is is in a physical machine would
+be by changing the contents of registers or  some stack.
+
+Our machine is not a register machine, but an object machine: it operates directly on objects and
+also has no separate stack, only objects. There are a number of objects which are accessible,
+and one can think of these (their addresses) as register contents.
+(And one wouldn't be far off as that is the implementation.)
+
+The objects the machine works on are:
+
+- Message
+- Frame
+- Self
+- NewMessage
+
+and working on means, these are the only objects which the machine accesses.
+Ie all others would have to be moved first.
+
+When a Method needs to make a call, or send a Message, it creates a NewMessage object.
+Messages contain return addresses and arguments.
+
+Then the machine must find the method to call.
+This is a function of the virtual machine and is implemented in ruby.
+
+Then a new Method receives the Message, creates a Frame for local and temporary variables
+and continues execution.
+
+The important thing here is that Messages and Frames are normal objects.
+
+And interestingly we can partly use ruby to find the method, so in a way it is not just a top
+down transformation. Instead the sending goes back up and then down again.
+
+The Message object is the second parameter to the compile method, the run-time part as it were.
+Why? Since it only exists at runtime: to make compile time analysis possible
+(it is after all the Virtual version, not Parfait. ie compile-time, not run-time).
+Especially for those times when we can resolve the method at compile time.
+
+
+*
+As ruby is a dynamic language, it also compiles at run-time. This line of thought does not help
+though as it sort of mixes the seperate times up, even they are not.
+Even in a running ruby programm the stages of compile and run are seperate.
+Similarly it does not help to argue that the code is static too, not dynamic,
+as that leaves us with a worse working model.

lib/compiler/basic_expressions.rb

@@ -0,0 +1,86 @@
+# collection of the simple ones, int and strings and such
+
+module Compiler
+
+  # Constant expressions can by definition be evaluated at compile time.
+  # But that does not solve their storage, ie they need to be accessible at runtime from _somewhere_
+  # So we view ConstantExpressions like functions that return the value of the constant.
+  # In other words, their storage is the return slot as it would be for a method
+
+  # The current approach moves the constant into a variable before using it
+  # But in the future (in the one that holds great things) we optimize those unneccesay moves away
+
+#    attr_reader :value
+    def compile_integer expession , method , message
+      int = Virtual::IntegerConstant.new(value)
+      to = Virtual::NewReturn.new(Virtual::Integer , int)
+      method.add_code Virtual::Set.new( to , int)
+      to
+    end
+
+    def compile_true expession , method , message
+      value = Virtual::TrueConstant.new
+      to = Virtual::Return.new(Virtual::Reference , value)
+      method.add_code Virtual::Set.new( to , value )
+      to
+    end
+
+    def compile_false expession , method , message
+      value = Virtual::FalseConstant.new
+      to = Virtual::Return.new(Virtual::Reference , value)
+      method.add_code Virtual::Set.new( to , value )
+      to
+    end
+
+    def compile_nil expession , method , message
+      value = Virtual::NilConstant.new
+      to = Virtual::Return.new(Virtual::Reference , value)
+      method.add_code Virtual::Set.new( to , value )
+      to
+    end
+
+#    attr_reader  :name
+    # compiling name needs to check if it's a variable and if so resolve it
+    # otherwise it's a method without args and a send is ussued.
+    # this makes the namespace static, ie when eval and co are implemented method needs recompilation
+    def compile_name expession , method , message
+      return Virtual::Self.new( Virtual::Mystery ) if expession.name == :self
+      if method.has_var(expession.name)
+        message.compile_get(method , expession.name )
+      else
+        raise "Unimplemented #{self}"
+        message.compile_send( method , expession.name ,  Virtual::Self.new( Virtual::Mystery ) )
+      end
+    end
+
+
+    def compile_module expession , method , message
+      clazz = Virtual::BootSpace.space.get_or_create_class name
+      raise "uups #{clazz}.#{name}" unless clazz
+      to = Virtual::Return.new(Virtual::Reference , clazz )
+      method.add_code Virtual::Set.new( to , clazz )
+      to
+    end
+
+#    attr_reader  :string
+    def compile_string expession , method , message
+      value = Virtual::StringConstant.new(expession.string)
+      to = Virtual::Return.new(Virtual::Reference , value)
+      Virtual::BootSpace.space.add_object value
+      method.add_code Virtual::Set.new( to , value )
+      to
+    end
+
+    #attr_reader  :left, :right
+    def compile_assignment expession , method , message
+      raise "must assign to NameExpression , not #{expession.left}" unless expession.left.instance_of? NameExpression
+      r = right.compile(method,message)
+      raise "oh noo, nil from where #{expession.right.inspect}" unless r
+      message.compile_set( method , expession.left.name , r )
+    end
+
+    def compile_variable expession, method , message
+      method.add_code Virtual::InstanceGet.new(expession.name)
+      Virtual::NewReturn.new( Virtual::Mystery )
+    end
+end

lib/compiler/call_site_expression.rb

@@ -0,0 +1,26 @@
+module Compiler
+  # operators are really function calls
+
+#    call_site - attr_reader  :name, :args , :receiver
+
+    def compile_call_site expession , method , message
+      me = expession.receiver.compile( method, message )
+      method.add_code Virtual::NewMessage.new
+      method.add_code Virtual::Set.new(Virtual::NewSelf.new(me.type), me)
+      method.add_code Virtual::Set.new(Virtual::NewName.new(), Virtual::StringConstant.new(expession.name))
+      compiled_args = []
+      expession.args.each_with_index do |arg , i|
+        #compile in the running method, ie before passing control
+        val = arg.compile( method, message)
+        # move the compiled value to it's slot in the new message
+        to = Virtual::NewMessageSlot.new(i ,val.type , val)
+        # (doing this immediately, not after the loop, so if it's a return it won't get overwritten)
+        method.add_code Virtual::Set.new(to , val )
+        compiled_args << to
+      end
+      method.add_code Virtual::MessageSend.new(expession.name , me , compiled_args) #and pass control
+      # the effect of the method is that the NewMessage Return slot will be filled, return it
+      # (this is what is moved _inside_ above loop for such expressions that are calls (or constants))
+      Virtual::NewReturn.new( method.return_type )
+    end
+end

lib/compiler/compound_expressions.rb

@@ -0,0 +1,14 @@
+module Compiler
+
+#    attr_reader :values
+    def compile_array expession, context
+      to.do
+    end
+#    attr_reader :key , :value
+    def compile_association context
+      to.do
+    end
+    def compile_hash context
+      to.do
+    end
+end

lib/compiler/expression_list.rb

@@ -0,0 +1,6 @@
+module Compiler
+#    list - attr_reader  :expressions
+    def compile_list expession , method , message
+      expession.expressions.collect { |part|  part.compile( method, message ) }
+    end
+end

lib/compiler/function_expression.rb

@@ -0,0 +1,69 @@
+module Compiler
+#    function attr_reader  :name, :params, :body , :receiver
+    def compile_function  expression, method , message
+      args = expession.params.collect do |p|
+        raise "error, argument must be a identifier, not #{p}" unless p.is_a? NameExpression
+        p.name
+      end
+      r = expession.receiver ? expession.receiver.compile(method,message) : Virtual::Self.new()
+      new_method = Virtual::CompiledMethod.new(expession.name , args , r )
+      new_method.class_name = r.is_a?(Virtual::BootClass) ? r.name : method.class_name
+      clazz = Virtual::BootSpace.space.get_or_create_class(new_method.class_name)
+      clazz.add_instance_method new_method
+
+      #frame = frame.new_frame
+      return_type = nil
+      expession.body.each do |ex|
+        return_type = ex.compile(new_method,message )
+        raise return_type.inspect if return_type.is_a? Virtual::Instruction
+      end
+      new_method.return_type = return_type
+      new_method
+    end
+    def scratch
+      args = []
+      locals = {}
+      expession.params.each_with_index do |param , index|
+        arg = param.name
+        register = Virtual::RegisterReference.new(Virtual::RegisterMachine.instance.receiver_register).next_reg_use(index + 1)
+        arg_value = Virtual::Integer.new(register)
+        locals[arg] = arg_value
+        args << arg_value
+      end
+      # class depends on receiver
+      me = Virtual::Integer.new( Virtual::RegisterMachine.instance.receiver_register )
+      if expession.receiver.nil?
+        clazz = context.current_class
+      else
+        c = context.object_space.get_or_create_class expession.receiver.name.to_sym
+        clazz = c.meta_class
+      end
+
+      function = Virtual::Function.new(name , me , args )
+      clazz.add_code_function function
+
+      parent_locals = context.locals
+      parent_function = context.function
+      context.locals = locals
+      context.function = function
+
+      last_compiled = nil
+      expession.body.each do |b|
+        puts "compiling in function #{b}"
+        last_compiled = b.compile(context)
+        raise "alarm #{last_compiled} \n #{b}" unless last_compiled.is_a? Virtual::Word
+      end
+
+      return_reg = Virtual::Integer.new(Virtual::RegisterMachine.instance.return_register)
+      if last_compiled.is_a?(Virtual::IntegerConstant) or last_compiled.is_a?(Virtual::ObjectConstant)
+        return_reg.load function , last_compiled if last_compiled.register_symbol != return_reg.register_symbol
+      else
+        return_reg.move( function, last_compiled ) if last_compiled.register_symbol != return_reg.register_symbol
+      end
+      function.set_return return_reg
+
+      context.locals = parent_locals
+      context.function = parent_function
+      function
+    end
+end

lib/compiler/if_expression.rb

@@ -0,0 +1,40 @@
+module Compiler
+#    if - attr_reader  :cond, :if_true, :if_false
+
+    def compile_if expression , method , message
+      # to execute the logic as the if states it, the blocks are the other way around
+      # so we can the jump over the else if true ,and the else joins unconditionally after the true_block
+      merge_block = method.new_block "if_merge"    # last one, created first
+      true_block =  method.new_block "if_true"     # second, linked in after current, before merge
+      false_block = method.new_block "if_false"    # directly next in order, ie if we don't jump we land here
+
+
+      is = cond.compile(method,message)
+      # TODO should/will use different branches for different conditions.
+      # just a scetch : cond_val = cond_val.is_true?(method) unless cond_val.is_a? Virtual::BranchCondition
+      method.add_code Virtual::IsTrueBranch.new( true_block )
+
+      # compile the true block (as we think of it first, even it is second in sequential order)
+      method.current true_block
+      last = is
+      if_true.each do |part|
+        last = part.compile(method,message )
+        raise part.inspect if last.nil?
+      end
+
+      # compile the false block
+      method.current false_block
+      if_false.each do |part|
+        puts "compiling in if false #{part}"
+        last = part.compile(method,message)
+        raise part.inspect if last.nil?
+      end
+      method.add_code Virtual::UnconditionalBranch.new( merge_block )
+
+      puts "compiled if: end"
+      method.current merge_block
+
+      #TODO should return the union of the true and false types
+      last
+    end
+end

lib/compiler/module_expression.rb

@@ -0,0 +1,22 @@
+module Compiler
+#    module attr_reader  :name ,:expressions
+    def compile_module expression , context
+      return clazz
+    end
+
+    def compile_class expression , method , message
+      clazz = ::Virtual::BootSpace.space.get_or_create_class name
+      puts "Created class #{clazz.name.inspect}"
+      expression.expressions.each do |expr|
+        # check if it's a function definition and add
+        # if not, execute it, but that does means we should be in salama (executable), not ruby. ie throw an error for now
+        raise "only functions for now #{expr.inspect}" unless expr.is_a? Ast::FunctionExpression
+        #puts "compiling expression #{expression}"
+        expression_value = expr.compile(method,message )
+        clazz.add_instance_method(expression_value)
+        #puts "compiled expression #{expression_value.inspect}"
+      end
+
+      return clazz
+    end
+end

lib/compiler/operator_expressions.rb

@@ -0,0 +1,7 @@
+module Compiler
+#    operator attr_reader  :operator, :left, :right
+    def compile_operator expression, method , message
+      call = CallSiteExpression.new( operator , [right] , left )
+      call.compile(method,message)
+    end
+end

lib/compiler/return_expression.rb

@@ -0,0 +1,22 @@
+module Compiler
+
+#    return attr_reader  :expression
+    def compile_return expression, scope ,method
+      Virtual::Reference.new
+    end
+    def old
+      into = context.function
+      puts "compiling return expression #{expression}, now return in return_regsiter"
+      expression_value = expression.compile(context)
+      # copied from function expression: TODO make function
+
+      return_reg = Virtual::Integer.new(Virtual::RegisterMachine.instance.return_register)
+      if expression_value.is_a?(Virtual::IntegerConstant) or expression_value.is_a?(Virtual::ObjectConstant)
+        return_reg.load into , expression_value
+      else
+        return_reg.move( into, expression_value ) if expression_value.register_symbol != return_reg.register_symbol
+      end
+      #function.set_return return_reg
+      return return_reg
+    end
+end

lib/compiler/while_expression.rb

@@ -0,0 +1,25 @@
+module Compiler
+
+#    while- attr_reader  :condition, :body
+    def compile_while expression, method , message
+      start = Virtual::Label.new("while_start")
+      method.add_code start
+      is = expression.condition.compile(method,message)
+      branch = Virtual::IsTrueBranch.new "while"
+      merge = Virtual::Label.new(branch.name)
+      branch.other = merge   #false jumps to end of while
+      method.add_code branch
+      last = is
+      expression.body.each do |part|
+        last = part.compile(method,message )
+        raise part.inspect if last.nil?
+      end
+      # unconditionally brnach to the start
+      merge.next = method.current.next
+      method.current.next = start
+      # here we add the end of while that the branch jumps to
+      #but don't link it in (not using add)
+      method.current =  merge
+      last
+    end
+end