move compiler to typed

starting to get rid of soml, bit by bit

lib/typed/compiler/README.md

@@ -0,0 +1,69 @@
+### Compiling
+
+The Ast (abstract syntax tree) is created by [salama-reader](https://github.com/salama/salama-reader)
+ gem and the classes defined there
+
+The code in this directory compiles the AST to the virtual machine code, and Parfait object structure.
+
+If this were an interpreter, we would just walk the tree and do what it says.
+Since it's not things are a little more difficult, especially in time.
+
+When compiling we deal with two times, compile-time and run-time.
+All the headache comes from mixing those two up.*
+
+Similarly, the result of compiling is two-fold: a static and a dynamic part.
+
+- the static part are objects like the constants, but also defined classes and their methods
+- the dynamic part is the code, which is stored as streams of instructions in the MethodSource
+
+Too make things a little simpler, we create a very high level instruction stream at first and then
+run transformation and optimization passes on the stream to improve it.
+
+The compiler has a method for each type for ast, named along on_xxx with xxx as the type
+
+#### Compiler holds scope
+
+The Compiler instance can hold arbitrary scope needed during the compilation. Since we compile Soml
+(a static language) things have become more simple.
+
+A class statement sets the current @clazz scope , a method definition the @method.
+If either are not set when needed compile errors will follow. So easy, so nice.
+
+All code is encoded as a stream of Instructions in the MethodSource.
+Instructions are stored as a list of Blocks, and Blocks are the smallest unit of code,
+which is always linear.
+
+Code is added to the method (using add_code), rather than working with the actual instructions.
+This is so each compiling method can just do it's bit and be unaware of the larger structure
+that is being created.
+The general structure of the instructions is a graph
+(with if's and whiles and breaks and what), but we build it to have one start and *one* end (return).
+
+
+#### Messages and frames
+
+Since the machine is virtual, we have to define it, and since it is oo we define it in objects.
+
+Also it is important to define how instructions operate, which is is in a physical machine would
+be by changing the contents of registers or some stack.
+
+Our machine is not a register machine, but an object machine: it operates directly on objects and
+also has no separate stack, only objects. There is only one object which is accessible,
+basically meaning pinned to a register, the Message.
+
+One can think of the Message as an oo replacement of the stack.
+
+When a Method needs to make a call, it creates a NewMessage object.
+Messages contain return addresses (yes, plural) and arguments.
+
+The important thing here is that Messages and Frames are normal objects.
+
+### Distinctly future proof
+
+Soml is designed to be used as an implementation language for a higher oo language. Some, or
+even many, features may not make sense on their own. But these features, like several return
+addresses, are important to implement the higher language.
+
+In fact, Soml's main purpose is not even to be written. The main purpose is to have a language to
+compile ruby to. In the same way that the assembler layer in salama is not designed to be written,
+we just need it to create our layers.

lib/typed/compiler/assignment.rb

@@ -0,0 +1,33 @@
+module Typed
+  Compiler.class_eval do
+
+    def on_Assignment statement
+      reset_regs # statements reset registers, ie have all at their disposal
+      #puts statement.inspect
+#      name , value = *statement
+      name_s = no_space statement.name
+      v = process(statement.value)
+      raise "Not register #{v}" unless v.is_a?(Register::RegisterValue)
+      code = nil
+      if( index = @method.has_arg(name_s.name))
+         # TODO, check type @method.arguments[index].type
+        code = Register.set_slot(statement , v , :message , Parfait::Message.get_indexed(index) )
+      else # or a local so it is in the frame
+        index = @method.has_local( name_s.name )
+        if(index)
+          # TODO, check type  @method.locals[index].type
+          frame = use_reg(:Frame)
+          add_code Register.get_slot(statement , :message , :frame , frame )
+          code = Register.set_slot(statement , v , frame , Parfait::Frame.get_indexed(index) )
+        end
+      end
+      if( code )
+        #puts "addin code #{code}"
+        add_code code
+      else
+        raise "must define variable #{name} before using it in #{@method.inspect}"
+      end
+    end
+
+  end
+end

lib/typed/compiler/basic_values.rb

@@ -0,0 +1,56 @@
+module Typed
+  # collection of the simple ones, int and strings and such
+
+  Compiler.class_eval do
+
+    # Constant expressions can by definition be evaluated at compile time.
+    # But that does not solve their storage, ie they need to be accessible at runtime from _somewhere_
+    # So expressions move the data into a Register.
+    # All expressions return registers
+
+    # But in the future (in the one that holds great things) we optimize those unneccesay moves away
+
+    def on_IntegerExpression expression
+      int = expression.value
+      reg = use_reg :Integer , int
+      add_code Register::LoadConstant.new( expression, int , reg )
+      return reg
+    end
+
+    def on_TrueExpression expression
+      reg = use_reg :Boolean
+      add_code Register::LoadConstant.new( expression, true , reg )
+      return reg
+    end
+
+    def on_FalseExpression expression
+      reg = use_reg :Boolean
+      add_code Register::LoadConstant.new( expression, false , reg )
+      return reg
+    end
+
+    def on_NilExpression expression
+      reg = use_reg :NilClass
+      add_code Register::LoadConstant.new( expression, nil , reg )
+      return reg
+    end
+
+    def on_StringExpression expression
+      value = Parfait.new_word expression.value.to_sym
+      reg = use_reg :Word
+      Register.machine.constants << value
+      add_code Register::LoadConstant.new( expression, value , reg )
+      return reg
+    end
+
+    def on_ClassExpression expression
+      name = expression.value
+      clazz = Parfait::Space.object_space.get_class_by_name! name
+      raise "No such class #{name}" unless clazz
+      reg = use_reg :MetaClass , clazz
+      add_code Register::LoadConstant.new( expression, clazz , reg )
+      return reg
+    end
+
+  end
+end

lib/typed/compiler/call_site.rb

@@ -0,0 +1,71 @@
+module Typed
+  Compiler.class_eval do
+
+    def on_CallSite statement
+      #puts statement
+#      name_s , arguments , receiver = *statement
+      raise "not inside method " unless @method
+      reset_regs
+      #move the new message (that we need to populate to make a call) to std register
+      new_message = Register.resolve_to_register(:new_message)
+      add_code Register.get_slot(statement, :message , :next_message , new_message )
+      if statement.receiver
+        me = process( statement.receiver  )
+      else
+        me = use_reg @method.for_class.name
+        add_code Register.get_slot(statement, :message , :receiver , me )
+      end
+      if(me.type == :MetaClass)
+        clazz = me.value.meta
+      else
+        # now we have to resolve the method name (+ receiver) into a callable method
+        clazz =  Register.machine.space.get_class_by_name(me.type)
+      end
+      # move our receiver there
+      add_code Register.set_slot( statement , me , :new_message , :receiver)
+
+      set_message_details(statement , statement.arguments)
+      set_arguments(statement.arguments)
+      ret = use_reg( :Integer ) #TODO real return type
+      do_call(clazz , statement)
+      # the effect of the method is that the NewMessage Return slot will be filled, return it
+      # but move it into a register too
+      add_code Register.get_slot(statement, :new_message , :return_value , ret )
+      ret
+    end
+
+    private
+    def do_call clazz , statement
+      name = statement.name
+      #puts "clazz #{clazz.name}"
+      raise "No such class" unless clazz
+      method = clazz.resolve_method(name)
+      #puts Register.machine.space.get_class_by_name(:Integer).method_names.to_a
+      raise "Method not implemented #{clazz.name}.#{name}" unless method
+      Register.issue_call( self , method )
+    end
+    def set_message_details name_s , arguments
+      name = name_s.name
+      # load method name and set to new message (for exceptions/debug)
+      name_tmp = use_reg(:Word)
+      add_code Register::LoadConstant.new(name_s, name , name_tmp)
+      add_code Register.set_slot( name_s , name_tmp , :new_message , :name)
+      # next arguments. first length then args
+      len_tmp = use_reg(:Integer , arguments.to_a.length )
+      add_code Register::LoadConstant.new(name_s, arguments.to_a.length , len_tmp)
+      add_code Register.set_slot( name_s , len_tmp , :new_message , :indexed_length)
+    end
+    def set_arguments arguments
+      # reset tmp regs for each and load result into new_message
+      arguments.to_a.each_with_index do |arg , i|
+        reset_regs
+        # processing should return the register with the value
+        val = process( arg)
+        raise "Not register #{val}" unless val.is_a?(Register::RegisterValue)
+        # which we load int the new_message at the argument's index (the one comes from c index)
+        set = Register.set_slot( arg , val , :new_message , Parfait::Message.get_indexed(i+1))
+        add_code set
+      end
+    end
+  end
+end

lib/typed/compiler/class_field.rb

@@ -0,0 +1,18 @@
+module Typed
+  Compiler.class_eval do
+
+    def on_ClassField statement
+      #puts statement.inspect
+      #type , name , value = *statement
+
+      for_class = @clazz
+      raise "no class" unless for_class
+      index = for_class.instance_type.variable_index(statement.name)
+      #raise "class field already defined:#{name} for class #{for_class.name}" if index
+      #puts "Define field #{name} on class #{for_class.name}"
+      for_class.instance_type.add_instance_variable( statement.name , statement.type )
+
+      return nil # statements don't reurn values, only expressions
+    end
+  end
+end

lib/typed/compiler/class_statement.rb

@@ -0,0 +1,14 @@
+module Typed
+  Compiler.class_eval do
+
+    def on_ClassStatement statement
+      #puts statement.inspect
+      raise "classes dont yet play babushka, get coding #{name}" if @clazz
+      @clazz = Parfait::Space.object_space.get_class_by_name! statement.name
+      #puts "Compiling class #{@clazz.name.inspect}"
+      statement_value = process(statement.statements).last
+      @clazz = nil
+      return statement_value
+    end
+  end
+end

lib/typed/compiler/collections.rb

@@ -0,0 +1,13 @@
+module Typed
+  Compiler.class_eval do
+
+#    attr_reader :values
+    def on_array statement, context
+    end
+#    attr_reader :key , :value
+    def on_association context
+    end
+    def on_hash context
+    end
+  end
+end

lib/typed/compiler/field_access.rb

@@ -0,0 +1,26 @@
+module Typed
+  Compiler.class_eval do
+
+    def on_FieldAccess statement
+#      receiver_ast , field_ast = *statement
+      receiver = process(statement.receiver)
+
+      clazz = Register.machine.space.get_class_by_name receiver.type
+
+      field_name = statement.field.name
+
+
+      index = clazz.instance_type.variable_index(field_name)
+      raise "field access, but no such field:#{field_name} for class #{clazz.name}" unless index
+      value = use_reg(clazz.instance_type.type_at(index))
+
+      add_code Register.get_slot(statement , receiver , index, value)
+
+      value
+    end
+
+    def on_receiver expression
+      process expression.first
+    end
+  end
+end

lib/typed/compiler/field_def.rb

@@ -0,0 +1,16 @@
+module Typed
+  Compiler.class_eval do
+    include AST::Sexp
+
+    def on_FieldDef statement
+      reset_regs # field_def is a statement, no return and all regs
+      #puts statement.inspect
+#      type , name , value = *statement
+      name_s = no_space( statement.name.value )
+      @method.ensure_local( name_s, statement.type ) unless( @method.has_arg(name_s))
+      # if there is a value assigned, process it as am assignemnt statement (kind of call on_assign)
+      process( Soml::Assignment.new(statement.name , statement.value )  ) if statement.value
+      return nil
+    end
+  end
+end

lib/typed/compiler/function_definition.rb

@@ -0,0 +1,43 @@
+module Typed
+  Compiler.class_eval do
+
+    def on_FunctionStatement  statement
+      #puts statement.inspect
+#      return_type , name , parameters, kids , receiver = *statement
+      name =  statement.name
+      raise "Already in method #{@method}" if @method
+
+      args = statement.parameters.collect do |p|
+        Parfait::Variable.new( *p )
+      end
+
+      class_method = nil
+      if(statement.receiver )
+        if( statement.receiver.first == :self)  #class method
+          class_method = @clazz
+          @clazz = @clazz.meta
+        else
+          raise "Not covered #{statement.receiver}"
+        end
+      end
+
+      @method = @clazz.get_instance_method( name )
+      if(@method)
+        #puts "Warning, redefining method #{name}" unless name == :main
+        #TODO check args / type compatibility
+        init_method
+      else
+        create_method_for(@clazz, name , args ).init_method
+      end
+      @method.source = statement
+      #puts "compile method #{@method.name}"
+
+      process(statement.statements)
+
+      @clazz = class_method if class_method
+      @method = nil
+      # function definition is a statement, does not return any value
+      return nil
+    end
+  end
+end

lib/typed/compiler/if_statement.rb

@@ -0,0 +1,35 @@
+module Typed
+  Compiler.class_eval do
+
+    # an if evaluates the condition and jumps to the true block if true
+    # so the else block is automatically after that.
+    # But then the else needs to jump over the true block unconditionally.
+    def on_IfStatement statement
+#      branch_type , condition , if_true , if_false = *statement
+#      condition = condition.first
+
+      reset_regs
+      process(statement.condition)
+
+      branch_class = Object.const_get "Register::Is#{statement.branch_type.capitalize}"
+      true_block = Register::Label.new(statement, "if_true")
+      add_code branch_class.new( statement.condition , true_block )
+
+      # compile the false block
+      reset_regs
+      process(statement.if_false) if statement.if_false.statements
+      merge = Register::Label.new(statement , "if_merge")
+      add_code Register::Branch.new(statement.if_false, merge )
+
+      # compile the true block
+      add_code true_block
+      reset_regs
+      process(statement.if_true)
+
+      #puts "compiled if: end"
+      add_code merge
+
+      nil # statements don't return anything
+    end
+  end
+end

lib/typed/compiler/name_expression.rb

@@ -0,0 +1,45 @@
+module Typed
+  Compiler.class_eval do
+
+      #    attr_reader  :name
+      # compiling name needs to check if it's a local variable
+      # or an argument
+      # whichever way this goes the result is stored in the return slot (as all compiles)
+      def on_NameExpression statement
+        name = statement.name
+        if( name == :self)
+          ret = use_reg @clazz.name
+          add_code Register.get_slot(statement , :message , :receiver , ret )
+          return ret
+        end
+        if(name == :space)
+          space = Parfait::Space.object_space
+          reg = use_reg :Space , space
+          add_code Register::LoadConstant.new( statement, space , reg )
+          return reg
+        end
+        if(name == :message)
+          reg = use_reg :Message
+          add_code Register::RegisterTransfer.new( statement, Register.message_reg , reg )
+          return reg
+        end
+        # either an argument, so it's stored in message
+        if( index = @method.has_arg(name))
+          ret = use_reg @method.arguments[index].value_type
+          add_code Register.get_slot(statement , :message , Parfait::Message.get_indexed(index), ret )
+          return ret
+        else # or a local so it is in the frame
+          index = @method.has_local( name )
+          if(index)
+            frame = use_reg :Frame
+            add_code Register.get_slot(statement , :message , :frame , frame )
+            ret = use_reg @method.locals[index].value_type
+            add_code Register.get_slot(statement , frame , Parfait::Frame.get_indexed(index), ret )
+            return ret
+          end
+        end
+        raise "must define variable '#{name}' before using it"
+      end
+
+  end #module
+end

lib/typed/compiler/operator_value.rb

@@ -0,0 +1,18 @@
+module Typed
+  Compiler.class_eval do
+
+    def on_OperatorExpression statement
+      #puts "operator #{statement.inspect}"
+#      operator , left_e , right_e = *statement
+      # left and right must be expressions. Expressions return a register when compiled
+      left_reg = process(statement.left_expression)
+      right_reg = process(statement.right_expression)
+      raise "Not register #{left_reg}" unless left_reg.is_a?(Register::RegisterValue)
+      raise "Not register #{right_reg}" unless right_reg.is_a?(Register::RegisterValue)
+      #puts "left #{left_reg}"
+      #puts "right #{right_reg}"
+      add_code Register::OperatorInstruction.new(statement,statement.operator,left_reg,right_reg)
+      return left_reg # though this has wrong value attached
+    end
+  end
+end

lib/typed/compiler/return_statement.rb

@@ -0,0 +1,10 @@
+module Typed
+  Compiler.class_eval do
+
+    def on_ReturnStatement statement
+      reg =  process(statement.return_value)
+      add_code Register.set_slot( statement, reg , :message , :return_value)
+      nil # statements don't return
+    end
+  end
+end

lib/typed/compiler/statement_list.rb

@@ -0,0 +1,9 @@
+module Typed
+  
+  Compiler.class_eval do
+
+    def on_statements statement
+      process_all(  statement.children  )
+    end
+  end
+end

lib/typed/compiler/while_statement.rb

@@ -0,0 +1,28 @@
+module Typed
+  Compiler.class_eval do
+
+    def on_WhileStatement statement
+      #puts statement.inspect
+      #branch_type , condition , statements = *statement
+
+      condition_label = Register::Label.new(statement.condition , "condition_label")
+      # unconditionally branch to the condition upon entering the loop
+      add_code Register::Branch.new(statement.condition,condition_label)
+
+      add_code  start = Register::Label.new(statement , "while_start" )
+      reset_regs
+      process(statement.statements)
+
+      # This is where the loop starts, though in subsequent iterations it's in the middle
+      add_code condition_label
+      reset_regs
+      process(statement.condition)
+
+      branch_class = Object.const_get "Register::Is#{statement.branch_type.capitalize}"
+      # this is where the while ends and both branches meet
+      add_code branch_class.new( statement.condition , start )
+
+      nil # statements don't return anything
+    end
+  end
+end