initial version externalized from salama

README.md

@@ -1,2 +1,59 @@
-# salama-object-file
+### Reading the code
-Yaml like object writer
+
+Knowing what's going on while coding salama is not so easy: Hence the need to look at code dumps
+
+Hence the need for a code/object file format
+(remember an oo program is just objects, some data, some code, all objects)
+
+I started with yaml, which is nice in that it has a solid implementation, reads and writes,
+handles arbitrary objects, handles graphs and is a sort of readable text format.
+
+But the "sort of" started to get to me, because 
+
+- 1) it's way to verbose (long files, object groups over many pages) and 
+- 2) does not allow for (easy) ordering.
+
+To fix this i started on Sof, with an eye to expand it.
+
+The main starting goal was quite like yaml, but with
+
+- more text per line, specifically objects with simple attributes to have a constructor like syntax
+- also short versions of arrays and hashes
+- Shorter class names (no ruby/object or even ruby/struct stuff)
+- references at the most shallow level
+- an easy way to order attributes and specify attributes that should not be serialized
+
+### Salama Object File
+
+Ok, so we all heard about object files, it's the things compilers create so we don't have to have 
+huge compiles and can link them later.
+
+Much fewer know what they include, and that is not because they are not very useful,
+but rather very complicated.
+
+An object machine must off course have it's own object files, because:
+
+- otherwise we'd have to express the object machine in c (nischt gut)
+- we would be forced to read the source every time (slow)
+- we would have no language independant format
+
+And i was going to get there, juust not now. I mean i think it's a great idea to have many languages 
+compile and run  on the same object machine. 
+Not neccessarily my idea, but i haven't seen it pulled off. Not that i will.
+
+I just want to be able to read my compiled code!!
+
+And so this is a little start, just some outputter.
+
+#### Direction
+
+The way this is meant to go (planned for 2020+) was a salama core with only a sof parser
+(as that is soo much simpler).
+
+Then to_ruby for all the ast classes to be able to roundtrip ruby code.
+
+Then go to storing sof in git, rather than ruby.
+
+Then write a python/java parser and respective runtime conversion. Extracting common features.
+With the respective to_python on the ast's to roundtrip that too.
+Have to since by now we work on sof's. Etc . ..

lib/sof/all.rb

@@ -0,0 +1,39 @@
+require_relative "util"
+require_relative "node"
+require_relative "members"
+require_relative "volotile"
+require_relative "writer"
+require_relative "array"
+require_relative "hash"
+require_relative "occurence"
+
+Symbol.class_eval do
+  def to_sof()
+    ":#{to_s}"
+  end
+end
+TrueClass.class_eval do
+  def to_sof()
+    "true"
+  end
+end
+NilClass.class_eval do
+  def to_sof()
+    "nil"
+  end
+end
+FalseClass.class_eval do
+  def to_sof()
+    "false"
+  end
+end
+String.class_eval do
+  def to_sof()
+    "'" + self + "'"
+  end
+end
+Fixnum.class_eval do
+  def to_sof()
+    to_s
+  end
+end

lib/sof/array.rb

@@ -0,0 +1,51 @@
+module Sof
+  class ArrayNode < Node
+    def initialize ref
+      super(ref)
+      @children = []
+    end
+    attr_reader  :children
+    def add c
+      @children << c
+    end
+    def out io , level = 0
+      super
+      short = true
+      children.each do |c|
+        short = false unless c.is_a?(SimpleNode)
+      end
+      if(short and children.length < 7 )
+        short_out(io,level)
+      else
+        long_out(io , level)
+      end
+    end
+    
+    private
+    def short_out(io,level)
+      io.write("[")
+      @children.each_with_index do |child , i|
+        child.out(io , level + 1)
+        io.write ", " unless (i+1) == children.length
+      end
+      io.write("]")
+    end
+    def long_out io , level
+      indent = " " * level
+      @children.each_with_index do |child , i|
+        io.write "\n#{indent}" unless i == 0
+        io.write "-"
+        child.out(io , level + 1)
+      end
+    end
+  end
+end
+Array.class_eval do
+  def to_sof_node(writer , level , ref )
+    node = Sof::ArrayNode.new(ref)
+    each do |object|
+      node.add writer.to_sof_node( object , level + 1)
+    end
+    node
+  end
+end

lib/sof/hash.rb

@@ -0,0 +1,60 @@
+module Sof
+  class HashNode < Node
+    def initialize ref
+      super(ref)
+      @children = []
+    end
+    attr_reader  :children
+    def add key , val
+      @children << [key,val]
+    end
+    def out io , level = 0
+      super
+      short = true
+      children.each do |k,v|
+        short = false unless k.is_a?(SimpleNode)
+        short = false unless v.is_a?(SimpleNode)
+      end
+      if(short and children.length < 7 )
+        short_out(io,level)
+      else
+        long_out(io , level)
+      end
+    end
+    def short_out(io,level)
+      io.write("{")
+      children.each_with_index do |child , i|
+        key , val = child
+        key.out(io , level + 1)
+        io.write " => "
+        val.out(io , level + 1)
+        io.write ", " unless (i+1) == children.length
+      end
+      io.write("}")
+    end
+    def long_out io , level
+      indent = " " * level
+      children.each_with_index do |child , i|
+        key , val = child
+        io.write "\n#{indent}" unless i == 0
+        io.write "-"
+        key.out(io , level + 1)
+        io.write " => "
+        val.out(io , level + 1)
+      end
+    end
+  end
+end
+
+Hash.class_eval do
+  def to_sof_node(writer , level , ref)
+    node = Sof::HashNode.new(ref)
+    each do |key , object|
+      k = writer.to_sof_node( key ,level + 1)
+      v = writer.to_sof_node( object ,level +1)
+      node.add(k , v)
+    end
+    node
+  end
+end
+

lib/sof/members.rb

@@ -0,0 +1,49 @@
+module Sof
+  
+  class Members
+    include Util
+    
+    def initialize root
+      @root = root
+      @counter = 1
+      @objects = {}
+      @referenced = false
+      add(root , 0)
+    end
+    attr_reader :objects , :root , :referenced
+    
+    def add object , level
+      return if is_value?(object)
+      if( occurence = @objects[object.object_id] )
+        #puts "reset level #{level} at #{occurence.level}"
+        if occurence.level > level
+          occurence.level = level 
+        end
+        unless occurence.referenced
+          #puts "referencing #{@counter} , at level #{level}/#{occurence.level} "
+          occurence.set_reference(@counter)
+          @counter = @counter + 1
+        end
+        return
+      end
+      o = Occurence.new( object , level )
+      @objects[object.object_id] = o
+      attributes = attributes_for(object)
+      attributes.each do |a|
+        val = get_value( object , a)
+        add(val , level + 1)
+      end
+      if( object.is_a? Array )
+        object.each do |a|
+          add(a , level + 1)
+        end
+      end
+      if( object.is_a? Hash )
+        object.each do |a,b|
+          add(a , level + 1)
+          add(b , level + 1)
+        end
+      end
+    end
+  end
+end

lib/sof/node.rb

@@ -0,0 +1,58 @@
+# We transform objects into a tree of nodes
+
+module Sof
+  #abstract base class for nodes in the tree
+  # may be referenced (should be a simple name or number)
+  class Node
+    include Util
+    def initialize ref
+      @referenced = ref
+    end
+    # must be able to output to a stream
+    def out io ,level
+      io.write "&#{@referenced} " if @referenced
+    end
+    def as_string(level)
+      io = StringIO.new
+      out(io,level)
+      io.string
+    end
+    attr_reader :referenced
+  end
+  
+  class SimpleNode < Node
+    def initialize data , ref = nil
+      super(ref)
+      @data = data
+    end
+    attr_reader :data
+    def out io , level
+      super(io,level)
+      io.write(data)
+    end
+  end
+  
+  class ObjectNode < Node
+    def initialize data , ref
+      super(ref)
+      @data = data 
+      @children = []
+    end
+    attr_reader   :children ,  :data
+    def add k , v
+      @children << [k,v]
+    end
+    def out io , level = 0
+      super
+      io.write(@data)
+      indent = " " * (level + 1)
+      @children.each_with_index do |child , i|
+        k , v = child
+        io.write "\n#{indent}"
+        k.out(io , level + 2)
+        io.write " "
+        v.out(io , level + 2)
+      end
+    end
+  end
+end

lib/sof/occurence.rb

@@ -0,0 +1,16 @@
+module Sof
+
+  class Occurence
+    def initialize object , level
+      @object = object
+      @level = level
+      @referenced = nil      
+    end
+    def set_reference r
+      @referenced = r
+    end
+    attr_reader   :object , :referenced
+    attr_accessor :level
+  end
+
+end

lib/sof/util.rb

@@ -0,0 +1,25 @@
+module Sof
+  module Util
+    def is_value? o
+      return true if o == true
+      return true if o == false
+      return true if o == nil
+      return true if o.class == Fixnum
+      return true if o.class == Symbol
+      return true if o.class == String
+      return false
+    end
+
+    def get_value(object,name)
+      object.instance_variable_get "@#{name}".to_sym
+    end
+
+    def attributes_for object
+      Sof::Util.attributes(object)
+    end
+    def self.attributes( object )
+      atts = object.instance_variables.collect{|i| i.to_s[1..-1].to_sym } # chop of @
+      atts - Volotile.attributes(object.class)
+    end
+  end
+end

lib/sof/volotile.rb

@@ -0,0 +1,11 @@
+module Sof
+  class Volotile
+    @@mapping = {    }
+    def self.attributes clazz
+      @@mapping[clazz] || []
+    end
+    def self.add clazz , attributes
+      @@mapping[clazz] = attributes
+    end
+  end
+end

lib/sof/writer.rb

@@ -0,0 +1,60 @@
+module Sof
+  class Writer
+    include Util
+    def initialize members
+      @members = members
+    end
+
+    def write
+      node = to_sof_node(@members.root , 0)
+      io = StringIO.new
+      node.out( io , 0 )
+      io.string
+    end
+
+    def to_sof_node(object , level)
+      if is_value?(object)
+        return SimpleNode.new(object.to_sof())
+      end
+      occurence = @members.objects[object.object_id]
+      raise "no object #{object}" unless occurence
+      if(level > occurence.level )
+        #puts "ref #{occurence.referenced} level #{level} at #{occurence.level}"
+        return SimpleNode.new("*#{occurence.referenced}")
+      end
+      ref = occurence.referenced
+      if(object.respond_to? :to_sof_node) #mainly meant for arrays and hashes
+        object.to_sof_node(self , level , ref )
+      else
+        object_sof_node(object , level , ref )
+      end
+    end
+
+    def object_sof_node( object , level , ref)
+      if( object.is_a? Class )
+        return SimpleNode.new( object.name , ref )
+      end
+      head = object.class.name + "("
+      atts = {}
+      attributes_for(object).each() do |a| 
+        val = get_value(object , a)
+        next if val.nil?
+        atts[a] =  to_sof_node(val , level + 1)
+      end
+      immediate , extended = atts.partition {|a,val| val.is_a?(SimpleNode) }
+      head += immediate.collect {|a,val| "#{a.to_sof()} => #{val.as_string(level)}"}.join(", ") + ")"
+      return SimpleNode.new(head) if( ref.nil? and extended.empty? and head.length < 30 )
+      node = ObjectNode.new(head , ref)
+      extended.each do |a , val|
+        node.add( to_sof_node(a,level + 1) , val )
+      end
+      node
+    end
+
+    def self.write object
+      writer = Writer.new(Members.new(object) )
+      writer.write
+    end
+    
+  end
+end

test/sof.rb

@@ -0,0 +1,106 @@
+require_relative "helper"
+require "yaml"
+
+class ObjectWithAttributes
+  def initialize
+    @name = "some name"
+    @number = 1234
+  end
+  attr_accessor :extra
+end
+OBJECT_STRING = "ObjectWithAttributes(:name => 'some name', :number => 1234)"
+
+class BasicSof < MiniTest::Test
+  def check should
+    same = (should == @out)
+    puts "Shouldda\n#{@out}" unless same
+    assert_equal should , @out
+  end
+  def test_true
+    @out = Sof::Writer.write(true)
+    check "true" 
+  end
+  def test_num
+    @out = Sof::Writer.write(124)
+    check  "124" 
+  end
+  def test_simple_object
+    @out = Sof::Writer.write(ObjectWithAttributes.new)
+    check "#{OBJECT_STRING}" 
+  end
+  def test_object_extra_array
+    object = ObjectWithAttributes.new
+    object.extra = [:sym , 123]
+    @out = Sof::Writer.write(object)
+    check "#{OBJECT_STRING}\n :extra [:sym, 123]" 
+  end
+  def test_simple_array
+    @out = Sof::Writer.write([true, 1234])
+    check "[true, 1234]" 
+  end
+  def test_array_object
+    @out = Sof::Writer.write([true, 1234 , ObjectWithAttributes.new])
+    check "-true\n-1234\n-#{OBJECT_STRING}" 
+  end
+  def test_array_array
+    @out = Sof::Writer.write([true, 1 , [true , 12 ]])
+    check "-true\n-1\n-[true, 12]" 
+  end
+  def test_array_array_reverse
+    @out = Sof::Writer.write([ [true , 12 ], true, 1])
+    check "-[true, 12]\n-true\n-1" 
+  end
+  def test_array_array_array
+    @out = Sof::Writer.write([true, 1 , [true , 12 , [true , 123 ]]])
+    check "-true\n-1\n--true\n -12\n -[true, 123]" 
+  end
+  def test_array_array_object
+    @out = Sof::Writer.write([true, 1 , [true , 12 , ObjectWithAttributes.new]])
+    check "-true\n-1\n--true\n -12\n -#{OBJECT_STRING}" 
+  end
+  def test_simple_hash
+    @out = Sof::Writer.write({ one: 1 , tru: true })
+    check "{:one => 1, :tru => true}" 
+  end
+  def test_hash_object
+    @out = Sof::Writer.write({ one: 1 , two: ObjectWithAttributes.new })
+    check "-:one => 1\n-:two => #{OBJECT_STRING}" 
+  end
+  def test_array_hash
+    @out = Sof::Writer.write([true, 1 , { one: 1 , tru: true }])
+    check "-true\n-1\n-{:one => 1, :tru => true}" 
+  end
+  def test_hash_array
+    @out = Sof::Writer.write({ one: [1 , ObjectWithAttributes.new] , two: true })
+    check "-:one => -1\n -#{OBJECT_STRING}\n-:two => true" 
+  end
+  def test_array_recursive
+    ar = [true, 1 ]
+    ar << ar
+    @out = Sof::Writer.write(ar)
+    check "&1 [true, 1, *1]" 
+  end
+  def test_object_recursive
+    object = ObjectWithAttributes.new
+    object.extra = object
+    @out = Sof::Writer.write(object)
+    check "&1 ObjectWithAttributes(:name => 'some name', :number => 1234, :extra => *1)" 
+  end
+  def test_object_inline
+    object = ObjectWithAttributes.new
+    object.extra = Object.new
+    @out = Sof::Writer.write(object)
+    check "ObjectWithAttributes(:name => 'some name', :number => 1234, :extra => Object())" 
+  end
+  def test_class
+    @out = Sof::Writer.write(ObjectWithAttributes)
+    check "ObjectWithAttributes"
+  end
+  def test_class_ref
+    object = ObjectWithAttributes.new
+    object.extra = ObjectWithAttributes
+    ar = [object , ObjectWithAttributes]
+    @out = Sof::Writer.write(ar)
+    check "-ObjectWithAttributes(:name => 'some name', :number => 1234, :extra => *1)\n-&1 ObjectWithAttributes"
+  end
+end