rubyx/test/bench/results.md

# Benchmarks

hello - output hello world to measure kernel calls
add   - run integer adds by linear fibonacci of 20
call  - exercise calling by recursive fibonacci of 10
noop  - a baseline that does nothing

All programs (apart from noop) run 100k times to minimize startup impact.

C was linked statically as dynamic linked influences times.
Output was sent to /dev/null, so as to measure the calling and not the terminal.
Also output was unbuffered, because that is what rubyx implements.

# Results

Results were measured by a ruby script. Mean and variance was measured until variance was low,
always under one percent. Noop showed that program startup is a factor, so all programs loop
to 100k.

The machine was a virtual arm (qemu) run on a acer swift 5 (i5 8265 3.9GHz), performance roughly equivalent to a raspberry pi.
Results (in ms) should be seen as relative, not absolute.


language  |  noop   |  hello   |  add   |  call        
c         |    45   |  3480    |   72   |   591
go        |    53   |  4000    |   64   |   624
rubyx     |    47   |  1660    |  800   |  2000
ruby      |  1570   |  8240    | 2700   | 12370
mruby     |   108   | 11210    | 1580   | 23400
thats most of the benchmarking strange about the puts in soml, hard to see where a factor of 10 would come from 2015-11-20 22:51:58 +01:00			`# Benchmarks`

updating benchmarks on the new machine, consistent not rubyx yet 2019-07-23 19:14:28 +02:00			`hello - output hello world to measure kernel calls`
			`add - run integer adds by linear fibonacci of 20`
			`call - exercise calling by recursive fibonacci of 10`
			`noop - a baseline that does nothing`
thats most of the benchmarking strange about the puts in soml, hard to see where a factor of 10 would come from 2015-11-20 22:51:58 +01:00
fixing test to pass and extract own kind of same tests to bench bench tests will need some loop, but not as large as c 2019-07-25 20:25:15 +02:00			`All programs (apart from noop) run 100k times to minimize startup impact.`
thats most of the benchmarking strange about the puts in soml, hard to see where a factor of 10 would come from 2015-11-20 22:51:58 +01:00
redid the old benchmarks With #26 out of the way, was able to get meaningful rubyx benchmarks. Meaning loops large enough for the exec time to go significantly over the noop. Did mruby too and as expected got much lower noop 2019-07-30 12:55:29 +02:00			`C was linked statically as dynamic linked influences times.`
			`Output was sent to /dev/null, so as to measure the calling and not the terminal.`
			`Also output was unbuffered, because that is what rubyx implements.`

thats most of the benchmarking strange about the puts in soml, hard to see where a factor of 10 would come from 2015-11-20 22:51:58 +01:00			`# Results`

			`Results were measured by a ruby script. Mean and variance was measured until variance was low,`
fixing test to pass and extract own kind of same tests to bench bench tests will need some loop, but not as large as c 2019-07-25 20:25:15 +02:00			`always under one percent. Noop showed that program startup is a factor, so all programs loop`
			`to 100k.`
thats most of the benchmarking strange about the puts in soml, hard to see where a factor of 10 would come from 2015-11-20 22:51:58 +01:00
updating benchmarks on the new machine, consistent not rubyx yet 2019-07-23 19:14:28 +02:00			`The machine was a virtual arm (qemu) run on a acer swift 5 (i5 8265 3.9GHz), performance roughly equivalent to a raspberry pi.`
redid the old benchmarks With #26 out of the way, was able to get meaningful rubyx benchmarks. Meaning loops large enough for the exec time to go significantly over the noop. Did mruby too and as expected got much lower noop 2019-07-30 12:55:29 +02:00			`Results (in ms) should be seen as relative, not absolute.`
add ruby bench programs and numbers 2015-11-24 14:33:16 +01:00

redid the old benchmarks With #26 out of the way, was able to get meaningful rubyx benchmarks. Meaning loops large enough for the exec time to go significantly over the noop. Did mruby too and as expected got much lower noop 2019-07-30 12:55:29 +02:00			`language \| noop \| hello \| add \| call`
			`c \| 45 \| 3480 \| 72 \| 591`
			`go \| 53 \| 4000 \| 64 \| 624`
			`rubyx \| 47 \| 1660 \| 800 \| 2000`
			`ruby \| 1570 \| 8240 \| 2700 \| 12370`
			`mruby \| 108 \| 11210 \| 1580 \| 23400`