obench.m: an Octave-based benchmark obench.m is a simple Octave benchmark which performs the following timed tests:

• Nested FOR loops
• Integration/function calls
• Random matrix/FFT calculation
• Simple matrix operations
• Complex matrix operations

The timings are added to provide an approximate indicator of how fast the system can run Octave. It's a long test (over an hour on an AMD Athlon 2100) which requires about 512MB to run properly. If you wish to try it simply download obench.m and type the following:

     octave -q obench.m

That's it! If you send me the output of the above command (which includes timings plus some info about your computer) I will add it to the table below. I've only tested obench.m with Linux and Windows (under Cygwin), but I think it should be platform-agnostic except for the gathering of system information, (if the program cannot gather the system info please just send the timings and relevant specs e.g. CPU/RAM/etc). I'm releasing the obench.m to the public domain and without any warranties whatsoever, but please only send me the results provided by the original version you downloaded from here.

You can reach me at marco@reimeika.ca. Thank you in advance for any contributions!

Benchmarks

As you will see the benchmark is sensitive to the version of Octave and underlying OS, but even for a given Octave version there may have been different compile-time options (the use of fftw, for example). However, it is also sometimes interesting to see how different versions of Octave compare on a given machine. Below is a table comparing my "standard test" setup (Octave 2.1.71, see below) with Octave 2.1.73 and 2.9.4 running under Knoppix 5.0.1 (all times in seconds):

It's clear that version number, underlying OS and compile-time optimizations all play important roles regarding performance, and hence comparisons between different systems should be done with utmost care (if at all). It may be worthwhile to note that copying the entire live CD to memory (using [toram]) did not make a significant difference (which should be expected if I made the timing calculations properly).

Under certain standard conditions, however, the variations between runs are within 5% on a given system, usually less (so, although I'm writing timings to two decimal places only the two first digits are actually significant). To compare various computers the proper procedure is to boot the machine off Quantian Linux and then run the benchmark. For historical comparisons the same versions of Quantian and obench.m should be used (although I do not forsee changing the timed obench.m benchmarks ever again).

My "standard test" is running [octave -q obench.m] on a freshly booted instance of Quantian 0.7.9.1 (the version number is important). Those values are noted in red and have an associated "performance index". Timings for other systems are shown, but are not really comparable. If you send me your benchmark results I will add them to the table below, but system performance will be noted only if you run it as specified above (in which case please make that clear!)

The following graph summarizes the results, while the table below it shows the details (the order of the bars on the graph corresponds to the order of the entries in the table, only standard system performance is shown):

A quick note regarding Quantian

To burn the Quantian DVD I had to use the following command (as root):

nice -20 growisofs -use-the-force-luke=dao -dvd-compat -Z /dev/dvd=Quantian_0.7.9.1.iso -speed=2

Upon booting up the machine I encountered the following error after pressing [Enter] at the boot prompt:

Kernel panic -- not syncing : VFS : unable to mount root fs on unknown-block(1,0)

This seems to be a bug in the Quantian .iso. Fortunately the fix is simple: instead of [Enter] type [knoppix lang=us] and then [Enter] at the boot prompt and all is well again. This will probably get fixed in future ISO images (which will not be considered as a standard system).

Last updated: 16/March/2008
Return to the [index]