Erlang vs. Stackless vs. multitask: The Ring Benchmark Showdown

The Challenge

From Section 8.11 of Joe Armstrong's Programming Erlang:

Write a ring benchmark. Create N processes in a ring. Send a message round the ring M times so that a total of N * M messages get sent. Time how long this takes for different values of N and M.

Write a similar program in some other programming language you are familiar with. Compare the results. Write a blog, and publish the results on the internet!

The Contenders

Following the example of Muharem Hrnjadovic, I decided to pit Erlang against Stackless. Of course, since my own multitask package provides a Queue class similar to Stackless channels, I couldn't resist throwing it into the ring as well.

The Tests

The scripts I used to test the three different systems are ring.erl, ring_stackless.py, and ring_multitask.py. These were run under Erlang/OTP R11B-5, Stackless 3.1b3 (Python 2.5.1), and Python 2.5.1 with multitask revision 5286, respectively. The tests were conducted on a quad-core Mac Pro (2 x 2.66 GHz Dual-Core Intel Xeon, 3 GB RAM) running Mac OS X 10.4.10.

The Results

The following table shows, for various values of N and M, the total CPU time (in seconds) taken by each script, as well as the ratio of Stackless's and multitask's times to Erlang:

NM Erlang (s)Stackless (s)multitask (s) Stackless/Erlangmultitask/Erlang multitask/Stackless
100 100 0.0 0.0166667 1.06667 N/A N/A 64.0000719999
100 1000 0.02 0.166667 10.6167 8.33335 530.835 63.7000725999
100 10000 0.14 1.7 106.183 12.1428571429 758.45 62.4605882353
1000 100 0.02 0.183333 10.8333 9.16665 541.665 59.0908347106
1000 1000 0.14 1.68333 107.95 12.0237857143 771.071428571 64.1288398591
1000 10000 1.39 16.7 1078.48 12.0143884892 775.884892086 64.5796407186
10000 100 0.22 2.58333 109.933 11.7424090909 499.695454545 42.5547645868
10000 1000 2.05 24.9167 1095.2 12.1544878049 534.243902439 43.9544562482
10000 10000 20.48 248.533 10948.8 12.1354003906 534.609375 44.0537071536

So it looks like Stackless is generally about 12 times slower than Erlang, with multitask being another 40 to 60 times slower still. This suggests two conclusions, neither of which seems very surprising:

  1. Although the message passing in both Erlang and Stackless is (AFAIK) implemented in C, Erlang's implementation is far more efficient. This is good, since message passing is the basis of Erlang's concurrency model.
  2. Stackless's C implementation gives it a huge performance advantage over multitask, which is pure Python.

Clearly, multitask's showing here is not very impressive. However, I'm not too upset about it. I think the domain where multitask is really advantageous is asynchronous networking, not in-process message passing. Once the latency of the network enters into the equation, I think that 500-to-800-times performance hit over Erlang will drop substantially. Perhaps my next task should be an Erlang vs. multitask vs. Twisted comparison?

Tags: erlang, multitask, python
Tue, 18 Sep 2007 18:40 UTC

super and classmethod

Here's a nice example of why you have to use super in order for classmethod to work correctly in derived classes.

The code:

class Base(object):
    @classmethod
    def method(cls):
        print 'In base:   ', cls.__name__
        print

class DerivedWithoutSuper(Base):
    @classmethod
    def method(cls):
        print 'In derived:', cls.__name__
        Base.method()

class DerivedWithSuper(Base):
    @classmethod
    def method(cls):
        print 'In derived:', cls.__name__
        super(DerivedWithSuper, cls).method()

if __name__ == '__main__':
    DerivedWithoutSuper().method()
    DerivedWithSuper().method()

The output:

$ python super_classmethod.py
In derived: DerivedWithoutSuper
In base:    Base

In derived: DerivedWithSuper
In base:    DerivedWithSuper

Tags: python
Tue, 18 Sep 2007 16:05 UTC

tokens help considered unhelpful

$ tokens -help
Usage: tokens [-help]

So -help exists solely to advertise its existence. That seems sort of Darwinian.

Tags: humor
Fri, 14 Sep 2007 16:25 UTC

« Next    Previous »