Write a ring benchmark. Create N processes in a ring. Send a message round the ring M times so that a total of N * M messages get sent. Time how long this takes for different values of N and M.
Write a similar program in some other programming language you are familiar with. Compare the results. Write a blog, and publish the results on the internet!
Following the example of Muharem Hrnjadovic, I decided to pit Erlang against Stackless. Of course, since my own multitask package provides a Queue class similar to Stackless channels, I couldn't resist throwing it into the ring as well.
The scripts I used to test the three different systems are ring.erl, ring_stackless.py, and ring_multitask.py. These were run under Erlang/OTP R11B-5, Stackless 3.1b3 (Python 2.5.1), and Python 2.5.1 with multitask revision 5286, respectively. The tests were conducted on a quad-core Mac Pro (2 x 2.66 GHz Dual-Core Intel Xeon, 3 GB RAM) running Mac OS X 10.4.10.
The following table shows, for various values of N and M, the total CPU time (in seconds) taken by each script, as well as the ratio of Stackless's and multitask's times to Erlang:
|N||M||Erlang (s)||Stackless (s)||multitask (s)||Stackless/Erlang||multitask/Erlang||multitask/Stackless|
So it looks like Stackless is generally about 12 times slower than Erlang, with multitask being another 40 to 60 times slower still. This suggests two conclusions, neither of which seems very surprising:
- Although the message passing in both Erlang and Stackless is (AFAIK) implemented in C, Erlang's implementation is far more efficient. This is good, since message passing is the basis of Erlang's concurrency model.
- Stackless's C implementation gives it a huge performance advantage over multitask, which is pure Python.
Clearly, multitask's showing here is not very impressive. However, I'm not too upset about it. I think the domain where multitask is really advantageous is asynchronous networking, not in-process message passing. Once the latency of the network enters into the equation, I think that 500-to-800-times performance hit over Erlang will drop substantially. Perhaps my next task should be an Erlang vs. multitask vs. Twisted comparison?