Friday, April 08, 2011

Pyramid vs NodeJS vs Gevent

Summary: NodeJS wins.

Test Program
ab -n 10000 -c 5 http://localhost/


Gevent Code
from gevent import wsgi

class WebServer(object):
def application(self, environ, start_response):
start_response("200 OK", [])
return ["Hello world!"]

if __name__ == "__main__":
app = WebServer()
wsgi.WSGIServer(('', 8888), app.application, backlog=1024).serve_forever()


Pyramid Code
from pyramid.config import Configurator
from pyramid.response import Response
from paste.httpserver import serve

def hello_world(request):
return Response('Hello world!')

if __name__ == '__main__':
config = Configurator()
config.add_view(hello_world)
app = config.make_wsgi_app()
serve(app, host='0.0.0.0')


NodeJS Code
var http = require('http');
http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/plain'});
res.end('Hello World\n');
}).listen(8124, "127.0.0.1");


Gevent Results
Time taken for tests:   3.255 seconds
Complete requests: 10000
Failed requests: 0
Write errors: 0
Requests per second: 3072.53


Pyramid Results
Concurrency Level:      5
Time taken for tests: 14.650 seconds
Complete requests: 10000
Failed requests: 0
Write errors: 0
Requests per second: 682.57



NodeJS Results
Concurrency Level:      5
Time taken for tests: 2.953 seconds
Complete requests: 10000
Failed requests: 0
Write errors: 0
Requests per second: 3386.80

23 comments:

cammm said...

I'd be interested to see the same test done with gevent instead of pyramid.

http://www.gevent.org

Unknown said...

Pretty pointless.

Pyramid provides a hell of a lot more then NodeJS last I looked and Hello World tests are basically worthless to begin with.

Simon Wittber said...

No, this is not pointless.

I'm working on an MMO which requires some web services. I've written version of these services in Python and NodeJS, each with less than 500 lines of code. The entirety of the code is about providing functionality over HTTP, I don't care about templates, configurators, html blah blah.

Don't be so insular. The fact that your use-case does not fit my use-case does not make this kind of benchmark pointless.

Unknown said...

"I've written version of these services in Python and NodeJS, each with less than 500 lines of code."

If you ab test your own webservices - is there still a 5:1 advantage over pyramid? Could you share a real world benchmark?

cammm said...

A little bit slower? Not by much though.

You should implement the full 500 line server in both nodejs and gevent then run them side by side. Nobody does proper bench tests these days of production systems. The fact that you have a small enough but otherwise very real use case puts you in a unique position to try it.

Simon Wittber said...

OK, here are results from my services.

Gevent Requests per second: 2177.98

NodeJS Requests per second: 3051.92

Pyramid Not going to bother!

Unknown said...

"Pyramid Not going to bother!"

Interesting - why did you post that comparison in the first place?

Simon Wittber said...

Well, I've been out of the Python web engine sphere for almost a year, and I saw a friend talking about Python/Pyramid. I decided I should make sure that I'm still using the right tool. Because my use-case is so light, a simple hello world test would give me that information, and I decided to publish the results.

I'd already written my service using Gevent and NodeJS, hence I could run the seconds set of tests quickly. I'm not going to bother doing the same with Pyramid, as it will obviously be slower!

mongodb_is_webscale said...

But is it web scale? MongoDB is web scale. You should use MongoDB

boothead said...

How about running pyramid on top of gevent or gunicorn? That should speed things up considerably.

Simon Wittber said...

@boothead That would just needlessly complicate things for me. I don't need anything but HTTP.

Unknown said...

"I don't need anything but HTTP."

Pyramid uses paste.httpserver as its default httpserver - so in fact you are testing paste.httpserver vs. Gevent vs. NodeJS.

Anonymous said...

By no means I expect Bottle to be extremely fast - it will probably deliver half of NodeJS' Rps - but I like its design. Here's a snippet, can we have it added to the comparison? Thanks


from bottle import route, run

@route('/')
def index():
return 'Hello World'

run()

Anonymous said...

On my 64-bit Ubuntu 10.10 machine:

gevent: 5600 r/s
njs: 5400 r/s
pyramid + gevent: 4400 r/s
pyramid + paste: 700 r/s

Oh my, gevent wins, and it's not using any kind of JIT.

I expect the more interesting shakedown will occur with c100k and whether you can consistently get low latency.

Simon Wittber said...

Very nice! I'm not sure how to generate that many connections to create such a test, any ideas?

Anonymous said...

Simon,
perhaps you could try out meinheld web server (http://www.meinheld.org). It's been very fast for our web services.

- Michael

Anonymous said...

Oops, bad link, it's just http://meinheld.org :)

- Michael

peterbe said...

On my Macbook pro:

Tornado:
Requests per second: 2848.71 [#/sec] (mean)

NodeJS:
Requests per second: 6922.16 [#/sec] (mean)

Anonymous said...

Change this:

#wsgi.WSGIServer(('', 8888), app.application, backlog=1024).serve_forever()

to:

wsgi.WSGIServer(('', 8888), app.application, log=None).serve_forever()

* From 4000 req/s to 6200 req/s on my machine just disabling logging (node.js not logging in the example shown). Node.js: 5800 req/s.

Anonymous said...

Yes, do what this guy said. Turn off logging on gevent. Gevent prints out one string to sdtout by default. This slows down the "hello" test.

Turning it off will increase performance much more in this test.

I think, you should do it, retest and post new results.

Manto said...

You probably want to monkey.patch_all() gevent to get the full benefit of greenlets.

Unknown said...

lel, I can't understand what pyramid paster in that company does. ;) I think more smart add tornado's results or Eventlet cause noone setup production envirement on paster, Pyramid just create application and don't tell you how to serve it. I use nginx+uwsgi and it's works great.

Victor said...

Thanks for the post. I did my own benchmarking based on your examples and some comments.

Popular Posts