Web Framework Benchmark Description
To compare the performance of Snap to that of other web servers/frameworks, we ran some simple benchmarks using httperf: pong and file. In the pong benchmark, servers sent a 4-byte response of “PONG”. The file benchmark sent a 50KB PNG image file as its response. Our results and nodes are described below.
Most of the benchmarks were run with the following httperf command line options:
httperf --hog --num-conns 1000 --num-calls 1000 --burst-length 20 --port 3000 --rate 1000 --uri=/pong
All benchmarks were run on a dual quad-core Xeon E5345 2.33 GHz workstation with 8 gigs of RAM. In all these tests, we compiled Snap with “-threaded” and ran it with “+RTS -A4M -N4 -qg0 -qb -g1”. Using more than four threads caused the performance to decrease. This is certainly a good area for future improvement in Snap.
Disclaimer: We are not experts at optimizing these web frameworks. It is almost certain that the results for other frameworks are unfairly low. If you’re interested in the details, the code we used is in a github repository. We’re always willing to update our benchmarks with optimizations that don’t take an inordinate amount of effort to set up.
The pong benchmark is quite simple, but there was a large amount of variation in performance. Here is a chart of responses per second (higher numbers are better):
It should be noted that the Happstack benchmark is not multithreaded. We compiled Happstack with -threaded and when run with “+RTS -N4”, the test never finished. When we aborted httperf, it reported a rate of 12.6 requests/second. We also tried running it with with the same “+RTS -A4M -N4 -qg0 -qb -g1” runtime options that we used with Snap. This caused a stack overflow.
The file benchmark tests the ability to serve static content from disk. The results here were a little closer together than the pong results, but we still see the same general pattern.
In this test, we had more problems because Happstack does not support pipelining. So we switched to the following httperf line for Happstack:
httperf --hog --num-conns 100000 --num-calls 1 --burst-length 1 --port 8080 --rate 10000 --uri=/FiringGeometry.png
This means that Happstack’s results in this test are not really an apples-to-apples comparison with the other frameworks, but they’re still somewhat meaningful since they reflect a deficiency in the server.
See the raw test output for more detailed httperf output.