Measuring the Speed of Clouds

8 October 2008 — jaksa

I have started a project for benchmarking highly scalable technologies. My plan is to use cloud computing platforms and implement the same application using different stacks of technologies. The aspect I want to evaluate is performance when the number of nodes grows a lot.

Application specification
The application I’m using for benchmarking is a simplified model of Facebook.

Implementations
Currently I have 2 implementations:
Servlets + EJB3
Servlets + in memory Java

Benchmark
As benchmarking client I’m using JMeter.
The benchmark looks like this: a number of threads will start iterating over the following sequence

  1. load first page (redirection)
  2. load login page
  3. login
  4. load main page
  5. add post
  6. add friend
  7. logout

There are no pauses so in theory, this should test the capacity of the server.

Ideally there would be a very large number of threads (users) performing requests occasionally.

Configurations tested
I have tested everything on my machine before deploying on EC2. I have used JBoss and MySql for the EJB solution.
The performance I obtained was around 10 requests per second, which is dissapointing. Probably I have a bottleneck somewhere and need to configure things properly.
The In Memory Java implementation achieves more than 1000 requests per second, which is 100 times faster than using EJBs. However it occasionally throws ConcurrentModificationExceptions because I’m not synchronozing properly (fixing this will lower performance).

Then I deployed everything on one EC2 instance.
When I run the client on the outside I get around 100 requests per second, which may be due to my internet connection speed. But when I execute the client on another EC2 instance, I expect no communication bottlenecks. Instead I get 3 requests per second. Either there is a huge traffic jam on the EC2 internal network or I have misconfigured the benchmark client.
Running the benchmark on the EJB3 implementation shows a performance of 7 requests per second.

Next thing I need to do is some EJB3 tuning. I have the feeling I have not set up transactions correctly.
If you have any suggestion or wish your technology stack to be evaluated or perhaps want to contribute, please leave a comment.

Posted in Uncategorized. Tags: , , , , , , , , , , , , . 1 Comment »

One Response to “Measuring the Speed of Clouds”


Leave a Reply

«

»