Benchmarking Memcached and Redis clients

As some of you may know, I’m crazy about speed. So when I saw that people were happily using Predis as their choice of PHP client for Redis, I was a bit confused. 

Why use a client written in PHP for something that should be ‘fast’ like Redis?

That kind of defeats the purpose - unless you don’t really care about response times and scalability. 

I could understand using it if there were no alternatives such as PhpRedis, or if you wanted to add some sort of proprietary layer that you cannot add on top of a C extension.

Don’t get me wrong, if you have a valid reason to use the extension, then more power to you. I know both packages have contributors who have put tons of sweat into getting them to where they are now.

What I Analyzed

  • The performance difference piqued my interest. I wanted to find out just how much performance users are sacrificing by choosing one implementation over another.
  • Since Redis is usually found in the same stack as Memcached (which I will touch upon later in this post), I included benchmarks for Memcached that demonstrates how it stacks up against Redis while performing identical operations.
  • Also, for the fans of IgBinary, I covered both native and client level implementations of it too.

Benchmarking Framework

To automate and define a common benchmarking strategy for Memcache, Memcached, Predis and PhpRedis I decided to write a small framework that automatically runs a set of tests that client requests.

You can find it here: https://github.com/AlekseyKorzun/benchmarker-nosql-php

Hardware Set-Up

Tests were performed on VirtualBox with 2 processors and 1024MB of RAM allocated to it.

The host machine is Intel i7 2600K with 16 GB of RAM.

Software Set-Up

  • Ubuntu 12.04.2
  • PHP 5.3.10-1ubuntu3.6
  • Apache 2.2.22 
  • Redis server 2.2.12 
  • Memcached 1.4.13
  • Libmemcached 1.0.16
  • Memcache client 3.0.6
  • Memcached client 2.1.0
  • Redis client 2.2.2
  • Predis client 0.8
  • IgBinary extension 1.1.2-dev

Common Benchmarks

In a regular get/set benchmark every client except Predis performs on equal level. Memcached edges out PhpRedis and Memcache by ~1 r/s on average at ~83 r/s while Predis is trailing the pack around ~12 r/s.

This test is pretty hard core; if you look at the benchmarking framework we are testing get/set with a pretty huge object.

Both Memcache and Predis fail to complete the test and begin to fail once concurrency goes up to 100.

Redis and Memcached are pretty much even at 50 and 100 concurrent requests but once we go up to 150 requests Redis starts to trail Memcached by ~ 10.5 r/s which indicates that it prone to fail before Memcached gives in.

Pretty even performance for everybody except Predis, which is about 6x slower than the rest of the clients.

Again, every client except Predis performs at about the same level. Predis seems to average out at 11-12 r/s in every test as it seems that this is a limitation before it even starts to hand off requests to Redis daemon.

Predis failed to complete this test while every other client passed it with 77 r/s on average with PhpRedis leading the pack with a small margin.

List & Set Benchmarks

Predis once again, fails to go above 13 r/s while PhpRedis destroys it five way till Sunday.

IgBinary Benchmarks

Both Redis and Memcached clients support IgBinary, so obviously I had to test them since I’m a huge IgBinary fan.

There are two ways you can use IgBinary, natively (as in let client handle it) or directly in PHP (serialize object prior to passing it to client using IgBinary extension).

I tested both approaches, let’s start with native:

Memcached is performing extremely well with IgBinary but sees a minor performance drop over regular serializer  as we reach 150 concurrent connections.

PhpRedis sees a good jump in performance as well but it starts to even out as we increase connections and unfortunately the client was unable to complete the final 150 concurrent connections test.

And check out results when we serialize objects directly in PHP using igbinary extension:

While PhpRedis still fails to complete the final test both clients seem to process more requests when we handle serialization ourselves.

Let’s compare native IgBinary tests with regular PHP seriazer:

Keeping in mind that you can squeeze more juice out if you use IgBinary directly in your code I will have to say IgBinary is a winner even if it shows a minor drop as we reach 150 connections.

Memcached with IgBinary is a clear winner.

Conclusion

Provided benchmarks might not reflect real world (tm) performance so take everything you read below with a grain of salt.

I do not recommend using Predis if you care about performance, period. It’s a massive bottle neck and if you are not using features unique to Redis over traditional RDBMS you are already running then I would not even bother introducing Redis in your stack if you are going to use Predis as a client.

The Memcached client is faster than PhpRedis and will keep your site up (even if its slow) for a bit longer before starting to fail.

The Memcache client is not a snail by any means, while it failed the large keys test at 150 concurrent connections it still put up a really good fight and performed quite well.

If you do not need the features Memcached has to offer and are not scaling application that cached large objects you should be fine.

If you can, use IgBinary. It’s does make a big difference.

Words of Wisdom

  • If your web site can only serve X number of requests per second under load, when you do not factor in data calls, having a NoSQL client that can do XYZ requests per second will not magicly solve any of your problems.
  • Not everybody can afford to maintain Redis and Memcached servers in their stack. If you need data retention and/or features Redis has to offer over Memcached and can live with the performance level Redis offer then you do not need Memcached.

    If you care about performance, I advise to store ‘unimportant’ or ‘larger’ data in Memcached. For example template views/layouts or collections of models. 
  • Always test ‘theories’ and see if they apply to your case. Evaluate your options and strategy prior to committing to anything. 

You can view spreadsheet of benchmark data here: https://docs.google.com/spreadsheet/pub?key=0AhePUdRMAppIdHIzd2d3YU9oVE55MnctaGc3NTVvcVE&single=true&gid=0&output=html

8 notes

Show

  1. czytelnia reblogged this from alekseykorzun
  2. alekseykorzun posted this