Connect to Redis from Azure throwing Intermittent Exceptions

Question

I have a Windows Azure server that is running my C # application. It extends to 4 medium instances, and I use Redis for my L2 caching. The application handles a pretty decent amount of traffic (about 300,000 page views per day). I use BookSleeve to connect redis, and after the application gets up and builds, it will start throwing SocketExceptions from BookSleeve about four times per minute. The exception is:

Exception type: System.Net.Sockets.SocketException Exception message: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 

It seems to only happen when I read something from the server:

 using (var connection = ConnectionGateway.GetReadConnection()) { var task = connection.Hashes.GetString(App.RedisDatabase, CacheKeys.My_KEY); var result = connection.Wait(task); } 

My GetReadConnection is configured as follows:

  public static RedisConnection GetReadConnection() { RedisConnection conn = getNewConnection(); try { var openAsync = conn.Open(); conn.Wait(openAsync); return conn; } catch (SocketException ex) { throw new Exception(RedisConnectionFailed, ex); } } 

Now all my records use one connection, as the author describes, so this only happens on readings that require the use of connection.Wait (). Everything seems to be working fine. The entry uses code similar to Maintaining an Open Redis Connection Using BookSleeve

I tried changing the Redis server timeouts to account for Azure load balancing without success. I tried setting timeout 30 and setting timeout 0 as described here: Redis connection errors when using the Booklee Redis client in Azure VM

Any help would be really appreciated.

Answer 1

Looking at the code, the first thing that happens is that you create a connection for each read operation. This is not required and can adversely affect .net and redis: BookSleeve is a multiplexer - it is designed to handle many requests simultaneously on one connection. When you call Wait, only the waiting thread is blocked - BookSleeve internals will process other threads, etc.

In addition, creating a connection requires quite a bit of overhead: in addition to the cost of creating a new TCP connection, BookSleeve needs to talk to the redis server to find the key configuration information necessary for proper operation, so it is always recommended to reuse one common connection or a small one number of shared connections.