What is the "tcp-backlog" in redis.conf

Question

I am confused by tcp-backlog in redis.conf:

 # TCP listen() backlog. # # In high requests-per-second environments you need an high backlog in order # to avoid slow clients connections issues. Note that the Linux kernel # will silently truncate it to the value of /proc/sys/net/core/somaxconn so # make sure to raise both the value of somaxconn and tcp_max_syn_backlog # in order to get the desired effect. tcp-backlog 511 

Is the tcp-backlog size of the "full connection queue" (the three-way handshake is completed, described here ) or the "incomplete connection queue"?

If this means "full connection queue", then why should I raise tcp_max_syn_backlog , which limits the size of an incomplete connection queue?

Answer 1

Is tcp-backlog the size of a "full connection queue" (the three-way handshake that is described here is completed) or an "incomplete connection queue"?

tcp-backlog - the size of the full connection queue . In fact, Redis passes this configuration as the second parameter to the listen(int s, int backlog) call.

@GuangshengZuo already got a good answer to this question. Therefore, I will focus on the other.

If this means "full connection queue", then why should I raise tcp_max_syn_backlog, which limits the size of the incomplete connection queue?

Quote from the document you mentioned:

An implementation uses two queues, a SYN queue (or an incomplete connection queue) and a receive queue (or a full connection queue). Connections in the SYN RECEIVED state are added to the SYN queue and then moved to the accept queue when their state changes to ESTABLISHED, that is, when the ACK packet is received in a three-way handshake. As the name implies, the accept call is then implemented simply to consume connections from the accept queue. In this case, the backlog argument to syscall listening determines the size of the receiving queue.

We see that the elements in the complete connection queue moved from the incomplete connection queue .

If you have a large somaxconn with a small tcp_max_syn_backlog , you may not have enough items to move to the complete connection queue , and the complete connection queue will never be populated. Many queries may have already been removed from the first queue before they have the opportunity to move to the second.

So only increasing the somaxconn value may NOT work. You have to pick them both up.

Answer 2

Tcp-backlog is the size of the receive queue or the full connection queue.

As indicated in the mentioned documents:

On Linux, things are different, as mentioned in the syscall listening man page:

The behavior of the backlog argument on TCP sockets has changed in Linux 2.2. It now determines the queue length for fully installed sockets awaiting acceptance, instead of the number of outstanding connection requests. The maximum queue length for incomplete sockets can be set using / proc / sys / net / ipv 4 / tcp_max_syn_backlog.

This means that current versions of Linux use the second option with two separate queues: a SYN queue with the size specified by the system parameter, and a receiving queue with the size specified by the application.

The Redis server uses the tcp-backlog configuration to specify the size of the receiving queue using the listen () function. And the size of the SYN queue is determined by the Linux administrator.

If this means "full connection queue", then why should I raise tcp_max_syn_backlog, which limits the size of the incomplete connection queue?

The tcp_max_syn_backlog enhancement aims to avoid problems with slow client connections . If there is a slow client that does a three-way handshake with the redis server, and that client slowly reads the response and sends the request, they will wait a long time for the SYN redis queue because they are slow.

And in some cases, the SYN queue will be full because these are inefficient clients. If the SYN queue is full, the redis server cannot accept new clients. So you need to raise tcp_max_syn_backlog to handle this.

Answer 3

Yesterday I read Chapter 6 in Redis In Action, where Joshua Carlton writes: “One of the drawbacks of publishing and publishing the Redis model is that the client must always connect to messages, disconnections can lead to client loss by messages, and older versions of Redis can become unsuitable, crash or be killed if there was a slow subscriber. "

Joshua Carleton then states that “Although push messaging can be useful, we encounter problems where customers cannot keep in touch all the time for one reason or another. To address this limitation, write two different messaging methods that are good can use PUBLISH / SUBSCRIBE as a substitute. First, start by exchanging messages with one recipient, since he has a lot in common with our queues with the first entry, the first exit. Later in this section, go to the method in which we can do some Message recipients: With multiple recipients, we can replace Redis PUBLISH and SUBSCRIBE when we need our messages to reach all recipients, even if they have been disconnected. "We are interested to see if it will be more efficient to replace Redis PUBLISH and SUBSCRIBE with Joshua Carlton Section 6.5 .2. Multi-user publication / replacement subscription instead of using the UDP protocol to detect and correct disconnect loss.

  Could we set a high tcp_max_syn_backlog in redis.conf to prevent either of 

Joshua Carlson's single-user messaging and multi-drop messaging methods with a load of 20,000 messages per second, where each message is 20 bytes?