Socket Option SO_ATTACH_REUSEPORT_CBPF

Question

I am trying to use a port from two applications and each of them receives a packet from a different set of IP addresses. For this, I use the options SO_REUSEPORT and SO_ATTACH_REUSEPORT_CBPF. My code is as follows:

parentfd = socket(AF_INET, SOCK_STREAM, 0); if (parentfd < 0) error( "ERROR opening socket"); struct sock_filter code[]={ { 0x28, 0, 0, 0x0000000c }, { 0x15, 0, 3, 0x00000800 }, { 0x20, 0, 0, 0x0000001a }, { 0x15, 2, 0, 0xc0a8ff01 }, { 0x6, 0, 0, 0x00000000 }, { 0x6, 0, 0, 0x00040000 }, { 0x6, 0, 0, 0x00000001 }, }; struct sock_fprog bpf = { .len = ARRAY_SIZE(code), .filter = code, }; if (setsockopt(parentfd, SOL_SOCKET, SO_REUSEPORT, (const void *)&optval,sizeof(optval))) error("ERROR setting SO_REUSEPORT"); if (setsockopt(parentfd, SOL_SOCKET, SO_ATTACH_REUSEPORT_CBPF, (const void *)&bpf, sizeof(bpf))) error("ERROR setting SO_ATTACH_REUSEPORT_CBPF); 

I also have another process that listens on the same port using only the SO_REUSEPORT flag. On a machine with IP 192.168.255.1 I run echo 1234 | ncat 192.168.255.150 1234 echo 1234 | ncat 192.168.255.150 1234 . Based on my filter, I would expect all traffic from this IP address to be received by the second process. However, they are all accepted first. When I change the filter to simple:

struct sock_filter code[]={ { 0x6, 0, 0, 0x00000001 }, };

It works as expected, and all packets are accepted by the second process. Any idea why this might happen?

Answer 1

I found out what the problem is. The filter applies to all packets, even to TCP acknowledgment packets. In addition, the base pointer points to the first byte of the packet payload, not the headers. Therefore, when he performs

 ldh[12] 

it goes out of the packet limits (the SYN packet has 0 bytes of payload), and by default returns 0.

Answer 2

Inoperative code:

 l0: ldh [12] /* read EtherType (2 bytes), which is found at offset 12 (decimal) */ l1: jeq #0x800, l2, l5 /* if EtherType == `0x800` (IPv4), jump to `l2`, otherwise jump to `l5` */ l2: ld [26] /* read source IP address (4 bytes) */ l3: jeq #0xc0a8ff01, l6, l4 /* if source IP address == 192.168.255.1, jump to l6 (return 1), else jump to l4 (return 0) */ l4: ret #0 l5: ret #0x40000 l6: ret #0x1 

Work code:

 ret #0x1 

socket (7) says:

BPF should return an index between 0 and N-1 representing the socket that the packet should receive (where N is the number of sockets in the group). If the BPF program returns an invalid index, the socket selection will revert to the usual SO_REUSEPORT mechanisms.

On my machine, tcpdump -i lo -ddd 'src host 192.168.255.1' is created

 10 40 0 0 12 21 0 2 2048 32 0 0 26 21 4 5 3232300801 21 1 0 2054 21 0 3 32821 32 0 0 28 21 0 1 3232300801 6 0 0 262144 6 0 0 0 

What

 l0: ldh [12] l1: jeq #0x800, l2, l4 l2: ld [26] l3: jeq #0xc0a8ff01, l8, l9 l4: jeq #0x806, l6, l5 l5: jeq #0x8035, l6, l9 l6: ld [28] l7: jeq #0xc0a8ff01, l8, l9 l8: ret #0x40000 l9: ret #0 

I do not see anything that clearly does not match your code.

Have you tried running tcpdump on the server? Perhaps you forgot to remove the additional IP address on the client, or somewhere forgotten SNAT rule?

What kernel version are you using? Could you post a minimal C application that reproduces the problem?