Sources of delay when sending / receiving tcp / udp packets on linux

Question

What are the sources of delay when sending / receiving TCP / UDP packets in Linux 2.6?

I want to know the sources of latency in ping pong latency tests.

There are pretty good documents about ethernet latency, but they only cover the delay sources in the wiring and the switch (and rather fluently, only for a specific switch).

What processing steps does the package perform?

Papers with in-depth conventional ping delay analysis (icmp) will also be helpful.

I rely on the community :)

Answer 1

, , linux.

http://www.cs.unh.edu/cnrg/people/gherrin/linux-net.html

, , ​​ .

.

Answer 2

: perf probe perf script.

: . , TCP ping-pong ( netperf).

:

:

, ( , TCP). :

perf probe --add sock_sendmsg='sock_sendmsg'
perf probe --add sock_sendmsg_exit='sock_sendmsg%return'
perf probe --add inet_sendmsg='inet_sendmsg'
perf probe --add inet_sendmsg_exit='inet_sendmsg%return'
perf probe --add tcp_sendmsg_exit='tcp_sendmsg%return'
perf probe --add tcp_sendmsg='tcp_sendmsg'
perf probe --add tcp_sendmsg_locked='tcp_sendmsg_locked'
perf probe --add tcp_sendmsg_locked_exit='tcp_sendmsg_locked%return'
perf probe --add sk_stream_alloc_skb='sk_stream_alloc_skb'
perf probe --add sk_stream_alloc_skb_exit='sk_stream_alloc_skb%return'
perf probe --add tcp_push_exit='tcp_push%return'
perf probe --add tcp_push='tcp_push'
perf probe --add tcp_send_mss='tcp_send_mss'
perf probe --add tcp_send_mss_exit='tcp_send_mss%return'
perf probe --add __tcp_push_pending_frames='__tcp_push_pending_frames'
perf probe --add __tcp_push_pending_frames_exit='__tcp_push_pending_frames%return'
perf probe --add tcp_write_xmit_exit='tcp_write_xmit%return'
perf probe --add tcp_transmit_skb_exit='tcp_transmit_skb%return'
perf probe --add tcp_transmit_skb='tcp_transmit_skb'

, :

perf record -e probe:* -aR taskset -c 7 netperf -t TCP_RR -l 5 -T 7,7

script :

perf script -F time,event --ns

(1 ):

525987.403094082:                   probe:sock_sendmsg:
525987.403095586:                   probe:inet_sendmsg:
525987.403096192:                    probe:tcp_sendmsg:
525987.403098203:             probe:tcp_sendmsg_locked:
525987.403099296:                   probe:tcp_send_mss:
525987.403100002:              probe:tcp_send_mss_exit:
525987.403100740:            probe:sk_stream_alloc_skb:
525987.403101697:       probe:sk_stream_alloc_skb_exit:
525987.403103079:                       probe:tcp_push:
525987.403104284:      probe:__tcp_push_pending_frames:
525987.403105575:               probe:tcp_transmit_skb:
525987.403110178:               probe:tcp_transmit_skb:
525987.403111640:          probe:tcp_transmit_skb_exit:
525987.403112876:          probe:tcp_transmit_skb_exit:
525987.403114351:            probe:tcp_write_xmit_exit:
525987.403114768: probe:__tcp_push_pending_frames_exit:
525987.403115191:                  probe:tcp_push_exit:
525987.403115718:        probe:tcp_sendmsg_locked_exit:
525987.403117576:               probe:tcp_sendmsg_exit:
525987.403118082:              probe:inet_sendmsg_exit:
525987.403118568:              probe:sock_sendmsg_exit:

, . , , sock_sendmsg() inet_sendmsg() 1504 (ns) 1,504 () . , , sk_stream_alloc_skb 957 . ( sock_sendmsg ) ~ 24.5us. , , netperf, - .

.

, .

P.S. 4.14, 2.6. , , .