Re: TCP SACK issue, hung connection, tcpdump included

On Sun, Jul 29, 2007 at 12:28:04PM +0300, Ilpo Järvinen wrote:
(...)
Limitation for 48 byte segments? You have to be kidding... :-) But yes,
it seems that one of the directions is dropping packets for some reason
though I would not assume MTU limitation... Or did you mean some other
segment?

No, I was talking about the 1448 bytes segments. But in fact I don't
believe it much because the SACKs are always retransmitted just afterwards.

Ah, but it's ACKed correctly right below it...:

[...snip...]
09:21:39.490740 IP SERVER.ssh > CLIENT.50727: P 18200:18464(264) ack 2991
win 2728 <nop,nop,timestamp 7692910 800001727>
09:21:39.490775 IP CLIENT.50727 > SERVER.ssh: . ack 18464 win 378
<nop,nop,timestamp 800001755 7692910>
09:21:39.860245 IP SERVER.ssh > CLIENT.50727: . 12408:13856(1448) ack 2991
win 2728 <nop,nop,timestamp 7693293 800001749>

...segment below snd_una arrived => snd_una remains 18464, receiver
generates a duplicate ACK:

09:21:39.860302 IP CLIENT.50727 > SERVER.ssh: . ack 18464 win 378
<nop,nop,timestamp 800001847 7692910,nop,nop,sack sack 1 {12408:13856} >

The cumulative ACK field of it covers _everything_ below 18464 (i.e., it
ACKs them), including the 1448 bytes in 12408:13856... In addition, the
SACK block is DSACK information [RFC2883] telling explicitly the address
of the received duplicate block. However, if this ACK doesn't reach the
SERVER TCP, RTO is triggered and the first not yet cumulatively ACKed
segment is retransmitted (I guess cumulative ACKs up to 12408 arrived
without problems to the SERVER):

Oh yes, you're damn right. I did not notice that the ACK was higher than
the SACK, I'm more used to read traces with absolute rather than relative
seq/acks.

So I agree, it is this ACK which is lost between client and server,
reinforcing the supposition about the location of the capture (client side).

[...snip...]

BTW, some information are missing. It would have been better if the trace
had been read with tcpdump -Svv. We would have got seq numbers and ttl.
Also, we do not know if there's a firewall between both sides. Sometimes,
some IDS identify attacks in crypted traffic and kill connections. It
might have been the case here, with the connection closed one way on an
intermediate firewall.

Yeah, firewall or some other issue, I'd say it's quite unlikely a bug in
TCP because behavior to both directions indicate client -> sender
blackhole independently of each other...

It would also be possible that something stupid between both ends simply
drops packets with the SACK option set. Also something which sometimes
happen is that some firewalls automatically translate sequence numbers
but not necessarily SACK values, which could pretty well lead to this
packet being received but ignored on the server side.

I'm pretty sure that the same trace taken on the server side will reveal
the reason for the problem.

Willy

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Relevant Pages

  • Re: TCP SACK issue, hung connection, tcpdump included
    ... SACK in this scenario when tcp_dsack set to 1. ... when the tcpdump shows the ack covering it has been seen. ... The ACK got discarded by the SERVER ... We'll just have to figure out why it's discarding it. ...
    (Linux-Kernel)
  • Re: Multiple new packets with ACK or ACK,FIN bits set
    ... either ACK or ACK, FIN bits set. ... The packets are addressed to (mostly ... since this traffic seen here originates from US based server farms. ... At one stage I would send an email to 'whois' block owner informing ...
    (comp.os.linux.security)
  • problem with select() and sockets...
    ... sections and sends them to the server, ... ACK through the gateway, when the client sees the ack it sends the ... when no packets get dropped, ...
    (comp.lang.cpp)
  • problem with select() and sockets...
    ... sections and sends them to the server, ... ACK through the gateway, when the client sees the ack it sends the ... when no packets get dropped, ...
    (comp.unix.programmer)
  • Re: Problem with TCP connection not opening properly
    ... >>I have a problem that is happening fairly regularly between a client ... This ACK gets lost, so is never seen by the server. ... That packets get lost occasionally is not realy realy an issue here. ...
    (comp.os.linux.networking)