Skip to main content

TCP State Machine and What ss Shows

TL;DR
  • Every TCP socket is one row in a per-connection finite-state machine in the kernel. ss and netstat print that row's state, so you can read a host's connection health without a packet capture.
  • Eleven states (RFC 9293 section 3.3.2). Active closer (sent the first FIN): ESTABLISHED, FIN_WAIT_1, FIN_WAIT_2, TIME_WAIT, CLOSED. Passive closer: ESTABLISHED, CLOSE_WAIT, LAST_ACK, CLOSED. CLOSING appears only on a simultaneous close.
  • A pile of CLOSE_WAIT is your application's bug: the peer closed, your code never called close(). TIME_WAIT is normal (the owner page is port exhaustion and TIME_WAIT).
  • FIN_WAIT_2 stuck means the peer never sent its FIN. tcp_fin_timeout (60 s) only reaps orphaned sockets. SYN_RECV pile-up means a SYN flood or a full backlog.
  • On a LISTEN socket, Recv-Q is the accept queue length and Send-Q is its maximum. On a connected socket they are unread and unacknowledged bytes.
  • ESTABLISHED does not prove the peer is alive: an idle TCP connection sends no packets at all.

Mental model​

ANALOGYEveryday

Think of a phone call. Whoever hangs up first has to stay on the line for a moment to be sure the other person heard "goodbye". The person who hears the other one finish does not hang up automatically: they might still have something to say, and until they put the phone down the call is half open. So the roles at the end decide who waits where. The one who ended the call waits at the very end (TIME_WAIT); the one who was told "I am done" sits in CLOSE_WAIT until they choose to hang up.

Where it breaks: a phone call has no half-close and no states you can list for every call on a switchboard. TCP does both: either side can say "I will send no more" while still listening, and the kernel keeps an inspectable state for every connection.

ANALOGYFor engineers

It is an enum column in a table, one row per socket, advanced by a small state machine. Two kinds of events move a row: local calls (listen(), connect(), close()) and arriving segments (SYN, ACK, FIN, RST), plus a few timers. ss -tan is SELECT state, recv_q, send_q, local, peer FROM sockets, and counting by state is a GROUP BY. A leak shows up as one value of that enum growing without bound, like jobs stuck in a PROCESSING status.

Where it breaks: you only ever see your own row. The peer's state is inferred from segments, never read, so two hosts can disagree about a connection (one ESTABLISHED, the other with no record at all) and nothing in the table says so.

How it works​

TCP connection explains the segments of the handshake and the close. This page is about the bookkeeping underneath: the state the kernel keeps for each socket between those segments, and how to read it.

A socket moves because of one of three things:

  • An application call: listen() creates a LISTEN socket, connect() sends a SYN, close() sends a FIN.
  • A segment arriving: a SYN, an ACK of something sent, a FIN, or a RST.
  • A timer: most importantly the TIME_WAIT timer, which ends the connection after 2 * MSL.
active pathpassive pathCLOSEDSYN_SENTLISTENSYN_RCVDESTABLISHEDFIN_WAIT_1CLOSE_WAITCLOSINGFIN_WAIT_2LAST_ACKTIME_WAITCLOSEDpassive open: listen()active open: connect()snd SYNrcv SYNsnd SYN+ACKrcv SYN+ACK, snd ACKrcv ACKclose(): snd FINrcv FIN, snd ACKrcv ACK of FINrcv FIN, snd ACKrcv FIN, snd ACKrcv ACK of FIN2*MSL timeoutclose(): snd FINrcv ACK of FIN
The main path through the eleven states of RFC 9293 section 3.3.2, drawn as columns. Left, solid: the active path (connect(), then the side that sends the first FIN). Right, dashed: the passive path (listen(), then the side that receives the first FIN). Each arrow is labelled event, then action. Omitted for clarity: simultaneous open, the FIN+ACK shortcut from FIN_WAIT_1 to TIME_WAIT, and the RST and timeout edges that drop a half-open socket back to CLOSED.

The two columns are two roles, not two machines. For the open, the active role is the client (connect()) and the passive role is the server (listen()). For the close, the roles are decided by timing: the side whose application calls close() first is the active closer, whichever end that is. Both paths meet in ESTABLISHED and both finish in CLOSED.

Click events on either endpoint, or run a scenario, and watch both sockets move through the states:

current state (token names the side)state visited earlierevent not valid in this state
CLOSEDclientserver
LISTEN
SYN_SENT
SYN_RCVD
ESTABLISHED
FIN_WAIT_1
FIN_WAIT_2
CLOSE_WAIT
CLOSING
LAST_ACK
TIME_WAIT
client: CLOSED
server: CLOSED
$ ss -tan
State       Recv-Q Send-Q Local Address:Port   Peer Address:Port
(no socket)   (client)
(no socket)   (server)

Linux holds TIME_WAIT for a fixed 60 s; the RFC says 2 * MSL (MSL = 2 minutes). An RST from any synchronised state goes straight to CLOSED. In this view each endpoint is one socket row; a real server also keeps its LISTEN row.

Click an event on either endpoint, or pick a scenario. Both endpoints start in CLOSED.
event 0 / 5
Active closer (sends the first FIN)Passive closer (receives it)FINACKFINACKthe passive side may keep sending; CLOSE_WAIT lasts until its app calls close()TIME_WAIT: 2*MSL, 60 s on Linux (see the owner page)app calls close()rcv ACKrcv FIN2*MSL timer firesESTABLISHEDFIN_WAIT_1FIN_WAIT_2TIME_WAITCLOSEDrcv FINapp calls close()rcv ACKESTABLISHEDCLOSE_WAITLAST_ACKCLOSED
One connection being closed, one lifeline per host, time flowing downward. The side that sends the first FIN walks FIN_WAIT_1, FIN_WAIT_2, TIME_WAIT. The side that receives it walks CLOSE_WAIT, LAST_ACK. CLOSE_WAIT lasts until the passive side's application calls close(), which is why a leak there is an application bug.

Worked example: one close, state by state​

A client and a server are ESTABLISHED. The client's application finishes its request and calls close().

StepSegmentClient state afterServer state afterWhat the server's app has done
0noneESTABLISHEDESTABLISHEDserving the connection
1client sends FINFIN_WAIT_1CLOSE_WAIT (on receipt)nothing yet: it will see a read return 0 (end of file)
2server sends ACKFIN_WAIT_2 (on receipt)CLOSE_WAITmay still write; has not called close()
3server sends FINTIME_WAIT (on receipt)LAST_ACKcalled close()
4client sends ACKTIME_WAITCLOSED (on receipt)socket gone
5none; timer fires after 2 * MSLCLOSEDCLOSEDnone

The segment and sequence-number mechanics, and the three-segment variant where steps 2 and 3 merge into one FIN+ACK, are in TCP connection. Note what the table shows about time: steps 1 to 4 normally take about one round trip each, but step 2 to step 3 lasts as long as the server's application chooses to wait. That one gap is where most state-related incidents live.

Deep dive​

The eleven states​

StateWho is in itNormal durationA pile-up means
CLOSEDNobody: it is fictional, the state of having no TCB (RFC 9293 section 3.3.2). ss never lists itn/an/a
LISTENA server socket waiting for SYNsThe life of the server processNot a pile-up; read its Recv-Q and Send-Q (below)
SYN_SENTA client that sent a SYNAbout one RTT; up to 127 s with Linux's default SYN retries (131 s on recent kernels) (tcp-connection)The peer or a firewall is not answering. Many at once toward one address means an unreachable destination
SYN_RCVDA server that sent a SYN-ACK and awaits the final ACK (SYN-RECV in ss)About one RTT; up to 63 s of SYN-ACK retriesA SYN flood, a full backlog, or a client whose ACK never arrives
ESTABLISHEDBoth ends, for the whole data phaseAs long as the application keeps it openIdle ones are not a fault by themselves; see the idle section below
FIN_WAIT_1The active closer, FIN sent, not yet ACKedAbout one RTTThe peer or network is not acknowledging; usually accompanied by unacknowledged Send-Q
FIN_WAIT_2The active closer, FIN ACKed, waiting for the peer's FINUntil the peer's app calls close()The peer never closes. Orphaned ones are reaped by tcp_fin_timeout; the rest wait forever
CLOSE_WAITThe passive closer: peer's FIN received and ACKedMilliseconds, until the app calls close()Your application is not calling close(): a leak
CLOSINGBoth sides sent a FIN at once (simultaneous close)About one RTTRare to see even one; a stuck pile points at lost ACKs
LAST_ACKThe passive closer, own FIN sent, awaiting the ACKAbout one RTTACKs from the peer are not arriving; the kernel retransmits the FIN
TIME_WAITThe active closer, after the final ACK60 s on LinuxUsually normal; a client-side pile toward one destination is the exhaustion incident. Owner: port exhaustion

That is eleven rows, with CLOSED listed once. Many references list it twice, as a start state and an end state; they are the same state, the absence of a connection.

Names differ by tool. The RFC says SYN-RECEIVED, ss prints SYN-RECV, netstat prints SYN_RECV; ss also uses hyphens (CLOSE-WAIT, TIME-WAIT, ESTAB) where netstat uses underscores and ESTABLISHED. The filter keyword accepted by ss ... state is the lower-case hyphenated form, for example close-wait (ss(8)).

Reading Recv-Q and Send-Q, including on LISTEN​

The two numeric columns mean different things depending on the socket's state (sock_diag(7)):

Socket stateRecv-QSend-QWhat a large value means
ESTABLISHED and other connected statesBytes received and not yet read by the applicationBytes sent and not yet acknowledged by the peerLarge Recv-Q: your application is slow or stuck reading. Large Send-Q: the peer or the path is slow, or the peer's window is closed
LISTENConnections that completed the handshake and wait for accept(): the accept queue lengthThe maximum length of that queue: min(listen() backlog, net.core.somaxconn)Recv-Q near Send-Q: the application is not calling accept() fast enough, and new connections are about to be dropped

One wording trap: it is easy to read the LISTEN numbers as the SYN backlog. The kernel's ss backend reports the accept queue there (sock_diag(7) calls them the pending connections and the backlog size; the fields sk_ack_backlog and sk_max_ack_backlog, filled in net/ipv4/tcp_diag.c), the queue of already-established connections. The SYN queue for half-open connections is a separate structure with no column in ss. TCP connection describes both queues; Cloudflare's write-up shows them on a live host.

ss -tanStateRecv-QSend-QLocal Address:PortPeer Address:PortLISTEN31280.0.0.0:4430.0.0.0:*ESTAB0010.0.0.5:443203.0.113.9:51422CLOSE-WAIT1010.0.0.5:443203.0.113.44:60310TIME-WAIT0010.0.0.5:52344198.51.100.7:443FIN-WAIT-20010.0.0.5:52350198.51.100.7:443SYN-RECV0010.0.0.5:443192.0.2.77:401123 queued for accept(), at most 128healthy: nothing unread, nothing unackedpeer sent FIN, our app never called close()we closed first; held 60 s on Linuxour FIN was ACKed, peer has not sent its FINSYN seen, SYN-ACK sent, final ACK pending
Illustrative output for one host, one row per interesting state, with documentation-range addresses. Read the state first, then the two queue columns: they mean different things on the LISTEN row than on the connected rows.

CLOSE_WAIT pile-up: the application forgot close()​

CLOSE_WAIT means the kernel has done its part: the peer's FIN arrived, the kernel ACKed it, and the application can read end-of-file. Nothing in the state diagram moves a socket out of CLOSE_WAIT except the application calling close(). So a count that keeps growing is a descriptor leak in your process, not a network fault: a handler that returns on an error path without closing, a pool that drops connections without closing them, an HTTP client that never closes a response body.

Because each leaked socket holds a file descriptor, the leak ends in failure: once the process reaches its descriptor limit (ulimit -n, the per-process RLIMIT_NOFILE), accept() and open() fail with EMFILE ("too many open files", see accept(2)) and the service stops taking new connections while the machine looks idle. Find the owner with ss -tanp or lsof (see Hands-on), then fix the missing close().

FIN_WAIT_2: waiting for a peer that does not close​

The active closer in FIN_WAIT_2 is waiting for the peer's FIN, which only comes when the peer's application closes. Two cases, with different outcomes (ip-sysctl, tcp_fin_timeout):

  • Orphaned socket (the application has already called close() and no file descriptor refers to it any more): Linux aborts it after tcp_fin_timeout seconds, default 60. A system-wide cap, tcp_max_orphans, resets orphans immediately if exceeded and logs a warning.
  • Not orphaned (the application used shutdown(SHUT_WR) and still holds the socket, a legitimate half-close): FIN_WAIT_2 is "a perfectly valid receive only state" and there is no timeout. If the peer never closes, it stays forever.

So a long-lived pile of FIN_WAIT_2 is rarely a kernel problem. Either the peer (often a crashed or buggy client behind a load balancer) is not sending its FIN, or your own application half-closed and never finished. Note the 60 s is easy to confuse with the 60 s TIME_WAIT: they happen to match, but they are different constants, covered on the port exhaustion page.

SYN_RECV pile-up and the accept queue​

A server in SYN_RCVD is holding a slot for a half-open connection and retransmitting its SYN-ACK. A few at any instant are normal (roughly connection rate times RTT). A large, growing count has two usual causes:

  • A SYN flood: spoofed sources never send the final ACK. The mitigation is SYN cookies; the mechanism is in TCP connection. With cookies active the server stores nothing, so SYN-RECV can look calm during a flood and the evidence moves to nstat counters instead.
  • A full accept queue: the application is not calling accept() fast enough. When that queue is full, Linux drops incoming SYNs and, by default (tcp_abort_on_overflow=0), the final ACKs, without sending a RST. The client sees nothing but a retransmitting SYN, a silent timeout. The kernel counts these in TcpExtListenOverflows and TcpExtListenDrops (SNMP counters). Raising the backlog treats the symptom; the slow accept() loop is the cause.

ESTABLISHED does not mean alive​

A connection with nothing to send emits no packets. If the peer crashes, loses power, or a NAT or load balancer silently forgets the flow, your side still shows ESTABLISHED until it next writes (and gets a RST, or retransmits until tcp_retries2) or a keepalive probe fails. Counting ESTABLISHED sockets tells you how many connections your host believes it has, not how many peers are healthy. The mechanisms (half-open connections, keepalive at 7,200 s, TCP_USER_TIMEOUT, application heartbeats) are in TCP connection; the fix for idle timeouts at a load balancer is a keepalive interval shorter than the balancer's timeout.

close() with unread data, and SO_LINGER​

Normally close() returns immediately and the kernel sends a FIN after any queued data. If the receive buffer still holds unread bytes when the application closes, the stack sends a RST instead, to signal that data was lost: RFC 1122 section 4.2.2.13 says a TCP SHOULD do this, and RFC 2525 section 2.17 explains why. The peer sees connection reset by peer and can lose data it had already been sent. State-wise the socket skips FIN_WAIT_* and TIME_WAIT entirely and goes straight to CLOSED.

SO_LINGER (socket(7)) changes what close() does: with a non-zero timeout, close() blocks until queued data is sent or the timeout expires; with a zero timeout it sends a RST immediately and discards pending data, which also skips TIME_WAIT. That is a legitimate way to abort a connection and a dangerous way to "fix" a TIME_WAIT pile, because the data in flight is lost and the guard TIME_WAIT provides is gone.

Textbook vs real world​

The textbook saysProduction doesWhy
The state diagram is clean: every close ends FIN_WAIT_2 then TIME_WAITOrphaned FIN_WAIT_2 sockets are aborted after tcp_fin_timeout (60 s), and past tcp_max_orphans they are reset at onceThe RFC diagram has no timer for FIN_WAIT_2; a real kernel cannot hold memory for peers that never answer
TIME_WAIT lasts 2 * MSL, MSL being 2 minutesLinux hard-codes 60 s; you cannot tune itThe owner page port exhaustion and TIME_WAIT covers the constant and the knobs
Tuning TIME_WAIT: enable tcp_tw_recycletcp_tw_recycle was removed in Linux 4.12; tcp_tw_reuse helps outgoing connections onlyRecycle dropped SYNs from clients behind NAT because their timestamps are per host
LISTEN has no queues in the diagram; the handshake just completesThere are two queues, and when the accept queue is full SYNs and final ACKs are dropped silentlyA refused connection would tell the client; a silent drop makes it retry, which looks like latency
close() sends a FINclose() with unread receive data sends a RSTSilently discarding data would corrupt the peer's view, so the stack aborts instead
ESTABLISHED means the connection is upIt means this host has not heard otherwise; idle peers can be gone for hoursTCP has no heartbeat and keepalive is off by default
CLOSED is a state a connection returns toCLOSED is fictional: there is no socket, so no tool can show itRFC 9293 defines it as the state with no TCB

In production​

  • Cloudflare documents how its Linux hosts handle SYNs: the SYN queue, the accept queue, what ss shows for a listener, when the kernel switches on cookies, and where silent drops happen (SYN packet handling in the wild). It is the best primary source for the LISTEN-queue story on this page.
  • bcc's tcplife and tcpstates trace TCP sessions with eBPF instead of polling ss: tcplife prints each connection's duration and bytes when it closes, tcpstates prints every state transition with the time spent in the previous state (tcplife examples, tcpstates examples). They were written by Brendan Gregg and are the standard answer to "which connection sat in which state for how long" when a snapshot from ss is not enough.
  • AWS Network Load Balancer documents an idle timeout of 350 seconds for TCP flows (NLB documentation). A connection idle longer than that can be forgotten by the balancer while both endpoints still show ESTABLISHED, the exact failure the idle section above describes.
  • Go's net/http requires the caller to close the response body so the connection returns to the pool or is released; the documentation says so explicitly (http.Client.Do). Code that skips this leaks descriptors and is a textbook source of growing socket counts.
  • I did not find primary-source Datadog or Lyft postmortems on CLOSE_WAIT leaks and have left them out rather than cite them from memory.

History and evolution​

HISTORY
  • 1981 RFC 793 defines TCP and its state diagram: a connection is a state machine with the states and transitions drawn on this page, plus the 2 * MSL wait in TIME_WAIT.
  • 1989 RFC 1122 (Host Requirements) clarifies behaviour the diagram leaves open, including what a TCP does when the application closes with data still unread (send a RST, section 4.2.2.13).
  • 2014 Vincent Bernat documents that tcp_tw_recycle silently drops connections from clients behind NAT (Coping with the TCP TIME-WAIT state on busy Linux servers); advice flips from "enable it" to "never enable it".
  • 2017 Linux 4.12 removes tcp_tw_recycle entirely.
  • 2022 RFC 9293 restates TCP in one document, obsoleting RFC 793, and keeps the same eleven-state machine, noting that CLOSED is fictional.
  • Now ss reads sockets from the kernel over the netlink sock_diag interface (sock_diag(7)), and eBPF tools trace state changes live.
  • Next QUIC (RFC 9000) runs over UDP and has its own connection states inside user space, so ss shows a UDP socket and none of the TCP states on this page.

Hands-on​

Linux: ss is in the iproute2 package. macOS has no ss; use netstat -an -p tcp (states use underscores, for example CLOSE_WAIT) and lsof.

Look at everything, then count by state.

ss -tan # all TCP sockets, numeric
ss -Htan | awk '{print $1}' | sort | uniq -c | sort -rn
41 ESTAB
12 TIME-WAIT
3 CLOSE-WAIT
1 LISTEN

-H drops the header line, so the count does not include a bogus State entry. Run it twice a minute apart: a number that only ever grows is the signal. Counts are illustrative.

Read the summary.

ss -s
Total: 191
TCP: 57 (estab 41, closed 4, orphaned 0, timewait 12)

The orphaned figure is sockets whose application has closed them but the kernel is still finishing; compare it against tcp_max_orphans. Exact layout varies by iproute2 version; the output above is abbreviated.

Read a listener's queues.

ss -ltn
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 4096 0.0.0.0:443 0.0.0.0:*

Recv-Q is the current accept queue, Send-Q its maximum. A Recv-Q that sits near Send-Q means accept() is too slow.

List one state and find the owner.

ss -tanp state close-wait
lsof -nP -iTCP -sTCP:CLOSE_WAIT # also works on macOS
Recv-Q Send-Q Local Address:Port Peer Address:Port Process
1 0 10.0.0.5:443 203.0.113.44:60310 users:(("java",pid=2211,fd=57))

-p adds the owning process and file descriptor; seeing other users' processes needs root. Add -o to print the kernel timer for each socket (timer:(keepalive,...) or timer:(timewait,...)).

Check whether the accept queue is overflowing (Linux).

nstat -az TcpExtListenOverflows TcpExtListenDrops
TcpExtListenOverflows 0 0.0
TcpExtListenDrops 0 0.0

Non-zero and rising means the accept queue overflowed and SYNs were dropped; run it again after a minute to see the rate. Raise the limit only after you know why accept() is slow: check sysctl net.core.somaxconn and the backlog your application passes to listen().

Watch a leak form over time.

watch -n 5 'ss -Htan state close-wait | wc -l'

Gotchas​

GOTCHAS5
CLOSE_WAIT is your application's bug, not the network's
TIME_WAIT is normal, and it is not a leak
ESTABLISHED does not prove the peer is alive
Recv-Q and Send-Q change meaning on a LISTEN socket
tcp_tw_recycle is not a fix, it is gone

Interview questions​

INTERVIEWWe see CLOSE_WAIT sockets piling up on a server. What is happening and how do you find the cause?mid
Show answer

CLOSE_WAIT means the peer closed its side, our kernel ACKed the FIN, and our application has not called close(). Nothing in TCP ends that state except the application, so a growing count is a socket or file descriptor leak in our code, not a network problem. Find the owner with ss -tanp state close-wait or lsof -nP -iTCP -sTCP:CLOSE_WAIT, then look for code paths (error returns, unclosed response bodies, dropped pool entries) that skip the close. Left alone, the process hits its descriptor limit and accept() starts failing with EMFILE.

Follow-ups:

  • How do you find which process? ss -tanp shows users:((name,pid,fd)); lsof -iTCP -sTCP:CLOSE_WAIT does the same and works on macOS.
  • Could the network cause it? No: the FIN already arrived. The network only decides how fast new ones appear.
  • What breaks first? The descriptor limit (ulimit -n), so new connections fail.
INTERVIEWSockets are stuck in FIN_WAIT_2. Why, and when do they go away?senior
Show answer

FIN_WAIT_2 means our FIN was acknowledged and we are waiting for the peer's FIN, which only comes when the peer's application closes. If the peer never closes (crashed client, buggy proxy, lost FIN), the socket waits. For an orphaned socket (our application already closed it) Linux aborts it after tcp_fin_timeout, 60 seconds by default, and resets orphans immediately beyond tcp_max_orphans. For a socket our application still holds, for example after shutdown(SHUT_WR), there is no timeout and it can stay indefinitely.

Follow-ups:

  • Is tcp_fin_timeout the TIME_WAIT timer? No: it only governs orphaned FIN_WAIT_2; TIME_WAIT is a compiled 60 s constant that happens to match.
  • How do you tell the cases apart? The orphaned count in ss -s, and whether the owning process still appears in ss -p.
INTERVIEWA server shows a growing number of SYN_RECV sockets. What does it mean?senior
Show answer

SYN_RECV is a half-open connection: a SYN arrived, a SYN-ACK was sent, and the final ACK has not. A handful is normal. A growing pile means either a SYN flood (spoofed sources never reply) or a backlog problem, such as a full accept queue that makes the kernel drop the final ACKs and keep retrying the SYN-ACK. Check nstat for TcpExtListenOverflows, TcpExtListenDrops and TcpExtSyncookiesSent. Mitigations are SYN cookies (tcp_syncookies), a larger backlog, and fixing a slow accept() loop.

Follow-ups:

  • Why might SYN_RECV be empty during a flood? With cookies active the server stores no state per SYN.
  • What does the client see? Silent retransmitted SYNs and eventually a timeout, not a refusal.
INTERVIEWWhat do Recv-Q and Send-Q mean on a LISTEN socket versus an ESTABLISHED one?senior
Show answer

On an established socket they are bytes: Recv-Q is data received but not yet read by the application, Send-Q is data sent but not yet acknowledged. On a LISTEN socket they are connection counts: Recv-Q is the current length of the accept queue (completed handshakes waiting for accept()) and Send-Q is its maximum, min(listen() backlog, somaxconn). A listener whose Recv-Q approaches Send-Q is about to start dropping connections.

Follow-ups:

  • Large Recv-Q on ESTABLISHED? The application is not reading fast enough.
  • Large Send-Q on ESTABLISHED? The peer or the path is slow, or the peer's receive window is closed.
  • Where is the SYN queue? Not shown in ss; it holds half-open connections and is a separate structure.
INTERVIEWDraw the state path for the active closer and the passive closer.mid
Show answer

The active closer, whose application calls close() first, goes ESTABLISHED, sends FIN into FIN_WAIT_1, receives the ACK into FIN_WAIT_2, receives the peer's FIN and sends the last ACK into TIME_WAIT, then CLOSED after 2 * MSL. The passive closer goes ESTABLISHED, receives the FIN into CLOSE_WAIT, calls close() and sends FIN into LAST_ACK, receives the final ACK and is CLOSED. If both sides send FIN at once, each goes FIN_WAIT_1, CLOSING, TIME_WAIT.

Follow-ups:

  • Which side is the client? Either: the roles come from who closes first, not who connected.
  • Why LAST_ACK? The passive side must still receive the ACK of its own FIN.
INTERVIEWWhich side ends up in TIME_WAIT, and why does it live there?mid
Show answer

The active closer, the side that sent the first FIN. It holds the connection's 4-tuple for 2 * MSL so it can re-ACK a retransmitted FIN if its last ACK was lost and so stray segments from the old connection die out before the tuple is reused. On Linux the hold is 60 seconds. It is on the client when the client closes first, which is where it can exhaust ports; on a server that closes first it costs only kernel memory. The full story is on the port exhaustion page.

Follow-ups:

  • Can you shorten it? Not on Linux; it is a compile-time constant.
  • Can the passive side hold it? Only if it closes first, in which case it becomes the active closer.
INTERVIEWHow would you find which process is leaking sockets on a host?mid
Show answer

Count sockets by state twice a minute apart with ss -Htan | awk '{print $1}' | sort | uniq -c, and see which state grows. Then name the owner with ss -tanp state <that state> or lsof -nP -iTCP -sTCP:<STATE>, and watch its descriptor count in /proc/<pid>/fd. A growing CLOSE_WAIT points at the application; a growing ESTABLISHED to one peer points at a missing pool limit or idle timeout; a growing SYN-RECV at a flood or a slow accept().

Follow-ups:

  • Why root? -p only shows processes you may inspect.
  • Over time? watch, or bcc's tcpstates to see each transition with durations.
INTERVIEWWhat happens when an application calls close() while there is still unread data in the receive buffer?senior
Show answer

Instead of the normal FIN, the stack sends a RST and the socket goes straight to CLOSED, skipping FIN_WAIT_* and TIME_WAIT. RFC 1122 section 4.2.2.13 says a TCP should do this because the unread data would otherwise be lost silently. The peer sees connection reset by peer, and any data it had sent that was not yet acknowledged to the application can be lost. SO_LINGER with a zero timeout forces the same abort on purpose; with a non-zero timeout close() blocks until queued data is sent.

Follow-ups:

  • How do servers avoid truncating the last response? Drain the receive side, or shutdown(SHUT_WR) and wait for the peer to close.
  • Does a RST leave TIME_WAIT? No: the connection is gone at once, which is also why zero-linger is a poor TIME_WAIT fix.
INTERVIEWWhy is CLOSE_WAIT worse than TIME_WAIT?mid
Show answer

TIME_WAIT is the stack doing its job and ends on its own after the 60 second hold; it costs a little kernel memory, and it only hurts when the side holding it also needs the ports. CLOSE_WAIT has no timer at all: the peer has already closed and the kernel is waiting for our application to call close(), so the socket stays until the process exits. That makes it a leak in our code, and it eats file descriptors until the process hits its limit and cannot accept new connections.

Follow-ups:

  • Which one do you alert on? A steadily rising CLOSE_WAIT count; a large but flat TIME_WAIT count is usually normal (see port exhaustion and TIME_WAIT).
  • Does restarting fix it? It clears the sockets but not the bug: the code path that skips close() will leak again.

Fun facts​

FACTSState machine trivia
  • CLOSED is not a state any socket sits in: RFC 9293 calls it fictional because it stands for having no TCB at all, which is why no tool can list it.
  • CLOSING is the rarest state in practice: it only occurs when both sides send a FIN before receiving the other's.
  • The RFC says SYN-RECEIVED, ss prints SYN-RECV and netstat prints SYN_RECV: three spellings of one state.
  • A non-orphaned socket in FIN_WAIT_2 has no timeout in Linux; the kernel documentation calls it "a perfectly valid receive only state".
  • tcp_tw_recycle, once a popular tuning tip, is one of the few Linux sysctls removed outright (4.12), because it could not be made safe behind NAT.

Cheat sheet​

Symptom in ssMeaningFirst action
Growing CLOSE-WAITPeer closed, your app did notss -tanp state close-wait, find the leak
Growing FIN-WAIT-2Peer never sends its FINCheck the peer; ss -s orphaned vs tcp_max_orphans
Growing SYN-RECVSYN flood or full backlognstat -az TcpExtListenOverflows TcpExtSyncookiesSent
LISTEN Recv-Q near Send-Qaccept() is too slowProfile the accept loop before raising backlog
Many TIME-WAITNormal; client side toward one destination is the incidentPort exhaustion
Many ESTAB, no trafficPeers may be gone; no heartbeatApplication keepalive below every idle timeout
NUMBERSNumbers to remember
  • 11 - states in RFC 9293 (CLOSED is fictional, so no socket ever sits in it)
  • 60 s - tcp_fin_timeout default, orphaned FIN_WAIT_2 only
  • 60 s - Linux TIME_WAIT constant (separate from tcp_fin_timeout)
  • 63 s and 127 s - Linux SYN-ACK and SYN retry totals (tcp-connection)
  • 4096 - somaxconn default since Linux 5.4 (128 before), per listen(2)
  • 350 s - AWS NLB idle timeout for TCP flows
  • Linux 4.12 - tcp_tw_recycle removed

For the commands themselves, see the reference page ss, nc, lsof and ip.