Tcl Source Code

Initially reported by @daviddem on Stack Overflow in "tcl close does not gracefully terminate tcp/ip connection" (and in tclchat).

Current implementation of socket subsystem for Windows leaves the socket after close linger (remain open after a closesocket call), so lets flooding with sockets in TIME_WAIT state up to free descriptor or port range until TcpTimedWaitDelay is reached for that.

This may be especially worse because the the maximum number of file descriptors allowed per process is defined by macro FD_SETSIZE (default 1024 on Windows) and it can be reached very fast by many concurrent connection going "half" closed related to the issue (either on client or on server resp. both sides).

The simplest test-case looks like this:

  % puts [exec netstat -n | grep -c TIME_WAIT]
  2
  % timerate { time { close [socket localhost 80] } 50; after 100; puts [exec netstat -n | grep -c TIME_WAIT] } 1000
  52
  102
  152

MS docs ("Graceful Shutdown, Linger Options, and Socket Closure" and "LINGER (winsock.h)") are realy confusing, but...
I found how one can "solve" the entering TIME_WAIT state... Added this implicit before closesocket:

  /* set SO_DONTLINGER to 0 (yes, 0 :) - forces the socket will not remain open */
  BOOL val = 0;
  setsockopt(infoPtr->socket, SOL_SOCKET, SO_DONTLINGER, (const char *)&val, sizeof(BOOL));

  % puts [exec netstat -n | grep -c TIME_WAIT]
  1
  % timerate { time { close [socket localhost 80] } 50; after 100; puts [exec netstat -n | grep -c TIME_WAIT] } 1000
  1
  1
  1

Note that enabling a nonzero timeout on a nonblocking socket is not recommended.

So it is more to investigate. But as interim solution we can implement "-linger" option (so setting it to 0 would avoid that, and any other integer value would specify setting a linger timeout, where -1 can mean default system timeout is used).
One could also well safe provide this option firstly using some define TCL_FEATURE_SOCKET_LINGER (similar TCL_FEATURE_KEEPALIVE_NAGLE).

I think, I fixed it in [b960d1b71e] as good as possible:

on close it would firstly try a graceful disconnect and don't linger if in success case (as well as would perform a hard reset in case if socket get closed without data sent/received).

Test script is attached...
I don't see growing sockets in TIME-WAIT state anymore (regardless how it is closed on server side).

Tests (socket.test and *-io.test) passed too.

Further investigations shows that the kind of close doing from other side (peer) is also very important to fulfill graceful shutdown - if it is not following the strict rules of graceful disconnect process, many sockets would enter TIME-WAIT state (regardless proper lingering set for them).

For instance here is the diff illustrating bad (red) and good (green) server accept procedure (in sense of graceful disconnect), if a client peer closing it properly at its side (with WSASendDisconnect, wait for FD_CLOSE, etc):

socket -server {apply {{ch args} {
-  close $ch
+  chan event $ch readable [list apply {{ch} {
+    if {[catch {string length [read $ch 1024]} s] || $s == 0} {
+      close $ch
+    }
+  }} $ch]
}}} $port

> Apparently, this is an internal WIndows "feature", that Windows wants to give itself time to empty its buffers.

Agree and I thought also it's "normal" Windows behavior, unless I saw every single socket in the time-wait state without necessary sending buffers up-to default timeout (TcpTimedWaitDelay). This cannot be called "normal" anymore and either graceful shutdown should be implemented differently (at least up-to last receive after some send/flush) or there is really an option needed to control that by developer.

Serges, this sounds sensible, thank you.

David Graveraux might be a wizard to give more light to it.

Apparently, this is an internal WIndows "feature", that Windows wants to give itself time to empty its buffers.

A new parameter might be sensible. Also, the -nagle" parameter was always there but commented out.

Thank you, Harald