Tcl Source Code

Platform

Issue

Symptom

Remark

Demo 1

1 % proc accept {s a p} {
     puts $s bye
     close $s
     puts ok
}
2 % set server [socket -server accept -myaddr 127.0.0.1 0]
sock272
3 % set port [lindex [fconfigure $server -sockname] 2]
61680
4 % set client [socket -async localhost $port]
sock492
5 % fconfigure $client -sockname
:: :: 61681
6 % fconfigure $client -error
connection refused

Demo 2

wish1: server

1 % proc accept {s a p} {
     puts $s bye
     close $s
     puts ok
}
2 % set server [socket -server accept -myaddr 127.0.0.1 61680]
sock272
ok

wish2: client

1 % set client [socket localhost 61680]
sock468
5 % fconfigure $client -sockname
127.0.0.1 steinmeister 61720
6 % fconfigure $client -error

The original Windows Bug is fixed in Tcl 8.6.3 (suffering another bug [c6ed4acfd8] fixed in Tcl8.6.4 http://wiki.tcl.tk/39687). Thus, the bug is closed.

The test failures observed for unix described in the comments may now be discussed in the new bug ticket [42d50ebd0c].

Thank you all, Harald

The robustness issue was cured in branch [robust-async-connect-tests].

The remaining test failures are:

FreeBSD

==== socket-14.2 [socket -async] fileevent connection refused FAILED
==== Contents of test case:

        set client [socket -async localhost [randport]]
        fileevent $client writable {set x ok}
        set after [after $latency {set x timeout}]
        vwait x
        after cancel $after
        lappend x [fconfigure $client -error]
    
---- Test generated error; Return code was: 1
---- Return code should have been one of: 0 2
---- errorInfo: couldn't open socket: connection refused
    while executing
"socket -async localhost [randport]"
    ("uplevel" body line 2)
    invoked from within
"uplevel 1 $script"
---- errorCode: POSIX ECONNREFUSED {connection refused}
---- Test cleanup failed:
can not find channel named "sock801aaed90"
---- errorInfo(cleanup): can not find channel named "sock801aaed90"
    while executing
"close $client"
    ("uplevel" body line 3)
    invoked from within
"uplevel 1 $cleanup"
---- errorCode(cleanup): TCL LOOKUP CHANNEL sock801aaed90
==== socket-14.2 FAILED



==== socket-14.7.0 pending [socket -async] and blocking [gets], server is IPv4 FAILED
==== Contents of test case:

        set sock [socket -async localhost $port]
        list [fconfigure $sock -error] [gets $sock] [fconfigure $sock -error]
    
---- Result was:
{} {} {}
---- Result should have been (exact matching):
{} ok {}
==== socket-14.7.0 FAILED



==== socket-14.7.2 pending [socket -async] and blocking [gets], no listener FAILED
==== Contents of test case:

        set sock [socket -async localhost [randport]]
        catch {gets $sock} x
        list $x [fconfigure $sock -error] [fconfigure $sock -error]
    
---- Test generated error; Return code was: 1
---- Return code should have been one of: 0 2
---- errorInfo: couldn't open socket: connection refused
    while executing
"socket -async localhost [randport]"
    ("uplevel" body line 2)
    invoked from within
"uplevel 1 $script"
---- errorCode: POSIX ECONNREFUSED {connection refused}
---- Test cleanup failed:
can not find channel named "sock801c56210"
---- errorInfo(cleanup): can not find channel named "sock801c56210"
    while executing
"close $sock"
    ("uplevel" body line 2)
    invoked from within
"uplevel 1 $cleanup"
---- errorCode(cleanup): TCL LOOKUP CHANNEL sock801c56210
==== socket-14.7.2 FAILED



==== socket-14.8.2 pending [socket -async] and nonblocking [gets], no listener FAILED
==== Contents of test case:

        set sock [socket -async localhost [randport]]
        fconfigure $sock -blocking 0
        for {set i 0} {$i < 50} {incr i } {
            if {[catch {gets $sock} x] || $x ne "" || ![fblocked $sock]} break
            after 200
        }
        list $x [fconfigure $sock -error] [fconfigure $sock -error]
    
---- Test generated error; Return code was: 1
---- Return code should have been one of: 0 2
---- errorInfo: couldn't open socket: connection refused
    while executing
"socket -async localhost [randport]"
    ("uplevel" body line 2)
    invoked from within
"uplevel 1 $script"
---- errorCode: POSIX ECONNREFUSED {connection refused}
---- Test cleanup failed:
can not find channel named "sock801965f10"
---- errorInfo(cleanup): can not find channel named "sock801965f10"
    while executing
"close $sock"
    ("uplevel" body line 2)
    invoked from within
"uplevel 1 $cleanup"
---- errorCode(cleanup): TCL LOOKUP CHANNEL sock801965f10
==== socket-14.8.2 FAILED



==== socket-14.11.0 pending [socket -async] and nonblocking [puts], no listener, no flush FAILED
==== Contents of test case:

        set sock [socket -async localhost [randport]]
        fconfigure $sock -blocking 0
        puts $sock ok
        fileevent $sock writable {set x 1}
        vwait x
        close $sock
    
---- Result was:
couldn't open socket: connection refused
---- Result should have been (exact matching):
socket is not connected
==== socket-14.11.0 FAILED



==== socket-14.11.1 pending [socket -async] and nonblocking [puts], no listener, flush FAILED
==== Contents of test case:

        set sock [socket -async localhost [randport]]
        fconfigure $sock -blocking 0
        puts $sock ok
        flush $sock
        fileevent $sock writable {set x 1}
        vwait x
        close $sock
    
---- Result was:
couldn't open socket: connection refused
---- Result should have been (exact matching):
socket is not connected
---- Test cleanup failed:
can't unset "x": no such variable
---- errorInfo(cleanup): can't unset "x": no such variable
    while executing
"unset x"
    ("uplevel" body line 3)
    invoked from within
"uplevel 1 $cleanup"
---- errorCode(cleanup): TCL LOOKUP VARNAME x
==== socket-14.11.1 FAILED



==== socket-14.12 [socket -async] background progress triggered by [fconfigure -error] FAILED
==== Contents of test case:

        set s [socket -async localhost [randport]]
        for {set i 0} {$i < 50} {incr i} {
            set x [fconfigure $s -error]
            if {$x != ""} break
            after 200
        }
        set x
    
---- Test generated error; Return code was: 1
---- Return code should have been one of: 0 2
---- errorInfo: couldn't open socket: connection refused
    while executing
"socket -async localhost [randport]"
    ("uplevel" body line 2)
    invoked from within
"uplevel 1 $script"
---- errorCode: POSIX ECONNREFUSED {connection refused}
---- Test cleanup failed:
can not find channel named "sock801965e10"
---- errorInfo(cleanup): can not find channel named "sock801965e10"
    while executing
"close $s"
    ("uplevel" body line 2)
    invoked from within
"uplevel 1 $cleanup"
---- errorCode(cleanup): TCL LOOKUP CHANNEL sock801965e10
==== socket-14.12 FAILED



==== socket-14.14 testing fileevent readable on failed async socket connect FAILED
==== Contents of test case:

    # Test for bug 581937ab1e
    
    set a1 [after 5000 {set x timeout}]
    # This connect should fail
    set s [socket -async localhost [randport]]
    fileevent $s readable {set x readable}
    vwait x
    set x

---- Test generated error; Return code was: 1
---- Return code should have been one of: 0 2
---- errorInfo: couldn't open socket: connection refused
    while executing
"socket -async localhost [randport]"
    ("uplevel" body line 6)
    invoked from within
"uplevel 1 $script"
---- errorCode: POSIX ECONNREFUSED {connection refused}
==== socket-14.14 FAILED



==== socket-14.15 blocking read on async socket should not trigger event handlers FAILED
==== Contents of test case:

        set s [socket -async localhost [randport]]
        set x ok
        fileevent $s writable {set x fail}
        catch {read $s}
        set x
    
---- Test generated error; Return code was: 1
---- Return code should have been one of: 0 2
---- errorInfo: couldn't open socket: connection refused
    while executing
"socket -async localhost [randport]"
    ("uplevel" body line 2)
    invoked from within
"uplevel 1 $script"
---- errorCode: POSIX ECONNREFUSED {connection refused}
==== socket-14.15 FAILED

CentOS 5

==== http-4.14 http::Event FAILED
==== Contents of test case:

    set token [http::geturl $badurl/?timeout=10 -timeout 10000 -command \#]
    if {$token eq ""} {
	error "bogus return from http::geturl"
    }
    http::wait $token
    lindex [http::error $token] 0

---- Result was:

---- Result should have been (exact matching):
connect failed connection refused
==== http-4.14 FAILED

==== socket_inet-2.13 Bug 1758a0b603 FAILED
==== Contents of test case:

    file delete $path(script)
    set f [open $path(script) w]
    puts $f {
	set server [socket -server accept 0]
	puts [lindex [chan configure $server -sockname] 2]
	proc accept { client host port } {
	    chan configure $client -blocking  0 -buffering line -buffersize 1
	    puts $client [string repeat . 720000]
	    puts ready
	    chan event $client writable [list setup $client]
	}
	proc setup client {
	    chan event $client writable {set forever write}
	    after 5 {set forever timeout}
	}
	vwait forever
	puts $forever
    }
    close $f
    set pipe [open |[list [interpreter] $path(script)] r]
    gets $pipe port
    set sock [socket $localhost $port]
    chan configure $sock -blocking  0 -buffering line
    chan event $sock readable [list read_lines $sock $pipe ]
    proc read_lines { sock pipe } {
	gets $pipe
	gets $sock line
	after idle [list stop $sock $pipe]
	chan event $sock readable {}
    }
    proc stop {sock pipe} {
	variable done
	close $sock
	set done [gets $pipe]
    }
    variable done
    vwait [namespace which -variable done]
    close $pipe
    set done

---- Result was:
timeout
---- Result should have been (exact matching):
write
==== socket_inet-2.13 FAILED

==== socket_inet6-2.13 Bug 1758a0b603 FAILED
==== Contents of test case:

    file delete $path(script)
    set f [open $path(script) w]
    puts $f {
	set server [socket -server accept 0]
	puts [lindex [chan configure $server -sockname] 2]
	proc accept { client host port } {
	    chan configure $client -blocking  0 -buffering line -buffersize 1
	    puts $client [string repeat . 720000]
	    puts ready
	    chan event $client writable [list setup $client]
	}
	proc setup client {
	    chan event $client writable {set forever write}
	    after 5 {set forever timeout}
	}
	vwait forever
	puts $forever
    }
    close $f
    set pipe [open |[list [interpreter] $path(script)] r]
    gets $pipe port
    set sock [socket $localhost $port]
    chan configure $sock -blocking  0 -buffering line
    chan event $sock readable [list read_lines $sock $pipe ]
    proc read_lines { sock pipe } {
	gets $pipe
	gets $sock line
	after idle [list stop $sock $pipe]
	chan event $sock readable {}
    }
    proc stop {sock pipe} {
	variable done
	close $sock
	set done [gets $pipe]
    }
    variable done
    vwait [namespace which -variable done]
    close $pipe
    set done

---- Result was:
timeout
---- Result should have been (exact matching):
write
==== socket_inet6-2.13 FAILED

==== socket-14.2 [socket -async] fileevent connection refused FAILED
==== Contents of test case:

        set client [socket -async localhost [randport]]
        fileevent $client writable {set x ok}
        set after [after $latency {set x timeout}]
        vwait x
        after cancel $after
        lappend x [fconfigure $client -error]
    
---- Result was:
timeout {connection refused}
---- Result should have been (exact matching):
ok {connection refused}
==== socket-14.2 FAILED

Note that merging the trunk before widespread testing
would be a good idea.  Also I noted my displeasure with
this implementation on the chat:


dgp	At a minimum, if magical configurations cannot be avoided, we need to find some way to not let them leak out into the public interface.  That would likely take the form of a testing socket command in Tcltest.
oehhar	I see no other way to test what we want to test, sorry. For example: "set h [socket -async $h $p];fconfigure $h -blocking 0; puts $h A; flush $h" and then it should fail, was not possible without this because it sometimes failed in the "setup".
oehhar	That is ok. Reinhard has proposed the same thing which I like. I first want to know if this effectively makes the tests pass. Then we know what we do and can do it better. Reinhard has proposed to only include the command in a special test build.
oehhar	If you have any suggestions how to do it, they are welcomed. Unfortunately, I don't have the knowledge for that and I did what I could.
oehhar	But first I want to know if it passes on Mac...

oehhar	Testing socket command would be best for me. But how may a flag be set in the driver ?
oehhar	So "tcltest sockettest 1" sets the flag in the driver. But how to implement that ?
dgp	if it comes to that, we'd either extend [testchannel] or invent [testsocket] in generic/tclTest.c to gain access to whatever internal knobs are needed to reliably test what needs testing.
dgp	But augmenting every socket with a new [chan configure .... -unsupported] option is a No Go.

Whole list of tests which fail currently for me on branch [robust-async-connect-tests] on CentOS:

==== http-4.14 http::Event FAILED
==== Contents of test case:

    set token [http::geturl $badurl/?timeout=10 -timeout 10000 -command \#]
    if {$token eq ""} {
	error "bogus return from http::geturl"
    }
    http::wait $token
    lindex [http::error $token] 0

---- Result was:

---- Result should have been (exact matching):
connect failed connection refused
==== http-4.14 FAILED

==== socket-14.2 [socket -async] fileevent connection refused FAILED
==== Contents of test case:

        set client [socket -async localhost [randport]]
        fileevent $client writable {set x ok}
        set after [after $latency {set x timeout}]
        vwait x
        after cancel $after
        lappend x [fconfigure $client -error]
    
---- Result was:
timeout {connection refused}
---- Result should have been (exact matching):
ok {connection refused}
==== socket-14.2 FAILED

> fossil info
project-name: Tcl Source Code
repository:   /usr/home/cerutti/tcl-open/../tcl.fossil
local-root:   /usr/home/cerutti/tcl-open/
config-db:    /home/cerutti/.fossil
project-code: 1ec9da4c469c29f4717e2a967fe6b916d9c8c06e
checkout:     0c5005721e527d5c2f72dd5769ef6bba57f1eb0b 2014-06-05 19:09:51 UTC
parent:       8b99a29c3ff6448c43fbe82ab228937acda48509 2014-06-04 21:45:29 UTC
child:        109f86e24f78a1f3840a7a2ee66ab14f2049bc00 2014-06-16 21:01:12 UTC
tags:         robust-async-connect-tests
comment:      Robust async connect tests by temporarely switching off auto
              continuation. Ticket [13d3af3ad5] (user: oehhar)

> ./tclsh ../tests/socket.test
socket.test:    Total   158     Passed  157     Skipped 1       Failed  0
Number of tests skipped for each constraint:
        1       win

Thanks a lot!

About the unix & firends test crashes:

• Reinhard has reworked the tests in trunk
• There is a new fossil branch [robust-async-connect-tests] which should get the timing dependent things out (Test socket-14.11.0) and may allow nore advanced tests in future.

For me, only test socket-14.2 failes with a result of "timeout {connection refused}" instead of "ok {connection refused}".

I will open an extra bug for that as it looks as a real error to me.

Tests on MacOS and FreeBSD are pending on this branch.

Thank you, Harald

I have made some tests today with my CentOS system.

Issue: test socket-14.9.1 blocks forever.

Background: as CentOS does not resolve "::1" to localhost but only "127.0.0.1" due to the /etc/hosts contents:

127.0.0.1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6

Thus, the socket in the test should not connect. Test contents:

test socket-14.9.1 {pending [socket -async] and blocking [puts], server is IPv6} \
    -constraints {socket supported_inet supported_inet6} \
    -setup {
        makeFile {
            set server [socket -server accept -myaddr ::1 0]
            proc accept {s h p} {set ::x $s}
            puts [lindex [fconfigure $server -sockname] 2]
            flush stdout
            vwait x
            puts [gets $x]
        } script
        set fd [open |[list [interpreter] script] RDWR]
        set port [gets $fd]
    } -body {
        set sock [socket -async localhost $port]
        puts $sock ok
        flush $sock
        list [fconfigure $sock -error] [gets $fd]
    } -cleanup {
        # make sure the server exits
        catch {socket ::1 $port}
        close $sock
        close $fd
    } -result {{} ok}

The command which blocks is: "flush $sock".

What I have tried:

• No server at all (as it does not connect anyway) -> ok, no block but error
• Server and client in different shells -> ok, no block but error
• Checkout which was stated ok by dgp: 2014-04-08 15:15:31 2b3521353bf91817 GOOD CURRENT -> blocks
• Tried this test with tcl8.6.1 release -> blocks

I suppose that the issue is in the function "WaitForConnect".

So I have put "fprintf(stderr,...) or printf(...) in this function but nothing appeared on the test run:

make tests TESTFLAGS="-verbose bte -file socket2.tcl"

(socket2.tcl is my test version of socket.tcl only containing 14.9.1 and the boot code).

So I tired to use "ddd" to debug the function, but even with "configure --enable-symbols", ddd says to "WaitForConnect" -> No line number information.

So I am sorry, this is my bank holiday report, switch back to windows issues, Harald

The last comment by gahr is related to an obsolate test file. Test 14.6 uses internal information not exposed any more like that.

But he observed more or less the same issues as DGP with the current test file.

Thank you ! Keep testing on FreeBSD ! Harald

I don't know how it's related, but socket-14.6 fails on FreeBSD. 

==== socket-14.6 [socket -async] with no event loop and [fconfigure -error] before the socket is connected FAILED
==== Contents of test case:

        set client [socket -async localhost $port]
        foreach _ {1 2} {
            lappend x [lindex [fconfigure $client -sockname] 0]
            lappend x [fconfigure $client -error]
            update
        }
        lappend x [gets $client]
    
---- Result was:
127.0.0.1 {} 127.0.0.1 {} bye
---- Result should have been (exact matching):
::1 {connection refused} 127.0.0.1 {} bye
==== socket-14.6 FAILED

ok, test socket-14.11.0 does pass, it just requires
75 seconds to complete, which isn't a good fit for the
test suite if we have any chance to test more quickly.

Likewise socket-14.11.1 fails, but requires 75 seconds
to do so.

==== socket-14.11.1 pending [socket -async] and blocking [puts], no listener, flush FAILED
==== Contents of test case:

        set sock [socket -async localhost [randport]]
        fconfigure $sock -blocking 0
        puts $sock ok
        flush $sock
        fileevent $sock writable {set x 1}
        vwait x
        close $sock
    
---- Test completed normally; Return code was: 0
---- Return code should have been one of: 1
==== socket-14.11.1 FAILED

And socket-14.15 also passes, also requiring 75 seconds.

Additional failing trunk tests on OSX Mavericks:

==== socket-14.7.2 pending [socket -async] and blocking [gets], no listener FAILED
==== Contents of test case:

        set sock [socket -async localhost [randport]]
        catch {gets $sock} x
        list $x [fconfigure $sock -error] [fconfigure $sock -error]
    
---- Result was:
{error reading "sock7f9e038df590": socket is not connected} {connection timed out} {}
---- Result should have been (glob matching):
{error reading "sock*": socket is not connected} {connection refused} {}
==== socket-14.7.2 FAILED



==== socket-14.8.2 pending [socket -async] and nonblocking [gets], no listener FAILED
==== Contents of test case:

        set sock [socket -async localhost [randport]]
        fconfigure $sock -blocking 0
        for {set i 0} {$i < 50} {incr i } {
            if {[catch {gets $sock} x] || $x ne "" || ![fblocked $sock]} break
            after 200
        }
        list $x [fconfigure $sock -error] [fconfigure $sock -error]
    
---- Result was:
{} {} {}
---- Result should have been (glob matching):
{error reading "sock*": socket is not connected} {connection refused} {}
==== socket-14.8.2 FAILED

==== socket-14.12 [socket -async] background progress triggered by [fconfigure -error] FAILED
==== Contents of test case:

        set s [socket -async localhost [randport]]
        for {set i 0} {$i < 50} {incr i} {
            set x [fconfigure $s -error]
            if {$x != ""} break
            after 200
        }
        set x
    
---- Result was:

---- Result should have been (exact matching):
connection refused
==== socket-14.12 FAILED

==== socket-14.14 testing fileevent readable on failed async socket connect FAILED
==== Contents of test case:

    # Test for bug 581937ab1e
    
    set a1 [after 5000 {set x timeout}]
    # This connect should fail
    set s [socket -async localhost [randport]]
    fileevent $s readable {set x readable}
    vwait x
    set x

---- Result was:
timeout
---- Result should have been (exact matching):
readable
==== socket-14.14 FAILED

and the following tests appear to just hang:

socket-14.11.0
socket-14.11.1
socket-14.15

Same two tests fail on trunk on an OSX Mavericks system.

Bisecting yields:

2014-04-08 18:00:35 9b3b3adf47823f59 BAD
2014-04-08 15:15:31 2b3521353bf91817 GOOD CURRENT

At the start of the bugfix branch both tests were passing.

On the trunk (and on the bugfix branch just
before merging), on a CentOS 5 system, I see
two failing tests:

==== http-4.14 http::Event FAILED
==== Contents of test case:

    set token [http::geturl $badurl/?timeout=10 -timeout 10000 -command \#]
    if {$token eq ""} {
        error "bogus return from http::geturl"
    }
    http::wait $token
    lindex [http::error $token] 0

---- Result was:

---- Result should have been (exact matching):
connect failed connection refused
==== http-4.14 FAILED

==== socket-14.2 [socket -async] fileevent connection refused FAILED
==== Contents of test case:

        set client [socket -async localhost [randport]]
        fileevent $client writable {set x ok}
        set after [after $latency {set x timeout}]
        vwait x
        after cancel $after
        lappend x [fconfigure $client -error]

---- Result was:
timeout {connection refused}
---- Result should have been (exact matching):
ok {connection refused}
==== socket-14.2 FAILED

Branch [bug-13d3af3ad5] committed to trunk by checkin [fdd8917252]. Bug closed.

The commit includes the following additional changes, see also http://wiki.tcl.tk/39687

• Continue async connect process on any command touching the driver (gets, puts, fconfigure) (all Platforms)
• Fix read event on connection failure (windows) [581937ab1e]
• Connect ignored (windows) [336441ed59]
• Rename variables and functions to be closer to the tcl core which increases highly the diff, sorry.

Opened a new ticket [5425f2c082] with only the MAC OS X issue.

I propose to open a new bug with the issue observed by DKF on MAC OS.

What is the internal function to connect to IP-V6:

1. fetch the list of IP addresses to the destination. This is normally multiple IPV6 addresses and one IPV4 address.
2. try to open the first IP address
3. on a connection, return with ok
4. on connection failure, continue with next IP
5. only if all connection failed, flag an error state (fconfigure $client -error)

We have the following different issues:

• On Windows, only the first IP address is tried. So step 4 does not happen.
• On MAC-OS (and, I suppose, also on Linux) all addresses are tired, but an error is shown if a connection try fails. So step 5 is violated, that an error is also shown if there is still another try running.

DKF is right, for async sockets, the event loop might be necessary. So I retried [5d478dc916], the last checkin before commit [aef7ffe46d] fixing bug [3545363fffffffff] was submitted. The first tclsh:

% proc accept {s a p} {
     puts $s bye
     close $s
     puts ok
}
% set server [socket -server accept -myaddr 127.0.0.1 61680]
sock192
% vwait a

and in a second console

% set client [socket -async localhost 61680]
sock448
% vwait a

There is no "ok" shown in the server console so I suppose no connection.

On the plus side, this seems to be a somewhat more cross-platform bug…

I've just tried with 8.6.1 (ActiveState build for OSX). With that, I get:

% set client [socket -async localhost 61680]
sock10100b710
% update

% fconfigure $client -sockname
127.0.0.1 localhost 64134
% fconfigure $client -error
connection refused

The real issue with this sequence is the bogus error message.

But that's not all! This sequence is different:

% set client [socket -async localhost 61680]
sock103825910
% fconfigure $client -error
connection refused
% update
% fconfigure $client -error
% fconfigure $client -error
% fconfigure $client -sockname
::1 localhost 64153

I tried today [5d478dc916], the last checkin before this patch was submitted. I did not build a wish, only tclsh, so I ran multiple tclsh.exe, with the commands:

% proc accept {s a p} {
     puts $s bye
     close $s
     puts ok
}
% set server [socket -server accept -myaddr 127.0.0.1 61680]
sock192
% vwait a

and in a second console

% set client [socket -async localhost 61680]
sock448
% fconfigure $client -sockname
:: :: 51254
% fconfigure $client -error
connection refused

In a thiered console, use a syncroneous socket:

set client [socket localhost 61680]

IMHO, this version also shows the bug. Sorry, Harald

Reinhard Max reported by private chat, that this bug was introduced by commit [aef7ffe46d] fixing bug [3545363fffffffff].