Tcl Source Code

Channel transforms and (multibyte) encodings break the byte/char 1-to-1 mapping. As a consequence, there are cases where the low-level (resp. raw) channel has a byte ready (as detected by the select() in the notifier, letting a [fileevent readable] fire), but a [read 1] on the hi-level (resp. decoded) channel would block (if blocking) or return the empty string (if not).

The possibility of such "false positives" in readable fileevents is not advertised, and a recurring idiom is

   fconfigure $ch -blocking 0
   fileevent $ch readable foo
   proc foo {} {
      set data [read $::ch 1000]
      if {![string length $data]} {close $::ch;clean_things_up}
      # use $data
    }

This idiom, of course, leads to erroneous truncation. The same thing at C level leads to 2762041 and 1945538.
Note that this mechanism potentially applies to all multibyte encodings and channel transforms. TLS and Zlib are highly suspect of suffering from this, along with the ubiquitous [-encoding utf-8], as shown by the attached script.

Two options:

 (1) fix it the hard and clean way, by doing a read-ahead inside the NotifyProc (see 1945538 for details). Do this in the core for non-single-byte encodings, and in the transform-based extensions.

 (2) work around it, by strongly documenting the fact. Possibly many scripts to fix. Still some C code to update in extensions making the wrong assumption, to stop them from giving false EOFs.

Fixed in 8.6 trunk by option (2), documentation.

File Added - 428260: emptyread.tcl