I have proposed a fix for this bug, see [1516a46a81].


<b>I. Approach I used for solving this bug, and status</b>

  <b>1.</b>

  In LayoutDLine(), see if there is soft hyphen in the current segment; if so, only layout characters up to (and including) this character. That means that if there is a soft hyphen in a text segment then it will always be the last character of a chunk (chunks are produced from segments in TkTextCharLayoutProc()).

  This is the same approach as already done in the text widget for tabs (a tabs is always the last character of a chunk).

  <b>2.</b>

  TkTextCharLayoutProc() computes chunk data from segment information. In doing that, it includes the soft hyphen in the number of "bytes that fit" (in some allotted horizontal space on screen). However the pixel width on screen of the soft hyphen is always zero (this is done in MeasureChars()), regardless of the position of the chunk, i.e. regardless of whether the chunk is the last of the display line or not.

  <b>3.</b>

  Possible break locations include soft hyphens (\u00AD) in TkTextCharLayoutProc()

  <b>4.</b>

  When rendering a display line on screen, CharDisplayProc() actually draws the soft hyphen only if it is the last character of a continuing display line.


<b>II. Testing status with today's commit</b>

  <b>A.</b>

  No previously existing test fail.

  Several new tests have been added to the test suite to check for the behaviour of the text widget with soft hyphens. All these tests pass (for me on Windows), except textDisp-2.34.

  This latter one checks the width (in its bounding box) of a soft hyphen when it is visible on screen, i.e. when it is actually dislayed because it's the last character of a continuing display line. Currently the width of a soft hyphen is always zero, as explained above in 2., which makes this test fail. To fix this is not trivial at all. One needs to tell chunks containing soft hyphens to increase their width field if and only if these chunks terminate a continuing display line.

<b><i>--> I'm wondering if one could not just leave things as they are now and decide that it's good enough, in other words that soft hyphens always have zero width in their bounding boxes even if they are accidentally displayed. Your

  <b>B.</b>

  My fix is currently uncompiled and untested in the case where TK_LAYOUT_WITH_BASE_CHUNKS and TK_DRAW_IN_CONTEXT are defined. This seems to happen on OSX only.

<b><i>--> Could someone having an OSX platform please run the test suite (textDisp.test file is enough) in branch [bug-1096580fff] and report results


<b>III. Peer review</b>

  Not sure I did not mix bytes and chars.

<i><b>--> Peer review of the fix requested,


<b>IV. Demo script</b>

<small><verbatim>
package require Tk
pack [text .t -width 60 -height 5 -wrap word -font fixed \
        -insertofftime 0] -fill both -expand true
.t insert end "Now this is a sample soft-hyphen abc\u00AD123 text.\n"
.t insert end "Now this is a sample soft-hyphen abc\t123 text.\n"
.t insert end "Very good large line Test\twith soft\u00ADhyphen
.t insert end "Very good large line Test\u00ADwith soft\thyphen
# Change window width at will with the mouse
# Notice how soft hyphens show up and disappear from the screen
# depending on the window width