Tk Source Code

Subject: 1000x slowdown in tk X11 widget drawing speed Mac OS X 10.10+

In the course of compiling and testing an
X11/OpenGL cortical surface reconstruction program
on Mac OS X 10.10 and 10.11, I have documented an
extreme slowdown in the performance of simple X11 tk
widget commands on current fast Mac hardware
(e.g., MacPro, latest retina laptops) using default
compiles of the latest tcl/tk.

I have demonstrated this with a default unix
compile (N.B.: *X11* tk, not Aqua tk) compile using
both tcl/tk 8.5.18 and tcl/tk 8.6.4.  I have found
no workaround yet.

Here are instructions for reproducing it:

  tar xvfz tcl.8.5.17.tar.gz
  cd tcl8.5.17
  cd unix
  ./configure
  make
  make install

or:

  tar xvfz tk.8.5.17.tar.gz
  cd tk8.5.17
  cd unix
  ./configure
  make
  make install

and then making sure that the just-compiled wish
and current TCL/TK_LIBRARY's are being used.

In XQuartz 2.7.3 to 2.7.8, the following simple tk
program takes almost 10 *seconds* to display the
400 buttons (e.g., on a MacPro 6-core 3.5 GHz
machine with two high performance video cards):

# display 400 buttons
for {set j 0} {$j < 20} {incr j} {
  frame .f$j
  pack .f$j -side top
  for {set i 0} {$i < 20} {incr i} {
    button .f$j.b$i -text asdf
    pack .f$j.b$i -side left
    #update idletasks
  }
}

The exact same catastrophic slowdown has been
observed on Mac laptops upon upgrading to Mac OS X
10.10 or 10.11.

This program should finish in about 25 msec, not 10
seconds.  This is almost a 1000x slowdown.  If the
update idletasks is commented in, you can see each
button as it is very slowly drawn.

Adding -sync to the call to wish makes things run
even more slowly.

Ticking "Info X11.app" -> "Prevent App Nap" has
no effect.

There is well under 1% CPU usage during the draw.

OpenGL performance in a GLX window is *not*
affected.

I have also filed a bug report with XQuartz, but they
said it wasn't a bug because it was caused by the
OS upgrade.

I have also filed a bug report with Apple.

If anybody else has experienced this, or has any
ideas about leads to pursue to fix this, I would
be very interested, since it effectively disables
the interface of any software that uses X11/tk on
the Mac (my software is cross compiled on
Linux).

In absolute desperation, to make my cortical
surface reconstruction programs usable on Mac
(it is a branch of FreeSurfer, originally written by
Anders Dale and myself), I now pop up an xterm
in front of an in-progress tk window and then
immediately kill the xterm. Upon uncovering,
tk drawing goes faster.  The drawing speed increase
is proportional to how much of the tk/wish window
was occluded by the xterm. The occluding window
must come from a non-tk process.

It seems like it might be a bad interaction with
the new AppNap, but there is no effect of turning
it off for X11.app or globally with:

  defaults write NSGlobalDomain NSAppSleepDisabled -bool YES

I also tried compiling with:

  cd ~/Developer/tk8.6.4/unix
  make distclean
  ./configure --enable-64bit --enable-corefoundation=yes --enable-framework=no --enable aqua=no
  make

but the problem remained.

Cheers,
Marty Sereno

Extreme X11/tk slowdown still identically present on macOS Sierra 10.12.5 with XQuartz 2.7.11

Extreme X11/tk slowdown still identically present on macOS Sierra 10.12.1 with XQuartz 2.7.11

[desperately groping in complete darkness here]

I noticed that:

  fc-cache -f -v

(fontconfig version 2.11.1 in Quartz 2.7.9) uses 100% CPU for
75 seconds.  It takes about 50 seconds scanning /System/Library/Fonts and
25 seconds scanning /Library/Fonts (the X11 directories are scanned
instantaneously).

This looks like some kind of timeout/fail problem.
Could whatever is causing this somehow also result
in timeouts that explain the massive X11 tk drawing slowdown?

cheers,
Marty

Also XQuartz 2.7.9 is out, and the problem is still extant.

On the Mac, if you set the theme to match the linux theme, do you still need to adjust all the sizing for your application?  All of the various themes can be used in the quartz version of Tk.

The font sizes are handled completely differently in X11 and Quartz, so the font scaling would have to be set appropriately.

   ttk::style theme use alt

I checked if temporarily disabling the systemstatsd helped
with the Mac X11 tk slowdown (El Capitan).  It didn't.

# check
su -
launchctl list | grep systemstat
-	0	com.apple.systemstats.analysis
631	0	com.apple.systemstatsd
-	0	com.apple.systemstats.daily

# disable 
su -
servicelist="systemstatsd systemstats.daily systemstats.analysis"
for service in $servicelist; do
    launchctl disable system/$service
done
reboot

[test for tk slowdown -> same]

# re-enable (auto-restarts)
su -
servicelist="systemstatsd systemstats.daily systemstats.analysis"
for service in $servicelist; do
    launchctl enable system/$service
done

Thanks kevin and dgp for your responses.  I
realize that both of you, like me, have many other
more pressing things to attend to.

I just tested default compiles of tk 8.5.19 and
8.6.5 on Mac OS X 10.11.3 (MacPro) and the very
slow button drawing is exactly unchanged.  I tried
a number of combinations of earlier releases of Tk
and X11 on Mac OS X 10.10+ with no luck.

However, everything worked fine with any tk 8.5
and any XQuartz from 2.7.4 to 2.7.8 running on any
Mac OS X, up to and including, Mac OS X 10.9.

No response so far from the Apple bug report.  I
recently pinged Jeremy Huddleston Sequoia again
for pointers as to where I might poke around in
the XQuartz source.  Jeremy's first response last
year was that this was probably something caused
by an Apple change, which seems likely.  However,
set against this is that I did very occasionally see a
very similar slowdown before, on Mac OS X 10.6
with XQuartz 2.7.4 and 2.7.5, fixable by a reboot
(I never saw this happen on Mac OS X 10.6.8 with
XQuartz 2.7.6 and 2.7.7).

As dgp commented, Apple seems to be letting X11 on
Mac decay (I can hear the cracks about it already
being that...), which suggests that I find a path
outside of native X11/GLX/OpenGL.

One possibility is a VM.  As it turns out, I
actually compile my Linux distributions on 32-bit
and 64-bit CentOS 5.9 virtual machines
(VirtualBox) running on Mac OS X 10.6.8.

On Mac OS X 10.6.8, the tcl/tk/tix performance
inside the Linux VirtualBox VM is perfectly as snappy
as tcl/tk/tix running on the native XQuartz (I
haven't tried Linux VMs on Mac 10.11 yet, but will
do soon).

The GLX/OpenGL performance, as you might expect,
is a lot slower than the native GLX/OpenGL.  It's
usable, tho a bit icky for direct manipulate (250
msec per redraw in the VM vs. 25 msec per redraw
native, for a neocortical surface with 250K polygons).
However, if I changed over to shaders, it's possible
that that might improve.  I do sometimes manipulate
very large surfaces (e.g., a human cerebellum with
10M triangles).

As always, I am sure this is nothing that another
20,000 lines of code couldn't fix :-}

However, it is also possible that a small number
lines in XQuartz or tk might be able to get around this.

I found the following Obj-C code for disabling app nap
(for 10.9 where it wasn't yet a tk problem):

  http://stackoverflow.com/questions/19847293/disable-app-nap-in-macos-10-9-mavericks-application

@property (strong) id activity;

if ([[NSProcessInfo processInfo] respondsToSelector:@selector(beginActivityWithOptions:reason:)]) {
  self.activity = [[NSProcessInfo processInfo] beginActivityWithOptions:0x00FFFFFF reason:@"receiving OSC messages"];
}

but don't know if something like that
might be relevant here.

I'm guessing from the latest comment that the release of
Tk 8.6.5 didn't improve matters any?

I thought I was still on Mountain Lion out of laziness, but
maybe I had reasons to stay put without knowing it.

Are the bug reports submitted to XQuartz and Apple available
online anywhere for browsing?  Have they produced any reply
at all?

To grasp at a straw or two, are there any releases of Tk/X11
that are known not to suffer in this way?  If we could find one
there might be a prayer we could locate some source code 
change in Tk or Tcl themselves that's triggering the problem.

From the description it sounds like the blame lies in the
supporting systems though, and there's only limited things
we can do about that.  Even if we locate some workaround,
there's nothing to suggest that we might rely on that continuing
to work.

So I'm willing to look into this, but with limited hope for success.

Apple shows all signs of letting X on its systems decay into
disfunction.  So the message to all people deploying X programs
to OSX is to either adapt them to supported graphics systems
(and Tk makes this a great deal easier than it would be otherwise)
or to interpose some Unix emulation system that does a better
job presenting X on OSX.  That may not exist, or may have its own
performance woes, I cannot predict.  Even if we get Tk behaving
better on El Capitan, I advise you ought to move in one of these
directions.

The two reasons for me to stick with X11 tk both
involving minimizing coding effort.

First, I am cross-compiling for Mac and Linux.
Differences in all the widget sizes between X11
and Aqua would require zillions of tweaks to 
adjust the tightly packed Illustrator-like 
interfaces (and would double the work for new
changes).

Second, I am still using Tix, which would have to
be ported to Aqua, or replaced with more code.  I
have made small repairs to Tix, but would not be
capable of porting it to Aqua.  Together, these
would force a Mac/Linux fork in the interface
code.

I remain hopeful that there is a small tk or
XQuartz fix that could fix the 1000x slowdown.

I have hacked the X11 tk widget code a bit, but
made no progress toward a speedup.  I may try to 
hack the more difficult XQuartz code.

The extreme tk slowdown used to occasionally occur
way back on XQuartz 2.7.4 on Mac OS X 10.6.8, but
was fixable back then by a reboot.

There are other neuroimaging and statistical
codebases out there in exactly the same situation
as me and FreeSurfer (cross compiled Mac/Linux
with X11 and using tix: e.g., FSL, R).

Given that virtually everything *does* work great
in X11 tk on Mac, it seems a shame not to try to
fix this one perhaps little, but crucial, thing.

Though the usage of these scientific neuroimaging
applications can't compare to the latest app for
cataloging cat videos (and therefore, have 
difficulty attracting coding resources), they are
equally important for understanding how the brain
works :-}

Was ultimately able to build X11 Tk in MacPorts (8.6.3). I see the slow rendering, but my earlier query about why you are using X11 remains.

MacPorts isn't allowing me to build an X11 version of Tk (I prefer the sandboxed approach so I don't pollute my native build), but I did test your script with a version of Tk that uses the native windowing system and things render very quickly. Is there any particular reason you are using X11 on the Mac? It's a second-class citizen these days, XQuartz isn't very actively developed, and I am sure the problem resides somewhere in the interaction between Tk, X11, and the native windowing system.