Tcl Source Code

ubuntu 10.04 latest repos contain tcl 8.4.*; my machine had them installed.

then (without uninstallation of tcl 8.4.*), i built tcl 8.5.10 from sources on ubuntu 10.04 (ubuntu was updated with latest stuff).

tcl build line used was: $ ./configure --enable-threads --enable-shared --enable-symbols && make && make test && sudo make install

all completed successfully.

then, the program using tcl was built successfully. When ran, this program gave:
error while loading shared libraries: libtcl8.5.so: cannot open shared object file: No such file or directory

This seems to be a bug in tcl's make install: it didn't ran ldconfig at the end of installation (i.e. at the end of make install)... But I don't know for sure if this is tcl's duty to run ldconfig or not.

When I consulted ld.so.conf.d, it had /usr/local/lib listed in libc.conf:

$ cat /etc/ld.so.conf
include /etc/ld.so.conf.d/*.conf
$ cat /etc/ld.so.conf.d/libc.conf | grep local
/usr/local/lib
$ sudo ldconfig
$

After that, the program referencing libtcl8.5.so found it successfully.

It is not Tcl's responsibility to run ldconfig on install; not every system needs it and not every situation requires it.
According to mthuurne, "there seems to be a trend of libraries running ldconfig on installation".
Should Tcl's instructions mention ldconfig? Perhaps. Should Tcl run ldconfig? I'm not sure.
I'm not entirely closed to the idea. Patches/examples/convincing arguments are always welcome.
For the current situation:
If you have the privs to put Tcl into /usr/lib then you have the privs to run ldconfig, and you should understand the requirements of your OS with respect to such things.
"make test" and "make shell" should work; they can show you the way to point to Tcl's libs if you won't run ldconfig.
Leaving this open for further comment but I'd like to close it with "won't fix because there's nothing to fix" unless convinced otherwise.

1. As I said, the problems seems to be solvable for everyone, in a general but probably very transitionally hard way.

Combative is OK - perfect solutions are useful.

"Were you assuming that only root installs? --- No such assumption. "If you ... insist on the system installation approach..." --- No, I don't insist.

"If you're going to insist on the system installation approach (without relying on system-provided libraries, which would be a recommended method) then you're just going to have to do a little extra work and run 'ldconfig' with appropriate privilege elevation where necessary."

This could be a switch for TCL configure script to select non-root install (a local one) or an installing process with root privileges which would include running ldconfig (updating ld.so's database with TCL installation's internal paths is TCL's responsibility somehow (why?) made the TCL user's responsibility).

"Because there's just no way to completely satisfy both automatically ..." --- About automatic way --- 1) that behavior choice must be specifiable to avoid errors, and 2) some operating systems might have a way to diagnose if the user who is installing TCL has root access or not; for such OS types there could be a way to specify this diagnosis way -- I lack knowledge if modern operating systems allow for this.

So, the conclusion for the current situation is: A problem of underspecification of the standard way to load libtcl*.so for a given TCL version string (provided only this information: <TCL version, and OS type>). Problem type: swamp.


2. Underspecification of the clean and exact installation procedure.

TCL_SRC_FOLDER/unix/README says: "The release should compile and run either "out of the box" or with trivial changes on any
UNIX-like system that approximates POSIX, BSD, or System V."

It does not specify running ldconfig. It should be clear at specifying what "trivial changes" should be done and when. For perfection and reducing a number of TCL users' problems.

TCL_SRC_FOLDER/README doesn't specify cases to run ldconfig.

README also mentions http://www.tcl.tk/doc/howto/compile.html . It doesn't specify cases to run ldconfig.

(This page refers to ActiveTcl/ActiveState. I didn't try it.)

This page also refers to "configure --help". It doesn't specify cases to run ldconfig.

So, the conclusion for the current situation is: A problem of underspecification of the standard and complete way to install tcl from source package. Problem type: swamp.


P.S. But TCL is extremely close to perfection --- almost everything is done and worksforme. Almost :)

Hi! I'm the person who maintains the build of openMSX.

It might help to summarize the situation:
- joxy installed Tcl from source because Ubuntu LTS had Tcl 8.4 while the most recent openMSX release needs Tcl 8.5
- Ubuntu did have /usr/local/lib in the ld.so configuration, but because the cache was not updated it could not be found
- openMSX finds Tcl by looking for tclConfig.sh in typical locations that distros put it in; this has advantages and disadvantages, but it generally works well
- in its default build mode, openMSX links dynamically to system-wide libraries using -L and -l, no rpath

I think rpath is not really the issue here: openMSX does not support compilation against Tcl in non-standard locations anyway, since it will not look for tclConfig.sh there and therefore won't find Tcl at all. openMSX has a second build mode where we install all dependencies as non-root and link statically to them, so we do cover the non-root scenario but in a different way.

The situation that joxy encountered is where Tcl is installed in a standard location (/usr/local) and linked dynamically. So the dynamic loader can find libtcl without extra help from rpath or LD_LIBRARY_PATH, provided that its cache is up-to-date. The question is whether the user should update the ld.so cache by running ldconfig or whether the "make install" target of Tcl should do that.

I don't think there is a "correct" answer to that, however there seems to be a trend of libraries running ldconfig on installation. When I first started using Linux (10+ years ago) I always had to do it by hand, while nowadays it is rare that I have to do it by hand. Apparently joxy was not aware that the cache needed updating in the first place and this is probably true for more users.

A possible solution would be for the Tcl "make install" target to check for a GNU system and uid 0 and if both are the case, run "ldconfig" as the last command of the installation. This does not conflict with rpath and it does not conflict with non-root installations, but it does solve the issue joxy ran into.

As was said, there are two camps that don't see eye-to-eye at all. There are users who don't have root on their systems who want to just install Tcl locally - the -rpath support is for them - and there are the system distributors/admins who put libraries in a "standard" place. Because there's just no way to completely satisfy both automatically, we have the first camp as our primary objective as distro makers should be smarter, e.g., they can have a standard set of options to pass to configure (they need that anyway because they want to at least set the --prefix option, and they also need to prepare packages from the result).

The use of 'ldconfig' is *not* universal, is restricted to root (were you assuming that only root installs anything anywhere? not so!), and it doesn't fit at all with the use of the -rpath linker option. If you're going to insist on the system installation approach (without relying on system-provided libraries, which would be a recommended method) then you're just going to have to do a little extra work and run 'ldconfig' with appropriate privilege elevation where necessary.

Sorry to be combative but we can't solve everything. There are _conflicting_ demands on this space.

a. These two questions are to be answered later.

b. The ideal way would be to somehow have exact the "entire self-contained TCL system" installation IDs, and the OS central entrance for TCL users to select a TCL installation by ID. This would resolve all issues like those --- while making everyone happy --- but is most probably a very hard way to reimplement things.

And what would a patch look like that
would make Tcl source distributions
better match your expectations?

BTW, why is it that you're building from
source, rather than using what Ubuntu
distributes (which they've presumably
adapted to their system conventions
and expectations)?

So you're in the "don't use -rpath" camp.

There's also a "must use -rpath" camp.

I have no ability to resolve the long-raging debate.,
and it's impossible to make all happy.

I'll leave the ticket open for someone better able
than I am to comment further.  I've offered what
help I can.

1. openmsx developers said that this tip (echo 'puts [tcl::pkgconfig get libdir,runtime]' | /path/to/tclsh) doesn't work for cross compilation... "in openMSX, we treat every build as a cross build; a native build is just a special cross build"... They also said: "we do find the tclConfig.sh though, but by trying various typical locations, not by executing the interpreter". --- But well, I don't want to discuss this particular issue, this is just for general information, a side thing which seems to be irrelevant to this tracker issue.

2. We discussed this issue with openmsx developers, and decided that openmsx binary's indirect reference to libtcl8.5.so (indirection via ld.so) is saner than a direct hardcoded full-file-name reference (to /usr/local/lib/libtcl8.5.so).

So the conclusion I think is that TCL installer (TCL's make install) must configure the system and OS and ld.so so that ld.so would load libtcl8.5.so given the only string "libtcl8.5.so" string at appropriate place --- this string "libtcl8.5.so" must be the only information to be able to perform the indirect loading of "libtcl8.5.so".

And currently TCL doesn't provide this. This is a defect.

I think it is not the user's program's responsibility to configure the OS: to setup OS indirections related to TCL's internal configuration. And the path to the place where TCL's files reside is part of TCL's internal configuration. TCL must configure the OS so that libtcl8.5.so is findable.

Do you agree?

Untested, but I think this is the recipe:

LIB_RUNTIME_DIR should be in the Makefile
(or whatever build scripts) of the openmsx program.
It should be defined to hold the name of the directory
where you found the tclConfig.sh file of the Tcl
installation you are embedding in your program.
Having done that, including $TCL_LD_SEARCH_FLAGS
on your link line should take care of things.

You seem to have been able to find the tclConfig.sh
file, but if that's an issue, you can ask Tcl where
it is:

echo 'puts [tcl::pkgconfig get libdir,runtime]' | /path/to/tclsh

And the main point is to understand the best way to specify libtcl*.so exact filename for openmsx.

It is either via ldconfig, or via -rpath (if the latter is possible at all).

And the question in the first case (ldconfig's): who is responsible to run ldconfig? And when.

It's a failure of ld.so when called or referenced (directly or indirectly) by openmsx binary in order to load "libtcl8.5.so".

ld.so fails to find libtcl8.5.so because neither openmsx nor ld.so know the path to libtcl8.5.so at runtime.

Yes, kind of like that, it's "a failure of another program".

I don't know what tclsh does.

joxy, so your complaint is not about a
failure of the tclsh program installed by
Tcl?  It's a failure of another program that
embeds and links to Tcl?  I think that's a 
point of confusion.

If you specify where to dig, I could dig. Currently, the digging area is too broad -- please narrow... I'm not familiar with build systems of tcl and openmsx.

File Added - 422483: tclConfig.sh

Comment title: Linker options for both tcl's make and for a program.

(A program is openmsx, and openmsx chat log connected with this tcl tracker item is here -- might have some details: https://sourceforge.net/tracker/?func=detail&aid=3401010&group_id=38274&atid=421861 )

1. openmsx's ldflags: TCL_LDFLAGS:=-L/usr/local/lib

(So openmsx's build process found a correct location for TCL8.5.)

2. tcl's ldflags:

Lines from /usr/local/lib/tclConfig.sh (attached to this tracker item):

# Flags to pass to ld, such as "-R /usr/local/tcl/lib", that tell the
# run-time dynamic linker where to look for shared libraries such as
# libtcl.so.  Used when linking applications.  Only works if there
# is a variable "LIB_RUNTIME_DIR" defined in the Makefile.
TCL_CC_SEARCH_FLAGS='-Wl,-rpath,${LIB_RUNTIME_DIR}'
TCL_LD_SEARCH_FLAGS='-Wl,-rpath,${LIB_RUNTIME_DIR}'

This does not specify which Makefile must define this LIB_RUNTIME_DIR variable.

A typo: The item 2 in prev.msg. is supposed to be:
"It seems that in Linux, there is no standard location for tcl's libtcl8.*.so".

I don't comprehend what your pronouns refer to.

1. "What linker options did it actually use?" Here, "it"=tcl's 'make' process OR "it"=a program that is using tcl? (In my next comment, I will try to list linker options for both tcl's make and for a program.)

2. "the -rpath option". It seems that in Linux, there is not standard location for tcl's libtcl8.*.so: in Ubuntu lucid repos, it is /usr/lib, and in the defaults of source package of tcl8.5, it is /usr/local/lib:
$ ls /usr/lib/libtcl8.4.so
/usr/lib/libtcl8.4.so
$ ls /usr/local/lib/libtcl8.5.so 
/usr/local/lib/libtcl8.5.so
$ 

Therefore, the user program cannot know where to search for libtcl*.so in a 'standard' tcl installation. Therefore, some mechanism like ld.so's path database (/etc/ld.so.conf) must be used.

3. "this should be enabled by default; it needs to be switched off for a build for a Linux distro though)." Here, what phrase "this" refers to? And also: what phrase "it" refers to?

What linker options did it actually use? (The configuration system is tricky enough that I don't try to second guess what it did.) AIUI, we're supposed to be passing in the -rpath option to the linker so that the binary knows where the libs are without running ldconfig (and this should be enabled by default; it needs to be switched off for a build for a Linux distro though).

uname -a: Linux name1 2.6.32-33-generic #72-Ubuntu SMP Fri Jul 29 21:08:37 UTC 2011 i686 GNU/Linux

Problematic Behavior: ldconfig was not run by 'make install' process of tcl.
Expected Behavior: ldconfig is run by 'make install' process of tcl.