Tcl Source Code

The TIP 268 implementation
broke this:

% package provide demo 1.2.3
% package require -exact demo 1.2    
1.2.3

Contrast with Tcl 8.4:

% package provide demo 1.2.3
% package require -exact demo 1.2    
version conflict for package "demo": have 1.2.3, need 1.2
% info patch
8.4.14

And with what I recall being
specified in TIP 268.

Logged In: YES 
user_id=80530
Originator: YES


followup patch for Tcl
and similar for Tk committed
for 8.5b1

File Added - 245332: initstubs.patch

Logged In: YES 
user_id=80530
Originator: YES


Re-opened while we chase down
a few consequences...

1) Need to consider what backport
   might be appropriate for the
   TCL_TIP268-enabled portions
   on the core-8-4-branch.

2) It's been pointed out that this
   change dashes the expectations of
   those extensions that have a history
   of calling

      Tcl_InitStubs(interp, TCL_VERSION, 1);

   Compiled against the HEAD tcl.h, that
   call will currently produce a library
   that cannot be [load]ed into HEAD Tcl.
   At least surprising, if not alarming.

   Attached is a patch that makes followup
   revisions to Tcl_InitStubs() (and associated
   macros and support) itself to restore the
   expectations of such callers.

   I've tested this patch against Tk with
   its own Tcl_InitStubs() calls edited to
   be as above.  It appears to be effective.


File Added: initstubs.patch

Logged In: YES 
user_id=80530
Originator: YES


fixed for Tcl/Tk 8.5b1

File Added - 245181: 1578344-6.patch

Logged In: YES 
user_id=80530
Originator: YES


Updated patch removes some
lingering remnants of the
misguided Tcl_PkgRequirePrefixMatch
routine and support.


File Added: 1578344-6.patch

File Deleted - 245172: 



File Added - 245178: 1578344-tk.patch

Logged In: YES 
user_id=80530
Originator: YES


Updated Tk patch adds improvemed
pkgIndex.tcl file.

       
File Added: 1578344-tk.patch

Logged In: YES 
user_id=80530
Originator: YES


d'oh!  tripped up by
the `make shell` failure
again! (1788551)

Logged In: YES 
user_id=80530
Originator: YES


something still flawed
in that patch:

% package require foo 1.2-1.2
expected version number but got "1.2-1.2"

File Added - 245172: 1578344-tk.patch

Logged In: YES 
user_id=80530
Originator: YES


Here is that Tk patch.
 
File Added: 1578344-tk.patch

File Added - 245171: 1578344-5.patch

Logged In: YES 
user_id=80530
Originator: YES


New patch limited to removing
DWIMmery in [package] without
adding any new DWIMmery elsewhere.

Requires an accompanying patch
for Tk 8.5 to convert it to Say What
It Means in its Tcl_InitStubs() and
[package require Tcl] calls.

File Added: 1578344-5.patch

Logged In: YES 
user_id=80530
Originator: YES


Some discussion on the chat
concludes this solution is
heading in the wrong direction.
It's replacing one DWIM with
another DWIM.

We're just gonna have to expect
that extensions will update to
the new realities of what Tcl
and Tk version numbers are available
in 8.5.  The tools are there for
them to Say What They Mean.

File Added - 245031: 1578344-4.patch

Logged In: YES 
user_id=80530
Originator: YES

File Added: 1578344-4.patch

Logged In: YES 
user_id=75003
Originator: NO

I retract the question ... Checking the uses of the function I am reminded that this generates an error message, and for that the 'exactly' is exactly right (sic!). Reads nicer too, good.

Logged In: YES 
user_id=75003
Originator: NO

Reading over your changes to AddRequirementsToResult, do I understand correctly that you are translating the form 'V-V' back into the old-style form, i.e. '-exact V' ? If yes, is the 'exactly' not a typo ?

Logged In: YES 
user_id=80530
Originator: YES


I think these patches do the right thing.

They provide compatible support for
the sensible historic ways to call
Tcl_InitStubs():

  Tcl_InitStubs(i, TCL_VERSION, 1)

This would be called to get a Tcl stubs
table from the same TCL_VERSION series
as was used to compile the extension.
With this patch, that same meaning is
maintained, without the need to damage
the compatibility of what 
[package require -exact] has always meant.

  Tcl_InitStubs(i, TCL_VERSION, 0)

would get a stub table from the same
TCL_VERSION series as was used to
compile the extension, or from any
compatible later version.  Again this
patch does the same thing when
called in the same way.

In addition, this patch permits some
other cases identified by jenglish to
be addressed for the first time.

  Tcl_InitStubs(i, TCL_PATCH_LEVEL, 0)

can be used to set a more discriminating
floor on the set of acceptable stub tables,
so that, say, stub tables from only Tcl 8.5.2
and later would be accepted, and attempts to
call in a Tcl 8.5.0 interp would fail.

  Tcl_InitStubs(i, TCL_PATCH_LEVEL, 1)

could be used to demand lockstep requirement
for a stub table from a single release of Tcl.
I tend to dismiss this a lot, since it counters
the multiple version capabilities provided by
stubs.  But there can be other reasons to use stubs
(getting an architecture suitable for starkit inclusion,
for instance), and from a strict perspective of
"support only what I test", this can be a useful
tool for some developers and some needs.

There are some other more complex requirement
needs that cannot be expressed with a single
Tcl_InitStubs() call even with this
patch in place, but I really cannot imagine
completely addressing those without an entirely
new interface routine.

There more complex requirements *can* be resolved
with this patch, if one uses Tcl_InitStubs() to
first get an 8.5a7 stubs table, and then make
a call to Tcl_PkgRequireProc() to make a requirements
test with the full power and generality of the
TIP 268 enhanced [package require] command.

File Added - 230929: 1578344-3.patch

Logged In: YES 
user_id=80530
Originator: YES


Updated patch takes care of a few more
details.
File Added: 1578344-3.patch

Logged In: YES 
user_id=68433
Originator: NO

I don't think this is the right way to fix the problem.

Instead of changing the semantics of Tcl_InitStubs() to be what Tk wants it to mean, Tk 8.5 should instead say exactly what it means (possibly by calling Tcl_InitStubs(interp, TCL_VERSION, 0); to initialize the stubs table, followed by further calls to Tcl_PkgRequirePrefixMatch to express the exact requirement).

Extensions other than Tk might very well want Tcl_InitStubs(interp, TCL_PATCH_LEVEL, 1); to mean "the exact version that I was compiled against, no prefix matches".  In particular, extensions that reference private headers might want this behaviour.

(That's been a problem with the 8.4 series -- there was no way to say "I need Tcl 8.4.N or later, because I'm using a routine that was added to the internal stubs table in that version", and there was no way to say "I want Tcl 8.4.N exactly, because ditto and furthermore there's no guarantee that it will still be in Tcl 8.4.N+1").

File Added - 230284: 1578344-2.patch

Logged In: YES 
user_id=80530
Originator: YES

attached patch is a small tweak
on the first one that I think
corrects the failure mode.

jenglish, care to have another look?


File Added: 1578344-2.patch

Logged In: YES 
user_id=80530
Originator: YES


jenglish points out that
the changes to Tcl_InitStubs()
in that patch won't work in 
all the cases they need to work.

If this patch went into Tcl 8.5,
and then an extension was compiled
against 8.5 headers, and linked to
libtclstub8.5a, that extension's
shared library file would contain
the modified Tcl_InitStubs() routine.

If that shared library file were
[load]ed into a Tcl 8.3 interp
(and the prefixMatching argument
were true), then an attempt to call
Tcl_PkgRequirePrefixMatching through
the stubs table would be attempted,
and since no such routine is in
an 8.3 stubs table the program would
crash.

back to that drawing board...

File Added - 230146: 1578344.patch

Logged In: YES 
user_id=80530
Originator: YES


The attached patch corrects everything
except the docs.

It corrects the bug reported here, but
also restores the compatibility of
Tcl_InitStubs() that was lost with the
original TIP 268 implementation, which
prompted the DWIM redefine for -exact
matching.  I think this solution ought
to satisfy everyone.

A backport of this solution to the
TCL_TIP268 sections on the core-8-4-branch
might be a good idea as well, but it's best
I leave that conclustion to the actual users
of that fork.  I'll help with producing such
a patch if it's desired.

File Added: 1578344.patch

Logged In: YES 
user_id=80530
Originator: YES


Ok, I see (at least some of) the
motivation for this now.  It comes
down to a deficiency in the Tcl_InitStubs()
interface.

Consider Tk 8.4 as an example of a stubs
enabled extension.  It calls

  Tcl_InitStubs(interp, TCL_VERSION, 1)

The Tk header file forces TCL_VERSION
to be "8.4", so this call is looking to
initialize "exactly" version 8.4 of
the Tcl stub table.

Trouble is that the stub table itself
does not include any version field.
Instead the "version" of a stub table
is implicitly taken to be the same
as the version of the "Tcl" package
in the same interp, via a call in
Tcl_InitStubs:

  Tcl_PkgRequireEx(interp, "Tcl",
      version, exact, &pkgData);

where version and exact have the
values TCL_VERSION and 1 passed into
Tcl_InitStubs originally.

Since the history has been that
Tcl only [package provide]d a
Tcl version to M.m resolution, the
implicit rule is that the Tcl stub
table was version 8.4 throughout
the entire 8.4* series of releases.

This has a few implications.  First
a value of 1 in the "exact" argument
to Tcl_InitStubs() has always meant
that any stub table from any 8.4*
release of Tcl is acceptable, but
that any table from 8.3* or 8.5*
releases is not.  For extensions like
Tk that may be accessing volatile internals,
this is a good kind of requirement to be
able to express.

Note that the history of having a single
version identifier for all the stub tables
of all Tcl releases matching 8.4.*
indicates that the stub table is meant
to be unchanging through the whole
8.4* series of releases.  Except for the
backport of TIP 218, we've even honored
that (in the default build anyway).

After TIP 268, [package provide Tcl]
now returns the full patchlevel version
number.  Since the workings of
Tcl_InitStubs() were not modified by
that TIP, we now effectively have
a new version of the stub table with
every release of Tcl.  This means
that Tcl_InitStubs() calls with exact=1
can only match a single release of Tcl,
which is not desirable.  Furthermore
passing both TCL_VERSION and exact=1
can only match the stub table to be
found in Tcl 8.5.0 which does not yet
exist.  This means without source code
changes, Tk 8.5a6 could not [load] into
any existing Tcl interp.

Tk 8.5a6 has addressed this in part
by changing its Tcl_InitStubs call to:

  Tcl_InitStubs(interp, TCL_PATCH_LEVEL, 1)

With the traditional meaning of -exact
matching, this would permit a Tk 8.5a6
built against Tcl 8.5a6 header file to [load] into
a Tcl 8.5a6 interp, but no other release at all.
This level of lockstep forced even on the
stubs interface is undesirable.

The old ability of Tcl_InitStubs to
approve the stub table of all Tcl
releases match 8.5* was restored by
the incomptible interpretation of
-exact matching noted in the original
bug report here.  It's an example
of restoring one kind of compatibility
by destroying another kind.

I think for the 8.5 series of releases,
this issue can successfully be solved
by making appropriate changes to
Tcl_InitStubs.  I'll be looking into that.

Logged In: YES 
user_id=80530
Originator: YES


ok, the docs and bug went in
at the same time; I just didn't
notice the doc before.

Still needs fixing.  Who's
got a round tuit?

Logged In: YES 
user_id=80530
Originator: YES


Hey!  Documenting the brokenness
doesn't count as fixing it!

Logged In: YES 
user_id=80530


I'm certain that making
guesses about what users
might mean is the wrong
approach.  

TIP 268 offers a well-specified
langauge for expressing exactly
what version requirements are
desired.  Users of Tcl 8.5 should
use that language to say what they
mean.  We should avoid DWIMery guessing.

For the 8.4 backport/fork, it's
possible the right solution is
for Tcl to continue [package provide]ing
version 8.4 throughout all the 8.4.*
series of releases, since that's what
some collection of scripts is assuming.
That was the right answer for Tk.
Are you making use of Tcl patchlevel
requirements in that backport?

Logged In: YES 
user_id=72656

I'm not so sure that it is a bug for 8.5 either.  Perhaps it
is that -exact is just not good enough when compared to what
we have now for version handling.

If a user asks for "1.2", are they asking for "1.2.0.0..."
or are they asking for "1.2.x but not 1.3 or 1.1"?  I
believe 98% are asking for the latter.

Logged In: YES 
user_id=80530


It might make sense for
the backport/fork in the
8.4.* series (the compatibility
calculus differs there);  I'd
have to think more carefully to be
certain.

I'm confident it doesn't
make sense going forward,
so at a minimum, this code
doesn't belong in the
HEAD branch.

FWIW, though this is a bug,
I don't think it's so critical
to hold 8.5a5 for, so long as
it is otherwise ready.

Logged In: YES 
user_id=72656

I talked with aku about making at least the 8.4 backport
support -exact 1.2 allowing 1.2.x IFF 1.2.0 was not
requested.  In other words, exactness should only be to the
level of specificity of the request.  Did that make sense?