Tcl Source Code

OriginalBugID: 4087 Bug
Version: 8.3b2
SubmitDate: '2000-01-26'
LastModified: '2000-04-09'
Severity: SER
Status: Assigned
Submitter: techsupp
ChangedBy: hobbs
RelatedBugIDs: 653
OS: Solaris
OSVersion: Solaris 7 64 bit
Machine: Ultra Enterprise 4500
FixedDate: '2000-10-25'
ClosedDate: '2000-10-25'


Name: Matthew Finch

ReproducibleScript:
If I compile tcl8.3b2 in 64 bit mode on Solaris 7 using -xarch=v9 then
perform the following I get....

# ./tclsh
% format %X 4294967290
integer value too large to represent as non-long integer
% 

If I compile tcl8.3b2 in 32 bit mode on Solaris 7 then perform the
following I get...

# ./tclsh
% format %X 4294967290
FFFFFFFA
% 

The problem is relates to the casting in Tcl_GetIntFromObj()

    if (((long)((int)l)) == l) {
        *intPtr = (int)objPtr->internalRep.longValue;
        return TCL_OK;
    }

The results should be the same.

ObservedBehavior:
If I compile tcl8.3b2 in 64 bit mode on Solaris 7 using -xarch=v9 then
perform the following I get....

# ./tclsh
% format %X 4294967290
integer value too large to represent as non-long integer
%

DesiredBehavior:
I expect to see the same behaviour as when I compiled tclsh on Solaris 7 in 32 bit mode

e.g
# ./tclsh
% format %X 4294967290
FFFFFFFA
%



Actually, there is some level of correctness here.  The number
the user is using is just -6 as far as Tcl is concerned on a
32bit system.  When there are true longs, then Tcl handles the
number properly in certain contexts.  However, format currently
isn't one of them.  This should be revisited in 8.4 to look at
expanding the use of Tcl_GetLongFromObj, which would solve the
problem here.  We just need to make sure it won't confuse
something else down the chain that will choke on the long value. 
-- 04/09/2000 hobbs

Logged In: YES 
user_id=79902

OK, so I'll make the code always work with longs on 64-bit
platforms, and the 'l' modifier will have no effect there. 
This seems to be what the test suite expects (that %d works
with long) so I'll make that the defined behaviour
everywhere.

Logged In: YES 
user_id=99768

dgp says that sizeof(int) == 4, while 
              sizeof(long) == sizeof(void*) == 8.

UINT_MAX is 0xffffffff.

The problem is that 0xfffffffa < UINT_MAX.  The test at line 2108
of tclCmdAH.c reads:

if ((unsigned long) intValue > UINT_MAX)

This test detects that 0xfffffffa can be formatted with %d
instead of %ld.  It cannot.

In any case, the code at line 2211:

sprintf(dst, newFormat, intValue);

is surely not correct.  The case that we're testing here gets there
with "%d" in newFormat, but "intValue" is a long: that surely cannot
be expected to work!

Logged In: YES 
user_id=79902

What is UINT_MAX on Linux/Alpha?  I suspect even more
strongly that the problem is that that is that
sizeof(int)==sizeof(long), and since that's the case,
bodging in a fix is easy; the hard bit will be ensuring that
it doesn't cause masses of warnings, and stopping people who
think they know what they are doing from fiddling with it.

Logged In: YES 
user_id=79902

Hmm. Can't duplicate the behaviour on Solaris/SPARC at
all...

Logged In: YES 
user_id=79902

Drat, missed a change in a manpage!

Don't know what's up on 64-bit Linux-Alpha, but the problem
might be that 'int' is 64 bits wide there (fuzzy memory or
hearsay.)  Will need to see if I can duplicate on 64-bit
Solaris-SPARC...

Logged In: YES 
user_id=80530

No.  There are still problems here.

As noted before, we need documentation so folks are not
surprised by this:

% format %d 4294967290
-6

Make it explicit that signed 32-bit values are used
unless the 'l' modifier is added:

% format %ld 4294967290
4294967290

Also in format.n, point [3] of the last section says that
the 'l' modifier is ignored.  That is clearly no longer
true.

Getting to the actual bug, on a 64-bit Linux-Alpha:

% format %d 4294967290
-6
% format %ld 4294967290
-6

so things are not consistent.  I think the intent of
TIP 72 was that there be consistent results on 32- and
64-bit platforms.

Logged In: YES 
user_id=79902

Is this fixed now?  I *think* so, but I'm not sure...

Logged In: YES 
user_id=79902

This was why I'd not changed it previously.  This issue will
need resolving as part of the 32/64 cleanup...

Logged In: YES 
user_id=80530

OK, commits to both HEAD and core-8-3-1-branch on
2001-09-19 have fixed this, sort of.  The reported
error is corrected:

% format %X 4294967290 
FFFFFFFA 

works on both 32-bit and 64-bit platforms.

However, the change also has this effect:

% format %d 4294967290
-6

on *both* 32-bit and 64-bit platforms.  If this
behavior is to be kept, then we need to document
in the format(n) page that %d formatting as a
signed decimal string places its sign boundary
at the range of a signed 32-bit integer.

Logged In: YES 
user_id=75003

There are two errors in Tcllib modules which seem to
result from the problem reported here:

http://sourceforge.net/tracker/index.php?
func=detail&aid=446997&group_id=12883&atid=112883

http://sourceforge.net/tracker/index.php?
func=detail&aid=447037&group_id=12883&atid=112883

Logged In: YES 
user_id=80530

I confirm that this patch adds the ability to avoid
this bug.  It does so by adding new features, though,
so perhaps a short TIP is in order.

I'm troubled by the non-Portability of test format-1.13.
Neither of these results really makes sense to me:

    # 32-bit platform:
    % format %d 4294967290
    -6

    # 64-bit platform:
    % format %d 4294967290
    integer value too large to represent as non-long integer

I understand what's going on, but why should the Tcl script
level claim that -6 == 4294967290 anywhere?  And if it
claims such nonsense anywhere, why not everywhere?

What I see above is the very platforms which *can* handle
the integer value 4294967290 are refusing to do so and
throwing an error instead, and those that *can't* are
silently converting it into -6 without throwing an error.

I suppose if we open the topic up a bit to a discussion of
how Tcl ought to be handling integer values at the script
level, then a TIP is definitely in order.

Logged In: YES 
user_id=79902

Please review...

File Added - 4393: format64.patch

Logged In: YES 
user_id=79902

Trying to upload a patch to fix this...

Logged In: YES 
user_id=79902

I am developing a patch that fixes this by allowing the use
of the 'l' specifier in format and scan.

Patch #102663 (quite inadvertently) updates the Tcl test suite with a
regression test for this problem.  

There was a bug on 32-bit platforms as well: negative integers
[format]ted with %x or %o could not be recovered by [scan]ning them
with %x or %o. The [scan] would not return an error but would silently
replace them with MAX_INT.

The fix for this bug includes a test case that tries to do
complementary [format] and [scan] of -1, MAX_INT, and MIN_INT; this
test case fails on 64-bit platforms because it trips over this issue
with [format].