Tcl Source Code

OriginalBugID: 2818 RFE
Version: 8.0.4
SubmitDate: '1999-09-16'
LastModified: '2000-05-15'
Severity: CRIT
Status: Assigned
Submitter: techsupp
ChangedBy: hobbs
RelatedBugIDs: 5493
OS: All Unix
OSVersion: 4.3.2
Machine: RS6000
FixedDate: '2000-10-25'
ClosedDate: '2000-10-25'


Name:
John Wiersba

ReproducibleScript:
generic/tclInt.h typedefs TclStat_ to be struct stat but hasn't included
sys/stat.h so when compiling with "large files" (> 2G) on AIX 4.3.2
there is a type mismatch (generic/tclCmdAH.c 1018) since sys/stat.h
#defines stat to be stat64 but sys/stat.h hasn't been used by tclInt.h.

Fix: generic/tclInt.h should #include <sys/stat.h> somewhere before line
1357.


This was originally for AIX, but Tcl in general doesn't support
>2GB files because it needs to extend a lot of the internal
code to use longs (size_t's actually) instead of ints. 
-- 03/30/2000 hobbs

Logged In: YES 
user_id=72656

This is correctly handled with dkf's work in TIP #72 now in 
8.4a4.

Logged In: YES 
user_id=79902

Of course, I should comment that the real problem with
64-bit offsets on 32-bit machines comes when those offsets
get back to the command-implementation level, where it is
currently impossible to handle them... :^(

Logged In: YES 
user_id=79902

Are you going to write the code for tcl.m4 to determine the
correct set of definitions so we can handle all this
largefile stuff?

I object to Tcl_off_t on the grounds that it is inconsistent
with the way the rest of Tcl's types are named.  :^)

Logged In: YES 
user_id=79902

Reexamine source:
Solaris depends on _FILE_OFFSET_BITS or _LP64, in a fairly
complex fashion, and only defines off64_t for you if
_LARGEFILE64_SOURCE is defined.  I've no idea what the
defaults for those all are, as that nest of #defines and
#ifdefs is not at all clear...

Logged In: YES 
user_id=124643

So what about a Tcl_off_t that is defined as
off64_t where applicable and off_t otherwise?

Logged In: YES 
user_id=79902

Not (or doesn't seem to be) true on Solaris, which is a
major target for Tcl.

Logged In: YES 
user_id=124643

What do you mean when you say:"off_t precludes off64_t"?

For what I understand, off_t is 64 bit on 64bit platforms.
It is also 64 bit on 32 bit platforms that support LFS if
you
enable it at compile time.

--- snip (/usr/include/sys/types.h on SuSE Linux 7.2) ---
# ifndef __USE_FILE_OFFSET64
typedef __off_t off_t;
# else
typedef __off64_t off_t;
# endif
--- snap ---

So to me it looks like that's exactly what we need for
Tcl_Seek and Tcl_Tell.

File Deleted - 10773: 



File Added - 10781: longseektelldrvcmd.patch

Logged In: YES 
user_id=79902

Can't be off_t since that precludes off64_t.
Can't be long long since that's not portable.
An independent guaranteed 64-bit number type requires a lot
more work *and* will have much wider impact upon the core,
especially including much of the bytecode implementation and
both [format] and [binary format].

64-bit support is non-trivial to add.  Converting up to at
least a long here will allow platforms that differentiate
between long and int (Alpha for sure, and maybe others) to
benefit while not forcing everything else to suffer.

Damn. Forgot Tcl_TellObjCmd and Tcl_SeekObjCmd...

Logged In: YES 
user_id=124643

Shouldn't the ints here be changed to off_t or long long
instead of long?
On 32bit plattforms int <=> long so this patch is not really
an improvement for those.

File Added - 10773: longseektellanddrv.patch

Logged In: YES 
user_id=79902

Here's the version with the driver update; if only it didn't
force a public API change, I'd check it in without asking
anyone else to give it the once-over...

File Added - 10772: longseektell.patch

Logged In: YES 
user_id=79902

I've yet to go through and fix the Tcl_DriverSeekProc type
(which needs to be long-returning since it is the new offset
in the file) but here are the fixes to make Tcl_Seek and
Tcl_Tell work with longs instead of ints, which makes a
difference on platforms where sizeof(long)!=sizeof(int) as
many (all?) of those support large files natively.  Note
that there are still many fixes to be done...

NB: The patch does some tricky stuff with stubs, because the
type signature of stubbed procs is changed and extension
code that currently exists will be expecting ints and not
longs...

At some point, this bug's category will need to change back
to 'File System', but much of the current stuff is 'Channel
System' and so needs considering there...

Logged In: YES 
user_id=124643

At the Tcl/Tk conference this year, I talked with Jeff
Hobbs, Larry Virden, Andreas Kupries and others about the
LFS issue. According to Jeff a patch that brings Tcl support
for large files is more of a bug fix than a feature add and
therefore doesn't need a TIP to make it in.

Logged In: YES 
user_id=32170

Seek and Tell are channel, not fs related, although of 
course all of this is tied up in 'big file support'. 

All of these problems are going to have to wait until 
someone decides they want to TIP and tackle large file 
support in Tcl.  Is there a sensible sourceforge category 
we can use for these things?  Something like "Known 
limitation/bug awaiting a TIP and contributor".

There's also no point in it being assigned to me.  It needs 
to be assigned to 'unknown future contributor'.

Logged In: YES 
user_id=79902

I've just noticed that Tcl_Seek() and Tcl_Tell() deal with
int offsets. That's Just Plain Wrong.

Logged In: YES 
user_id=79902

It seems that both Solaris and Linux can support large files
through the use of (l)stat64() and struct stat64, but while
Solaris gives you access to those by default, you need to
wave the magic define wand called _LARGEFILE64_SOURCE (which
is an implicit part of _GNU_SOURCE.)

I *HATE* magic wands...  :^(

Some details about Large File Support in Linux:

http://www.suse.de/~aj/linux_lfs.html