Tcl Source Code

Check-in [33041ae916]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:merge trunk
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | tip-445
Files: files | file ages | folders
SHA1: 33041ae916adf51f6a304e7f3668dbf9c555c425
User & Date: dgp 2016-12-02 19:01:40
Context
2016-12-19
18:43
merge trunk check-in: 2dbe7ac399 user: dgp tags: tip-445
2016-12-02
19:01
merge trunk check-in: 33041ae916 user: dgp tags: tip-445
18:18
Added long comment explaining history and work in progress making bytearray interfaces usable. check-in: d42a114238 user: dgp tags: trunk
2016-11-30
03:16
merge trunk check-in: c2b9c28871 user: dgp tags: tip-445
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Changes to ChangeLog.2000.

410
411
412
413
414
415
416
417
418
419
420
421
422
423
424

	* generic/tcl.decls:
	* generic/tclIO.c: updated Tcl_IsChannelShared,
	Tcl_IsChannelRegistered, Tcl_CutChannel, Tcl_SpliceChannel,
	Tcl_IsChannelExisting, and Tcl_ClearChannelHandlers to conform to the
	new stacked channel implementation. Their stub slots were also moved
	to give preference to the new 8.3.2 stub functions. This will cause an
	incompatability with 8.4a1 only.
	(StopCopy): fixed a bug introduced by a partial fix in 8.3.2 that
	didn't set nonBlocking correctly when resetting the flags for the
	write side. [Bug: 6261]

	* doc/ChnlStack.3:
	* doc/CrtChannel.3:
	* generic/tcl.decls:







|







410
411
412
413
414
415
416
417
418
419
420
421
422
423
424

	* generic/tcl.decls:
	* generic/tclIO.c: updated Tcl_IsChannelShared,
	Tcl_IsChannelRegistered, Tcl_CutChannel, Tcl_SpliceChannel,
	Tcl_IsChannelExisting, and Tcl_ClearChannelHandlers to conform to the
	new stacked channel implementation. Their stub slots were also moved
	to give preference to the new 8.3.2 stub functions. This will cause an
	incompatibility with 8.4a1 only.
	(StopCopy): fixed a bug introduced by a partial fix in 8.3.2 that
	didn't set nonBlocking correctly when resetting the flags for the
	write side. [Bug: 6261]

	* doc/ChnlStack.3:
	* doc/CrtChannel.3:
	* generic/tcl.decls:

Changes to ChangeLog.2002.

843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
	TCL_MEM_DEBUG is used. [Bug 583445]

	* win/tclWinConsole.c (ConsoleCloseProc): only wait on writable pipe
	if there was something to write. This may prevent infinite wait on
	exit.

	* tests/exec.test: marked exec-18.1 unixOnly until the Windows
	incompatability (in the test, not the core) can be resolved.

	* tests/http.test (http-3.11): added close $fp that was causing an
	error on Windows because the file was not closed before deleting.

	* unix/tclUnixInit.c (Tcl_MacOSXGetLibraryPath): made this static
	function only appear when HAVE_CFBUNDLE is defined.








|







843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
	TCL_MEM_DEBUG is used. [Bug 583445]

	* win/tclWinConsole.c (ConsoleCloseProc): only wait on writable pipe
	if there was something to write. This may prevent infinite wait on
	exit.

	* tests/exec.test: marked exec-18.1 unixOnly until the Windows
	incompatibility (in the test, not the core) can be resolved.

	* tests/http.test (http-3.11): added close $fp that was causing an
	error on Windows because the file was not closed before deleting.

	* unix/tclUnixInit.c (Tcl_MacOSXGetLibraryPath): made this static
	function only appear when HAVE_CFBUNDLE is defined.

3638
3639
3640
3641
3642
3643
3644
3645
3646
3647
3648
3649
3650
3651
3652
	[regsub] returns the modified string.
	* doc/regsub.n: Updated docs.
	* tests/regexp.test: Updated and added tests.

	* compat/strtoll.c (strtoll):
	* compat/strtoull.c (strtoull):
	* unix/tclUnixPort.h:
	* win/tclWinPort.h: Const-ing 64-bit compatability declarations. Note
	that the return pointer is non-const because it is entirely legal for
	the functions to be called from somewhere that owns the string being
	passed. Fixes problem reported by Larry Virden.

2002-02-21  David Gravereaux <[email protected]>

	* win/mkd.bat (removed):







|







3638
3639
3640
3641
3642
3643
3644
3645
3646
3647
3648
3649
3650
3651
3652
	[regsub] returns the modified string.
	* doc/regsub.n: Updated docs.
	* tests/regexp.test: Updated and added tests.

	* compat/strtoll.c (strtoll):
	* compat/strtoull.c (strtoull):
	* unix/tclUnixPort.h:
	* win/tclWinPort.h: Const-ing 64-bit compatibility declarations. Note
	that the return pointer is non-const because it is entirely legal for
	the functions to be called from somewhere that owns the string being
	passed. Fixes problem reported by Larry Virden.

2002-02-21  David Gravereaux <[email protected]>

	* win/mkd.bat (removed):
3775
3776
3777
3778
3779
3780
3781
3782
3783
3784
3785
3786
3787
3788
3789

	+----------------------+
	| TIP #72 IMPLEMENTED. |
	+----------------------+

	There are a lot of changes from this TIP, so please see
	http://purl.org/tcl/tip/72.html for discussion of
	backward-compatability issues, but the main ones modifications are in:

	* generic/tcl.h: New types.
	* generic/tcl.decls: New public functions.
	* generic/tclExecute.c: 64-bit aware bytecode engine.
	* generic/tclBinary.c: 64-bit handling in [binary] command.
	* generic/tclScan.c: 64-bit handling in [scan] command.
	* generic/tclCmdAH.c: 64-bit handling in [file] and [format]







|







3775
3776
3777
3778
3779
3780
3781
3782
3783
3784
3785
3786
3787
3788
3789

	+----------------------+
	| TIP #72 IMPLEMENTED. |
	+----------------------+

	There are a lot of changes from this TIP, so please see
	http://purl.org/tcl/tip/72.html for discussion of
	backward-compatibility issues, but the main ones modifications are in:

	* generic/tcl.h: New types.
	* generic/tcl.decls: New public functions.
	* generic/tclExecute.c: 64-bit aware bytecode engine.
	* generic/tclBinary.c: 64-bit handling in [binary] command.
	* generic/tclScan.c: 64-bit handling in [scan] command.
	* generic/tclCmdAH.c: 64-bit handling in [file] and [format]

Changes to ChangeLog.2003.

943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
	function in multiple interfaces simultaneously.

	* generic/tcl.decls: Duplicated some namespace declarations from
	tclInt.decls here, as mandated by TIP #139. This is OK since the
	declarations match and will end up using the declarations in the
	public code from now on because of #include ordering. Keeping the old
	declarations in tclInt.decls; there's no need to gratuitously break
	compatability for those extensions which are already clients of the
	namespace code.

2003-08-23  Zoran Vasiljevic  <[email protected]>

	* generic/tclIOUtil.c: merged fixes for thread-unsafe handling of
	filesystem records [Bug 753315]. This also fixed the [Bug 788780]
	* generic/tclPathObj.c: merged fixes for thread-unsafe handling of







|







943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
	function in multiple interfaces simultaneously.

	* generic/tcl.decls: Duplicated some namespace declarations from
	tclInt.decls here, as mandated by TIP #139. This is OK since the
	declarations match and will end up using the declarations in the
	public code from now on because of #include ordering. Keeping the old
	declarations in tclInt.decls; there's no need to gratuitously break
	compatibility for those extensions which are already clients of the
	namespace code.

2003-08-23  Zoran Vasiljevic  <[email protected]>

	* generic/tclIOUtil.c: merged fixes for thread-unsafe handling of
	filesystem records [Bug 753315]. This also fixed the [Bug 788780]
	* generic/tclPathObj.c: merged fixes for thread-unsafe handling of
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288

	* generic/tclCmdMZ.c (Tcl_StringObjCmd): Made [string map] accept
	dictionaries for maps.  This is much trickier than it looks, since map
	entry ordering is significant. [Bug 759936]

	* generic/tclVar.c (Tcl_ArrayObjCmd, TclArraySet): Made [array get]
	and [array set] work with dictionaries, producing them and consuming
	them. Note that for compatability reasons, you will never get a dict
	from feeding a string literal to [array set] since that alters the
	trace behaviour of "multi-key" sets. [Bug 759935]

2003-06-23  Vince Darley  <[email protected]>

	* generic/tclTrace.c: fix to Window debug build compilation error.








|







1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288

	* generic/tclCmdMZ.c (Tcl_StringObjCmd): Made [string map] accept
	dictionaries for maps.  This is much trickier than it looks, since map
	entry ordering is significant. [Bug 759936]

	* generic/tclVar.c (Tcl_ArrayObjCmd, TclArraySet): Made [array get]
	and [array set] work with dictionaries, producing them and consuming
	them. Note that for compatibility reasons, you will never get a dict
	from feeding a string literal to [array set] since that alters the
	trace behaviour of "multi-key" sets. [Bug 759935]

2003-06-23  Vince Darley  <[email protected]>

	* generic/tclTrace.c: fix to Window debug build compilation error.

Changes to ChangeLog.2004.

1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
	code splitting [Bug 925620] removing the need for several #ifdef's,
	and tests and fix for an unreported Windows glob problem ('glob -dir
	C: -tails *').

2004-10-07  Donal K. Fellows  <[email protected]>

	* *.3: Convert CONST to const and VOID to void so we document how
	people should actually use the Tcl API and not the compatability hacks
	that it has to have.

	* doc/man.macros, *.3: Update .AS macro so it can know how wide to
	make the third column of the argument list. Update documentation for C
	API (only users) to take advantage of this.

	* doc/FileSystem.3: Formatting fixes for greater documentation







|







1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
	code splitting [Bug 925620] removing the need for several #ifdef's,
	and tests and fix for an unreported Windows glob problem ('glob -dir
	C: -tails *').

2004-10-07  Donal K. Fellows  <[email protected]>

	* *.3: Convert CONST to const and VOID to void so we document how
	people should actually use the Tcl API and not the compatibility hacks
	that it has to have.

	* doc/man.macros, *.3: Update .AS macro so it can know how wide to
	make the third column of the argument list. Update documentation for C
	API (only users) to take advantage of this.

	* doc/FileSystem.3: Formatting fixes for greater documentation

Changes to generic/tcl.h.

1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
 * TCL_ONE_WORD_KEYS:		The keys are pointers, the pointer is stored
 *				in the entry.
 * TCL_CUSTOM_TYPE_KEYS:	The keys are arbitrary types which are copied
 *				into the entry.
 * TCL_CUSTOM_PTR_KEYS:		The keys are pointers to arbitrary types, the
 *				pointer is stored in the entry.
 *
 * While maintaining binary compatability the above have to be distinct values
 * as they are used to differentiate between old versions of the hash table
 * which don't have a typePtr and new ones which do. Once binary compatability
 * is discarded in favour of making more wide spread changes TCL_STRING_KEYS
 * can be the same as TCL_CUSTOM_TYPE_KEYS, and TCL_ONE_WORD_KEYS can be the
 * same as TCL_CUSTOM_PTR_KEYS because they simply determine how the key is
 * accessed from the entry and not the behaviour.
 */

#define TCL_STRING_KEYS		(0)







|

|







1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
 * TCL_ONE_WORD_KEYS:		The keys are pointers, the pointer is stored
 *				in the entry.
 * TCL_CUSTOM_TYPE_KEYS:	The keys are arbitrary types which are copied
 *				into the entry.
 * TCL_CUSTOM_PTR_KEYS:		The keys are pointers to arbitrary types, the
 *				pointer is stored in the entry.
 *
 * While maintaining binary compatibility the above have to be distinct values
 * as they are used to differentiate between old versions of the hash table
 * which don't have a typePtr and new ones which do. Once binary compatibility
 * is discarded in favour of making more wide spread changes TCL_STRING_KEYS
 * can be the same as TCL_CUSTOM_TYPE_KEYS, and TCL_ONE_WORD_KEYS can be the
 * same as TCL_CUSTOM_PTR_KEYS because they simply determine how the key is
 * accessed from the entry and not the behaviour.
 */

#define TCL_STRING_KEYS		(0)

Changes to generic/tclBasic.c.

3019
3020
3021
3022
3023
3024
3025
3026
3027
3028
3029
3030
3031
3032
3033
3034
3035
3036
3037
3038
3039
    Tcl_Command cmd)		/* Token for command to delete. */
{
    Interp *iPtr = (Interp *) interp;
    Command *cmdPtr = (Command *) cmd;
    ImportRef *refPtr, *nextRefPtr;
    Tcl_Command importCmd;

    /*
     * Bump the command epoch counter. This will invalidate all cached
     * references that point to this command.
     */

    cmdPtr->cmdEpoch++;

    /*
     * The code here is tricky. We can't delete the hash table entry before
     * invoking the deletion callback because there are cases where the
     * deletion callback needs to invoke the command (e.g. object systems such
     * as OTcl). However, this means that the callback could try to delete or
     * rename the command. The deleted flag allows us to detect these cases
     * and skip nested deletes.







<
<
<
<
<
<
<







3019
3020
3021
3022
3023
3024
3025







3026
3027
3028
3029
3030
3031
3032
    Tcl_Command cmd)		/* Token for command to delete. */
{
    Interp *iPtr = (Interp *) interp;
    Command *cmdPtr = (Command *) cmd;
    ImportRef *refPtr, *nextRefPtr;
    Tcl_Command importCmd;








    /*
     * The code here is tricky. We can't delete the hash table entry before
     * invoking the deletion callback because there are cases where the
     * deletion callback needs to invoke the command (e.g. object systems such
     * as OTcl). However, this means that the callback could try to delete or
     * rename the command. The deleted flag allows us to detect these cases
     * and skip nested deletes.
3048
3049
3050
3051
3052
3053
3054








3055
3056
3057
3058
3059
3060
3061
	 * three times, everything goes up in smoke. [Bug 1220058]
	 */

	if (cmdPtr->hPtr != NULL) {
	    Tcl_DeleteHashEntry(cmdPtr->hPtr);
	    cmdPtr->hPtr = NULL;
	}








	return 0;
    }

    /*
     * We must delete this command, even though both traces and delete procs
     * may try to avoid this (renaming the command etc). Also traces and
     * delete procs may try to delete the command themsevles. This flag







>
>
>
>
>
>
>
>







3041
3042
3043
3044
3045
3046
3047
3048
3049
3050
3051
3052
3053
3054
3055
3056
3057
3058
3059
3060
3061
3062
	 * three times, everything goes up in smoke. [Bug 1220058]
	 */

	if (cmdPtr->hPtr != NULL) {
	    Tcl_DeleteHashEntry(cmdPtr->hPtr);
	    cmdPtr->hPtr = NULL;
	}

	/*
	 * Bump the command epoch counter. This will invalidate all cached
	 * references that point to this command.
	 */

	cmdPtr->cmdEpoch++;

	return 0;
    }

    /*
     * We must delete this command, even though both traces and delete procs
     * may try to avoid this (renaming the command etc). Also traces and
     * delete procs may try to delete the command themsevles. This flag
3150
3151
3152
3153
3154
3155
3156







3157
3158
3159
3160
3161
3162
3163
     * cmdPtr->hptr, and make sure that no-one else has already deleted the
     * hash entry.
     */

    if (cmdPtr->hPtr != NULL) {
	Tcl_DeleteHashEntry(cmdPtr->hPtr);
	cmdPtr->hPtr = NULL;







    }

    /*
     * A number of tests for particular kinds of commands are done by checking
     * whether the objProc field holds a known value. Set the field to NULL so
     * that such tests won't have false positives when applied to deleted
     * commands.







>
>
>
>
>
>
>







3151
3152
3153
3154
3155
3156
3157
3158
3159
3160
3161
3162
3163
3164
3165
3166
3167
3168
3169
3170
3171
     * cmdPtr->hptr, and make sure that no-one else has already deleted the
     * hash entry.
     */

    if (cmdPtr->hPtr != NULL) {
	Tcl_DeleteHashEntry(cmdPtr->hPtr);
	cmdPtr->hPtr = NULL;

	/*
	 * Bump the command epoch counter. This will invalidate all cached
	 * references that point to this command.
	 */

	cmdPtr->cmdEpoch++;
    }

    /*
     * A number of tests for particular kinds of commands are done by checking
     * whether the objProc field holds a known value. Set the field to NULL so
     * that such tests won't have false positives when applied to deleted
     * commands.

Changes to generic/tclBinary.c.

53
54
55
56
57
58
59


60
61
62

63
64
65
66
67
68
69

/*
 * Prototypes for local procedures defined in this file:
 */

static void		DupByteArrayInternalRep(Tcl_Obj *srcPtr,
			    Tcl_Obj *copyPtr);


static int		FormatNumber(Tcl_Interp *interp, int type,
			    Tcl_Obj *src, unsigned char **cursorPtr);
static void		FreeByteArrayInternalRep(Tcl_Obj *objPtr);

static int		GetFormatSpec(const char **formatPtr, char *cmdPtr,
			    int *countPtr, int *flagsPtr);
static Tcl_Obj *	ScanNumber(unsigned char *buffer, int type,
			    int flags, Tcl_HashTable **numberCachePtr);
static int		SetByteArrayFromAny(Tcl_Interp *interp,
			    Tcl_Obj *objPtr);
static void		UpdateStringOfByteArray(Tcl_Obj *listPtr);







>
>



>







53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72

/*
 * Prototypes for local procedures defined in this file:
 */

static void		DupByteArrayInternalRep(Tcl_Obj *srcPtr,
			    Tcl_Obj *copyPtr);
static void		DupProperByteArrayInternalRep(Tcl_Obj *srcPtr,
			    Tcl_Obj *copyPtr);
static int		FormatNumber(Tcl_Interp *interp, int type,
			    Tcl_Obj *src, unsigned char **cursorPtr);
static void		FreeByteArrayInternalRep(Tcl_Obj *objPtr);
static void		FreeProperByteArrayInternalRep(Tcl_Obj *objPtr);
static int		GetFormatSpec(const char **formatPtr, char *cmdPtr,
			    int *countPtr, int *flagsPtr);
static Tcl_Obj *	ScanNumber(unsigned char *buffer, int type,
			    int flags, Tcl_HashTable **numberCachePtr);
static int		SetByteArrayFromAny(Tcl_Interp *interp,
			    Tcl_Obj *objPtr);
static void		UpdateStringOfByteArray(Tcl_Obj *listPtr);
152
153
154
155
156
157
158
159


160





161



162
163
164
165
166

167
168
169

























170
171
172

173







174







175
176
177
178







179



180









181








182
183
184
185
186
187
188
189
190
191
192
193
194
    { "hex",      BinaryDecodeHex, TclCompileBasic1Or2ArgCmd, NULL, NULL, 0 },
    { "uuencode", BinaryDecodeUu,  TclCompileBasic1Or2ArgCmd, NULL, NULL, 0 },
    { "base64",   BinaryDecode64,  TclCompileBasic1Or2ArgCmd, NULL, NULL, 0 },
    { NULL, NULL, NULL, NULL, NULL, 0 }
};

/*
 * The following object type represents an array of bytes. An array of bytes


 * is not equivalent to an internationalized string. Conceptually, a string is





 * an array of 16-bit quantities organized as a sequence of properly formed



 * UTF-8 characters, while a ByteArray is an array of 8-bit quantities.
 * Accessor functions are provided to convert a ByteArray to a String or a
 * String to a ByteArray. Two or more consecutive bytes in an array of bytes
 * may look like a single UTF-8 character if the array is casually treated as
 * a string. But obtaining the String from a ByteArray is guaranteed to

 * produced properly formed UTF-8 sequences so that there is a one-to-one map
 * between bytes and characters.
 *

























 * Converting a ByteArray to a String proceeds by casting each byte in the
 * array to a 16-bit quantity, treating that number as a Unicode character,
 * and storing the UTF-8 version of that Unicode character in the String. For

 * ByteArrays consisting entirely of values 1..127, the corresponding String







 * representation is the same as the ByteArray representation.







 *
 * Converting a String to a ByteArray proceeds by getting the Unicode
 * representation of each character in the String, casting it to a byte by
 * truncating the upper 8 bits, and then storing the byte in the ByteArray.







 * Converting from ByteArray to String and back to ByteArray is not lossy, but



 * converting an arbitrary String to a ByteArray may be.









 */









const Tcl_ObjType tclByteArrayType = {
    "bytearray",
    FreeByteArrayInternalRep,
    DupByteArrayInternalRep,
    UpdateStringOfByteArray,
    SetByteArrayFromAny
};

/*
 * The following structure is the internal rep for a ByteArray object. Keeps
 * track of how much memory has been used and how much has been allocated for
 * the byte array to enable growing and shrinking of the ByteArray object with







|
>
>
|
>
>
>
>
>
|
>
>
>
|
<
<
<
<
>
|
|
|
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|
<
|
>
|
>
>
>
>
>
>
>
|
>
>
>
>
>
>
>

|
|
|
>
>
>
>
>
>
>
|
>
>
>
|
>
>
>
>
>
>
>
>
>

>
>
>
>
>
>
>
>





|







155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175




176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205

206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
    { "hex",      BinaryDecodeHex, TclCompileBasic1Or2ArgCmd, NULL, NULL, 0 },
    { "uuencode", BinaryDecodeUu,  TclCompileBasic1Or2ArgCmd, NULL, NULL, 0 },
    { "base64",   BinaryDecode64,  TclCompileBasic1Or2ArgCmd, NULL, NULL, 0 },
    { NULL, NULL, NULL, NULL, NULL, 0 }
};

/*
 * The following object types represent an array of bytes. The intent is
 * to allow arbitrary binary data to pass through Tcl as a Tcl value
 * without loss or damage. Such values are useful for things like
 * encoded strings or Tk images to name just two.
 *
 * It's strange to have two Tcl_ObjTypes in place for this task when
 * one would do, so a bit of detail and history how we got to this point
 * and where we might go from here.
 *
 * A bytearray is an ordered sequence of bytes. Each byte is an integer
 * value in the range [0-255].  To be a Tcl value type, we need a way to
 * encode each value in the value set as a Tcl string.  The simplest
 * encoding is to represent each byte value as the same codepoint value.
 * A bytearray of N bytes is encoded into a Tcl string of N characters




 * where the codepoint of each character is the value of corresponding byte.
 * This approach creates a one-to-one map between all bytearray values
 * and a subset of Tcl string values.
 * 
 * When converting a Tcl string value to the bytearray internal rep, the
 * question arises what to do with strings outside that subset?  That is,
 * those Tcl strings containing at least one codepoint greater than 255?
 * The obviously correct answer is to raise an error!  That string value
 * does not represent any valid bytearray value. Full Stop.  The
 * setFromAnyProc signature has a completion code return value for just
 * this reason, to reject invalid inputs.
 * 
 * Unfortunately this was not the path taken by the authors of the
 * original tclByteArrayType.  They chose to accept all Tcl string values
 * as acceptable string encodings of the bytearray values that result
 * from masking away the high bits of any codepoint value at all. This
 * meant that every bytearray value had multiple accepted string
 * representations.
 *
 * The implications of this choice are truly ugly.  When a Tcl value has
 * a string representation, we are required to accept that as the true
 * value.  Bytearray values that possess a string representation cannot
 * be processed as bytearrays because we cannot know which true value
 * that bytearray represents.  The consequence is that we drag around
 * an internal rep that we cannot make any use of.  This painful price
 * is extracted at any point after a string rep happens to be generated
 * for the value.  This happens even when the troublesome codepoints
 * outside the byte range never show up.  This happens rather routinely
 * in normal Tcl operations unless we burden the script writer with the
 * cognitive burden of avoiding it.  The price is also paid by callers

 * of the C interface.  The routine
 *
 *	unsigned char *Tcl_GetByteArrayFromObj(objPtr, lenPtr)
 *
 * has a guarantee to always return a non-NULL value, but that value
 * points to a byte sequence that cannot be used by the caller to  
 * process the Tcl value absent some sideband testing that objPtr
 * is "pure".  Tcl offers no public interface to perform this test,
 * so callers either break encapsulation or are unavoidably buggy.  Tcl
 * has defined a public interface that cannot be used correctly. The
 * Tcl source code itself suffers the same problem, and has been buggy,
 * but progressively less so as more and more portions of the code have
 * been retrofitted with the required "purity testing".  The set of values
 * able to pass the purity test can be increased via the introduction of
 * a "canonical" flag marker, but the only way the broken interface itself
 * can be discarded is to start over and define the Tcl_ObjType properly.
 * Bytearrays should simply be usable as bytearrays without a kabuki
 * dance of testing.
 *
 * The Tcl_ObjType "properByteArrayType" is (nearly) a correct 
 * implementation of bytearrays.  Any Tcl value with the type
 * properByteArrayType can have its bytearray value fetched and
 * used with confidence that acting on that value is equivalent to
 * acting on the true Tcl string value.  This still implies a side
 * testing burden -- past mistakes will not let us avoid that
 * immediately, but it is at least a conventional test of type, and
 * can be implemented entirely by examining the objPtr fields, with
 * no need to query the intrep, as a canonical flag would require.
 *
 * Until Tcl_GetByteArrayFromObj() and Tcl_SetByteArrayLength() can
 * be revised to admit the possibility of returning NULL when the true
 * value is not a valid bytearray, we need a mechanism to retain
 * compatibility with the deployed callers of the broken interface.
 * That's what the retained "tclByteArrayType" provides.  In those
 * unusual circumstances where we convert an invalid bytearray value
 * to a bytearray type, it is to this legacy type.  Essentially any
 * time this legacy type gets used, it's a signal of a bug being ignored.
 * A TIP should be drafted to remove this connection to the broken past
 * so that Tcl 9 will no longer have any trace of it.  Prescribing a
 * migration path will be the key element of that work.  The internal
 * changes now in place are the limit of what can be done short of
 * interface repair.  They provide a great expansion of the histories
 * over which bytearray values can be useful in the meanwhile.
 */

static const Tcl_ObjType properByteArrayType = {
    "bytearray",
    FreeProperByteArrayInternalRep,
    DupProperByteArrayInternalRep,
    UpdateStringOfByteArray,
    NULL
};

const Tcl_ObjType tclByteArrayType = {
    "bytearray",
    FreeByteArrayInternalRep,
    DupByteArrayInternalRep,
    NULL,
    SetByteArrayFromAny
};

/*
 * The following structure is the internal rep for a ByteArray object. Keeps
 * track of how much memory has been used and how much has been allocated for
 * the byte array to enable growing and shrinking of the ByteArray object with
207
208
209
210
211
212
213






214
215
216
217
218
219
220

#define BYTEARRAY_SIZE(len) \
		((unsigned) (TclOffset(ByteArray, bytes) + (len)))
#define GET_BYTEARRAY(irPtr) ((ByteArray *) (irPtr)->twoPtrValue.ptr1)
#define SET_BYTEARRAY(irPtr, baPtr) \
		(irPtr)->twoPtrValue.ptr1 = (void *) (baPtr)








/*
 *----------------------------------------------------------------------
 *
 * Tcl_NewByteArrayObj --
 *
 *	This procedure is creates a new ByteArray object and initializes it







>
>
>
>
>
>







283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302

#define BYTEARRAY_SIZE(len) \
		((unsigned) (TclOffset(ByteArray, bytes) + (len)))
#define GET_BYTEARRAY(irPtr) ((ByteArray *) (irPtr)->twoPtrValue.ptr1)
#define SET_BYTEARRAY(irPtr, baPtr) \
		(irPtr)->twoPtrValue.ptr1 = (void *) (baPtr)

int
TclIsPureByteArray(
    Tcl_Obj * objPtr)
{
    return (NULL != Tcl_FetchIntRep(objPtr, &properByteArrayType));
}

/*
 *----------------------------------------------------------------------
 *
 * Tcl_NewByteArrayObj --
 *
 *	This procedure is creates a new ByteArray object and initializes it
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
    byteArrayPtr->allocated = length;

    if ((bytes != NULL) && (length > 0)) {
	memcpy(byteArrayPtr->bytes, bytes, (size_t) length);
    }
    SET_BYTEARRAY(&ir, byteArrayPtr);

    Tcl_StoreIntRep(objPtr, &tclByteArrayType, &ir);
}

/*
 *----------------------------------------------------------------------
 *
 * Tcl_GetByteArrayFromObj --
 *







|







421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
    byteArrayPtr->allocated = length;

    if ((bytes != NULL) && (length > 0)) {
	memcpy(byteArrayPtr->bytes, bytes, (size_t) length);
    }
    SET_BYTEARRAY(&ir, byteArrayPtr);

    Tcl_StoreIntRep(objPtr, &properByteArrayType, &ir);
}

/*
 *----------------------------------------------------------------------
 *
 * Tcl_GetByteArrayFromObj --
 *
367
368
369
370
371
372
373
374
375
376


377


378


379
380
381
382
383
384
385
unsigned char *
Tcl_GetByteArrayFromObj(
    Tcl_Obj *objPtr,		/* The ByteArray object. */
    int *lengthPtr)		/* If non-NULL, filled with length of the
				 * array of bytes in the ByteArray object. */
{
    ByteArray *baPtr;
    const Tcl_ObjIntRep *irPtr = Tcl_FetchIntRep(objPtr, &tclByteArrayType);

    if (irPtr == NULL) {


	SetByteArrayFromAny(NULL, objPtr);


	irPtr = Tcl_FetchIntRep(objPtr, &tclByteArrayType);


    }
    baPtr = GET_BYTEARRAY(irPtr);

    if (lengthPtr != NULL) {
	*lengthPtr = baPtr->used;
    }
    return baPtr->bytes;







|


>
>
|
>
>
|
>
>







449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
unsigned char *
Tcl_GetByteArrayFromObj(
    Tcl_Obj *objPtr,		/* The ByteArray object. */
    int *lengthPtr)		/* If non-NULL, filled with length of the
				 * array of bytes in the ByteArray object. */
{
    ByteArray *baPtr;
    const Tcl_ObjIntRep *irPtr = Tcl_FetchIntRep(objPtr, &properByteArrayType);

    if (irPtr == NULL) {
	irPtr = Tcl_FetchIntRep(objPtr, &tclByteArrayType);
	if (irPtr == NULL) {
	    SetByteArrayFromAny(NULL, objPtr);
	    irPtr = Tcl_FetchIntRep(objPtr, &properByteArrayType);
	    if (irPtr == NULL) {
		irPtr = Tcl_FetchIntRep(objPtr, &tclByteArrayType);
	    }
	}
    }
    baPtr = GET_BYTEARRAY(irPtr);

    if (lengthPtr != NULL) {
	*lengthPtr = baPtr->used;
    }
    return baPtr->bytes;
418
419
420
421
422
423
424



425
426
427


428


429
430
431
432
433
434
435

    assert(length >= 0);
    newLength = (unsigned int)length;

    if (Tcl_IsShared(objPtr)) {
	Tcl_Panic("%s called with shared object", "Tcl_SetByteArrayLength");
    }



    irPtr = Tcl_FetchIntRep(objPtr, &tclByteArrayType);
    if (irPtr == NULL) {
	SetByteArrayFromAny(NULL, objPtr);


	irPtr = Tcl_FetchIntRep(objPtr, &tclByteArrayType);


    }

    byteArrayPtr = GET_BYTEARRAY(irPtr);
    if (newLength > byteArrayPtr->allocated) {
	byteArrayPtr = ckrealloc(byteArrayPtr, BYTEARRAY_SIZE(newLength));
	byteArrayPtr->allocated = newLength;
	SET_BYTEARRAY(irPtr, byteArrayPtr);







>
>
>
|
|
|
>
>
|
>
>







506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530

    assert(length >= 0);
    newLength = (unsigned int)length;

    if (Tcl_IsShared(objPtr)) {
	Tcl_Panic("%s called with shared object", "Tcl_SetByteArrayLength");
    }

    irPtr = Tcl_FetchIntRep(objPtr, &properByteArrayType);
    if (irPtr == NULL) {
	irPtr = Tcl_FetchIntRep(objPtr, &tclByteArrayType);
	if (irPtr == NULL) {
	    SetByteArrayFromAny(NULL, objPtr);
	    irPtr = Tcl_FetchIntRep(objPtr, &properByteArrayType);
	    if (irPtr == NULL) {
		irPtr = Tcl_FetchIntRep(objPtr, &tclByteArrayType);
	    }
	}
    }

    byteArrayPtr = GET_BYTEARRAY(irPtr);
    if (newLength > byteArrayPtr->allocated) {
	byteArrayPtr = ckrealloc(byteArrayPtr, BYTEARRAY_SIZE(newLength));
	byteArrayPtr->allocated = newLength;
	SET_BYTEARRAY(irPtr, byteArrayPtr);
456
457
458
459
460
461
462
463
464
465
466
467
468



469


470
471
472
473
474
475
476
477
478

479
480
481
482
483
484
485
486

487
488
489
490
491
492
493
 */

static int
SetByteArrayFromAny(
    Tcl_Interp *interp,		/* Not used. */
    Tcl_Obj *objPtr)		/* The object to convert to type ByteArray. */
{
    int length;
    const char *src, *srcEnd;
    unsigned char *dst;
    ByteArray *byteArrayPtr;
    Tcl_ObjIntRep ir;




    assert (NULL == Tcl_FetchIntRep(objPtr, &tclByteArrayType));



    src = TclGetStringFromObj(objPtr, &length);
    srcEnd = src + length;

    byteArrayPtr = ckalloc(BYTEARRAY_SIZE(length));
    for (dst = byteArrayPtr->bytes; src < srcEnd; ) {
	Tcl_UniChar ch;

	src += Tcl_UtfToUniChar(src, &ch);

	*dst++ = UCHAR(ch);
    }

    byteArrayPtr->used = dst - byteArrayPtr->bytes;
    byteArrayPtr->allocated = length;

    SET_BYTEARRAY(&ir, byteArrayPtr);
    Tcl_StoreIntRep(objPtr, &tclByteArrayType, &ir);

    return TCL_OK;
}

/*
 *----------------------------------------------------------------------
 *
 * FreeByteArrayInternalRep --







|





>
>
>
|
>
>







<

>







|
>







551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576

577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
 */

static int
SetByteArrayFromAny(
    Tcl_Interp *interp,		/* Not used. */
    Tcl_Obj *objPtr)		/* The object to convert to type ByteArray. */
{
    int length, improper = 0;
    const char *src, *srcEnd;
    unsigned char *dst;
    ByteArray *byteArrayPtr;
    Tcl_ObjIntRep ir;

    if (Tcl_FetchIntRep(objPtr, &properByteArrayType)) {
	return TCL_OK;
    }
    if (Tcl_FetchIntRep(objPtr, &tclByteArrayType)) {
	return TCL_OK;
    }

    src = TclGetStringFromObj(objPtr, &length);
    srcEnd = src + length;

    byteArrayPtr = ckalloc(BYTEARRAY_SIZE(length));
    for (dst = byteArrayPtr->bytes; src < srcEnd; ) {
	Tcl_UniChar ch;

	src += Tcl_UtfToUniChar(src, &ch);
	improper = improper || (ch > 255);
	*dst++ = UCHAR(ch);
    }

    byteArrayPtr->used = dst - byteArrayPtr->bytes;
    byteArrayPtr->allocated = length;

    SET_BYTEARRAY(&ir, byteArrayPtr);
    Tcl_StoreIntRep(objPtr,
	    improper ? &tclByteArrayType : &properByteArrayType, &ir);
    return TCL_OK;
}

/*
 *----------------------------------------------------------------------
 *
 * FreeByteArrayInternalRep --
506
507
508
509
510
511
512







513
514
515
516
517
518
519

static void
FreeByteArrayInternalRep(
    Tcl_Obj *objPtr)		/* Object with internal rep to free. */
{
    ckfree(GET_BYTEARRAY(Tcl_FetchIntRep(objPtr, &tclByteArrayType)));
}








/*
 *----------------------------------------------------------------------
 *
 * DupByteArrayInternalRep --
 *
 *	Initialize the internal representation of a ByteArray Tcl_Obj to a







>
>
>
>
>
>
>







607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627

static void
FreeByteArrayInternalRep(
    Tcl_Obj *objPtr)		/* Object with internal rep to free. */
{
    ckfree(GET_BYTEARRAY(Tcl_FetchIntRep(objPtr, &tclByteArrayType)));
}

static void
FreeProperByteArrayInternalRep(
    Tcl_Obj *objPtr)		/* Object with internal rep to free. */
{
    ckfree(GET_BYTEARRAY(Tcl_FetchIntRep(objPtr, &properByteArrayType)));
}

/*
 *----------------------------------------------------------------------
 *
 * DupByteArrayInternalRep --
 *
 *	Initialize the internal representation of a ByteArray Tcl_Obj to a
544
545
546
547
548
549
550





















551
552
553
554
555
556
557
    copyArrayPtr->used = length;
    copyArrayPtr->allocated = length;
    memcpy(copyArrayPtr->bytes, srcArrayPtr->bytes, (size_t) length);

    SET_BYTEARRAY(&ir, copyArrayPtr);
    Tcl_StoreIntRep(copyPtr, &tclByteArrayType, &ir);
}






















/*
 *----------------------------------------------------------------------
 *
 * UpdateStringOfByteArray --
 *
 *	Update the string representation for a ByteArray data object.







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
    copyArrayPtr->used = length;
    copyArrayPtr->allocated = length;
    memcpy(copyArrayPtr->bytes, srcArrayPtr->bytes, (size_t) length);

    SET_BYTEARRAY(&ir, copyArrayPtr);
    Tcl_StoreIntRep(copyPtr, &tclByteArrayType, &ir);
}

static void
DupProperByteArrayInternalRep(
    Tcl_Obj *srcPtr,		/* Object with internal rep to copy. */
    Tcl_Obj *copyPtr)		/* Object with internal rep to set. */
{
    unsigned int length;
    ByteArray *srcArrayPtr, *copyArrayPtr;
    Tcl_ObjIntRep ir;

    srcArrayPtr = GET_BYTEARRAY(Tcl_FetchIntRep(srcPtr, &properByteArrayType));
    length = srcArrayPtr->used;

    copyArrayPtr = ckalloc(BYTEARRAY_SIZE(length));
    copyArrayPtr->used = length;
    copyArrayPtr->allocated = length;
    memcpy(copyArrayPtr->bytes, srcArrayPtr->bytes, (size_t) length);

    SET_BYTEARRAY(&ir, copyArrayPtr);
    Tcl_StoreIntRep(copyPtr, &properByteArrayType, &ir);
}

/*
 *----------------------------------------------------------------------
 *
 * UpdateStringOfByteArray --
 *
 *	Update the string representation for a ByteArray data object.
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
 */

static void
UpdateStringOfByteArray(
    Tcl_Obj *objPtr)		/* ByteArray object whose string rep to
				 * update. */
{
    const Tcl_ObjIntRep *irPtr = Tcl_FetchIntRep(objPtr, &tclByteArrayType);
    ByteArray *byteArrayPtr = GET_BYTEARRAY(irPtr);
    unsigned char *src = byteArrayPtr->bytes;
    unsigned int i, length = byteArrayPtr->used;
    unsigned int size = length;

    /*
     * How much space will string rep need?







|







696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
 */

static void
UpdateStringOfByteArray(
    Tcl_Obj *objPtr)		/* ByteArray object whose string rep to
				 * update. */
{
    const Tcl_ObjIntRep *irPtr = Tcl_FetchIntRep(objPtr, &properByteArrayType);
    ByteArray *byteArrayPtr = GET_BYTEARRAY(irPtr);
    unsigned char *src = byteArrayPtr->bytes;
    unsigned int i, length = byteArrayPtr->used;
    unsigned int size = length;

    /*
     * How much space will string rep need?
639
640
641
642
643
644
645

646
647


648
649
650


651


652
653
654
655
656
657
658
	Tcl_Panic("%s must be called with definite number of bytes to append",
		"TclAppendBytesToByteArray");
    }
    if (len == 0) {
	/* Append zero bytes is a no-op. */
	return;
    }

    length = (unsigned int)len;



    irPtr = Tcl_FetchIntRep(objPtr, &tclByteArrayType);
    if (irPtr == NULL) {
	SetByteArrayFromAny(NULL, objPtr);


	irPtr = Tcl_FetchIntRep(objPtr, &tclByteArrayType);


    }
    byteArrayPtr = GET_BYTEARRAY(irPtr);

    if (length > INT_MAX - byteArrayPtr->used) {
	Tcl_Panic("max size for a Tcl value (%d bytes) exceeded", INT_MAX);
    }








>


>
>
|
|
|
>
>
|
>
>







768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
	Tcl_Panic("%s must be called with definite number of bytes to append",
		"TclAppendBytesToByteArray");
    }
    if (len == 0) {
	/* Append zero bytes is a no-op. */
	return;
    }

    length = (unsigned int)len;

    irPtr = Tcl_FetchIntRep(objPtr, &properByteArrayType);
    if (irPtr == NULL) {
	irPtr = Tcl_FetchIntRep(objPtr, &tclByteArrayType);
	if (irPtr == NULL) {
	    SetByteArrayFromAny(NULL, objPtr);
	    irPtr = Tcl_FetchIntRep(objPtr, &properByteArrayType);
	    if (irPtr == NULL) {
		irPtr = Tcl_FetchIntRep(objPtr, &tclByteArrayType);
	    }
	}
    }
    byteArrayPtr = GET_BYTEARRAY(irPtr);

    if (length > INT_MAX - byteArrayPtr->used) {
	Tcl_Panic("max size for a Tcl value (%d bytes) exceeded", INT_MAX);
    }

Changes to generic/tclIO.c.

7137
7138
7139
7140
7141
7142
7143
7144
7145
7146
7147
7148
7149
7150
7151
}

/*
 *---------------------------------------------------------------------------
 *
 * Tcl_SeekOld, Tcl_TellOld --
 *
 *	Backward-compatability versions of the seek/tell interface that do not
 *	support 64-bit offsets. This interface is not documented or expected
 *	to be supported indefinitely.
 *
 * Results:
 *	As for Tcl_Seek and Tcl_Tell respectively, except truncated to
 *	whatever value will fit in an 'int'.
 *







|







7137
7138
7139
7140
7141
7142
7143
7144
7145
7146
7147
7148
7149
7150
7151
}

/*
 *---------------------------------------------------------------------------
 *
 * Tcl_SeekOld, Tcl_TellOld --
 *
 *	Backward-compatibility versions of the seek/tell interface that do not
 *	support 64-bit offsets. This interface is not documented or expected
 *	to be supported indefinitely.
 *
 * Results:
 *	As for Tcl_Seek and Tcl_Tell respectively, except truncated to
 *	whatever value will fit in an 'int'.
 *

Changes to generic/tclInt.h.

4437
4438
4439
4440
4441
4442
4443
4444
4445
4446
4447
4448
4449
4450
4451
4452
 * but we don't do that at the moment since this is purely about efficiency.
 * The ANSI C "prototype" for this macro is:
 *
 * MODULE_SCOPE int	TclIsPureByteArray(Tcl_Obj *objPtr);
 *----------------------------------------------------------------
 */

#define TclIsPureByteArray(objPtr) \
	(((objPtr)->typePtr==&tclByteArrayType) && ((objPtr)->bytes==NULL))

/*
 *----------------------------------------------------------------
 * Macro used by the Tcl core to compare Unicode strings. On big-endian
 * systems we can use the more efficient memcmp, but this would not be
 * lexically correct on little-endian systems. The ANSI C "prototype" for
 * this macro is:







|
<







4437
4438
4439
4440
4441
4442
4443
4444

4445
4446
4447
4448
4449
4450
4451
 * but we don't do that at the moment since this is purely about efficiency.
 * The ANSI C "prototype" for this macro is:
 *
 * MODULE_SCOPE int	TclIsPureByteArray(Tcl_Obj *objPtr);
 *----------------------------------------------------------------
 */

MODULE_SCOPE int	TclIsPureByteArray(Tcl_Obj *objPtr);


/*
 *----------------------------------------------------------------
 * Macro used by the Tcl core to compare Unicode strings. On big-endian
 * systems we can use the more efficient memcmp, but this would not be
 * lexically correct on little-endian systems. The ANSI C "prototype" for
 * this macro is:

Changes to generic/tclStringObj.c.

2468
2469
2470
2471
2472
2473
2474




2475
2476
2477
2478
2479
2480
2481
		    Tcl_ListObjAppendElement(NULL, list, Tcl_NewLongObj(
			    (long) va_arg(argList, int)));
		    break;
		case 1:
		    Tcl_ListObjAppendElement(NULL, list, Tcl_NewLongObj(
			    va_arg(argList, long)));
		    break;




		}
		break;
	    case 'e':
	    case 'E':
	    case 'f':
	    case 'g':
	    case 'G':







>
>
>
>







2468
2469
2470
2471
2472
2473
2474
2475
2476
2477
2478
2479
2480
2481
2482
2483
2484
2485
		    Tcl_ListObjAppendElement(NULL, list, Tcl_NewLongObj(
			    (long) va_arg(argList, int)));
		    break;
		case 1:
		    Tcl_ListObjAppendElement(NULL, list, Tcl_NewLongObj(
			    va_arg(argList, long)));
		    break;
		case 2:
		    Tcl_ListObjAppendElement(NULL, list, Tcl_NewWideIntObj(
			    va_arg(argList, Tcl_WideInt)));
		    break;
		}
		break;
	    case 'e':
	    case 'E':
	    case 'f':
	    case 'g':
	    case 'G':
2496
2497
2498
2499
2500
2501
2502
2503
2504
2505
2506
2507
2508
2509
2510
2511
2512
		p = end;
		break;
	    }
	    case '.':
		gotPrecision = 1;
		p++;
		break;
	    /* TODO: support for wide (and bignum?) arguments */
	    case 'l':
		size = 1;
		p++;
		break;
	    case 'h':
		size = -1;
	    default:
		p++;
	    }







|

|







2500
2501
2502
2503
2504
2505
2506
2507
2508
2509
2510
2511
2512
2513
2514
2515
2516
		p = end;
		break;
	    }
	    case '.':
		gotPrecision = 1;
		p++;
		break;
	    /* TODO: support for bignum arguments */
	    case 'l':
		++size;
		p++;
		break;
	    case 'h':
		size = -1;
	    default:
		p++;
	    }
2704
2705
2706
2707
2708
2709
2710
2711
2712
2713
2714
2715
2716
2717
2718
2719
	    TclInvalidateStringRep(objPtr);
	    objResultPtr = objPtr;
	}

        if (0 == Tcl_AttemptSetObjLength(objResultPtr, count*length)) {
	    if (interp) {
		Tcl_SetObjResult(interp, Tcl_ObjPrintf(
			"string size overflow: unable to alloc %lu bytes",
			STRING_SIZE(count*length)));
		Tcl_SetErrorCode(interp, "TCL", "MEMORY", NULL);
	    }
	    return TCL_ERROR;
	}
	Tcl_SetObjLength(objResultPtr, length);
	while (count - done > done) {
	    Tcl_AppendObjToObj(objResultPtr, objResultPtr);







|
|







2708
2709
2710
2711
2712
2713
2714
2715
2716
2717
2718
2719
2720
2721
2722
2723
	    TclInvalidateStringRep(objPtr);
	    objResultPtr = objPtr;
	}

        if (0 == Tcl_AttemptSetObjLength(objResultPtr, count*length)) {
	    if (interp) {
		Tcl_SetObjResult(interp, Tcl_ObjPrintf(
			"string size overflow: unable to alloc %llu bytes",
			(Tcl_WideUInt)STRING_SIZE(count*length)));
		Tcl_SetErrorCode(interp, "TCL", "MEMORY", NULL);
	    }
	    return TCL_ERROR;
	}
	Tcl_SetObjLength(objResultPtr, length);
	while (count - done > done) {
	    Tcl_AppendObjToObj(objResultPtr, objResultPtr);
2891
2892
2893
2894
2895
2896
2897
2898
2899
2900
2901
2902
2903
2904
2905
	/* Efficiently produce a pure byte array result */
	unsigned char *dst;

	/*
	 * Broken interface! Byte array value routines offer no way
	 * to handle failure to allocate enough space. Following
	 * stanza may panic.
	 */ 
	if (inPlace && !Tcl_IsShared(*objv)) {
	    int start;

	    objResultPtr = *objv++; objc--;
	    Tcl_GetByteArrayFromObj(objResultPtr, &start);
	    dst = Tcl_SetByteArrayLength(objResultPtr, length) + start;
	} else {







|







2895
2896
2897
2898
2899
2900
2901
2902
2903
2904
2905
2906
2907
2908
2909
	/* Efficiently produce a pure byte array result */
	unsigned char *dst;

	/*
	 * Broken interface! Byte array value routines offer no way
	 * to handle failure to allocate enough space. Following
	 * stanza may panic.
	 */
	if (inPlace && !Tcl_IsShared(*objv)) {
	    int start;

	    objResultPtr = *objv++; objc--;
	    Tcl_GetByteArrayFromObj(objResultPtr, &start);
	    dst = Tcl_SetByteArrayLength(objResultPtr, length) + start;
	} else {
2927
2928
2929
2930
2931
2932
2933
2934
2935
2936
2937
2938
2939
2940
2941
2942
2943
2944
2945
2946
2947
2948
2949
2950
2951
2952
2953
2954
2955
2956
2957

	    /* Ugly interface! Force resize of the unicode array. */
	    Tcl_GetUnicodeFromObj(objResultPtr, &start);
	    Tcl_InvalidateStringRep(objResultPtr);
	    if (0 == Tcl_AttemptSetObjLength(objResultPtr, length)) {
		if (interp) {
		    Tcl_SetObjResult(interp, Tcl_ObjPrintf(
		    	"concatenation failed: unable to alloc %lu bytes",
			STRING_SIZE(length)));
		    Tcl_SetErrorCode(interp, "TCL", "MEMORY", NULL);
		}
		return TCL_ERROR;
	    }
	    dst = Tcl_GetUnicode(objResultPtr) + start;
	} else {
	    Tcl_UniChar ch = 0;

	    /* Ugly interface! No scheme to init array size. */
	    objResultPtr = Tcl_NewUnicodeObj(&ch, 0);	/* PANIC? */
	    if (0 == Tcl_AttemptSetObjLength(objResultPtr, length)) {
		if (interp) {
		    Tcl_SetObjResult(interp, Tcl_ObjPrintf(
		    	"concatenation failed: unable to alloc %lu bytes",
			STRING_SIZE(length)));
		    Tcl_SetErrorCode(interp, "TCL", "MEMORY", NULL);
		}
		return TCL_ERROR;
	    }
	    dst = Tcl_GetUnicode(objResultPtr);
	}
	while (objc--) {







|
|













|
|







2931
2932
2933
2934
2935
2936
2937
2938
2939
2940
2941
2942
2943
2944
2945
2946
2947
2948
2949
2950
2951
2952
2953
2954
2955
2956
2957
2958
2959
2960
2961

	    /* Ugly interface! Force resize of the unicode array. */
	    Tcl_GetUnicodeFromObj(objResultPtr, &start);
	    Tcl_InvalidateStringRep(objResultPtr);
	    if (0 == Tcl_AttemptSetObjLength(objResultPtr, length)) {
		if (interp) {
		    Tcl_SetObjResult(interp, Tcl_ObjPrintf(
		    	"concatenation failed: unable to alloc %llu bytes",
			(Tcl_WideUInt)STRING_SIZE(length)));
		    Tcl_SetErrorCode(interp, "TCL", "MEMORY", NULL);
		}
		return TCL_ERROR;
	    }
	    dst = Tcl_GetUnicode(objResultPtr) + start;
	} else {
	    Tcl_UniChar ch = 0;

	    /* Ugly interface! No scheme to init array size. */
	    objResultPtr = Tcl_NewUnicodeObj(&ch, 0);	/* PANIC? */
	    if (0 == Tcl_AttemptSetObjLength(objResultPtr, length)) {
		if (interp) {
		    Tcl_SetObjResult(interp, Tcl_ObjPrintf(
		    	"concatenation failed: unable to alloc %llu bytes",
			(Tcl_WideUInt)STRING_SIZE(length)));
		    Tcl_SetErrorCode(interp, "TCL", "MEMORY", NULL);
		}
		return TCL_ERROR;
	    }
	    dst = Tcl_GetUnicode(objResultPtr);
	}
	while (objc--) {

Changes to generic/tclStringRep.h.

68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
    (int)(((size_t)UINT_MAX - sizeof(String))/sizeof(Tcl_UniChar))
#define STRING_SIZE(numChars) \
    (sizeof(String) + ((numChars) * sizeof(Tcl_UniChar)))
#define stringCheckLimits(numChars) \
    do {								\
	if ((numChars) < 0 || (numChars) > STRING_MAXCHARS) {		\
	    Tcl_Panic("max length for a Tcl unicode value (%d chars) exceeded", \
		      STRING_MAXCHARS);					\
	}								\
    } while (0)
#define stringAttemptAlloc(numChars) \
    (String *) attemptckalloc((unsigned) STRING_SIZE(numChars))
#define stringAlloc(numChars) \
    (String *) ckalloc((unsigned) STRING_SIZE(numChars))
#define stringRealloc(ptr, numChars) \
    (String *) ckrealloc((ptr), (unsigned) STRING_SIZE(numChars))
#define stringAttemptRealloc(ptr, numChars) \
    (String *) attemptckrealloc((ptr), (unsigned) STRING_SIZE(numChars))
#define GET_STRING(objPtr) \
    ((String *) (objPtr)->internalRep.twoPtrValue.ptr1)
#define SET_STRING(objPtr, stringPtr) \
    ((objPtr)->internalRep.twoPtrValue.ptr1 = (void *) (stringPtr))

/*
 * Local Variables:







|



|

|

|

|







68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
    (int)(((size_t)UINT_MAX - sizeof(String))/sizeof(Tcl_UniChar))
#define STRING_SIZE(numChars) \
    (sizeof(String) + ((numChars) * sizeof(Tcl_UniChar)))
#define stringCheckLimits(numChars) \
    do {								\
	if ((numChars) < 0 || (numChars) > STRING_MAXCHARS) {		\
	    Tcl_Panic("max length for a Tcl unicode value (%d chars) exceeded", \
		      (int)STRING_MAXCHARS);					\
	}								\
    } while (0)
#define stringAttemptAlloc(numChars) \
    (String *) attemptckalloc(STRING_SIZE(numChars))
#define stringAlloc(numChars) \
    (String *) ckalloc(STRING_SIZE(numChars))
#define stringRealloc(ptr, numChars) \
    (String *) ckrealloc((ptr), STRING_SIZE(numChars))
#define stringAttemptRealloc(ptr, numChars) \
    (String *) attemptckrealloc((ptr), STRING_SIZE(numChars))
#define GET_STRING(objPtr) \
    ((String *) (objPtr)->internalRep.twoPtrValue.ptr1)
#define SET_STRING(objPtr, stringPtr) \
    ((objPtr)->internalRep.twoPtrValue.ptr1 = (void *) (stringPtr))

/*
 * Local Variables:

Changes to generic/tclVar.c.

2782
2783
2784
2785
2786
2787
2788
2789
2790
2791
2792
2793
2794
2795
2796
		return TCL_ERROR;
	    }
	}
	return TCL_OK;
    } else {
	/*
	 * Not a dictionary, so assume (and convert to, for backward-
	 * -compatability reasons) a list.
	 */

	int elemLen;
	Tcl_Obj **elemPtrs, *copyListObj;

	result = TclListObjGetElements(interp, arrayElemObj,
		&elemLen, &elemPtrs);







|







2782
2783
2784
2785
2786
2787
2788
2789
2790
2791
2792
2793
2794
2795
2796
		return TCL_ERROR;
	    }
	}
	return TCL_OK;
    } else {
	/*
	 * Not a dictionary, so assume (and convert to, for backward-
	 * -compatibility reasons) a list.
	 */

	int elemLen;
	Tcl_Obj **elemPtrs, *copyListObj;

	result = TclListObjGetElements(interp, arrayElemObj,
		&elemLen, &elemPtrs);

Changes to library/http/http.tcl.

1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
    if {$http(-urlencoding) ne ""} {
	set string [encoding convertto $http(-urlencoding) $string]
	return [string map $formMap $string]
    }
    set converted [string map $formMap $string]
    if {[string match "*\[\u0100-\uffff\]*" $converted]} {
	regexp "\[\u0100-\uffff\]" $converted badChar
	# Return this error message for maximum compatability... :^/
	return -code error \
	    "can't read \"formMap($badChar)\": no such element in array"
    }
    return $converted
}

# http::ProxyRequired --







|







1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
    if {$http(-urlencoding) ne ""} {
	set string [encoding convertto $http(-urlencoding) $string]
	return [string map $formMap $string]
    }
    set converted [string map $formMap $string]
    if {[string match "*\[\u0100-\uffff\]*" $converted]} {
	regexp "\[\u0100-\uffff\]" $converted badChar
	# Return this error message for maximum compatibility... :^/
	return -code error \
	    "can't read \"formMap($badChar)\": no such element in array"
    }
    return $converted
}

# http::ProxyRequired --

Changes to unix/tcl.m4.

2702
2703
2704
2705
2706
2707
2708
2709
2710
2711
2712
2713
2714
2715
2716
#	Check for broken function.
#
# Arguments:
#	funcName - function to test for
#	advancedTest - the advanced test to run if the function is present
#
# Results:
#	Might cause compatability versions of the function to be used.
#	Might affect the following vars:
#		USE_COMPAT	(implicit)
#
#--------------------------------------------------------------------

AC_DEFUN([SC_TCL_CHECK_BROKEN_FUNC],[
    AC_CHECK_FUNC($1, tcl_ok=1, tcl_ok=0)







|







2702
2703
2704
2705
2706
2707
2708
2709
2710
2711
2712
2713
2714
2715
2716
#	Check for broken function.
#
# Arguments:
#	funcName - function to test for
#	advancedTest - the advanced test to run if the function is present
#
# Results:
#	Might cause compatibility versions of the function to be used.
#	Might affect the following vars:
#		USE_COMPAT	(implicit)
#
#--------------------------------------------------------------------

AC_DEFUN([SC_TCL_CHECK_BROKEN_FUNC],[
    AC_CHECK_FUNC($1, tcl_ok=1, tcl_ok=0)

Changes to win/makefile.vc.

109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
#		above may be used (comma separated).  'none' will over-ride
#		everything to nothing.
#
#		compdbg  = Enables byte compilation logging.
#		memdbg   = Enables the debugging memory allocator.
#
#	CHECKS=64bit,fullwarn,nodep,none
#		Sets special macros for checking compatability.
#
#		64bit    = Enable 64bit portability warnings (if available)
#		fullwarn = Builds with full compiler and link warnings enabled.
#			    Very verbose.
#		nodep	 = Turns off compatability macros to ensure the core
#			    isn't being built with deprecated functions.
#
#	MACHINE=(ALPHA|AMD64|IA64|IX86)
#		Set the machine type used for the compiler, linker, and
#		resource compiler.  This hook is needed to tell the tools
#		when alternate platforms are requested.  IX86 is the default
#		when not specified. If the CPU environment variable has been







|




|







109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
#		above may be used (comma separated).  'none' will over-ride
#		everything to nothing.
#
#		compdbg  = Enables byte compilation logging.
#		memdbg   = Enables the debugging memory allocator.
#
#	CHECKS=64bit,fullwarn,nodep,none
#		Sets special macros for checking compatibility.
#
#		64bit    = Enable 64bit portability warnings (if available)
#		fullwarn = Builds with full compiler and link warnings enabled.
#			    Very verbose.
#		nodep	 = Turns off compatibility macros to ensure the core
#			    isn't being built with deprecated functions.
#
#	MACHINE=(ALPHA|AMD64|IA64|IX86)
#		Set the machine type used for the compiler, linker, and
#		resource compiler.  This hook is needed to tell the tools
#		when alternate platforms are requested.  IX86 is the default
#		when not specified. If the CPU environment variable has been