Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
Comment: | * generic/tclIO.c: Revised ReadChars and FilterInputBytes routines to permit reads to continue up to the string limits of Tcl values. Before revisions, large read attempts could panic when as little as half the limiting value length was reached. [Patch 2107634] Thanks to Sean Morrison and Bob Parker for their roles in the fix. |
---|---|
Downloads: | Tarball | ZIP archive | SQL archive |
Timelines: | family | ancestors | descendants | both | core-8-5-branch |
Files: | files | file ages | folders |
SHA1: |
cdf00220922c7d8dbeba4940d4e474c1 |
User & Date: | dgp 2009-10-19 21:59:17 |
Context
2009-10-20
| ||
21:45 | describe backported fixes check-in: cb2104a015 user: ferrieux tags: core-8-5-branch | |
2009-10-19
| ||
21:59 |
* generic/tclIO.c: Revised ReadChars and FilterInputBytes routines to permit re...check-in: cdf0022092 user: dgp tags: core-8-5-branch | |
2009-10-18
| ||
11:21 | Fix for [Bug 988703, 1565466] check-in: 3a43f003a8 user: mistachkin tags: core-8-5-branch | |
Changes
Changes to ChangeLog.
1 2 3 4 5 6 7 | 2009-10-18 Joe Mistachkin <[email protected]> * tests/thread.test (thread-4.[345]): [Bug 1565466]: Correct tests to save their error state before the final call to threadReap just in case it triggers an "invalid thread id" error. This error can occur if one or more of the target threads has exited prior to the attempt to send it an asynchronous exit command. | > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | 2009-10-19 Don Porter <[email protected]> * generic/tclIO.c: Revised ReadChars and FilterInputBytes routines to permit reads to continue up to the string limits of Tcl values. Before revisions, large read attempts could panic when as little as half the limiting value length was reached. [Patch 2107634] Thanks to Sean Morrison and Bob Parker for their roles in the fix. 2009-10-18 Joe Mistachkin <[email protected]> * tests/thread.test (thread-4.[345]): [Bug 1565466]: Correct tests to save their error state before the final call to threadReap just in case it triggers an "invalid thread id" error. This error can occur if one or more of the target threads has exited prior to the attempt to send it an asynchronous exit command. |
︙ | ︙ |
Changes to generic/tclIO.c.
1 2 3 4 5 6 7 8 9 10 11 12 | /* * tclIO.c -- * * This file provides the generic portions (those that are the same on * all platforms and for all channel types) of Tcl's IO facilities. * * Copyright (c) 1998-2000 Ajuba Solutions * Copyright (c) 1995-1997 Sun Microsystems, Inc. * * See the file "license.terms" for information on usage and redistribution of * this file, and for a DISCLAIMER OF ALL WARRANTIES. * | | | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | /* * tclIO.c -- * * This file provides the generic portions (those that are the same on * all platforms and for all channel types) of Tcl's IO facilities. * * Copyright (c) 1998-2000 Ajuba Solutions * Copyright (c) 1995-1997 Sun Microsystems, Inc. * * See the file "license.terms" for information on usage and redistribution of * this file, and for a DISCLAIMER OF ALL WARRANTIES. * * RCS: @(#) $Id: tclIO.c,v 1.137.2.12 2009/10/19 21:59:18 dgp Exp $ */ #include "tclInt.h" #include "tclIO.h" #include <assert.h> /* |
︙ | ︙ | |||
4609 4610 4611 4612 4613 4614 4615 | Channel *chanPtr, /* Channel to read. */ GetsState *gsPtr) /* Current state of gets operation. */ { ChannelState *statePtr = chanPtr->state; /* State info for channel */ ChannelBuffer *bufPtr; char *raw, *rawStart, *dst; | | | 4609 4610 4611 4612 4613 4614 4615 4616 4617 4618 4619 4620 4621 4622 4623 | Channel *chanPtr, /* Channel to read. */ GetsState *gsPtr) /* Current state of gets operation. */ { ChannelState *statePtr = chanPtr->state; /* State info for channel */ ChannelBuffer *bufPtr; char *raw, *rawStart, *dst; int offset, toRead, dstNeeded, spaceLeft, result, rawLen; Tcl_Obj *objPtr; #define ENCODING_LINESIZE 20 /* Lower bound on how many bytes to convert at * a time. Since we don't know a priori how * many bytes of storage this many source * bytes will use, we actually need at least * ENCODING_LINESIZE * TCL_MAX_UTF bytes of * room. */ |
︙ | ︙ | |||
4675 4676 4677 4678 4679 4680 4681 | dst = *gsPtr->dstPtr; offset = dst - objPtr->bytes; toRead = ENCODING_LINESIZE; if (toRead > rawLen) { toRead = rawLen; } | | | > > > | > | | < < | > > | | 4675 4676 4677 4678 4679 4680 4681 4682 4683 4684 4685 4686 4687 4688 4689 4690 4691 4692 4693 4694 4695 4696 4697 4698 4699 4700 4701 4702 4703 4704 4705 4706 4707 4708 4709 | dst = *gsPtr->dstPtr; offset = dst - objPtr->bytes; toRead = ENCODING_LINESIZE; if (toRead > rawLen) { toRead = rawLen; } dstNeeded = toRead * TCL_UTF_MAX; spaceLeft = objPtr->length - offset; if (dstNeeded > spaceLeft) { int length = offset + ((offset < dstNeeded) ? dstNeeded : offset); if (Tcl_AttemptSetObjLength(objPtr, length) == 0) { length = offset + dstNeeded; if (Tcl_AttemptSetObjLength(objPtr, length) == 0) { dstNeeded = TCL_UTF_MAX - 1 + toRead; length = offset + dstNeeded; Tcl_SetObjLength(objPtr, length); } } spaceLeft = length - offset; dst = objPtr->bytes + offset; *gsPtr->dstPtr = dst; } gsPtr->state = statePtr->inputEncodingState; result = Tcl_ExternalToUtf(NULL, gsPtr->encoding, raw, rawLen, statePtr->inputEncodingFlags, &statePtr->inputEncodingState, dst, spaceLeft+1, &gsPtr->rawRead, &gsPtr->bytesWrote, &gsPtr->charsWrote); /* * Make sure that if we go through 'gets', that we reset the * TCL_ENCODING_START flag still. [Bug #523988] */ |
︙ | ︙ | |||
5415 5416 5417 5418 5419 5420 5421 5422 5423 5424 5425 5426 5427 5428 | * have been seen, EOF is seen, or the channel would block. Raw bytes * from the channel are converted to UTF-8 and stored in objPtr. EOL and * EOF translation is done. * * 'charsToRead' can safely be a very large number because space is only * allocated to hold data read from the channel as needed. * * Results: * The return value is the number of characters appended to the object, * *offsetPtr is filled with the number of bytes that were appended, and * *factorPtr is filled with the expansion factor used to guess how many * bytes of UTF-8 to allocate to hold N source bytes. * * Side effects: | > > | 5419 5420 5421 5422 5423 5424 5425 5426 5427 5428 5429 5430 5431 5432 5433 5434 | * have been seen, EOF is seen, or the channel would block. Raw bytes * from the channel are converted to UTF-8 and stored in objPtr. EOL and * EOF translation is done. * * 'charsToRead' can safely be a very large number because space is only * allocated to hold data read from the channel as needed. * * 'charsToRead' may *not* be 0. * * Results: * The return value is the number of characters appended to the object, * *offsetPtr is filled with the number of bytes that were appended, and * *factorPtr is filled with the expansion factor used to guess how many * bytes of UTF-8 to allocate to hold N source bytes. * * Side effects: |
︙ | ︙ | |||
5452 5453 5454 5455 5456 5457 5458 | * used. */ int *factorPtr) /* On input, contains a guess of how many * bytes need to be allocated to hold the * result of converting N source bytes to * UTF-8. On output, contains another guess * based on the data seen so far. */ { | | | 5458 5459 5460 5461 5462 5463 5464 5465 5466 5467 5468 5469 5470 5471 5472 | * used. */ int *factorPtr) /* On input, contains a guess of how many * bytes need to be allocated to hold the * result of converting N source bytes to * UTF-8. On output, contains another guess * based on the data seen so far. */ { int toRead, factor, offset, spaceLeft, srcLen, dstNeeded; int srcRead, dstWrote, numChars, dstRead; ChannelBuffer *bufPtr; char *src, *dst; Tcl_EncodingState oldState; int encEndFlagSuppressed = 0; factor = *factorPtr; |
︙ | ︙ | |||
5477 5478 5479 5480 5481 5482 5483 | /* * 'factor' is how much we guess that the bytes in the source buffer will * expand when converted to UTF-8 chars. This guess comes from analyzing * how many characters were produced by the previous pass. */ | | | > > > | > | | > | > < < | 5483 5484 5485 5486 5487 5488 5489 5490 5491 5492 5493 5494 5495 5496 5497 5498 5499 5500 5501 5502 5503 5504 5505 5506 5507 5508 5509 5510 5511 5512 5513 5514 5515 5516 5517 | /* * 'factor' is how much we guess that the bytes in the source buffer will * expand when converted to UTF-8 chars. This guess comes from analyzing * how many characters were produced by the previous pass. */ dstNeeded = TCL_UTF_MAX - 1 + toRead * factor / UTF_EXPANSION_FACTOR; spaceLeft = objPtr->length - offset; if (dstNeeded > spaceLeft) { /* * Double the existing size of the object or make enough room to hold * all the characters we want from the source buffer, whichever is * larger. */ int length = offset + ((offset < dstNeeded) ? dstNeeded : offset); if (Tcl_AttemptSetObjLength(objPtr, length) == 0) { length = offset + dstNeeded; if (Tcl_AttemptSetObjLength(objPtr, length) == 0) { dstNeeded = TCL_UTF_MAX - 1 + toRead; length = offset + dstNeeded; Tcl_SetObjLength(objPtr, length); } } spaceLeft = length - offset; } if (toRead == srcLen) { /* * Want to convert the whole buffer in one pass. If we have enough * space, convert it using all available space in object rather than * using the factor. */ |
︙ | ︙ | |||
5585 5586 5587 5588 5589 5590 5591 | statePtr->inputEncodingFlags |= TCL_ENCODING_END; } return 1; } Tcl_ExternalToUtf(NULL, statePtr->encoding, src, srcLen, statePtr->inputEncodingFlags, &statePtr->inputEncodingState, dst, | | | 5595 5596 5597 5598 5599 5600 5601 5602 5603 5604 5605 5606 5607 5608 5609 | statePtr->inputEncodingFlags |= TCL_ENCODING_END; } return 1; } Tcl_ExternalToUtf(NULL, statePtr->encoding, src, srcLen, statePtr->inputEncodingFlags, &statePtr->inputEncodingState, dst, dstNeeded + 1, &srcRead, &dstWrote, &numChars); if (encEndFlagSuppressed) { statePtr->inputEncodingFlags |= TCL_ENCODING_END; } if (srcRead == 0) { /* |
︙ | ︙ |