Tcl Source Code

View Ticket
Login
Ticket UUID: 1c7f179710d5dd791f4a161f2e6d34040d17bf4e
Title: undefined behavior for INST_LSHIFT in ExecuteExtendedBinaryMathOp()
Type: Patch Version: core-8-6-branch
Submitter: chrstphrchvz Created on: 2022-02-18 03:43:30
Subsystem: 48. Number Handling Assigned To: jan.nijtmans
Priority: 5 Medium Severity: Minor
Status: Closed Last Modified: 2022-03-08 15:32:38
Resolution: Fixed Closed By: jan.nijtmans
    Closed on: 2022-03-08 15:32:38
Description:

Analogous to [b0f84119c8] for left-shifting a value represented by a Tcl_WideInt (and not a long or a bignum) when Tcl_WideInt is long long (because long is only 32-bit):

  • Left-shifting by 0 causes signed integer overflow (undefined behavior detected by UBSan -fsanitize=shift-base) in the expression:
        ((Tcl_WideInt)1) << (CHAR_BIT*sizeof(Tcl_WideInt) - 1 - shift)
    

    i.e. 1 is shifted left (64-1-0)=63 times, which overflows; and signed integer overflow (undefined behavior detected by UBSan -fsanitize=signed-integer-overflow) when the overflowed result WIDE_MIN of the above expression is negated.

  • If the left-shifted value is negative, then there is left-shifting of a negative value (undefined behavior detected by UBSan -fsanitize=shift-base).

Example command (left operand is in the range [WIDE_MIN,LONG_MIN) ∪ (LONG_MAX,WIDE_MAX]); the test suite does not contain an affected example):

% ::tcl::mathop::<< -0x80000001 0
tcl/generic/tclExecute.c:8665:4: runtime error: left shift of 1 by 63 places cannot be represented in type 'long long int'
tcl/generic/tclExecute.c:8664:6: runtime error: negation of -9223372036854775808 cannot be represented in type 'long long int'; cast to an unsigned type to negate this value to itself
tcl/generic/tclExecute.c:8666:7: runtime error: left shift of negative value -2147483649
-2147483649

The attached patch avoids this by casting to Tcl_WideUInt in a couple of places before shifting.

User Comments: jan.nijtmans added on 2022-03-08 15:32:38:
Fixed [8a68239de9dfc6ce|here]. Thanks!

Attachments: