• Varun Gupta's avatar
    MDEV-24033: SIGSEGV in __memcmp_avx2_movbe from queue_insert | SIGSEGV in... · 5a0c34e4
    Varun Gupta authored
     MDEV-24033: SIGSEGV in __memcmp_avx2_movbe from queue_insert | SIGSEGV in __memcmp_avx2_movbe from native_compare
    
    The issue here was the system variable max_sort_length was being applied
    to decimals and it was truncating the value for decimals to the number
    of bytes set by max_sort_length.
    This was leading to a buffer overflow as the values were written
    to the buffer without truncation and then we moved the offset to
    the number of bytes(set by max_sort_length), that are needed for comparison.
    
    The fix is to not apply max_sort_length for fixed size types like INT,
    DECIMALS and only apply max_sort_length for CHAR, VARCHARS, TEXT and
    BLOBS.
    5a0c34e4
sql_type.h 19.8 KB