• Alexander Barkov's avatar
    MDEV-23535 SIGSEGV, SIGABRT and SIGILL in typeinfo for Item_func_set_collation... · f0a57acb
    Alexander Barkov authored
    MDEV-23535 SIGSEGV, SIGABRT and SIGILL in typeinfo for Item_func_set_collation (on optimized builds)
    
    This piece of the code in Item_func_or_sum::agg_item_set_converter:
    
     if (!conv && ((*arg)->collation.repertoire == MY_REPERTOIRE_ASCII))
       conv= new (thd->mem_root) Item_func_conv_charset(thd, *arg, coll.collation, 1);
    
    was wrong because:
    
    1. It could change Item_cache to Item_func_conv_charset
      (with the old Item_cache in args[0]).
      Such Item type change is not always supported, e.g.
      the code in Item_singlerow_subselect::reset() expects only Item_cache,
      to be able to call Item_cache::set_null(). So it erroneously
      reinterpreted Item_func_conv_charset to Item_cache and called
      a non-existing method set_null(), which crashed the server.
    
    2. The 1 in the last parameter to Item_func_conv_charset() was also
      a problem. In MariaDB versions where the reported query did not crash,
      it erroneously returned "empty set" instead of one row, because
      the 1 made subselects execute too earlier and return NULL.
    
    Fix:
    
    1. Removing the above two lines from Item_func_or_sum::agg_item_set_converter()
    
    2. Adding the repertoire test inside the constructor of Item_func_conv_charset,
       so it now detects itself as "safe" in more cases than before.
       This is needed to avoid new "Illegal mix of collations" after
       removing the wrong code in various scenarios when character set
       conversion from pure ASCII happens, including the reported scenario.
    
    So now this sequence:
    
       Item_cache -> Item_func_concat
    
    is replaced to this compatible sequence (the top Item is still Item_cache):
    
       new Item_cache -> Item_func_conv_charset -> Item_func_concat
    
    Before the fix it was replaced to this incompatible sequence:
    
       Item_func_conv_charset -> old Item_cache -> Item_func_concat
    f0a57acb
subselect_innodb.test 18.3 KB