1. 19 May, 2021 40 commits
    • Monty's avatar
      MDEV-20017 Implement TO_CHAR() Oracle compatible function · 81d9bed3
      Monty authored
      TO_CHAR(expr, fmt)
      - expr: required parameter, data/time/timestamp type expression
      - fmt: optional parameter, format string, supports
        YYYY/YYY/YY/RRRR/RR/MM/MON/MONTH/MI/DD/DY/HH/HH12/HH24/SS and special
        characters. The default value is "YYYY-MM-DD HH24:MI:SS"
      
      In Oracle, TO_CHAR() can also be used to convert numbers to strings, but
      this is not supported. This will gave an error in this patch.
      
      Other things:
      - If format strings is a constant, it's evaluated only once and if there
        is any errors in it, they are given at once and the statement will abort.
      
      Original author: woqutech
      Lots of optimizations and cleanups done as part of review
      81d9bed3
    • Monty's avatar
      MDEV-20021 sql_mode="oracle" does not support MINUS set operator · cf93209c
      Monty authored
      MINUS is mapped to EXCEPT
      One consequence of the patch is that MINUS becomes a reserved word in
      Oracle mode.
      
      Author: woqutech
      cf93209c
    • Monty's avatar
      MDEV-24285 support oracle build-in function: sys_guid · b8c31595
      Monty authored
      SYS_GUID() returns same as UUID(), but without any '-'
      
      author: woqutech
      b8c31595
    • Monty's avatar
      MDEV-24089 support oracle syntax: rownum · be093c81
      Monty authored
      The ROWNUM() function is for SELECT mapped to JOIN->accepted_rows, which is
      incremented for each accepted rows.
      For Filesort, update, insert, delete and load data, we map ROWNUM() to
      internal variables incremented when the table is changed.
      The connection between the row counter and Item_func_rownum is done
      in sql_select.cc::fix_items_after_optimize() and
      sql_insert.cc::fix_rownum_pointers()
      
      When ROWNUM() is used anywhere in query, the optimization to ignore ORDER
      BY in sub queries are disabled. This was done to get the following common
      Oracle query to work:
      select * from (select * from t1 order by a desc) as t where rownum() <= 2;
      MDEV-3926 "Wrong result with GROUP BY ... WITH ROLLUP" contains a discussion
      about this topic.
      
      LIMIT optimization is enabled when in a top level WHERE clause comparing
      ROWNUM() with a numerical constant using any of the following expressions:
      - ROWNUM() < #
      - ROWNUM() <= #
      - ROWNUM() = 1
      ROWNUM() can be also be the right argument to the comparison function.
      
      LIMIT optimization is done in two cases:
      - For the current sub query when the ROWNUM comparison is done on the top
        level:
        SELECT * from t1 WHERE rownum() <= 2 AND t1.a > 0
      - For an inner sub query, when the upper level has only a ROWNUM comparison
        in the WHERE clause:
        SELECT * from (select * from t1) as t WHERE rownum() <= 2
      
      In Oracle mode, one can also use ROWNUM without parentheses.
      
      Other things:
      - Fixed bug where the optimizer tries to optimize away sub queries
        with RAND_TABLE_BIT set (non-deterministic queries). Now these
        sub queries will not be converted to joins.  This bug fix was also
        needed to get rownum() working inside subqueries.
      - In remove_const() remove setting simple_order to FALSE if ROLLUP is
        USED. This code was disable a long time ago because of wrong assignment
        in the following code.  Instead we set simple_order to false if
        RAND_TABLE_BIT was used in the SELECT list.  This ensures that
        we don't delete ORDER BY if the result set is not deterministic, like
        in 'SELECT RAND() AS 'r' FROM t1 ORDER BY r';
      - Updated parameters for Sort_param::init_for_filesort() to be able
        to provide filesort with information where the number of accepted
        rows should be stored
      - Reordered fields in class Filesort to optimize storage layout
      - Added new error messsage to tell that a function can't be used in HAVING
      - Added field 'with_rownum' to THD to mark that ROWNUM() is used in the
        query.
      
      Co-author: Oleksandr Byelkin <sanja@mariadb.com>
                 LIMIT optimization for sub query
      be093c81
    • Alexander Barkov's avatar
      f16b8590
    • Monty's avatar
      Make LEX::can_not_use_merged more general · 7b134ffa
      Monty authored
      7b134ffa
    • Monty's avatar
      f2e3f0db
    • Monty's avatar
      5ac05a61
    • Monty's avatar
      Rename all external ddl_log function to start with ddl_log_ prefix · 188b0b99
      Monty authored
      Rename deactivate_ddl_log_entry to ddl_log_increment_phase
      188b0b99
    • Monty's avatar
      Move all ddl log code to ddl_log.cc and ddl_log.h · 02b6cef4
      Monty authored
      Part of prepration for: MDEV-17567 Atomic DDL
      
      No notable code changes except moving code around
      02b6cef4
    • Monty's avatar
      Indentation cleanups (break long lines) · a28ea028
      Monty authored
      a28ea028
    • Monty's avatar
      Replace find_temporary_table() with is_temporary_table() · f671a9de
      Monty authored
      DROP TABLE opens all temporary tables at start, but then
      uses find_temporary_table() to check if a table is temporary
      instead of is_temporary_table() which is much faster.
      
      This patch fixes this issue.
      f671a9de
    • Monty's avatar
      Don't reset StringBuffers in loops when not needed · 949d10be
      Monty authored
      - Moved out creating StringBuffers in loops and instead create them
        outside and just reset the buffer if it was not allocated (to avoid
        a possible malloc/free for every entry)
      
      Other things related to set_buffer_if_not_allocated()
      - Changed Valuebuffer to not call set_buffer_if_not_allocated() when
        it is created.
      - Fixed geometry functions to reset string length before calling
        String::reserve().  This is because one should not access length()
        of an undefined.
      - Added Item_func_conv_charset::save_in_field() as the item is using
        str_value to store cached values, which conflicts with
        Item::save_str_in_field().
      - Changed Item_proc_string to not store the string value in sql_string
        as this clashes with Item::save_str_in_field().
      - Locally store value of full_name_cstring() in analyse::end_of_records()
        as Item::save_str_in_field() may overwrite it.
      - Marked some strings as set_thread_specific()
      - Added String::free_buffer() to be used internally in String functions
        to just free the buffer but not reset other String values.
      - Fixed uses_buffer_owned_by() to check for allocated length instead of
        strlength, which could be marked MEM_UNDEFINED().
      949d10be
    • Monty's avatar
      Change CHARSET_INFO character set and collaction names to LEX_CSTRING · a206658b
      Monty authored
      This change removed 68 explict strlen() calls from the code.
      
      The following renames was done to ensure we don't use the old names
      when merging code from earlier releases, as using the new variables
      for print function could result in crashes:
      - charset->csname renamed to charset->cs_name
      - charset->name renamed to charset->coll_name
      
      Almost everything where mechanical changes except:
      - Changed to use the new Protocol::store(LEX_CSTRING..) when possible
      - Changed to use field->store(LEX_CSTRING*, CHARSET_INFO*) when possible
      - Changed to use String->append(LEX_CSTRING&) when possible
      
      Other things:
      - There where compiler issues with ensuring that all character set names
        points to the same string: gcc doesn't allow one to use integer constants
        when defining global structures (constant char * pointers works fine).
        To get around this, I declared defines for each character set name
        length.
      a206658b
    • Monty's avatar
      b0910ddd
    • Sergei Golubchik's avatar
      cleanup: Item::can_eval_in_optimize() · 6de84e6f
      Sergei Golubchik authored
      a helper method to check whether an item can be evaluated
      in the query optimization phase (in and below JOIN::optimize()).
      6de84e6f
    • Monty's avatar
      Added override to all releveant methods in Item (and a few other classes) · 30f0a246
      Monty authored
      Other things:
      - Remove inline and virtual for methods that are overrides
      - Added a 'final' to some Item classes
      30f0a246
    • Monty's avatar
      Added full_name_cstring() · 53b43f30
      Monty authored
      This returns a LEX_CSTRING and allows one to avoid strlen() calls.
      53b43f30
    • Monty's avatar
      Reduce usage of strlen() · b6ff139a
      Monty authored
      Changes:
      - To detect automatic strlen() I removed the methods in String that
        uses 'const char *' without a length:
        - String::append(const char*)
        - Binary_string(const char *str)
        - String(const char *str, CHARSET_INFO *cs)
        - append_for_single_quote(const char *)
        All usage of append(const char*) is changed to either use
        String::append(char), String::append(const char*, size_t length) or
        String::append(LEX_CSTRING)
      - Added STRING_WITH_LEN() around constant string arguments to
        String::append()
      - Added overflow argument to escape_string_for_mysql() and
        escape_quotes_for_mysql() instead of returning (size_t) -1 on overflow.
        This was needed as most usage of the above functions never tested the
        result for -1 and would have given wrong results or crashes in case
        of overflows.
      - Added Item_func_or_sum::func_name_cstring(), which returns LEX_CSTRING.
        Changed all Item_func::func_name()'s to func_name_cstring()'s.
        The old Item_func_or_sum::func_name() is now an inline function that
        returns func_name_cstring().str.
      - Changed Item::mode_name() and Item::func_name_ext() to return
        LEX_CSTRING.
      - Changed for some functions the name argument from const char * to
        to const LEX_CSTRING &:
        - Item::Item_func_fix_attributes()
        - Item::check_type_...()
        - Type_std_attributes::agg_item_collations()
        - Type_std_attributes::agg_item_set_converter()
        - Type_std_attributes::agg_arg_charsets...()
        - Type_handler_hybrid_field_type::aggregate_for_result()
        - Type_handler_geometry::check_type_geom_or_binary()
        - Type_handler::Item_func_or_sum_illegal_param()
        - Predicant_to_list_comparator::add_value_skip_null()
        - Predicant_to_list_comparator::add_value()
        - cmp_item_row::prepare_comparators()
        - cmp_item_row::aggregate_row_elements_for_comparison()
        - Cursor_ref::print_func()
      - Removes String_space() as it was only used in one cases and that
        could be simplified to not use String_space(), thanks to the fixed
        my_vsnprintf().
      - Added some const LEX_CSTRING's for common strings:
        - NULL_clex_str, DATA_clex_str, INDEX_clex_str.
      - Changed primary_key_name to a LEX_CSTRING
      - Renamed String::set_quick() to String::set_buffer_if_not_allocated() to
        clarify what the function really does.
      - Rename of protocol function:
        bool store(const char *from, CHARSET_INFO *cs) to
        bool store_string_or_null(const char *from, CHARSET_INFO *cs).
        This was done to both clarify the difference between this 'store' function
        and also to make it easier to find unoptimal usage of store() calls.
      - Added Protocol::store(const LEX_CSTRING*, CHARSET_INFO*)
      - Changed some 'const char*' arrays to instead be of type LEX_CSTRING.
      - class Item_func_units now used LEX_CSTRING for name.
      
      Other things:
      - Fixed a bug in mysql.cc:construct_prompt() where a wrong escape character
        in the prompt would cause some part of the prompt to be duplicated.
      - Fixed a lot of instances where the length of the argument to
        append is known or easily obtain but was not used.
      - Removed some not needed 'virtual' definition for functions that was
        inherited from the parent. I added override to these.
      - Fixed Ordered_key::print() to preallocate needed buffer. Old code could
        case memory overruns.
      - Simplified some loops when adding char * to a String with delimiters.
      b6ff139a
    • Monty's avatar
      Added ErrConvString.lex_cstring() to simplify code · b3bc02f9
      Monty authored
      This allows us to use String::append() without using strlen().
      
      The changes to the ErrConvString class where done by Alexander Barkov
      b3bc02f9
    • Monty's avatar
      Add support for minimum field width for strings to my_vsnprintf() · 5c7d243b
      Monty authored
      This patch adds support for right aligned strings and numbers.
      Left alignment is left as an exercise for anyone needing it.
      
      MDEV-25612 "Assertion `to <= end' failed in process_args" fixed.
      (Was caused by the original version of this patch)
      5c7d243b
    • Alexander Barkov's avatar
      Replaced base_flags_t::IS_AUTOGENERATED_NAME with IS_EXPLICT_NAME · 8dd6ad57
      Alexander Barkov authored
      The name change was to make the intention of the flag more clear and
      also because most usage of the old flag was to test for
      NOT IS_AUTOGENERATED_NAME.
      
      Note that the new flag is the inverse of the old one!
      8dd6ad57
    • Monty's avatar
      Split item->flags into base_flags and with_flags · 6079b46d
      Monty authored
      This was done to simplify copying of with_* flags
      
      Other things:
      - Changed Flags to C++ enums, which enables gdb to print
        out bit values for the flags. This also enables compiler
        errors if one tries to manipulate a non existing bit in
        a variable.
      - Added set_maybe_null() as a shortcut as setting the
        MAYBE_NULL flags was used in a LOT of places.
      - Renamed PARAM flag to SP_VAR to ensure it's not confused with persistent
        statement parameters.
      6079b46d
    • Monty's avatar
      Removed Item::is_fixed() and Item::has_subquery() · 7ca4e381
      Monty authored
      One should instead use Item::fixed() and Item::with_subquery()
      
      Removed Item::is_fixed() and has_subquery() and did the following replace:
      replace is_fixed() fixed() -- *.*
      replace 'has_subquery()' 'with_subquery()' -- *.*
      7ca4e381
    • Michael Widenius's avatar
      Remove calls to current_thd() in Item functions · 94485484
      Michael Widenius authored
      - Added THD argument to functions that calls current_thd() or
        new without a mem_root argument:
        make_same(), set_comparator_func(), set_cmp_func(), set_cmp_func*(),
        set_aggregator() and prepare_sum_aggregators()
      - Changed "new Class" to "new (thd->mem_root) Class"
      
      Almost all changes mechanical, no logic changes.
      94485484
    • Michael Widenius's avatar
      Change bitfields in Item to an uint16 · 3105c9e7
      Michael Widenius authored
      The reason for the change is that neither clang or gcc can do efficient
      code when several bit fields are change at the same time or when copying
      one or more bits between identical bit fields.
      Updated bits explicitely with & and | is MUCH more efficient than what
      current compilers can do.
      3105c9e7
    • Michael Widenius's avatar
      Renamed 'flags' variables in Item_class · 451c4ae5
      Michael Widenius authored
      This is a preparation for adding a flags variable to Item class
      451c4ae5
    • Michael Widenius's avatar
      Revert MDEV-14517 Cleanup for Item::with_subselect · 189d03da
      Michael Widenius authored
      Added back variable 'with_subquery' to Item class as a bit field.
      
      This made the code shorter, faster (removed some virtual methods,
      less code to create an initialized item etc) and made many Item's 7 bytes
      smaller.
      
      This is the last set of my patches the decreases the size of Item.
      
      Some examples from gdb:
      sizeof(Item):        144 -> 120
      sizeof(Item_func)    208 -> 184
      sizeof(Item_sum_max) 368 -> 344
      189d03da
    • Michael Widenius's avatar
      Revert MDEV-16592 "Change Item::with_sum_func to a virtual method" · ae39f4f6
      Michael Widenius authored
      Added back variable 'with_sum_func' to Item class as a bit field.
      
      This made the code shorter, faster (removed some virtual methods,
      less code to create an initialized item etc) and made many Item's 7 bytes
      smaller.
      
      The code is also easier to understand as 'with_sum_func' is threated as any
      other Item variable when creating or copying items.
      ae39f4f6
    • Monty's avatar
      963e5e40
    • Michael Widenius's avatar
      Removed Item::common_flags and replaced it with bit fields · 00d13069
      Michael Widenius authored
      This is to make the Item instances smaller
      00d13069
    • Monty's avatar
      Renamed Type_all_attributes::set_maybe_null() to set_type_maybe_null() · cd1782d2
      Monty authored
      This was done to have Item::set_maybe_null() as an inline function
      cd1782d2
    • Monty's avatar
      c6bf04ab
    • Monty's avatar
      Removed not used Item_type_holder() function · 61822a9b
      Monty authored
      61822a9b
    • Monty's avatar
      Enable BUILD scripts to work with clang · 60335ba1
      Monty authored
      60335ba1
    • Monty's avatar
    • Monty's avatar
      Fixes that enables my_new.cc (new wrapper using my_malloc) · e42130e9
      Monty authored
      This is not enabled by default, as there are leaks in the
      server that needs to be fixed first. One can compile
      with -DUSE_MYSYS_NEW to find the memory leaks from
      'new'. More comments can be found in mysys/my_new.cc
      e42130e9
    • Monty's avatar
      Fixed my_addr_resolve · a93c5145
      Monty authored
      When a server is compiled with -fPIE, my_addr_resolve needs to subtract the info.dli_fbase from symbol addresses in memory for addr2line to recognize them.
      When a server is compiled without -fPIE, my_addr_resolve should not do it.
      Unfortunately not all compilers define __PIE__ when -fPIE was used
      (e.g. older gcc doesn't), so we have to resort to run-time detection.
      a93c5145
    • Monty's avatar
      Report memory leaks from mariadbd if -T or --debug is used · 942a5a89
      Monty authored
      Before memory leaks was only reported if server stopped normally.
      This made it harder to find out where the leaks happened when
      debugging test cases.
      942a5a89
    • Monty's avatar
      Optimize usage of c_ptr(), c_ptr_quick() and String::alloc() · 36cdd5c3
      Monty authored
      The problem was that when one used String::alloc() to allocate a string,
      the String ensures that there is space for an extra NULL byte in the
      buffer and if not, reallocates the string. This is a problem with the
      String::set_int() that calls alloc(21), which forces extra
      malloc/free calls to happen.
      
      - We do not anymore re-allocate String if alloc() is called with the
        Allocated_length. This reduces number of malloc() allocations,
        especially one big re-allocation in Protocol::send_result_Set_metadata()
        for almost every query that produced a result to the connnected client.
      - Avoid extra mallocs when using LONGLONG_BUFFER_SIZE
        This can now be done as alloc() doesn't increase buffers if new length is
        not bigger than old one.
      - c_ptr() is redesigned to be safer (but a bit longer) than before.
      - Remove wrong usage of c_ptr_quick()
        c_ptr_quick() was used in many cases to get the pointer to the used
        buffer, even when it didn't need to be \0 terminated. In this case
        ptr() is a better substitute.
        Another problem with c_ptr_quick() is that it did not guarantee that
        the string would be \0 terminated.
      - item_val_str(), an API function not used currently by the server,
        now always returns a null terminated string (before it didn't always
        do that).
      - Ensure that all String allocations uses STRING_PSI_MEMORY_KEY. The old
        mixed usage of performance keys caused assert's when String buffers
        where shrunk.
      - Binary_string::shrink() is simplifed
      - Fixed bug in String(const char *str, size_t len, CHARSET_INFO *cs) that
        used Binary_string((char *) str, len) instead of Binary_string(str,len).
      - Changed argument to String() creations and String.set() functions to use
        'const char*' instead of 'char*'. This ensures that Alloced_length is
        not set, which gives safety against someone trying to change the
        original string. This also would allow us to use !Alloced_length in
        c_ptr() if needed.
      - Changed string_ptr_cmp() to use memcmp() instead of c_ptr() to avoid
        a possible malloc during string comparision.
      36cdd5c3