1. 05 Jun, 2020 9 commits
    • Eugene Kosov's avatar
      fix compilation with VS2019, preview of 16.7 version · 7a695d8a
      Eugene Kosov authored
      Compiler tells something about argument-dependent lookup. I do not
      understand how that ADL works. But I know that such operators should
      be free functions, instead of methods:
      http://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Ro-symmetric
      
      Such syntax defines 'friend' free functions.
      7a695d8a
    • Igor Babaev's avatar
      MDEV-22042 Server crash in Item_field::print on ANALYZE FORMAT=JSON · a8c200c7
      Igor Babaev authored
      When processing a query with a recursive CTE a temporary table is used for
      each recursive reference of the CTE. As any temporary table it uses its own
      mem-root for table definition structures. Due to specifics of the current
      implementation of ANALYZE stmt command this mem-root can be freed only at
      the very of query processing. Such deallocation of mem-root memory happens
      in close_thread_tables(). The function looks through the list of the tmp
      tables rec_tables attached to the THD of the query and frees corresponding
      mem-roots. If the query uses a stored function then such list is created
      for each query of the function. When a new rec_list has to be created the
      old one has to be saved and then restored at the proper moment.
      The bug occurred because only one rec_list for the query containing CTE was
      created. As a result close_thread_tables() freed tmp mem-roots used for
      rec_tables prematurely destroying some data needed for the output produced
      by the ANALYZE command.
      a8c200c7
    • Julius Goryavsky's avatar
    • Julius Goryavsky's avatar
      Merge 10.1 into 10.2 · 5f55f69e
      Julius Goryavsky authored
      5f55f69e
    • Vladislav Vaintroub's avatar
      Fix appveyor build. · 15cdcb2a
      Vladislav Vaintroub authored
      15cdcb2a
    • Marko Mäkelä's avatar
      MDEV-22769 Shutdown hang or crash due to XA breaking locks · efc70da5
      Marko Mäkelä authored
      The background drop table queue in InnoDB is a work-around for
      cases where the SQL layer is requesting DDL on tables on which
      transactional locks exist.
      
      One such case are XA transactions. Our test case exploits the
      fact that the recovery of XA PREPARE transactions will
      only resurrect InnoDB table locks, but not MDL that should
      block any concurrent DDL.
      
      srv_shutdown_t: Introduce the srv_shutdown_state=SRV_SHUTDOWN_INITIATED
      for the initial part of shutdown, to wait for the background drop
      table queue to be emptied.
      
      srv_shutdown_bg_undo_sources(): Assign
      srv_shutdown_state=SRV_SHUTDOWN_INITIATED
      before waiting for the background drop table queue to be emptied.
      
      row_drop_tables_for_mysql_in_background(): On slow shutdown, if
      no active transactions exist (excluding ones that are in
      XA PREPARE state), skip any tables on which locks exist.
      
      row_drop_table_for_mysql(): Do not unnecessarily attempt to
      drop InnoDB persistent statistics for tables that have
      already been added to the background drop table queue.
      
      row_mysql_close(): Relax an assertion, and free all memory
      even if innodb_force_recovery=2 would prevent the background
      drop table queue from being emptied.
      efc70da5
    • Marko Mäkelä's avatar
      MDEV-22790 Race between btr_page_mtr_lock() dropping AHI on the same block · 138c11cc
      Marko Mäkelä authored
      This race condition was introduced by
      commit ad6171b9 (MDEV-22456).
      
      In the observed case, two threads were executing
      btr_search_drop_page_hash_index() on the same block,
      to free a stale entry that was attached to a dropped index.
      Both threads were holding an S latch on the block.
      
      We must prevent the double-free of block->index by holding
      block->lock in exclusive mode.
      
      btr_search_guess_on_hash(): Do not invoke
      btr_search_drop_page_hash_index(block) to get rid of
      stale entries, because we are not necessarily holding
      an exclusive block->lock here.
      
      buf_defer_drop_ahi(): New function, to safely drop stale
      entries in buf_page_mtr_lock(). We will skip the call to
      btr_search_drop_page_hash_index(block) when only requesting
      bufferfixing (no page latch), because in that case, we should
      not be accessing the adaptive hash index, and we might get
      a deadlock if we acquired the page latch.
      138c11cc
    • Marko Mäkelä's avatar
      MDEV-22646: Fix a memory leak · 3677dd5c
      Marko Mäkelä authored
      btr_search_sys_free(): Free btr_search_sys->hash_tables.
      
      The leak was introduced in commit ad2bf112.
      3677dd5c
    • Vladislav Vaintroub's avatar
      Windows, build tweak. · 1828196f
      Vladislav Vaintroub authored
      Allow targets for building "noinstall" zip, and debuginfo zip.
      1828196f
  2. 04 Jun, 2020 5 commits
    • Sergei Golubchik's avatar
      29ed04cb
    • Sergey Vojtovich's avatar
      MDEV-22339 - Assertion `str_length < len' failed · dce4c0f9
      Sergey Vojtovich authored
      When acquiring SNW/SNRW/X MDL lock DDL/admin statements may abort pending
      thr lock in concurrent connection with open HANDLER (or delayed insert
      thread).
      
      This may lead to a race condition when table->alias is accessed
      concurrently by such threads. Either assertion failure or memory leak
      is a practical consequence of this race condition.
      
      Specifically HANDLER is opening a table and issuing alias.copy(), while
      DDL executing get_lock_data()/alias.c_ptr()/realloc()/realloc_raw().
      
      Fixed by perforimg table->init() before it is published via
      thd->open_tables.
      dce4c0f9
    • Marko Mäkelä's avatar
      dict_check_sys_tables(): Do not rely on buf_page_optimistic_get() · c5883deb
      Marko Mäkelä authored
      We are supposed to commit and restart the mini-transaction
      between records. There is no point to store and restore the
      persistent cursor position otherwise.
      
      If buf_page_optimistic_get() is patched to always fail, the
      debug build would fail to start up due to trying to re-acquire
      an already S-latched block.
      
      This bug (which should not have visible impact to users, because
      the code is only executed during startup, while no other threads
      are accessing B-trees or causing pages to be evicted from the
      buffer pool) was caught as part of a debugging effort for
      something else.
      
      The debugging approach was: Make buf_page_optimistic_get()
      always return FALSE, and add ut_a(block->lock.lock_word == X_LOCK_DECR)
      to both buf_LRU_get_free_only() and buf_LRU_block_free_non_file_page().
      This would catch misuse of the buffer pool. If it were not for
      buf_page_optimistic_get(), no buf_block_t::lock of any BUF_BLOCK_NOT_USED
      block would ever be acquired.
      c5883deb
    • Varun Gupta's avatar
      MDEV-16230: Server crashes when Analyze format=json is run with a window... · f69278bc
      Varun Gupta authored
      MDEV-16230: Server crashes when Analyze format=json is run with a window function with empty PARTITION BY and ORDER BY clauses
      
      Currently when both PARTITION BY and ORDER BY clauses are empty then we create a Item
      with the first field in the select list and sort with that field.
      It should be created as an Item_temptable_field instead of Item_field because the
      print() function continues to work even if the table has been dropped.
      f69278bc
    • Marko Mäkelä's avatar
      MDEV-22721 Remove bloat caused by InnoDB logger class · eba2d10a
      Marko Mäkelä authored
      Introduce a new ATTRIBUTE_NOINLINE to
      ib::logger member functions, and add UNIV_UNLIKELY hints to callers.
      
      Also, remove some crash reporting output. If needed, the
      information will be available using debugging tools.
      
      Furthermore, remove some fts_enable_diag_print output that included
      indexed words in raw form. The code seemed to assume that words are
      NUL-terminated byte strings. It is not clear whether a NUL terminator
      is always guaranteed to be present. Also, UCS2 or UTF-16 strings would
      typically contain many NUL bytes.
      eba2d10a
  3. 03 Jun, 2020 4 commits
    • Julius Goryavsky's avatar
      3f019d17
    • sjaakola's avatar
      MDEV-22763 backporting MDEV-20225 fix into 10.1 · 8ec0e911
      sjaakola authored
      Backported the support for aborting and replaying stored procedure and fix for trigger
      key assigments from 10.4 version.
      Backported also two mtr tests: wsrep_sp_bf_abort and MDEV-20225
      8ec0e911
    • Thirunarayanan Balathandayuthapani's avatar
      MDEV-22646 Assertion `table2->cached' failed in dict_table_t::add_to_cache · ad2bf112
      Thirunarayanan Balathandayuthapani authored
      Problem:
      ========
        During buffer pool resizing, InnoDB recreates the dictionary hash
      tables. Dictionary hash table reuses the heap of AHI hash tables.
      It leads to memory corruption.
      
      Fix:
      ====
      - While disabling AHI, free the heap and AHI hash tables. Recreate the
      AHI hash tables and assign new heap when AHI is enabled.
      
      - btr_blob_free() access invalid page if page was reallocated during
      buffer poolresizing. So btr_blob_free() should get the page from
      buf_pool instead of using existing block.
      
      - btr_search_enabled and block->index should be checked after
      acquiring the btr_search_sys latch
      
      - Moved the buffer_pool_scan debug sync to earlier before accessing the
      btr_search_sys latches to avoid the hang of truncate_purge_debug
      test case
      
      - srv_printf_innodb_monitor() should acquire btr_search_sys latches
      before AHI hash tables.
      ad2bf112
    • Marko Mäkelä's avatar
      MDEV-22577 innodb_fast_shutdown=0 fails to report purge progress · ca3aa679
      Marko Mäkelä authored
      srv_purge_should_exit(): Report progress on slow shutdown
      not only to systemd, but also to the error log.
      ca3aa679
  4. 02 Jun, 2020 1 commit
  5. 01 Jun, 2020 5 commits
  6. 31 May, 2020 1 commit
  7. 29 May, 2020 11 commits
    • Sergey Vojtovich's avatar
      Attempt fixing mroonga gcc 8 build failure · 49854811
      Sergey Vojtovich authored
      Part of MDEV-19061 - table_share used for reading statistical tables is
                           not protected
      49854811
    • Sergey Vojtovich's avatar
      Thread safe histograms loading · c2798784
      Sergey Vojtovich authored
      Previously multiple threads were allowed to load histograms concurrently.
      There were no known problems caused by this. But given amount of data
      races in this code, it'd happen sooner or later.
      
      To avoid scalability bottleneck, histograms loading is protected by
      per-TABLE_SHARE atomic variable.
      
      Whenever histograms were loaded by preceding statement (hot-path), a
      scalable load-acquire check is performed.
      
      Whenever histograms have to be loaded anew, mutual exclusion for loaders
      is established by atomic variable. If histograms are being loaded
      concurrently, statement waits until load is completed.
      
      - Table_statistics::total_hist_size moved to TABLE_STATISTICS_CB: only
        meaningful within TABLE_SHARE (not used for collected stats).
      - TABLE_STATISTICS_CB::histograms_can_be_read and
        TABLE_STATISTICS_CB::histograms_are_read are replaced with a tri state
        atomic variable.
      - Simplified away alloc_histograms_for_table_share().
      
      Note: there's still likely a data race if a thread attempts accessing
      histograms data after it failed to load it (because of concurrent load).
      It was there previously and goes out of the scope of this effort. One way
      of fixing it could be reviving TABLE::histograms_are_read and adding
      appropriate checks whenever it is needed.
      
      Part of MDEV-19061 - table_share used for reading statistical tables is
                           not protected
      c2798784
    • Sergey Vojtovich's avatar
      Thread safe statistics loading · 609a0d3d
      Sergey Vojtovich authored
      Previously multiple threads were allowed to load statistics concurrently.
      There were no known problems caused by this. But given amount of data
      races in this code, it'd happen sooner or later.
      
      To avoid scalability bottleneck, statistics loading is protected by
      per-TABLE_SHARE atomic variable.
      
      Whenever statistics were loaded by preceding statement (hot-path), a
      scalable load-acquire check is performed.
      
      Whenever statistics have to be loaded anew, mutual exclusion for loaders
      is established by atomic variable. If statistics are being loaded
      concurrently, statement waits until load is completed.
      
      TABLE_STATISTICS_CB::stats_can_be_read and
      TABLE_STATISTICS_CB::stats_is_read are replaced with a tri state atomic
      variable.
      
      Part of MDEV-19061 - table_share used for reading statistical tables is
                           not protected
      609a0d3d
    • Sergey Vojtovich's avatar
      Simplified away statistics_for_tables_is_needed() · 1055a7f4
      Sergey Vojtovich authored
      Removed redundant loops, integrated logics into the caller instead.
      Unified condition in read_statistics_for_tables(), less
      "table_share != NULL" checks, no more potential "table_share == NULL"
      dereferencing.
      
      Part of MDEV-19061 - table_share used for reading statistical tables is
                           not protected
      1055a7f4
    • Kentoku SHIBA's avatar
    • Kentoku SHIBA's avatar
    • Alexander Barkov's avatar
      MDEV-22744 *SAN: sql/item_xmlfunc.cc:791:43: runtime error: downcast of... · a2932e86
      Alexander Barkov authored
      MDEV-22744 *SAN: sql/item_xmlfunc.cc:791:43: runtime error: downcast of address ... which does not point to an object of type 'Item_func' note: object is of type 'Item_bool' (on optimized builds)
      
      In Item_nodeset_func_predicate::val_nodeset, args[1] is not necessarily
      an Item_func descendant. It can be Item_bool.
      
      Removing a wrong cast. It was not really needed anyway.
      a2932e86
    • Vladislav Vaintroub's avatar
    • Vladislav Vaintroub's avatar
      MDEV-22743 Windows 10 MSI installer : port in use is not determined · b00cd3e4
      Vladislav Vaintroub authored
      when checking for free port, use the same logic (IPv6 socket address
      / dual socket), like the server would.
      
      Previous solution for testing whether port is free was trying to bind
      IPv4 socket on INADDR_ANY.
      
      This not work now on some reason, that attempt succeeds, even if there is
      an existing IPv6-dual socket listening on 0.0.0.0:3306
      b00cd3e4
    • Vladislav Vaintroub's avatar
      MSI installer : Use CAQuietExec64 on Win64 , not CAQuietExec · ff72f369
      Vladislav Vaintroub authored
      It works, but irritates people who look into the log and see
      traces of 32bit custom action server.
      ff72f369
    • Vladislav Vaintroub's avatar
      Remove unused WiX source file · e2d7da49
      Vladislav Vaintroub authored
      e2d7da49
  8. 28 May, 2020 2 commits
    • Anel Husakovic's avatar
      fix pre-definition for embedded server for find_user_or_anon() · a1b3bebe
      Anel Husakovic authored
      Pre-definitions are allowed for non-embedded.
      Failur catched with:
      ```
      cmake ../../10.1 -DCMAKE_BUILD_TYPE=Debug -DCMAKE_CXX_COMPILER=g++-9
      -DCMAKE_C_COMPILER=gcc-9 -DWITH_EMBEDDED_SERVER=ON -DCMAKE_BUILD_TYPE=Debug
      -DPLUGIN_{ARCHIVE,TOKUDB,MROONGA,OQGRAPH,ROCKSDB,PERFSCHEMA,SPIDER,SPHINX}=N
      -DMYSQL_MAINTAINER_MODE=ON -DNOT_FOR_DISTRIBUTION=ON
      ```
      Alternative fix would be
      ```
      --- a/sql/sql_acl.cc
      +++ b/sql/sql_acl.cc
      @@ -201,8 +201,10 @@ LEX_STRING current_user= { C_STRING_WITH_LEN("*current_user") };
       LEX_STRING current_role= { C_STRING_WITH_LEN("*current_role") };
       LEX_STRING current_user_and_current_role= { C_STRING_WITH_LEN("*current_user_and_current_role") };
      
      +#ifndef EMBEDDED_LIBRARY
       class ACL_USER;
       static ACL_USER *find_user_or_anon(const char *host, const char *user, const char *ip);
      +#endif
      ```
      a1b3bebe
    • Anel Husakovic's avatar
      MDEV-22312: Bad error message for SET DEFAULT ROLE when user account is not granted the role · 957cb7b7
      Anel Husakovic authored
      - `SET DEFAULT ROLE xxx [FOR yyy]` should say:
        "User yyy has not been granted a role xxx" if:
          - The current user (not the user `yyy` in the FOR clause) can see the
          role xxx. It can see the role if:
            * role exists in `mysql.roles_mappings` (traverse the graph),
            * If the current user has read access on `mysql.user` table - in
          that case, it can see all roles, granted or not.
          - Otherwise it should be "Invalid role specification".
      
      In other words, it should not be possible to use `SET DEFAULT ROLE` to discover whether a specific role exist or not.
      957cb7b7
  9. 27 May, 2020 2 commits