1. 24 Feb, 2017 3 commits
    • Marko Mäkelä's avatar
      Clean up some Galera tests. · b513e371
      Marko Mäkelä authored
      Note: At least one test is unstable, failing with the following:
      ./mtr --mysqld=--innodb-purge-threads=9 --big-test --no-reorder \
      galera.galera_parallel_autoinc_largetrx galera.galera_var_slave_threads
      
      The result difference is dependent on innodb_purge_threads.
      b513e371
    • Marko Mäkelä's avatar
      MDEV-11454 post-merge fix: · 51af1985
      Marko Mäkelä authored
      buf_dump(): Correct the printf format passed to buf_dump_status()
      to match the argument types.
      
      Revert the changes to storage/xtradb. XtraDB is not being compiled
      for 10.2. The unused copy that we have in the 10.2 branch is only
      getting merges from 10.1.
      
      Disable the test sys_vars.innodb_buffer_pool_dump_pct_function
      because it is unstable on buildbot.
      51af1985
    • Marko Mäkelä's avatar
      Merge pull request #264 from grooverdan/10.2-MDEV-11454-innodb_buffer_pool_dump_pct-entire-pool · 342b48b7
      Marko Mäkelä authored
      MDEV-11454: Make innodb_buffer_pool_dump_pct refer to the entire buffer pool size
      342b48b7
  2. 21 Feb, 2017 1 commit
  3. 20 Feb, 2017 1 commit
    • Marko Mäkelä's avatar
      MDEV-11802 innodb.innodb_bug14676111 fails · a13a636c
      Marko Mäkelä authored
      The function trx_purge_stop() was calling os_event_reset(purge_sys->event)
      before calling rw_lock_x_lock(&purge_sys->latch). The os_event_set()
      call in srv_purge_coordinator_suspend() is protected by that X-latch.
      
      It would seem a good idea to consistently protect both os_event_set()
      and os_event_reset() calls with a common mutex or rw-lock in those
      cases where os_event_set() and os_event_reset() are used
      like condition variables, tied to changes of shared state.
      
      For each os_event_t, we try to document the mutex or rw-lock that is
      being used. For some events, frequent calls to os_event_set() seem to
      try to avoid hangs. Some events are never waited for infinitely, only
      timed waits, and os_event_set() is used for early termination of these
      waits.
      
      os_aio_simulated_put_read_threads_to_sleep(): Define as a null macro
      on other systems than Windows. TODO: remove this altogether and disable
      innodb_use_native_aio on Windows.
      
      os_aio_segment_wait_events[]: Initialize only if innodb_use_native_aio=0.
      
      log_write_flush_to_disk_low(): Invoke log_mutex_enter() at the end, to
      avoid race conditions when changing the system state. (No potential
      race condition existed before MySQL 5.7.)
      a13a636c
  4. 19 Feb, 2017 1 commit
    • Marko Mäkelä's avatar
      MDEV-11802 preparation: Clean up the purge tests. · cc4b2b18
      Marko Mäkelä authored
      Revert the MDEV-4396 tweak to innodb.innodb_bug14676111.
      We must fix the root cause instead.
      
      Allow gcol.innodb_virtual_purge to run on a non-debug build
      (If wait_innodb_all_purged.inc is used in a non-debug test,
      it will have no effect.)
      
      Add the test innodb.index_merge_threshold from MySQL 5.7.
      cc4b2b18
  5. 17 Feb, 2017 1 commit
  6. 16 Feb, 2017 3 commits
    • Marko Mäkelä's avatar
      Simplify a WL#6494/WL#7142 test. · 34bbc76f
      Marko Mäkelä authored
      The test innodb.log_file_size_checkpoint was originally added to
      MySQL 5.7 by me in a bug fix, to fix the interaction of WL#6494
      (redo log resizing, introduced in MySQL 5.6) and WL#7142
      (data file discovery based on MLOG_FILE_NAME records,
      introduced in MySQL 5.7):
      
      commit 70f9ef4e1220827132b50275ca7272f2bcca1864
      Author: Marko Mäkelä <marko.makela@oracle.com>
      Date:   Wed May 21 13:31:29 2014 +0300
      
          Bug#18755095 REDO LOG SIZE CHANGE AFTER CRASH RESULTS IN CHECKPOINT AGE
          ERROR MESSAGE
      
          This is a regression from fixing
          Bug#18730524 REPEATED KILL+RESTART FAILS DUE TO MISSING MLOG_FILE_NAME
          RECORD
      
          innobase_start_or_create_for_mysql(): Invoke fil_names_clear() before
          creating the "checkpoint" when changing redo log files.
      
          Approved by Jimmy Yang on IM.
      
      The relevant part of the test is that fil_names_clear() is invoked to
      emit an MLOG_CHECKPOINT record before the redo log files are deleted.
      In case the server is killed before ib_logfile0 has been deleted,
      the old (not-yet-resized) redo log will be treated as valid. We do not
      need to create a large number of tables for that.
      34bbc76f
    • Marko Mäkelä's avatar
      MDEV-12072 Do not unnecessarily construct rec_printer objects · 7a528801
      Marko Mäkelä authored
      I introduced the rec_printer object in MySQL to pretty-print raw InnoDB
      records and index tuples in diagnostic messages. These objects are being
      constructed unconditionally, even though the DBUG_PRINT is not enabled.
      
      The unnecessary work is avoided by simply passing rec_printer(…).str()
      to the DBUG_LOG macro that was introduced in MDEV-11713.
      7a528801
    • Igor Babaev's avatar
      Fixed bug mdev-9924. · 37925c6c
      Igor Babaev authored
      Supported queries with window functions when GROUP BY could be
      optimized away.
      37925c6c
  7. 15 Feb, 2017 11 commits
    • Sergei Petrunia's avatar
      MDEV-10694 - SIGFPE and/or huge memory allocation in maria_create ... · e688d814
      Sergei Petrunia authored
      The issue was that JOIN::rollup_write_data() used
      JOIN::tmp_table_param::[start_]recinfo, which had uninitialized data.
      
      These fields have uninitialized data, because JOIN::tmp_table_param
      currently only stores some grouping-related data fields.  The data about
      the work (temporary) tables themselves is stored in
      join->join_tab[...].tmp_table_param.
      
      The fix is to make JOIN::rollup_write_data follow this convention
      and look at the right TMP_TABLE_PARAM object
      e688d814
    • Vicențiu Ciorbaru's avatar
      MariaDB 10.2 can now start from MySQL 5.7 data directory · 24911cee
      Vicențiu Ciorbaru authored
      An upgrade is still necessary however.
      24911cee
    • Vicențiu Ciorbaru's avatar
      MDEV-10700: 10.2.2 windowing function returns incorrect result · eb54d86b
      Vicențiu Ciorbaru authored
      This bug is fixed by MDEV-10092. Add test case to check for regressions.
      eb54d86b
    • Vicențiu Ciorbaru's avatar
      MDEV-11868: min ( distinct ) over ( ) returns wrong value · d06a44e0
      Vicențiu Ciorbaru authored
      The bug was not visible in current HEAD. Introduced test case to catch
      regressions. Also improve error messages regarding distinct usage in
      window functions.
      d06a44e0
    • Vicențiu Ciorbaru's avatar
      MDEV-11697: Lead Window Function Returns Incorrect Results · 88ddb1ea
      Vicențiu Ciorbaru authored
      This issue is fixed by the patch for MDEV-10092. Add test case to check
      for regressions though.
      88ddb1ea
    • Vicențiu Ciorbaru's avatar
      MDEV-10092: Server crashes in in ha_heap::rnd_pos / Table_read_cursor::get_next · d4746422
      Vicențiu Ciorbaru authored
      The bug was caused by several issues.
      2 problems in seek_io_cache. Due to wrong offsets used, we would end up
      seeking way too much (first change), or over the intended seek point
      (second change). Fixing it requires correctly detecting available data
      in buffer (first change), and not using "IO_SIZE alligned" reads. The
      second is needed because _my_b_cache_read adjusts the pos_in_file itself
      based on read_pos and read_end. Pretending buffer is empty when we want
      to force a read will aleviate this problem.
      
      Secondly, the big-table cursors didn't repect the interface definitions
      of always returning the rownumber that Table_read_cursor::fetch() would activate.
      
      At the same time, next(), prev() and move_to() should not perform any
      row activation.
      d4746422
    • Vicențiu Ciorbaru's avatar
      MDEV-10859: Wrong result of aggregate window function in query with HAVING and no ORDER BY · 9fe9fb68
      Vicențiu Ciorbaru authored
      Window functions need to be computed after applying the HAVING clause.
      An optimization that we have for regular, non-window function, cases is
      to apply having only during sending of the rows to the client. This
      allows rows that should be filtered from the temporary table used to
      store aggregation results to be stored there.
      
      This behaviour is undesireable for window functions, as we have to
      compute window functions on the result-set after HAVING is applied.
      Storing extra rows in the table leads to wrong values as the frame
      bounds might capture those -to be filtered afterwards- rows.
      9fe9fb68
    • Marko Mäkelä's avatar
      MDEV-11641 innobase_get_stmt_safe() does not copy the last byte of thd->query_string · a90066b1
      Marko Mäkelä authored
      innobase_get_stmt_safe(): Copy also the last byte.
      a90066b1
    • Marko Mäkelä's avatar
      Fix some InnoDB memory leaks. · 703d0985
      Marko Mäkelä authored
      dict_init_free(): Make global, and move the call from
      dict_close() to srv_free(), because this is initialized
      earlier than dict_sys.
      
      innobase_space_shutdown(): Do not leak srv_allow_writes_event.
      703d0985
    • Marko Mäkelä's avatar
      MDEV-11782: Redefine the innodb_encrypt_log format · 2af28a36
      Marko Mäkelä authored
      Write only one encryption key to the checkpoint page.
      Use 4 bytes of nonce. Encrypt more of each redo log block,
      only skipping the 4-byte field LOG_BLOCK_HDR_NO which the
      initialization vector is derived from.
      
      Issue notes, not warning messages for rewriting the redo log files.
      
      recv_recovery_from_checkpoint_finish(): Do not generate any redo log,
      because we must avoid that before rewriting the redo log files, or
      otherwise a crash during a redo log rewrite (removing or adding
      encryption) may end up making the database unrecoverable.
      Instead, do these tasks in innobase_start_or_create_for_mysql().
      
      Issue a firm "Missing MLOG_CHECKPOINT" error message. Remove some
      unreachable code and duplicated error messages for log corruption.
      
      LOG_HEADER_FORMAT_ENCRYPTED: A flag for identifying an encrypted redo
      log format.
      
      log_group_t::is_encrypted(), log_t::is_encrypted(): Determine
      if the redo log is in encrypted format.
      
      recv_find_max_checkpoint(): Interpret LOG_HEADER_FORMAT_ENCRYPTED.
      
      srv_prepare_to_delete_redo_log_files(): Display NOTE messages about
      adding or removing encryption. Do not issue warnings for redo log
      resizing any more.
      
      innobase_start_or_create_for_mysql(): Rebuild the redo logs also when
      the encryption changes.
      
      innodb_log_checksums_func_update(): Always use the CRC-32C checksum
      if innodb_encrypt_log. If needed, issue a warning
      that innodb_encrypt_log implies innodb_log_checksums.
      
      log_group_write_buf(): Compute the checksum on the encrypted
      block contents, so that transmission errors or incomplete blocks can be
      detected without decrypting.
      
      Rewrite most of the redo log encryption code. Only remember one
      encryption key at a time (but remember up to 5 when upgrading from the
      MariaDB 10.1 format.)
      2af28a36
    • Marko Mäkelä's avatar
      MDEV-12061 Allow innodb_log_files_in_group=1 · 743ac7c2
      Marko Mäkelä authored
      The InnoDB redo log consists of a list of files that logically form
      a bigger file, as if the individual files were concatenated together.
      
      The first file will always be written on redo log checkpoint, because
      the two checkpoint pages are at the start of the single logical
      redo log file.
      
      There is no technical reason why InnoDB requires at least 2 files
      to exist. Let us reduce the minimum number to 1. In that way,
      restoring from backups will become easier, since InnoDB can directly
      deal with a single backed-up redo log file.
      743ac7c2
  8. 14 Feb, 2017 17 commits
    • Sergei Golubchik's avatar
      sporadic failures of rpl.rpl_binlog_errors · 3d85292a
      Sergei Golubchik authored
      the test restarts the server, giving it 60 seconds to shutdown
      and then killing it mercilessly.
      
      make sure the server closes all MyISAM tables before shutdown,
      as we cannot reliably expect it to make the deadline.
      3d85292a
    • Sergei Golubchik's avatar
      sporadic failures of main.default_debug · 96d097a7
      Sergei Golubchik authored
      wait for the first connection to reach the debug sync point
      before letting the second connection do its stuff
      96d097a7
    • Sergei Golubchik's avatar
      cleanup: mtr startup warning · 41f42dff
      Sergei Golubchik authored
      options --log-slow-admin-statements, --log-queries-not-using-indexes and --log-slow-slave-statements have no effect if --log_slow_queries is not set
      
      1. s/--log_slow_queries/--log-slow-queries/
      2. disable log-slow-admin-statement/etc in mytr when doing mysqld --help
      41f42dff
    • Sergei Golubchik's avatar
      MDEV-11439 No data type JSON, but CAST(something AS JSON) pretends to work · 6f6d0531
      Sergei Golubchik authored
      Use JSON_COMPACT(X) instead of CAST(X AS JSON).
      Bonus - X is validated to be a valid JSON.
      Fix a typo in the error message.
      6f6d0531
    • Sergei Golubchik's avatar
      bugfix: uninitialized variable · e0fa2ce4
      Sergei Golubchik authored
      fixes crashes on win32-debug
      e0fa2ce4
    • Vladislav Vaintroub's avatar
      MDEV-12056 mysql_config outputs non-existing mysqlclient · 8877f1c3
      Vladislav Vaintroub authored
      Change output of mysql_config --libs to have -lmariadb instead of
      -lmysqlclient
      
      Static linking is not yet supported, because static library has different
      base name ( libmariadbclient.a static vs libmariadb.so shared)
      8877f1c3
    • Alexey Botchkov's avatar
      MDEV-11439 No data type JSON, but CAST(something AS JSON) pretends to · f76d5fef
      Alexey Botchkov authored
      work.
      
              json_detailed() fixed
      f76d5fef
    • Marko Mäkelä's avatar
      Post-push fix for MDEV-12057. · 2bf07556
      Marko Mäkelä authored
      Fix a test result for the narrower type of innodb_fast_shutdown.
      2bf07556
    • Marko Mäkelä's avatar
      MDEV-12057 Embedded server shutdown hangs in InnoDB · 1b4b4f68
      Marko Mäkelä authored
      Ever since MDEV-5800 enabled indexed virtual columns for InnoDB,
      the InnoDB shutdown relied on close_connections() that would set
      thd->killed for the InnoDB purge threads. Alas, the embedded server
      shutdown is not invoking close_connections(), and thus InnoDB purge
      threads fail to initiate shutdown, causing a hang.
      
      innodb_inited: Remove. Use srv_was_started instead.
      
      innobase_fast_shutdown: Remove. Use srv_fast_shutdown instead.
      
      srv_running: Renamed from thd_destructor_myvar, and made global.
      The value NULL means that shutdown was requested or the purge threads
      should not be running because of innodb_read_only_mode=1.
      
      innobase_init(): Set srv_was_started after ensuring that srv_running
      was initialized. (In innodb_read_only mode, the purge threads are not
      started and we do not care if srv_running==NULL.)
      
      innobase_start_or_create_for_mysql(): Do not set srv_was_started.
      Let it be set by the only caller innobase_init().
      
      srv_purge_should_exit(): Check also srv_was_started and srv_running
      when evaluating thd->killed.
      1b4b4f68
    • Vicențiu Ciorbaru's avatar
      MDEV-11170: MariaDB 10.2 cannot start on MySQL 5.7 datadir · d731ce21
      Vicențiu Ciorbaru authored
      PART 2 of the fix adds the logic of not using password column, unless it
      exists. If password column is missing we attempt to use plugin &&
      authentication_string columns.
      d731ce21
    • Vicențiu Ciorbaru's avatar
      MDEV-11170: MariaDB 10.2 cannot start on MySQL 5.7 datadir · 5ab93737
      Vicențiu Ciorbaru authored
      PART 1 of the fix requires a bit of refactoring to not use hard-coded
      field indices any more. Create classes that express the grant tables structure,
      without exposing the underlying field indices.
      
      Most of the code is converted to use these classes, except parts which
      are not directly affected by the MDEV-11170. These however are TODO
      items for subsequent refactoring.
      5ab93737
    • Vicențiu Ciorbaru's avatar
      Define a helper class to allow for saving sql_mode using RAII · dc90e249
      Vicențiu Ciorbaru authored
      On construction the Sql_mode_save class stores the current THD's
      sql_mode. On destruction, the THD's mode is restored.
      dc90e249
    • Vicențiu Ciorbaru's avatar
    • Vicențiu Ciorbaru's avatar
      MDEV-11746: Wrong result upon using FIRST_VALUE with a window frame · 5bf33843
      Vicențiu Ciorbaru authored
      The same approach is needed for LAST_VALUE, otherwise the LAST_VALUE sum
      functions are not cleared correctly. Now LAST_VALUE behaves as NTH_VALUE
      with 0 offset, only that the frame that it is examining is the bottom bound,
      not the top bound.
      5bf33843
    • Vicențiu Ciorbaru's avatar
      MDEV-11746: Wrong result upon using FIRST_VALUE with a window frame · 57341852
      Vicențiu Ciorbaru authored
      Reimplement FIRST_VALUE to act as NTH_VALUE with 0 offset. The previous
      implementation was flawed when the window frame would remove values.
      57341852
    • Vicențiu Ciorbaru's avatar
      MDEV-10122: MariaDB does not support group functions in some contexts where MySQL does · f675eab7
      Vicențiu Ciorbaru authored
      The problematic queries involve unions. For unions we have an
      optimization where we skip the ORDER BY clause in a query from one side
      of the union if it will be performed later due to UNION.
      EX:
      (SELECT a from t1 ORDER BY a) ORDER BY b;
      The first ordering by a is not necessary and it gets removed.
      
      The problem is that we still need to resolve the Items before removing the
      ORDER BY list from the
      SELECT_LEX structure. During this final resolve step however, we forgot to
      allow SET functions within the ORDER BY clause. This caused us to return
      an "Invalid use of group function" error during the checking performed
      by fix_fields in Item_sum::init_sum_func_check.
      f675eab7
    • Vicențiu Ciorbaru's avatar
      MDEV-11170: MariaDB 10.2 cannot start on MySQL 5.7 datadir: · fdfdea40
      Vicențiu Ciorbaru authored
      Fatal error: mysql.user table is damaged or in unsupported 3.20 format
      
      The problem stems from MySQL 5.7.6. According to MySQL documentation:
      In MySQL 5.7.6, the Password column was removed and all credentials are
      stored in the authentication_string column.
      
      If opening a MySQL 5.7.6 (and up) datadir with MariaDB 10.2, the user table
      appears corrupted. In order to fix this, the server must be started with
      --skip-grant-tables and then a subsequent mysql_upgrade command must be
      issued.
      
      This patch updates the mysql_upgrade command to also add the removed
      Password column. The password column is necessary, otherwise
      the mysql_upgrade script fails due to the Event_scheduler not being able
      to start, as it can't find Event_priv in the table where it ought to be.
      MySQL's version has column position 28 (0 index) vs our datadir version
      expects position 29.
      fdfdea40
  9. 13 Feb, 2017 2 commits