1. 19 Jun, 2017 1 commit
  2. 16 Jun, 2017 1 commit
  3. 15 Jun, 2017 2 commits
    • Marko Mäkelä's avatar
      Remove some fields from dict_table_t · 58f87a41
      Marko Mäkelä authored
      dict_table_t::thd: Remove. This was only used by btr_root_block_get()
      for reporting decryption failures, and it was only assigned by
      ha_innobase::open(), and never cleared. This could mean that if a
      connection is closed, the pointer would become stale, and the server
      could crash while trying to report the error. It could also mean
      that an error is being reported to the wrong client. It is better
      to use current_thd in this case, even though it could mean that if
      the code is invoked from an InnoDB background operation, there would
      be no connection to which to send the error message.
      
      Remove dict_table_t::crypt_data and dict_table_t::page_0_read.
      These fields were never read.
      
      fil_open_single_table_tablespace(): Remove the parameter "table".
      58f87a41
    • Marko Mäkelä's avatar
      88b96181
  4. 13 Jun, 2017 3 commits
  5. 12 Jun, 2017 7 commits
  6. 09 Jun, 2017 3 commits
    • Marko Mäkelä's avatar
      MDEV-13039 innodb_fast_shutdown=0 may fail to purge all undo log · 417434f1
      Marko Mäkelä authored
      When a slow shutdown is performed soon after spawning some work for
      background threads that can create or commit transactions, it is possible
      that new transactions are started or committed after the purge has finished.
      This is violating the specification of innodb_fast_shutdown=0, namely that
      the purge must be completed. (None of the history of the recent transactions
      would be purged.)
      
      Also, it is possible that the purge threads would exit in slow shutdown
      while there exist active transactions, such as recovered incomplete
      transactions that are being rolled back. Thus, the slow shutdown could
      fail to purge some undo log that becomes purgeable after the transaction
      commit or rollback.
      
      srv_undo_sources: A flag that indicates if undo log can be generated
      or the persistent, whether by background threads or by user SQL.
      Even when this flag is clear, active transactions that already exist
      in the system may be committed or rolled back.
      
      innodb_shutdown(): Renamed from innobase_shutdown_for_mysql().
      Do not return an error code; the operation never fails.
      Clear the srv_undo_sources flag, and also ensure that the background
      DROP TABLE queue is empty.
      
      srv_purge_should_exit(): Do not allow the purge to exit if
      srv_undo_sources are active or the background DROP TABLE queue is not
      empty, or in slow shutdown, if any active transactions exist
      (and are being rolled back).
      
      srv_purge_coordinator_thread(): Remove some previous workarounds
      for this bug.
      
      innobase_start_or_create_for_mysql(): Set buf_page_cleaner_is_active
      and srv_dict_stats_thread_active directly. Set srv_undo_sources before
      starting the purge subsystem, to prevent immediate shutdown of the purge.
      Create dict_stats_thread and fts_optimize_thread immediately
      after setting srv_undo_sources, so that shutdown can use this flag to
      determine if these subsystems were started.
      
      dict_stats_shutdown(): Shut down dict_stats_thread. Backported from 10.2.
      
      srv_shutdown_table_bg_threads(): Remove (unused).
      417434f1
    • Marko Mäkelä's avatar
      Correct a merge error of MDEV-11626 · a9117c90
      Marko Mäkelä authored
      a9117c90
    • Jan Lindström's avatar
      MDEV-12610: MariaDB start is slow · 58c56dd7
      Jan Lindström authored
      Problem appears to be that the function fsp_flags_try_adjust()
      is being unconditionally invoked on every .ibd file on startup.
      Based on performance investigation also the top function
      fsp_header_get_crypt_offset() needs to addressed.
      
      Ported implementation of fsp_header_get_encryption_offset()
      function from 10.2 to fsp_header_get_crypt_offset().
      
      Introduced a new function fil_crypt_read_crypt_data()
      to read page 0 if it is not yet read.
      
      fil_crypt_find_space_to_rotate(): Now that page 0 for every .ibd
      file is not read on startup we need to check has page 0 read
      from space that we investigate for key rotation, if it is not read
      we read it.
      
      fil_space_crypt_get_status(): Now that page 0 for every .ibd
      file is not read on startup here also we need to read page 0
      if it is not yet read it. This is needed
      as tests use IS query to wait until background encryption
      or decryption has finished and this function is used to
      produce results.
      
      fil_crypt_thread(): Add is_stopping condition for tablespace
      so that we do not rotate pages if usage of tablespace should
      be stopped. This was needed for failure seen on regression
      testing.
      
      fil_space_create: Remove page_0_crypt_read and extra
      unnecessary info output.
      
      fil_open_single_table_tablespace(): We call fsp_flags_try_adjust
      only when when no errors has happened and server was not started
      on read only mode and tablespace validation was requested or
      flags contain other table options except low order bits to
      FSP_FLAGS_POS_PAGE_SSIZE position.
      
      fil_space_t::page_0_crypt_read removed.
      
      Added test case innodb-first-page-read to test startup when
      encryption is on and when encryption is off to check that not
      for all tables page 0 is read on startup.
      58c56dd7
  7. 08 Jun, 2017 5 commits
    • Marko Mäkelä's avatar
      Cleanup of MDEV-12600: crash during install_db with innodb_page_size=32K and ibdata1=3M · fbeb9489
      Marko Mäkelä authored
      The doublewrite buffer pages must fit in the first InnoDB system
      tablespace data file. The checks that were added in the initial patch
      (commit 112b21da)
      were at too high level and did not cover all cases.
      
      innodb.log_data_file_size: Test all innodb_page_size combinations.
      
      fsp_header_init(): Never return an error. Move the change buffer creation
      to the only caller that needs to do it.
      
      btr_create(): Clean up the logic. Remove the error log messages.
      
      buf_dblwr_create(): Try to return an error on non-fatal failure.
      Check that the first data file is big enough for creating the
      doublewrite buffers.
      
      buf_dblwr_process(): Check if the doublewrite buffer is available.
      Display the message only if it is available.
      
      recv_recovery_from_checkpoint_start_func(): Remove a redundant message
      about FIL_PAGE_FILE_FLUSH_LSN mismatch when crash recovery has already
      been initiated.
      
      fil_report_invalid_page_access(): Simplify the message.
      
      fseg_create_general(): Do not emit messages to the error log.
      
      innobase_init(): Revert the changes.
      
      trx_rseg_create(): Refactor (no functional change).
      fbeb9489
    • Jan Lindström's avatar
      d03abc71
    • Marko Mäkelä's avatar
      Revert part of MDEV-12113 · 68890fe7
      Marko Mäkelä authored
      commit 1af8bf39 added unnecessary
      calls to fil_write_flushed_lsn() during redo log resizing at
      InnoDB server startup.
      
      Because fil_write_flushed_lsn() is neither redo-logged nor doublewrite
      buffered, the call is risky and should be avoided, because if the
      server killed during the write call, the whole InnoDB instance can
      become inaccessible (corrupted page 0 in the system tablespace).
      
      In the best case, this call might prevent a diagnostic message from
      being emitted to the error log on the next startup.
      68890fe7
    • Jan Lindström's avatar
      Merge pull request #404 from grooverdan/10.1-MDEV-13032-galera_new_cluster · 18f62d94
      Jan Lindström authored
      MDEV-13032: fix galera_new_cluster to be POSIX
      18f62d94
    • Daniel Black's avatar
      d8515829
  8. 06 Jun, 2017 3 commits
    • Marko Mäkelä's avatar
      Correct a merge error · 6439238c
      Marko Mäkelä authored
      6439238c
    • Marko Mäkelä's avatar
      Merge 10.0 into 10.1 · 30df297c
      Marko Mäkelä authored
      Rewrite the test encryption.innodb-checksum-algorithm not to
      require any restarts or re-bootstrapping, and to cover all
      innodb_page_size combinations.
      
      Test innodb.101_compatibility with all innodb_page_size combinations.
      30df297c
    • Marko Mäkelä's avatar
      Follow-up to MDEV-12042 (test innodb_page_size variants) · d8d39721
      Marko Mäkelä authored
      innodb_page_size_small: A new set of combinations, for
      innodb_page_size up to 16k. In MariaDB 10.0, this does not
      make a difference, but in 10.1 and later, innodb_page_size
      would cover 32k and 64k, for which ROW_FORMAT=COMPRESSED
      is not available.
      
      Enable these combinations in a few InnoDB tests.
      d8d39721
  9. 05 Jun, 2017 2 commits
  10. 01 Jun, 2017 4 commits
    • Elena Stepanova's avatar
    • Jan Lindström's avatar
      MDEV-12600: crash during install_db with innodb_page_size=32K and ibdata1=3M; · 112b21da
      Jan Lindström authored
      Problem was that all doublewrite buffer pages must fit to first
      system datafile.
      
      Ported commit 27a34df7882b1f8ed283f22bf83e8bfc523cbfde
      Author: Shaohua Wang <shaohua.wang@oracle.com>
      Date:   Wed Aug 12 15:55:19 2015 +0800
      
          BUG#21551464 - SEGFAULT WHILE INITIALIZING DATABASE WHEN
          INNODB_DATA_FILE SIZE IS SMALL
      
      To 10.1 (with extended error printout).
      
      btr_create(): If ibuf header page allocation fails report error and
      return FIL_NULL. Similarly if root page allocation fails return a error.
      
      dict_build_table_def_step: If fsp_header_init fails return
      error code.
      
      fsp_header_init: returns true if header initialization succeeds
      and false if not.
      
      fseg_create_general: report error if segment or page allocation fails.
      
      innobase_init: If first datafile is smaller than 3M and could not
      contain all doublewrite buffer pages report error and fail to
      initialize InnoDB plugin.
      
      row_truncate_table_for_mysql: report error if fsp header init
      fails.
      
      srv_init_abort: New function to report database initialization errors.
      
      srv_undo_tablespaces_init, innobase_start_or_create_for_mysql: If
      database initialization fails report error and abort.
      
      trx_rseg_create: If segment header creation fails return.
      112b21da
    • Jan Lindström's avatar
      MDEV-12114: install_db shows corruption for rest encryption and... · 6b698715
      Jan Lindström authored
      MDEV-12114: install_db shows corruption for rest encryption and innodb_checksum_algorithm=strict_none
      
      Problem was that checksum check resulted false positives that page is
      both not encrypted and encryted when checksum_algorithm was
      strict_none.
      
      Encrypton checksum will use only crc32 regardless of setting.
      
      buf_zip_decompress: If compression fails report a error message
      containing the space name if available (not available during import).
      And note if space could be encrypted.
      
      buf_page_get_gen: Do not assert if decompression fails,
      instead unfix the page and return NULL to upper layer.
      
      fil_crypt_calculate_checksum: Use only crc32 method.
      
      fil_space_verify_crypt_checksum: Here we need to check
      crc32, innodb and none method for old datafiles.
      
      fil_space_release_for_io: Allow null space.
      
      encryption.innodb-compressed-blob is now run with crc32 and none
      combinations.
      
      Note that with none and strict_none method there is not really
      a way to detect page corruptions and page corruptions after
      decrypting the page with incorrect key.
      
      New test innodb-checksum-algorithm to test different checksum
      algorithms with encrypted, row compressed and page compressed
      tables.
      6b698715
    • Jan Lindström's avatar
      MDEV-12113: install_db shows corruption for rest encryption with innodb_data_file_path=ibdata1:3M; · 1af8bf39
      Jan Lindström authored
      Problem was that FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION field that for
      encrypted pages even in system datafiles should contain key_version
      except very first page (0:0) is after encryption overwritten with
      flush lsn.
      
      Ported WL#7990 Repurpose FIL_PAGE_FLUSH_LSN to 10.1
      The field FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION is consulted during
      InnoDB startup.
      
      At startup, InnoDB reads the FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION
      from the first page of each file in the InnoDB system tablespace.
      If there are multiple files, the minimum and maximum LSN can differ.
      These numbers are passed to InnoDB startup.
      
      Having the number in other files than the first file of the InnoDB
      system tablespace is not providing much additional value. It is
      conflicting with other use of the field, such as on InnoDB R-tree
      index pages and encryption key_version.
      
      This worklog will stop writing FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION to
      other files than the first file of the InnoDB system tablespace
      (page number 0:0) when system tablespace is encrypted. If tablespace
      is not encrypted we continue writing FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION
      to all first pages of system tablespace to avoid unnecessary
      warnings on downgrade.
      
      open_or_create_data_files(): pass only one flushed_lsn parameter
      
      xb_load_tablespaces(): pass only one flushed_lsn parameter.
      
      buf_page_create(): Improve comment about where
      FIL_PAGE_FIL_FLUSH_LSN_OR_KEY_VERSION is set.
      
      fil_write_flushed_lsn(): A new function, merged from
      fil_write_lsn_and_arch_no_to_file() and
      fil_write_flushed_lsn_to_data_files().
      Only write to the first page of the system tablespace (page 0:0)
      if tablespace is encrypted, or write all first pages of system
      tablespace and invoke fil_flush_file_spaces(FIL_TYPE_TABLESPACE)
      afterwards.
      
      fil_read_first_page(): read flush_lsn and crypt_data only from
      first datafile.
      
      fil_open_single_table_tablespace(): Remove output of LSN, because it
      was only valid for the system tablespace and the undo tablespaces, not
      user tablespaces.
      
      fil_validate_single_table_tablespace(): Remove output of LSN.
      
      checkpoint_now_set(): Use fil_write_flushed_lsn and output
      a error if operation fails.
      
      Remove lsn variable from fsp_open_info.
      
      recv_recovery_from_checkpoint_start(): Remove unnecessary second
      flush_lsn parameter.
      
      log_empty_and_mark_files_at_shutdown(): Use fil_writte_flushed_lsn
      and output error if it fails.
      
      open_or_create_data_files(): Pass only one flushed_lsn variable.
      1af8bf39
  11. 31 May, 2017 1 commit
  12. 29 May, 2017 5 commits
  13. 27 May, 2017 1 commit
  14. 26 May, 2017 2 commits