An error occurred fetching the project authors.
  1. 18 Jan, 2019 1 commit
    • Marko Mäkelä's avatar
      MDEV-18152 Assertion 'num_fts_index <= 1' failed · 4e75bfcb
      Marko Mäkelä authored
      InnoDB does not allow creating multiple FULLTEXT INDEX
      in ALGORITHM=INPLACE. This constraint was not being properly
      enforced after MariaDB started to support ALGORITHM=INSTANT
      and instant ADD COLUMN.
      
      As a side effect of this bug, we again allow ALGORITHM=INPLACE
      to rebuild a table when one FULLTEXT INDEX survives.
      Also, we are returning a more accurate reason for refusing LOCK=NONE.
      
      innobase_fulltext_exist(): Return the number of fulltext indexes.
      
      ha_innobase::check_if_supported_inplace_alter(): If the table
      needs to be rebuilt, refuse the operation if multiple fulltext
      indexes would remain.
      4e75bfcb
  2. 29 Dec, 2018 1 commit
    • Eugene Kosov's avatar
      MDEV-18041 Database corruption after renaming a prefix-indexed column · 802ce967
      Eugene Kosov authored
      This is a regression after MDEV-13671.
      
      The bug is related to key part prefix lengths wich are stored in SYS_FIELDS.
      Storage format is not obvious and was handled incorrectly which led to data
      dictionary corruption.
      
      SYS_FIELDS.POS actually contains prefix length too in case if any key part
      has prefix length.
      
      innobase_rename_column_try(): fixed prefixes handling
      
      Tests for prefixed indexes added too.
      
      Closes #1063
      802ce967
  3. 05 Nov, 2018 1 commit
  4. 19 Oct, 2018 1 commit
    • Marko Mäkelä's avatar
      MDEV-15662 Instant DROP COLUMN or changing the order of columns · 0e5a4ac2
      Marko Mäkelä authored
      Allow ADD COLUMN anywhere in a table, not only adding as the
      last column.
      
      Allow instant DROP COLUMN and instant changing the order of columns.
      
      The added columns will always be added last in clustered index records.
      In new records, instantly dropped columns will be stored as NULL or
      empty when possible.
      
      Information about dropped and reordered columns will be written in
      a metadata BLOB (mblob), which is stored before the first 'user' field
      in the hidden metadata record at the start of the clustered index.
      The presence of mblob is indicated by setting the delete-mark flag in
      the metadata record.
      
      The metadata BLOB stores the number of clustered index fields,
      followed by an array of column information for each field.
      For dropped columns, we store the NOT NULL flag, the fixed length,
      and for variable-length columns, whether the maximum length exceeded
      255 bytes. For non-dropped columns, we store the column position.
      
      Unlike with MDEV-11369, when a table becomes empty, it cannot
      be converted back to the canonical format. The reason for this is
      that other threads may hold cached objects such as
      row_prebuilt_t::ins_node that could refer to dropped or reordered
      index fields.
      
      For instant DROP COLUMN and ROW_FORMAT=COMPACT or ROW_FORMAT=DYNAMIC,
      we must store the n_core_null_bytes in the root page, so that the
      chain of node pointer records can be followed in order to reach the
      leftmost leaf page where the metadata record is located.
      If the mblob is present, we will zero-initialize the strings
      "infimum" and "supremum" in the root page, and use the last byte of
      "supremum" for storing the number of null bytes (which are allocated
      but useless on node pointer pages). This is necessary for
      btr_cur_instant_init_metadata() to be able to navigate to the mblob.
      
      If the PRIMARY KEY contains any variable-length column and some
      nullable columns were instantly dropped, the dict_index_t::n_nullable
      in the data dictionary could be smaller than it actually is in the
      non-leaf pages. Because of this, the non-leaf pages could use more
      bytes for the null flags than the data dictionary expects, and we
      could be reading the lengths of the variable-length columns from the
      wrong offset, and thus reading the child page number from wrong place.
      This is the result of two design mistakes that involve unnecessary
      storage of data: First, it is nonsense to store any data fields for
      the leftmost node pointer records, because the comparisons would be
      resolved by the MIN_REC_FLAG alone. Second, there cannot be any null
      fields in the clustered index node pointer fields, but we nevertheless
      reserve space for all the null flags.
      
      Limitations (future work):
      
      MDEV-17459 Allow instant ALTER TABLE even if FULLTEXT INDEX exists
      MDEV-17468 Avoid table rebuild on operations on generated columns
      MDEV-17494 Refuse ALGORITHM=INSTANT when the row size is too large
      
      btr_page_reorganize_low(): Preserve any metadata in the root page.
      Call lock_move_reorganize_page() only after restoring the "infimum"
      and "supremum" records, to avoid a memcmp() assertion failure.
      
      dict_col_t::DROPPED: Magic value for dict_col_t::ind.
      
      dict_col_t::clear_instant(): Renamed from dict_col_t::remove_instant().
      Do not assert that the column was instantly added, because we
      sometimes call this unconditionally for all columns.
      Convert an instantly added column to a "core column". The old name
      remove_instant() could be mistaken to refer to "instant DROP COLUMN".
      
      dict_col_t::is_added(): Rename from dict_col_t::is_instant().
      
      dtype_t::metadata_blob_init(): Initialize the mblob data type.
      
      dtuple_t::is_metadata(), dtuple_t::is_alter_metadata(),
      upd_t::is_metadata(), upd_t::is_alter_metadata(): Check if info_bits
      refer to a metadata record.
      
      dict_table_t::instant: Metadata about dropped or reordered columns.
      
      dict_table_t::prepare_instant(): Prepare
      ha_innobase_inplace_ctx::instant_table for instant ALTER TABLE.
      innobase_instant_try() will pass this to dict_table_t::instant_column().
      On rollback, dict_table_t::rollback_instant() will be called.
      
      dict_table_t::instant_column(): Renamed from instant_add_column().
      Add the parameter col_map so that columns can be reordered.
      Copy and adjust v_cols[] as well.
      
      dict_table_t::find(): Find an old column based on a new column number.
      
      dict_table_t::serialise_columns(), dict_table_t::deserialise_columns():
      Convert the mblob.
      
      dict_index_t::instant_metadata(): Create the metadata record
      for instant ALTER TABLE. Invoke dict_table_t::serialise_columns().
      
      dict_index_t::reconstruct_fields(): Invoked by
      dict_table_t::deserialise_columns().
      
      dict_index_t::clear_instant_alter(): Move the fields for the
      dropped columns to the end, and sort the surviving index fields
      in ascending order of column position.
      
      ha_innobase::check_if_supported_inplace_alter(): Do not allow
      adding a FTS_DOC_ID column if a hidden FTS_DOC_ID column exists
      due to FULLTEXT INDEX. (This always required ALGORITHM=COPY.)
      
      instant_alter_column_possible(): Add a parameter for InnoDB table,
      to check for additional conditions, such as the maximum number of
      index fields.
      
      ha_innobase_inplace_ctx::first_alter_pos: The first column whose position
      is affected by instant ADD, DROP, or changing the order of columns.
      
      innobase_build_col_map(): Skip added virtual columns.
      
      prepare_inplace_add_virtual(): Correctly compute num_to_add_vcol.
      Remove some unnecessary code. Note that the call to
      innodb_base_col_setup() should be executed later.
      
      commit_try_norebuild(): If ctx->is_instant(), let the virtual
      columns be added or dropped by innobase_instant_try().
      
      innobase_instant_try(): Fill in a zero default value for the
      hidden column FTS_DOC_ID (to reduce the work needed in MDEV-17459).
      If any columns were dropped or reordered (or added not last),
      delete any SYS_COLUMNS records for the following columns, and
      insert SYS_COLUMNS records for all subsequent stored columns as well
      as for all virtual columns. If any virtual column is dropped, rewrite
      all virtual column metadata. Use a shortcut only for adding
      virtual columns. This is because innobase_drop_virtual_try()
      assumes that the dropped virtual columns still exist in ctx->old_table.
      
      innodb_update_cols(): Renamed from innodb_update_n_cols().
      
      innobase_add_one_virtual(), innobase_insert_sys_virtual(): Change
      the return type to bool, and invoke my_error() when detecting an error.
      
      innodb_insert_sys_columns(): Insert a record into SYS_COLUMNS.
      Refactored from innobase_add_one_virtual() and innobase_instant_add_col().
      
      innobase_instant_add_col(): Replace the parameter dfield with type.
      
      innobase_instant_drop_cols(): Drop matching columns from SYS_COLUMNS
      and all columns from SYS_VIRTUAL.
      
      innobase_add_virtual_try(), innobase_drop_virtual_try(): Let
      the caller invoke innodb_update_cols().
      
      innobase_rename_column_try(): Skip dropped columns.
      
      commit_cache_norebuild(): Update table->fts->doc_col.
      
      dict_mem_table_col_rename_low(): Skip dropped columns.
      
      trx_undo_rec_get_partial_row(): Skip dropped columns.
      
      trx_undo_update_rec_get_update(): Handle the metadata BLOB correctly.
      
      trx_undo_page_report_modify(): Avoid out-of-bounds access to record fields.
      Log metadata records consistently.
      Apparently, the first fields of a clustered index may be updated
      in an update_undo vector when the index is ID_IND of SYS_FOREIGN,
      as part of renaming the table during ALTER TABLE. Normally, updates of
      the PRIMARY KEY should be logged as delete-mark and an insert.
      
      row_undo_mod_parse_undo_rec(), row_purge_parse_undo_rec():
      Use trx_undo_metadata.
      
      row_undo_mod_clust_low(): On metadata rollback, roll back the root page too.
      
      row_undo_mod_clust(): Relax an assertion. The delete-mark flag was
      repurposed for ALTER TABLE metadata records.
      
      row_rec_to_index_entry_impl(): Add the template parameter mblob
      and the optional parameter info_bits for specifying the desired new
      info bits. For the metadata tuple, allow conversion between the original
      format (ADD COLUMN only) and the generic format (with hidden BLOB).
      Add the optional parameter "pad" to determine whether the tuple should
      be padded to the index fields (on ALTER TABLE it should), or whether
      it should remain at its original size (on rollback).
      
      row_build_index_entry_low(): Clean up the code, removing
      redundant variables and conditions. For instantly dropped columns,
      generate a dummy value that is NULL, the empty string, or a
      fixed length of NUL bytes, depending on the type of the dropped column.
      
      row_upd_clust_rec_by_insert_inherit_func(): On the update of PRIMARY KEY
      of a record that contained a dropped column whose value was stored
      externally, we will be inserting a dummy NULL or empty string value
      to the field of the dropped column. The externally stored column would
      eventually be dropped when purge removes the delete-marked record for
      the old PRIMARY KEY value.
      
      btr_index_rec_validate(): Recognize the metadata record.
      
      btr_discard_only_page_on_level(): Preserve the generic instant
      ALTER TABLE metadata.
      
      btr_set_instant(): Replaces page_set_instant(). This sets a clustered
      index root page to the appropriate format, or upgrades from
      the MDEV-11369 instant ADD COLUMN to generic ALTER TABLE format.
      
      btr_cur_instant_init_low(): Read and validate the metadata BLOB page
      before reconstructing the dictionary information based on it.
      
      btr_cur_instant_init_metadata(): Do not read any lengths from the
      metadata record header before reading the BLOB. At this point, we
      would not actually know how many nullable fields the metadata record
      contains.
      
      btr_cur_instant_root_init(): Initialize n_core_null_bytes in one
      of two possible ways.
      
      btr_cur_trim(): Handle the mblob record.
      
      row_metadata_to_tuple(): Convert a metadata record to a data tuple,
      based on the new info_bits of the metadata record.
      
      btr_cur_pessimistic_update(): Invoke row_metadata_to_tuple() if needed.
      Invoke dtuple_convert_big_rec() for metadata records if the record is
      too large, or if the mblob is not yet marked as externally stored.
      
      btr_cur_optimistic_delete_func(), btr_cur_pessimistic_delete():
      When the last user record is deleted, do not delete the
      generic instant ALTER TABLE metadata record. Only delete
      MDEV-11369 instant ADD COLUMN metadata records.
      
      btr_cur_optimistic_insert(): Avoid unnecessary computation of rec_size.
      
      btr_pcur_store_position(): Allow a logically empty page to contain
      a metadata record for generic ALTER TABLE.
      
      REC_INFO_DEFAULT_ROW_ADD: Renamed from REC_INFO_DEFAULT_ROW.
      This is for the old instant ADD COLUMN (MDEV-11369) only.
      
      REC_INFO_DEFAULT_ROW_ALTER: The more generic metadata record,
      with additional information for dropped or reordered columns.
      
      rec_info_bits_valid(): Remove. The only case when this would fail
      is when the record is the generic ALTER TABLE metadata record.
      
      rec_is_alter_metadata(): Check if a record is the metadata record
      for instant ALTER TABLE (other than ADD COLUMN). NOTE: This function
      must not be invoked on node pointer records, because the delete-mark
      flag in those records may be set (it is garbage), and then a debug
      assertion could fail because index->is_instant() does not necessarily
      hold.
      
      rec_is_add_metadata(): Check if a record is MDEV-11369 ADD COLUMN metadata
      record (not more generic instant ALTER TABLE).
      
      rec_get_converted_size_comp_prefix_low(): Assume that the metadata
      field will be stored externally. In dtuple_convert_big_rec() during
      the rec_get_converted_size() call, it would not be there yet.
      
      rec_get_converted_size_comp(): Replace status,fields,n_fields with tuple.
      
      rec_init_offsets_comp_ordinary(), rec_get_converted_size_comp_prefix_low(),
      rec_convert_dtuple_to_rec_comp(): Add template<bool mblob = false>.
      With mblob=true, process a record with a metadata BLOB.
      
      rec_copy_prefix_to_buf(): Assert that no fields beyond the key and
      system columns are being copied. Exclude the metadata BLOB field.
      
      rec_convert_dtuple_to_metadata_comp(): Convert an alter metadata tuple
      into a record.
      
      row_upd_index_replace_metadata(): Apply an update vector to an
      alter_metadata tuple.
      
      row_log_allocate(): Replace dict_index_t::is_instant()
      with a more appropriate condition that ignores dict_table_t::instant.
      Only a table on which the MDEV-11369 ADD COLUMN was performed
      can "lose its instantness" when it becomes empty. After
      instant DROP COLUMN or reordering columns, we cannot simply
      convert the table to the canonical format, because the data
      dictionary cache and all possibly existing references to it
      from other client connection threads would have to be adjusted.
      
      row_quiesce_write_index_fields(): Do not crash when the table contains
      an instantly dropped column.
      
      Thanks to Thirunarayanan Balathandayuthapani for discussing the design
      and implementing an initial prototype of this.
      Thanks to Matthias Leich for testing.
      0e5a4ac2
  5. 11 May, 2018 1 commit
    • Sachin Agarwal's avatar
      Bug #26334149 - MYSQL CRASHES WHEN FULL TEXT INDEXES IBD FILES ARE ORPHANED DUE TO RENAME TABLE · 197bf0fe
      Sachin Agarwal authored
      Problem:
      When FTS index is added into a table which doesn't have 'FTS_DOC_ID'
      column, Innodb rebuilds table to add column 'FTS_DOC_ID'. when this FTS
      index is dropped from this table. Innodb doesn't not rebuild table to
      remove 'FTS_DOC_ID' column and deletes FTS index auxiliary tables.
      But it doesn't delete FTS common auxiliary tables.
      Later when the database having this table is renamed, FTS auxiliary
      tables are not renamed because table's flags2 (dict_table_t.flags2)
      has been resetted for DICT_TF2_FTS flag during FTS index drop operation.
      Now when we drop old database, it leads to an assert.
      
      Fix:
      During renaming of FTS auxiliary tables, ORed a condition to check if
      table has DICT_TF2_FTS_HAS_DOC_ID flag set.
      
      RB: 18769
      Reviewed by : Jimmy.Yang@oracle.com
      197bf0fe
  6. 02 Nov, 2017 1 commit
    • Marko Mäkelä's avatar
      MDEV-14016 Allow instant ADD COLUMN, ADD INDEX, LOCK=NONE · 6402ca78
      Marko Mäkelä authored
      Ideally, we would move some code from
      ha_innobase::prepare_inplace_alter_table() to
      ha_innobase::check_if_supported_inplace_alter(),
      but the API does not really allow us to return errors; it can
      only inform which forms of ALGORITHM and LOCK are allowed.
      
      So, we have to duplicate some logic between the "check" and "prepare"
      phases. We do the duplication by calling common functions.
      
      instant_alter_column_possible(): Check if instant column operation
      is possible. Invoked from both
      ha_innobase::check_if_supported_inplace_alter() and
      prepare_inplace_alter_table_dict().
      
      ha_innobase::check_if_supported_inplace_alter(): Before refusing
      certain operations if FULLTEXT INDEX exist, check if instant ALTER TABLE
      is possible and return early if it is the case.
      
      prepare_inplace_alter_table_dict(): Before checking the limitations
      on FULLTEXT INDEX, check if instant ALTER TABLE is possible, and suppress
      the checks if it is the case. If instant ADD COLUMN is used when the
      table already contains FULLTEXT INDEX, do account for a
      hidden FTS_DOC_ID_INDEX in a debug assertion.
      6402ca78
  7. 18 Oct, 2017 1 commit
    • Sergei Golubchik's avatar
      MDEV-14081 ALTER TABLE CHANGE COLUMN Corrupts Index Leading to Crashes in 10.2 · 607d8f9e
      Sergei Golubchik authored
      remove remnants of 10.0 bugfix, incorrectly merged into 10.2
      
      Using col_names[i] was obviously, wrong, must've been col_names[ifield->col_no].
      incorrect column name resulted in innodb having index unique_id2(id1),
      while the server thought it's unique_id2(id4).
      But col_names[ifield->col_no] is wrong too, because `table` has non-renamed
      columns, so the correct column name is always dict_table_get_col_name(table, ifield->col_no)
      607d8f9e
  8. 11 Oct, 2017 1 commit
  9. 10 Oct, 2017 1 commit
    • Jan Lindström's avatar
      MDEV-13838: Wrong result after altering a partitioned table · fc9ff695
      Jan Lindström authored
      Reverted incorrect changes done on MDEV-7367 and MDEV-9469. Fixes properly
      also related bugs:
      
      MDEV-13668: InnoDB unnecessarily rebuilds table when renaming a column and adding index
      MDEV-9469: 'Incorrect key file' on ALTER TABLE
      MDEV-9548: Alter table (renaming and adding index) fails with "Incorrect key file for table"
      MDEV-10535: ALTER TABLE causes standalone/wsrep cluster crash
      MDEV-13640: ALTER TABLE CHANGE and ADD INDEX on auto_increment column fails with "Incorrect key file for table..."
      
      Root cause for all these bugs is the fact that MariaDB .frm file
      can contain virtual columns but InnoDB dictionary does not and
      previous fixes were incorrect or unnecessarily forced table
      rebuilt. In index creation key_part->fieldnr can be bigger than
      number of columns in InnoDB data dictionary. We need to skip not
      stored fields when calculating correct column number for InnoDB
      data dictionary.
      
      dict_table_get_col_name_for_mysql
              Remove
      
      innobase_match_index_columns
              Revert incorrect change done on MDEV-7367
      
      innobase_need_rebuild
              Remove unnecessary rebuild force when column is renamed.
      
      innobase_create_index_field_def
              Calculate InnoDB column number correctly and remove
              unnecessary column name set.
      
      innobase_create_index_def, innobase_create_key_defs
              Remove unneeded fields parameter. Revert unneeded memset.
      
      prepare_inplace_alter_table_dict
              Remove unneeded col_names parameter
      
      index_field_t
              Remove unneeded col_name member.
      
      row_merge_create_index
              Remove unneeded col_names parameter and resolution.
      
      Effected tests:
               innodb-alter-table : Add test case for MDEV-13668
               innodb-alter : Remove MDEV-13668, MDEV-9469 FIXMEs
                              and restore original tests
               innodb-wl5980-alter : Remove MDEV-13668,  MDEV-9469 FIXMEs
                              and restore original tests
      fc9ff695
  10. 29 Aug, 2017 2 commits
    • Marko Mäkelä's avatar
      Adjust the imported MySQL 5.6 tests for MariaDB · f56bd70f
      Marko Mäkelä authored
      FIXME: MDEV-13668 InnoDB unnecessarily rebuilds table
      
      FIXME: MDEV-13671 InnoDB should use case-insensitive column name comparisons
      like the rest of the server
      
      FIXME: MDEV-13640 / Properly fix MDEV-9469 'Incorrect key file' on ALTER TABLE
      
      FIXME: investigate result difference in innodb.innodb-alter-autoinc
      and ensure that MariaDB does the right thing with auto_increment_increment
      and auto_increment_offset, for both ALGORITHM=INPLACE and ALGORITHM=COPY
      (Oracle MySQL behaviour differs between those two).
      f56bd70f
    • Marko Mäkelä's avatar
      MDEV-13625 Merge InnoDB test cases from MySQL 5.6 (part 1) · 8d929816
      Marko Mäkelä authored
      Import some ALTER TABLE test cases from MySQL 5.6 without modification.
      The adjustments will be in a separate commit.
      8d929816