- 25 Aug, 2020 1 commit
-
-
Aleksey Midenkov authored
Field::make_new_field() resets invisible property (needed for "CREATE .. SELECT" f.ex.). Recover invisible property in Delayed_insert::get_local_table() (unireg_check works by the same principle).
-
- 21 Aug, 2020 6 commits
-
-
Marko Mäkelä authored
-
Marko Mäkelä authored
In trx_free() we used to declare the entire trx_t unaccessible and then declare that some data members are accessible. This involves a race condition with other threads that may concurrently access the data members that must remain accessible. One type of error is "AddressSanitizer: unknown-crash", whose exact cause we have not determined. Another type of error (reported in MDEV-23472) is "use-after-poison", where the reported shadow bytes would in fact be 00, indicating that the memory was no longer poisoned. The poison-access-unpoison race condition was confirmed by "rr replay". We eliminate the race condition by invoking MEM_NOACCESS on each individual data member of trx_t before freeing the memory to the pool. The memory would not be unpoisoned until the pool is freed or the memory is being reused for another allocation. trx_t::free(): Replaces trx_free(). trx_t::active_commit_ordered: Changed to bool, so that MEM_NOACCESS can be invoked. Removed some accessor functions. Pool: Remove all MEM_ instrumentation. TrxFactory: Move the MEM_ instrumentation from Pool. TrxFactory::debug(): Removed. Moved to trx_t::free(). Because the memory was already marked unaccessible in trx_t::free(), the Factory::debug() call in Pool::putl() would be unable to access it. trx_allocate_for_background(): Replaces trx_create_low(). trx_t::free(): Perform all consistency checks while avoiding duplication, and declare most data members unaccessible.
-
Andrei Elkin authored
Shutdown of mtr tests may be too impatient, esp on CI environment where 10 seconds of `arg` of `shutdown_server arg` may not be enough for the clean shutdown to complete. This is fixed to remove explicit non-zero timeout argument to `shutdown_server` from all mtr tests. mysqltest computes 60 seconds default value for the timeout for the argless `shutdown_server` command. This policy is additionally ensured with a compile time assert.
-
Marko Mäkelä authored
-
Marko Mäkelä authored
-
Jan Lindström authored
This reverts commit 57960211.
-
- 20 Aug, 2020 5 commits
-
-
Marko Mäkelä authored
-
Thirunarayanan Balathandayuthapani authored
in buf_page_set_sticky - Adding os_thread_yield() in buf_page_create() to avoid the continuous buffer pool mutex acquistions.
-
Thirunarayanan Balathandayuthapani authored
in buf_page_set_sticky commit a1f899a8 (MDEV-23233) added the code to make page sticky. So that InnoDB can't allow the page to be grabbed by other thread while doing lazy drop of ahi. But the block could be in flush list and it could have io_fix value as BUF_IO_WRITE. It could lead to the failure in buf_page_set_sticky(). buf_page_create(): If btr_search_drop_page_hash_index() must be invoked, take x-latch on the block. If the block io_fix value is other than BUF_IO_NONE, release the buffer pool mutex and page hash lock and wait for I/O to complete.
-
Marko Mäkelä authored
Since commit 15093639 (MDEV-23484) the rollback of InnoDB transactions is no longer protected by dict_operation_lock. Removing that protection revealed a race condition between transaction rollback and the rollback of an online table-rebuilding operation (OPTIMIZE TABLE, or any online ALTER TABLE that is rebuilding the table). row_undo_mod_clust(): Re-check dict_index_is_online_ddl() after acquiring index->lock, similar to how row_undo_ins_remove_clust_rec() is doing it. Because innobase_online_rebuild_log_free() is holding exclusive index->lock while invoking row_log_free(), this re-check will ensure that row_log_table_low() will not be invoked when index->online_log=NULL. A different race condition is possible between the rollback of a recovered transaction and the start of online secondary index creation. Because prepare_inplace_alter_table_dict() is not acquiring an InnoDB table lock in this case, and because recovered transactions are not covered by metadata locks (MDL), the dict_table_t::indexes could be modified by prepare_inplace_alter_table_dict() while the rollback of a recovered transaction is being executed. Normal transactions would be covered by MDL, and during prepare_inplace_alter_table_dict() we do hold MDL_EXCLUSIVE, that is, an online ALTER TABLE operation may not execute concurrently with other transactions that have accessed the table. row_undo(): To prevent a race condition with prepare_inplace_alter_table_dict(), acquire dict_operation_lock for all recovered transactions. Before MDEV-23484 we used to acquire it for all transactions, not only recovered ones. Note: row_merge_drop_indexes() would not invoke dict_index_remove_from_cache() while transactional locks exist on the table, or while any thread is holding an open table handle. OK, it does that for FULLTEXT INDEX, but ADD FULLTEXT INDEX is not supported as an online operation, and therefore prepare_inplace_alter_table_dict() would acquire a table S lock, which cannot succeed as long as recovered transactions on the table exist, because they would hold a conflicting IX lock on the table.
-
Marko Mäkelä authored
-
- 19 Aug, 2020 1 commit
-
-
Marko Mäkelä authored
In commit fe39d02f (MDEV-20638) we removed some wake-up signaling of the master thread that should have been there, to ensure a steady log checkpointing workload. Common sense suggests that the commit omitted some necessary calls to srv_inc_activity_count(). But, an attempt to add the call to trx_flush_log_if_needed_low() as well as to reinstate the function innobase_active_small() did not restore the performance for the case where sync_binlog=1 is set. Therefore, we will revert the entire commit in MariaDB Server 10.2. In MariaDB Server 10.5, adding a srv_inc_activity_count() call to trx_flush_log_if_needed_low() did restore the performance, so we will not revert MDEV-20638 across all versions.
-
- 18 Aug, 2020 6 commits
-
-
Marko Mäkelä authored
InnoDB transaction rollback includes an unnecessary work-around for a data corruption bug that was fixed by me in MySQL 5.6.12 mysql/mysql-server@935ba09d52c1908bde273ad1940b5ab919d9763d and ported to MariaDB 10.0.8 by commit c291ddfd in 2013 and 2014, respectively. By acquiring and releasing dict_operation_lock in shared mode, row_undo() hopes to prevent the table from being dropped while the undo log record is being rolled back. But, thanks to mentioned fix, debug assertions (that we are adding) show that the rollback is protected by transactional locks (table IX lock, in addition to implicit or explicit exclusive locks on the records that had been modified). Because row_drop_table_for_mysql() would invoke row_add_table_to_background_drop_list() if any locks exist on the table, the mere existence of locks (which is guaranteed during ROLLBACK) is enough to protect the table from disappearing. Hence, acquiring and releasing dict_operation_lock for every row that is being rolled back is unnecessary. row_undo(): Remove the unnecessary acquisition and release of dict_operation_lock. Note: row_add_table_to_background_drop_list() is mostly working around bugs outside InnoDB: MDEV-21175 (insufficient MDL protection of FOREIGN KEY operations) MDEV-21602 (incorrect error handling of CREATE TABLE...SELECT).
-
Marko Mäkelä authored
Regretfully, the parameter innodb_log_checksums was introduced in MySQL 5.7.9 (the first GA release of that series) by mysql/mysql-server@af0acedd885eb7103e319f79d25fda7386ef1506 which partly replaced a parameter that had been introduced in 5.7.8 mysql/mysql-server@22ba38218e1d76c24f69b5a5595ad3bf5933acb0 as innodb_log_checksum_algorithm. Given that the CRC-32C operations are accelerated on many processor implementations (AMD64 with SSE4.2; since MDEV-22669 also on IA-32 with SSE4.2, POWER 8 and later, ARMv8 with some extensions) and by lookup tables when only generic SISD instructions are available, there should be no valid reason to disable checksums. In MariaDB 10.5.2, as a preparation for MDEV-12353, MDEV-19543 deprecated and ignored the parameter innodb_log_checksums altogether. This should imply that after a clean shutdown with innodb_log_checksums=OFF one cannot upgrade to MariaDB Server 10.5 at all. Due to these problems, let us deprecate the parameter innodb_log_checksums and honor it only during server startup. The command SET GLOBAL innodb_log_checksums will always set the parameter to ON.
-
Thirunarayanan Balathandayuthapani authored
Problem: ======= InnoDB drops the column which has foreign key relations on it. So it tries to load the foreign key during rename process of copy algorithm even though the foreign_key_check is disabled. Solution: ======== During alter copy algorithm, InnoDB ignores the error while loading the foreign key constraint if foreign key check is disabled. It should throw the warning about failure of the foreign key constraint when foreign key check is disabled.
-
Thirunarayanan Balathandayuthapani authored
This problem is caused by 6697135c (MDEV-21572). During recovery, InnoDB prefetches the siblings of change buffer index leaf page. It does asynchronous page read and recovery scenario wasn't handled in buf_read_page_background(). It leads to the refusal of startup of the server. Solution: ========= InnoDB shouldn't allow the change buffer index page siblings to be prefetched.
-
Oleksandr Byelkin authored
Remove __bss_start & Co, because systen call "write" check buffer address and return EFAULT if it is wrong.
-
Julius Goryavsky authored
Adding any unknown option to the "[mysqld_safe]" section makes mysqld impossible to start with mysqld_multi. For example, after adding the unknown option "numa_interleave" to the "[mysqld_safe]" section, mysqld_multi exits with the following diagnostics: [ERROR] /usr/local/mysql/bin/mysqld: unknown option '--numa_interleave' To get rid of this behavior, this patch by default adds the "--loose-" prefix to all unknown (for mysqld_safe) options. This behavior can be enabled explicitly with the --ignore-unknown option and disabled with the --no-ignore-unknown option.
-
- 15 Aug, 2020 2 commits
-
-
Eugene Kosov authored
btr_validate_index(): do not stop checking after some level failed. That way it'll become possible to see errors in leaf pages even when uppers layers are corrupted too. page_validate(): check info_bits and status_bits more
-
Daniel Black authored
Since MDEV-18778, timezone tables get changed to innodb to allow them to be replicated to other galera nodes. Even without galera, timezone tables could be declared innodb. With the standalone innodb tables, the mysql_tzinfo_to_sql takes approximately 27 seconds. With the transactions enabled in this patch, 1.2 seconds is the approximate load time. While explicit checks for the engine of the time zone tables could be done, or checks against !opt_skip_write_binlog, non-transactional storage engines will just ignore the transactional state without even a warning so its safe to enact globally. Leap seconds are pretty much ignored as they are a single insert statement and have gone out of favour as they have caused MariaDB stalls in the past.
-
- 13 Aug, 2020 5 commits
-
-
Marko Mäkelä authored
Fix some more cases after merging commit 31aef3ae. Some warnings look possibly genuine, others are clearly bogus.
-
Marko Mäkelä authored
-
Marko Mäkelä authored
-
Marko Mäkelä authored
fil_page_decompress(): Remove a rather useless debug check. We should have test coverage for reading page_compressed pages from files, either due to buffer pool page eviction or due to server restarts. A similar check was removed from fil_space_encrypt() in commit 0b36c27e (MDEV-20307).
-
Marko Mäkelä authored
-
- 12 Aug, 2020 5 commits
-
-
Marko Mäkelä authored
The usage message for the innodb_compression_algorithm system variable did not list snappy, which was added as an optional compression algorithm in MariaDB 10.1.3 and might actually work since commit 90c52e52 (MDEV-12615) in MariaDB 10.1.24. Unfortunately, we will include also unavailable compression algorithms in the list, because ENUM parameters allow numeric values, and we do not want innodb_compression_algorithm=3 to change meaning depending on the way how the source code was compiled.
-
Marko Mäkelä authored
InnoDB only reserves 13 bits for the heap number in the record header, limiting the heap number to be at most 8191. But, when using innodb_page_size=64k and secondary index records of 7 bytes each, it is possible to exceed the maximum heap number. btr_cur_optimistic_insert(): Let the operation fail if the maximum number of records would be exceeded. page_mem_alloc_heap(): Move to the same compilation unit with the only caller, and let the operation fail if the maximum heap number has been allocated already.
-
Marko Mäkelä authored
The debug assertion is bogus, and we had removed it in commit b1ab211d (MDEV-15053) in the MariaDB Server 10.5 branch. For a small data file, fil_space_extend_must_retry() would always allocate a minimum size of 4*innodb_page_size. It is possible that random read-ahead will be triggered for a smaller file than this. In the observed case, the read-ahead was triggered for a 6-page file that used ROW_FORMAT=COMPRESSED with 8KiB page size. So, the desired file size was 49152 bytes, but the actual size was 65536 bytes.
-
Marko Mäkelä authored
In 10.3, DBUG_ASSERT() may expand to something that includes __builtin_expect(), which expects integer arguments, not pointers. To avoid any compiler warnings, let us use an explicit rather than implicit comparison to the null pointer.
-
Eugene Kosov authored
-
- 11 Aug, 2020 9 commits
-
-
Eugene Kosov authored
-
Marko Mäkelä authored
innobase_pk_order_preserved(): Treat an added AUTO_INCREMENT column in the same way as an added existing column. In either case, the column values are not guaranteed to be constant, and thus the ordering may change if such a column is added before any existing PRIMARY KEY columns. prepare_inplace_alter_table_dict(): Initialize dict_table_t::persistent_autoinc before invoking innobase_pk_order_preserved().
-
Anel Husakovic authored
Patch 4aaa38d2 is replacing `my_config.h` with `my_global.h` which is included in in-source build, but not for out-of-source build tree. Closes #1466
-
Marko Mäkelä authored
fil_system_t::keyrotate_next(): If space && space->is_in_rotation_list does not hold, iterate from the start of the list. In debug builds, we would typically have hit SIGSEGV because the iterator would have wrapped a null pointer. It might also be that we are dereferencing a stale pointer. There is no test case, because the encryption is very nondeterministic in nature, due to the use of background threads. This scenario can be hit by setting the following: SET GLOBAL innodb_encryption_threads=5; SET GLOBAL innodb_encryption_rotate_key_age=0;
-
Marko Mäkelä authored
For some reason, GCC emits more -Wmaybe-uninitialized warnings when using the flag -Og than when using -O2. Many of the warnings look genuine.
-
Julius Goryavsky authored
The problem is caused by the fact that adding the --defaults-group-suffix option to fix MDEV-18863 causes mysqld to read all options from the appropriate sections of the config file, including options specific to mysqld_multi. Reading unknown options (which are not supported by mysqld) causes mysqld to terminate with an error. However, the MDEV-18863 problem has been completely fixed by passing options on the command line, and now there is no need to specify the --defaults-group-suffix option (we just need to give priority to options passed through the command line, so as not to break MDEV-18863).
-
Jan Lindström authored
During SST we need to let FTWRL to use normal timeout method even when client is disconnected.
-
Alexander Barkov authored
Some tests relied on the fact that DATETIME->DATE conversion always produce a truncation (with a warning). This is not the case when the SQL statement is executed at current time '00:00:00' sharp. Adding a new SET TIMESTAMP statements to make sure time is not '00:00:00'.
-
Marko Mäkelä authored
-