- 23 Dec, 2020 2 commits
-
-
Otto Kekäläinen authored
ColumnStore seems to build by default, so it must be explicitly disabled with a build flag, so that it does not build at all and thus build machine disk space and CPU will be spared. This reverts commit 113f1868.
-
Marko Mäkelä authored
-
- 22 Dec, 2020 1 commit
-
-
Aleksey Midenkov authored
Add missed PSI_CALL_drop_table_share().
-
- 21 Dec, 2020 10 commits
-
-
Otto Kekäläinen authored
Refactor previous commit to fix mistake revealed by Buildbot. We can't have a structure where PLUGIN_COLUMNSTORE would ever be 'YES' on an arch that does not support it, as the flag overrides any potential platform detection code and builds on non-amd64 would all fail.
-
Otto Kekäläinen authored
- Use new Salsa-CI templates just like downstream does - Also apply adaptations needed for native MariaDB to build correctly on Salsa-CI - Adapt for mysql-8.0 now in Debian Sid, keep also mysql-5.7 test - Test TLSv1.3 to ensure it does not regress - Remove Snappy compression from autopkgtest as it not longer works
-
Otto Kekäläinen authored
ColumnStore files and debian/control stanza was removed in 1edd2243, and thus will not be included in a native build. Also adapt the debian/rules to follow this same policy and only build ColumnStore in builds triggered from autobake-deb.sh. Avoiding building ColumnStore in vain saves a lot of build time and disk space.
-
Otto Kekäläinen authored
Maintaining the same deterministic file order in both upstream (this) and downstream (Debian) repositories makes it easier to compare the delta.
-
Marko Mäkelä authored
We are seeing !buf_pool.any_io_pending() assertion failures in srv_start() ever since MDEV-21452 in 10.6. But, the problem appears to be older. In 10.5 since MDEV-19514 removed writes from the precursor of buf_page_read_complete(), it seems that the debug assertion failure could have been harmless. recv_sys_t::apply(): At the end of each batch, wait not only for all log records to have been processed, but also for all pending reads to complete, so that the buffer pool will be in an idle state.
-
Sergei Golubchik authored
list all supported options in the comment. remove wsrep-specific hack of EXPORT_SYMBOLS, wsrep-specific hacks belong to wsrep
-
Sergei Golubchik authored
Reverts 10.5 commit 6033cc85 The fix a587ded2 will be merged from 10.2
-
Sergei Golubchik authored
-
Sergei Golubchik authored
-
Marko Mäkelä authored
In commit 0c23e32d (MDEV-24445) we forgot to keep m_freed_space in sync with m_freed_pages in one case.
-
- 18 Dec, 2020 3 commits
-
-
Marko Mäkelä authored
In the rewrite of MDEV-8139 (based on MDEV-15528), we introduced a wrong assumption that any persistent tablespace that is not an .ibd file is the system tablespace. This assumption is broken when innodb_undo_tablespaces (files undo001, undo002, ...) are being used. By default, we have innodb_undo_tablespaces=0 (the persistent undo log is being stored in the system tablespace). In MDEV-15528 and MDEV-8139 we rewrote the page scrubbing logic so that it will follow the tried-and-true write-ahead logging protocol, first writing FREE_PAGE records and then in the page flushing, zerofilling or hole-punching freed pages. Unfortunately, the implementation included a wrong assumption that that anything that is not in an .ibd file must be the system tablespace. This wrong assumption would cause overwrites of valid data pages in the system tablespace. mtr_t::m_freed_in_system_tablespace: Remove. mtr_t::m_freed_space: The tablespace associated with m_freed_pages. buf_page_free(): Take the tablespace and page number as a parameter, instead of taking a page identifier.
-
Marko Mäkelä authored
A race condition between deleting an .ibd file and fil_crypt_thread marking pages dirty was introduced in commit 118e258a (part of MDEV-23855). fil_space_t::acquire_if_not_stopped(): Correctly return false if the STOPPING flag is set, indicating that any further activity on the tablespace must be avoided. Also, remove the constant parameter have_mutex=true and move the function declaration to the same compilation unit with the only callers. fil_crypt_flush_space(): Remove an unused variable.
-
Marko Mäkelä authored
fil_crypt_find_space_to_rotate(): Always treat the sentinel value that indicates that we have run out of work, even if at the same time the thread should shut down due to other reasons. Thanks to Matthias Leich for reproducing this bug with RQG.
-
- 17 Dec, 2020 1 commit
-
-
Marko Mäkelä authored
After MDEV-15528, two modes of operation in the fil_crypt_thread remains, depending on whether innodb_encryption_rotate_key_age=0 (whether key rotation is disabled). If the key rotation is disabled, the fil_crypt_thread miss the opportunity to sleep, which will result in lots of wasted CPU usage. fil_crypt_return_iops(): Add a parameter to specify whether other fil_crypt_thread should be woken up. fil_system_t::keyrotate_next(): Return the special value fil_system.temp_space to indicate that no work is to be done. fil_space_t::next(): Propagage the special value fil_system.temp_space to the caller. fil_crypt_find_space_to_rotate(): If no work is to be done, do not wake up other threads.
-
- 15 Dec, 2020 12 commits
-
-
Etienne Guesnet authored
-
Etienne Guesnet authored
-
Etienne Guesnet authored
-
Etienne Guesnet authored
-
Etienne Guesnet authored
-
Etienne Guesnet authored
-
Etienne Guesnet authored
-
Etienne Guesnet authored
-
Etienne Guesnet authored
-
Etienne Guesnet authored
-
Etienne Guesnet authored
-
zhaorenhai authored
Move the S3 test case variables to suite.pm to use environment variables. Use minio credentials if a TCP connection to localhost:9000 is accepted so the current build works corrected. Reviewer: Daniel Black closes #1711
-
- 14 Dec, 2020 5 commits
-
-
Stepan Patryshev authored
-
Marko Mäkelä authored
-
Marko Mäkelä authored
-
Marko Mäkelä authored
In commit 5e62b6a5 (MDEV-16264) the logic of os_aio_init() was changed so that it will never fail, but instead automatically disable innodb_use_native_aio (which is enabled by default) if the io_setup() system call would fail due to resource limits being exceeded. This is questionable, especially because falling back to simulated AIO may lead to significantly reduced performance. srv_n_file_io_threads, srv_n_read_io_threads, srv_n_write_io_threads: Change the data type from ulong to uint. os_aio_init(): Remove the parameters, and actually return an error code. thread_pool::configure_aio(): Do not silently fall back to simulated AIO. Reviewed by: Vladislav Vaintroub
-
Marko Mäkelä authored
After commit a5a2ef07 (part of MDEV-23855) implemented asynchronous doublewrite, it is possible that the server will hang when the following parametes are in effect: innodb_doublewrite=1 (default) innodb_write_io_threads=1 innodb_use_native_aio=0 Note: In commit 5e62b6a5 (MDEV-16264) the logic of os_aio_init() was changed so that it will never fail, but instead automatically disable innodb_use_native_aio (which is enabled by default) if the io_setup() system call would fail due to resource limits being exceeded. Before commit a5a2ef07, we used a synchronous write for the doublewrite buffer batches, always at most 64 pages at a time. So, upon completing a doublewrite batch, a single thread would submit at most 64 page writes (for the individual pages that were first written to the doublewrite buffer). With that commit, we may submit up to 128 page writes at a time. The maximum number of outstanding requests per thread is 256. Because the maximum number of asynchronous write submissions per thread was roughly doubled, it is now possible that buf_dblwr_t::flush_buffered_writes_completed() will hang in io_slots::acquire(), called via os_aio() and fil_space_t::io(), when submitting writes of the individual blocks. We will prevent this type of hang by increasing the minimum number of innodb_write_io_threads from 1 to 2, so that this type of hang would only become possible when 512 outstanding write requests are exceeded.
-
- 11 Dec, 2020 1 commit
-
-
Marko Mäkelä authored
We observed a race condition that involved two threads executing fil_flush_file_spaces() and one thread executing fil_delete_tablespace(). After one of the fil_flush_file_spaces() observed that space.needs_flush_not_stopping() is set and was releasing the fil_system.mutex, the other fil_flush_file_spaces() would complete the execution of fil_space_t::flush_low() on the same tablespace. Then, fil_delete_tablespace() would destroy the object, because the value of fil_space_t::n_pending did not prevent that. Finally, the fil_flush_file_spaces() would resume execution and invoke fil_space_t::flush_low() on the freed object. This race condition was introduced in commit 118e258a of MDEV-23855. fil_space_t::flush(): Add a template parameter that indicates whether the caller is holding a reference to prevent the tablespace from being freed. buf_dblwr_t::flush_buffered_writes_completed(), row_quiesce_table_start(): Acquire a reference for the duration of the fil_space_t::flush_low() operation. It should be impossible for the object to be freed in these code paths, but we want to satisfy the debug assertions. fil_space_t::flush_low(): Do not increment or decrement the reference count, but instead assert that the caller is holding a reference. fil_space_extend_must_retry(), fil_flush_file_spaces(): Acquire a reference before releasing fil_system.mutex. This is what will fix the race condition.
-
- 09 Dec, 2020 2 commits
-
-
Marko Mäkelä authored
Since commit ea21d630 we conditionally define a variable that only plays a role on systems that support hole-punching (explicit creation of sparse files). However, that broke debug builds on such systems. It turns out that the debug_dbug label "ignore_punch_hole" is not at all used in MariaDB server. It would be covered by the MySQL 5.7 test innodb.table_compress. (Note: MariaDB 10.1 implemented page_compressed tables before something comparable appeared in MySQL 5.7.)
-
Marko Mäkelä authored
The flushing of the InnoDB temporary tablespace is unnecessarily tied to the write-ahead redo logging and redo log checkpoints, which must be tied to the page writes of persistent tablespaces. Let us simply omit any pages of temporary tables from buf_pool.flush_list. In this way, log checkpoints will never incur any 'collateral damage' of writing out unmodified changes for temporary tables. After this change, pages of the temporary tablespace can only be written out by buf_flush_lists(n_pages,0) as part of LRU eviction. Hopefully, most of the time, that code will never be executed, and instead, the temporary pages will be evicted by buf_release_freed_page() without ever being written back to the temporary tablespace file. This should improve the efficiency of the checkpoint flushing and the buf_flush_page_cleaner thread. Reviewed by: Vladislav Vaintroub
-
- 08 Dec, 2020 3 commits
-
-
Marko Mäkelä authored
-
Marko Mäkelä authored
MDEV-24278 improved the page cleaner so that it will no longer wake up once per second on an idle server. However, with innodb_adaptive_flushing (the default) the function page_cleaner_flush_pages_recommendation() could initially return 0 even if there is work to do. af_get_pct_for_dirty(): Remove. Based on a comment here, it appears that an initial intention of innodb_max_dirty_pages_pct_lwm=0.0 (the default value) was to disable something. That ceased to hold in MDEV-23855: the value is a pure threshold; the page cleaner will not perform any work unless the threshold is exceeded. page_cleaner_flush_pages_recommendation(): Add the parameter dirty_blocks to ensure that buf_pool.flush_list will eventually be emptied.
-
Sergei Petrunia authored
..causes error on slave. Cause: if the master doesn't have the frm file for the table, DROP TABLE code will call ha_delete_table_force() to drop the table in all available storage engines. The issue was that this code path didn't check for HTON_TABLE_MAY_NOT_EXIST_ON_SLAVE flag for the storage engine, and so did not add "... IF EXISTS" to the statement that's written to the binary log. This can cause error on the slave when it tries to drop a table that's already gone.
-