fil_space_t::acquire_low(): Introduce a parameter that specifies
which flags should be avoided. At all times, referenced() must not
be incremented if the STOPPING flag is set. When fil_system.mutex
is not being held by the current thread, the reference must not be
incremented if the CLOSING flag is set (unless NEEDS_FSYNC is set,
in fil_space_t::flush()).

fil_space_t::acquire(): Invoke acquire_low(STOPPING | CLOSING).
In this way, the reference count cannot be incremented after
fil_space_t::try_to_close() invoked fil_space_t::set_closing().

If the CLOSING flag was set, we must retry acquire_low() after
acquiring fil_system.mutex.

fil_space_t::prepare_acquired(): Replaces prepare(true).

fil_space_t::acquire_and_prepare(): Replaces prepare().
This basically retries fil_space_t::acquire() after
acquiring fil_system.mutex.

MDEV-28806 Assertion `flag == 1' failure in row_build_index_entry_low upon concurrent ALTER and UPDATE

- During online ADD INDEX, InnoDB was incorrectly writing log for an
UPDATE that does not affect the being-created index.

recv_sys_t::recover_deferred(): Hold the exclusive page latch until
the tablespace has been set up. Otherwise, the write of the page
may be lost due to non-existent tablespace. This race only affects
the recovery of the first page in a newly created tablespace.

This race condition was introduced in MDEV-24626.

Ever since commit 9608773f
the InnoDB persistent statistics are enabled on all InnoDB tables
by default. We must filter out any output that indicates that the
statistics tables are being internally accessed by InnoDB.

The issue was that flush_tables() didn't take a MDL lock on cached
TABLE_SHARE before calling open_table() to do a HA_EXTRA_FLUSH call.
Most engines seams to have no issue with it, but apparantly this conflicts
with InnoDB in 10.6 when using TRUNCATE

Fixed by taking a MDL lock before trying to open the table in
flush_tables().

There is no test case as it hard to repeat the scheduling that causes
the error. I did run the test case in MDEV-28897 to verify
that the bug is fixed.

This test failure is similar to encryption.innodb-redo-badkey,
which was fixed in commit 0f0a45b2.

The heap-use-after-free is caused by the following mechanism:

  * In the execution of FLUSH TABLE WITH READ LOCK, the function
    spider_free_trx_conn() is called and the connections held by
    SPIDER_TRX::trx_conn_hash are freed.

  * Then, an instance of ha_spider maintains the freed connections
    because they are also referenced from ha_spider::conns.
    The ha_spider instance is kept in a lock structure until the
    corresponding table is unlocked.

  * Spider accesses ha_spider::conns on the implicit UNLOCK TABLE
    issued by BEGIN.

In the first place, when the connections have been freed, it means
that there are really no remote table locked by Spider.
Thus, there is no need for Spider to access ha_spider::cons on the
implicit UNLOCK TABLE.

We can fix the bug by removing the above mentioned access to
ha_spider::conns. We also modified spider_free_trx_conn() so that it
frees the connections only when no table is locked to reduce the
chance of another heap-use-after-free on ha_spider::conns.

i_s_dict_fill_sys_tablestats(): Read all fields of dict_table_t
while holding dict_sys.latch.

dict_sys_t::allow_eviction(): Remove.

prepare_inplace_alter_table_dict(): If the table will not be rebuilt,
preserve all of the original ROW_FORMAT, including the compressed
page size flags related to ROW_FORMAT=COMPRESSED.

hex_to_ascii(): Add #if around the definition to avoid
clang -Wunused-function. Avoid GCC 5 -Wconversion with a cast.

btr_root_raise_and_insert(), btr_lift_page_up(),
rtr_page_split_and_insert(): Reset DB_FAIL from a failure to
copy records on a ROW_FORMAT=COMPRESSED page to DB_SUCCESS
before retrying.

This fixes a regression that was introduced by
commit 0b47c126 (MDEV-13542).

btr_root_raise_and_insert(): Remove a redundant condition.
btr_page_split_and_insert() will invoke btr_page_split_and_insert()
if needed.

On FreeBSD, tests run on persistent storage, and no asynchronous I/O
has been implemented. Warnings about 205-second waits on dict_sys.latch
may occur.

buf_page_print(): Dump the buffer page 32 bytes (64 hexadecimal digits)
per line. In this way, the limitation in mtr
("Data too long for column 'line'") will not be triggered.

Also, do not bother decoding the page contents, because everything
is present in the hexadecimal output.

dict_index_find_on_id_low(): Merge to dict_index_get_if_in_cache_low().
The direct call in buf_page_print() was prone to crashing, in case the
table definition was concurrently evicted or dropped from the
data dictionary cache.

Spider supports (or at least allows) INSERT DELAYED but the
documentation does not specify spider as a storage engine that supports
"INSERT DELAYED".
Also, although not mentioned in the documentation, "INSERT DELAYED" is
not intended to be executed inside a transaction, as can be seen from
the list of supported storage engines.
The current implementation allows executing a delayed insert on a
remote transactional table and this breaks the consistency ensured by
the transaction.

We too remove "internal_delayed", one of the Spider table parameters.
Documentation says,

> Whether to transmit existence of delay to remote servers when
> executing an INSERT DELAYED statement on local server.

This table parameter is only used for "INSERT DELAYED".

Reviewed by: Nayuta Yanagisawa

MDEV-22590 SIGSEGV in flush_all_key_blocks when changing key_buffer_size / ASAN: heap-use-after-free in flush_all_key_blocks

Take into account that in preparation of a simple key cache for resizing no disk blocks might be assigned to it.

Reviewer: IgorBabaev <igor@mariadb.com>

Revert "TSAN: data race on vptr (ctor/dtor vs virtual call)"

This reverts commit 78084fa7.

This commit was done to please TSAN, which falsely reported an error
where there was none.
Yet as consequence, it could cause a real error, a crash in os_aio_free on
shutdown

fil_name_process(): If the recovery of a tablespace was deferred,
do invoke fil_ibd_load() even though the name in recv_spaces is
not changing. This allows us to recover from a situation where
there are many FILE_RENAME records, renaming a tablespace back
and forth, and a FILE_MODIFY record that had been written by
fil_names_clear().

Co-developed with: Thirunarayanan Balathandayuthapani

Fixed tpool timer implementation on POSIX.
Prior to this patch, under some specific rare circumstances (concurrency
related), timer callback execution might be skipped.

When attempting to recover a database with an incorrect encryption key,
the unencrypted page contents should be expected to differ from what
was written before recovery. Let us suppress some more messages.
This caused intermittent failures, depending on when the latest
log checkpoint was triggered.

trx_undo_rec_copy(): Return nullptr if the undo record is corrupted.

trx_undo_rec_get_undo_no(): Define inline with the declaration.

trx_purge_dummy_rec: Replaced with a -1 pointer.

row_undo_rec_get(), UndorecApplier::apply_undo_rec(): Check
if trx_undo_rec_copy() returned nullptr.

trx_purge_get_next_rec(): Return nullptr upon encountering any
corruption, to signal the end of purge.

On GNU/Linux, even though the C11 aligned_alloc() appeared in
GNU libc early on, some custom memory allocators did not
implement it until recently. For example, before
gperftools/gperftools@d406f2285390c402e824dd28e6992f7f890dcdf9
the free() in tcmalloc would fail to free memory that was
returned by aligned_alloc(), because the latter would map to the
built-in allocator of libc. The Linux specific memalign() has a
similar interface and is safer to use, because it has been
available for a longer time. For AddressSanitizer, we will use
aligned_alloc() so that the constraint on size can be enforced.

buf_tmp_reserve_compression_buf(): When HAVE_ALIGNED_ALLOC holds,
round up the size to be an integer multiple of the alignment.

pfs_malloc(): In the unit test stub, round up the size to be an
integer multiple of the alignment.

Table_cache_instance: Define the structure aligned at
the CPU cache line, and remove a pad[] data member.
Krunal Bauskar reported this to improve performance on ARMv8.

aligned_malloc(): Wrapper for the Microsoft _aligned_malloc()
and the ISO/IEC 9899:2011 <stdlib.h> aligned_alloc().
Note: The parameters are in the Microsoft order (size, alignment),
opposite of aligned_alloc(alignment, size).
Note: The standard defines that size must be an integer multiple
of alignment. It is enforced by AddressSanitizer but not by GNU libc
on Linux.

aligned_free(): Wrapper for the Microsoft _aligned_free() and
the standard free().

HAVE_ALIGNED_ALLOC: A new test. Unfortunately, support for
aligned_alloc() may still be missing on some platforms.
We will fall back to posix_memalign() for those cases.

HAVE_MEMALIGN: Remove, along with any use of the nonstandard memalign().

PFS_ALIGNEMENT (sic): Removed; we will use CPU_LEVEL1_DCACHE_LINESIZE.

PFS_ALIGNED: Defined using the C++11 keyword alignas.

buf_pool_t::page_hash_table::create(),
lock_sys_t::hash_table::create():
lock_sys_t::hash_table::resize(): Pad the allocation size to an
integer multiple of the alignment.

Reviewed by: Vladislav Vaintroub

There was a race condition between log_checkpoint_low() and
deleting or renaming data files. The scenario is as follows:

1. The buffer pool does not contain dirty pages.
2. A FILE_DELETE or FILE_RENAME record is written.
3. The checkpoint LSN will be moved ahead of the write of the record.
4. The server is killed before the file is actually renamed or deleted.

We will prevent this race condition by ensuring that a log checkpoint
cannot occur between the durable write and the file system operation:

1. Durably write the FILE_DELETE or FILE_RENAME record.
2. Perform the file system operation.
3. Allow any log checkpoint to proceed.

mtr_t::commit_file(): Implement the DELETE or RENAME logic.

fil_delete_tablespace(): Delegate some of the logic to
mtr_t::commit_file().

fil_space_t::rename(): Delegate some logic to mtr_t::commit_file().
Remove the debug injection point fil_rename_tablespace_failure_2
because we do test RENAME failures without any debug injection.

fil_name_write_rename_low(), fil_name_write_rename(): Remove.

Tested by Matthias Leich

This commit contains workaround for a bug known as 'Red Hat issue 1870279'
(connection reset by peer issue in socat versions 1.7.3.3 to 1.7.4.0) which
further causes crashes during SST using mariabackup (when openssl is used).

Also fixed broken logic of automatic generation of the Diffie-Hellman parameters
for socat version less than 1.7.3 (which defaults to 512-bit values instead of
2048-bit ones).

buf_page_create_low(): Before retrying, release the exclusive page latch
in order to prevent an infinite loop in buf_pool_t::corrupted_evict().

Disable static build for encryption plugin file_key_management