settings.

RocksDB will only build with libzstd support if libzstd version is
>=1.1.13. Unfortunately CMake's FindPackage claims it found version
1.1.13 when we have 1.1.12-1 installed, so a workaround with
CheckFunctionExists is used to properly check for correct libzstd
support.

            IN predicate defined with non-constant values is pushed down

The problem appears because of wrong changes made in MDEV-16090 in the
Item_func_in::build_clone() method.
For the clone of the IN predicate it copied 'cmp_fields' array values
that become dirty after Item::cleanup_excluding_const_fields_processor
has worked in pushdown. That causes crash.
There is no need to copy 'cmp_fields' field, the array values should be
NULLs in order to fix_fields() for the cloned IN predicate can set them
correctly. fix_fields() computes values for 'cmp_fields' array only
if they were not set earlier.

Added flush tables to ensure that MyISAM tables are
properly flushed before reboot

The bug was that innobase_get_computed_value() trashed record[0] and data
in Field_blob::value

Fixed by using a record on the heap for innobase_get_computed_value()

Reviewer: Marko Mäkelä

This is to mark that a field is indirectly part of a key, which simplifes
checking if we need to have this field up to date to evaluate a key.

For example:
CREATE TABLE t1 (a int, b int as (a) virtual,
                 c int as (b) virtual, index(c))
would mark a and b with PART_INDIRECT_KEY_FLAG.
c is marked with PART_KEY_FLAG as before.

The optimizer can create various item's over the original one,
    so we can't count on the exact item's type inside the comparison.

The following conditions will decide the query cache retrieval or
storing inside innodb:

    (1) There should not be any locks on the table.
    (2) Some other trx shouldn't invalidated the cache before the
    transaction started.
    (3) Read view shouldn't exist. If exists then the view
    low_limit_id should be greater than or equal to the transaction that
    invalidates the cache for the particular table.

 For read-only transaction: should satisfy  the above (1) and (3)
 For read-write transaction: should satisfy the above (1), (2), (3).

 - Changed the variable from query_cache_inv_id to query_cache_inv_trx_id.

 - Moved the function row_search_check_if_query_cache_permitted from
    row0sel.h and made it as static function in ha_innodb.cc

"Truncated incorrect DOUBLE value: 'true'".

JSON_VALUE_TRUE and JSON_VALUE_FALSE should be handled specifically
in Item_json_value.

srv_print_verbose_log: Introduce the value 2 to refer to
mariabackup --verbose.

recv_recover_page(), recv_parse_log_recs(): Add output for
mariabackup --verbose.

Commit dc9c5554 moved the final phase of
the redo log copying to the background thread. This would sometimes cause
too little redo log to be copied at the end of the backup. We would only
guarantee copying up to the latest redo log checkpoint. This would produce
a consistent backup, but it could refer to a too old point of time.

xtrabackup_copy_log(), xtrabackup_copy_logfile(): Add the parameter 'last'.

xtrabackup_backup_low(): Copy any remaining part of the log after the
backup threads have terminated.

MDEV-11071: Assertion `thd->transaction.stmt.is_empty()' failed in Locked_tables_list::unlock_locked_table

fix_length_and_dec now return result (error/OK)

When altering from DECIMAL to *INT UNIGNED or to BIT, go through val_decimal(),
to avoid truncation to the biggest possible signed integer
(0x7FFFFFFFFFFFFFFF / 9223372036854775807).

            materialized derived table/view that uses aliases is done

The problem appears when a column alias inside the materialized derived
table/view t1 definition coincides with the column name used in the
GROUP BY clause of t1. If the condition that can be pushed into t1
uses that ambiguous column name this column is determined as a column that
is used in the GROUP BY clause instead of the alias used in the projection
list of t1. That causes wrong result.
To prevent it resolve_ref_in_select_and_group() was changed.

Since MariaDB Server 10.2.2 (and MySQL 5.7), the default value of
innodb_checksum_algorithm is crc32 (CRC-32C), not the inefficient "innodb"
checksum. Change Mariabackup to use the same default, so that checksum
validation (when using the default algorithm on the server) will take less
time during mariabackup --backup. Also, mariabackup --prepare should be
a little faster, and the server should read backups faster, because the
page checksums would only be validated against CRC-32C.

fil_page_decompress(): Replaces fil_decompress_page().
Allow the caller detect errors. Remove
duplicated code. Use the "safe" instead of "fast" variants of
decompression routines.

fil_page_compress(): Replaces fil_compress_page().
The length of the input buffer always was srv_page_size (innodb_page_size).
Remove printouts, and remove the fil_space_t* parameter.

buf_tmp_buffer_t::reserved: Make private; the accessors acquire()
and release() will use atomic memory access.

buf_pool_reserve_tmp_slot(): Make static. Remove the second parameter.
Do not acquire any mutex. Remove the allocation of the buffers.

buf_tmp_reserve_crypt_buf(), buf_tmp_reserve_compression_buf():
Refactored away from buf_pool_reserve_tmp_slot().

buf_page_decrypt_after_read(): Make static, and simplify the logic.
Use the encryption buffer also for decompressing.

buf_page_io_complete(), buf_dblwr_process(): Check more failures.

fil_space_encrypt(): Simplify the debug checks.

fil_space_t::printed_compression_failure: Remove.

fil_get_compression_alg_name(): Remove.

fil_iterate(): Allocate a buffer for compression and decompression
only once, instead of allocating and freeing it for every page
that uses compression, during IMPORT TABLESPACE. Also, validate the
page checksum before decryption, and reduce the scope of some variables.

fil_page_is_index_page(), fil_page_is_lzo_compressed(): Remove (unused).

AbstractCallback::operator()(): Remove the parameter 'offset'.
The check for it in FetchIndexRootPages::operator() was basically
redundant and dead code since the previous refactoring.

Problem:
The problem was most likely introduced by a fix for MDEV-11597
(commit 5f0c31f9) which removed
the assignment "killed= KILL_BAD_DATA" from THD::raise_condition().

Before MDEV-11597, sp_head::execute() tested thd->killed after
looping through the SP instructions and exited with an error
if thd->killed is set. After MDEV-11597, sp_head::execute()
stopped to notice errors and set the OK status on top of the
error status, which crashed on assert.

Fix:
Making sp_cursor::fetch() return -1 if server_side_cursor->fetch(1)
left an error in the diagnostics area. This makes the statement
"err_status= i->execute(thd, &ip)" in sp_head::execute() set the
error code and correctly break the SP instruction loop and
return on error without setting the OK status.

Adjust the patch to match the variant accepted into the upstream:
undo the changes in ha_rocksdb::load_hidden_pk_value().

Added a testcase

This problem was earlier fixed by MDEV-14603.
Only adding 10.2 specific tests.

When attempting to rename a table to a non-existing database,
InnoDB would misleadingly report "OS error 71" when in fact the
error code is InnoDB's own (OS_FILE_NOT_FOUND), and not report
both pathnames. Errors on rename could occur due to reasons
connected to either pathname.

os_file_handle_rename_error(): New function, to report errors in
renaming files.

Add variable rocksdb_remove_mariabackup_checkpoint.
If set, it will remove $rocksdb_datadir/mariabackup-checkpoint directory.
The variable is to be used by exclusively by mariabackup,
to remove temporary checkpoints.

log_copying_thread(): Keep copying redo log until the end has been
reached. (Previously, we would stop copying as soon as
the first batch of xtrabackup_copy_logfile() returned.)

log_copying: Remove. Use log_copying_running instead.

copy_logfile: Remove. Log copying will now only be invoked from
2 places: from xtrabackup_backup_func() for the initial batch,
and from log_copying_thread() until all of the log has been read.
Use the global variable metadata_to_lsn for determining if the
final part of the log is being copied.

xtrabackup_copy_log(): Add diagnostic messages for terminating
the copying. These messages should be dead code, because
log_group_read_log_seg() should be checking for the same.

xtrabackup_copy_logfile(): Correct the retrying logic.
If anything was successfully read, process the portion that
was read. On failure, let the caller close dst_log_file.

io_watching_thread(): Stop throttling during the last phase
of copying the log (metadata_to_lsn!=0). The final copying
of the log will now be performed in log_copying_thread().

stop_backup_threads(): Clean up the message about stopping
the log copying thread.

xtrabackup_backup_low(): Read metadata_to_lsn from the latest
checkpoint header page, even if it is the first page.
Let the log_copying_thread take care of copying all of
the redo log.

InnoDB insisted on closing the file handle before renaming a file.
Renaming a file should never be a problem on POSIX systems. Also on
Windows it should work if the file was opened in FILE_SHARE_DELETE
mode.

fil_space_t::stop_ios: Remove. We no longer need to stop file access
during rename operations.

fil_mutex_enter_and_prepare_for_io(): Remove the wait for stop_ios.

fil_rename_tablespace(): Remove the retry logic; do not close the
file handle. Remove the unused fault injection that was added along
with the DATA DIRECTORY functionality (MySQL WL#5980).

os_file_create_simple_func(), os_file_create_func(),
os_file_create_simple_no_error_handling_func(): Include FILE_SHARE_DELETE
in the share_mode. (We will still prevent multiple InnoDB instances
from using the same files by not setting FILE_SHARE_WRITE.)

ha_innobase::optimize(): If both innodb_defragment and
innodb_optimize_fulltext_only are at their default settings (OFF),
fall back to ALTER TABLE. Else process one or both options.

ha_innobase::set_partition_owner_stats(): Remove (unused function).

ha_innobase::ha_partition_stats: Remove (the variable is never read).

Remove unused ut_timer functions.

After a failed ADD INDEX, dict_index_remove_from_cache_low()
could iterate the index fields and dereference a freed virtual
column object when trying to remove the index from the v_indexes
of the virtual column.

This regression was caused by a merge of
MDEV-16119 InnoDB lock->index refers to a freed object.

ha_innobase_inplace_ctx::clear_added_indexes(): Detach the
indexes of uncommitted indexes from virtual columns, so that
the iteration in dict_index_remove_from_cache_low() can be avoided.

ha_innobase::prepare_inplace_alter_table(): Ignore uncommitted
corrupted indexes when rejecting ALTER TABLE. (This minor bug was
revealed by the extension of the test case.)

dict_index_t::detach_columns(): Detach an index from virtual columns.
Invoked by both dict_index_remove_from_cache_low() and
ha_innobase_inplace_ctx::clear_added_indexes().

dict_col_t::detach(const dict_index_t& index): Detach an index from
a column.

dict_col_t::is_virtual(): Replaces dict_col_is_virtual().

dict_index_t::has_virtual(): Replaces dict_index_has_virtual().

log_crypt_101_read_block(): Mimic MariaDB 10.1, and use the first
encryption key if the key for the checkpoint cannot be found.
Redo log encryption key rotation was ultimately disabled
in MDEV-9422 (MariaDB 10.1.13) due to design issues. So, from
MariaDB 10.1.13 onwards only one log encryption key should matter.

recv_log_format_0_recover(): Add the parameter 'bool crypt'.
Indicate when the log cannot be decrypted for upgrade, instead of
making a possibly false claim that the log requires crash recovery.

init_crypt_key(): Remove extra space from a message.

Just spacing out the links on separate linkes, they were running together
in a block of text and could be a little hard for some people to
differentiate where one begins and the next ends. Seems silly, but just
trying to help in the formatting a bit.

This bug caused crashes for queries with unreferenced non-recursive
CTEs specified by unions.It happened because the function
st_select_lex_unit::prepare() tried to use the value of the field 'derived'
that could not be set for unferenced CTEs as there was no derived
table associated with an unreferenced CTE.