MDEV-12926 encryption.innodb_onlinealter_encryption, encryption.innodb-bad-key-change failed in buildbot with valgrind

row_merge_write(): Pass the correct (possibly encrypted) buffer
to os_file_write_int_fd().

This bug was introduced in commit 65e1399e
which included a commit to merge changes from MySQL 5.6.36 to
MariaDB Server 10.0.

btr_defragment_thread(): Create the thread in the same place as other
threads. Do not invoke btr_defragment_shutdown(), because
row_drop_tables_for_mysql_in_background() in the master thread can still
keep invoking btr_defragment_remove_table().

logs_empty_and_mark_files_at_shutdown(): Wait for btr_defragment_thread()
to exit.

innobase_start_or_create_for_mysql(), innobase_shutdown_for_mysql():
Skip encryption and scrubbing in innodb_read_only_mode.

srv_export_innodb_status(): Do not export encryption or scrubbing
statistics in innodb_read_only mode, because the threads will not
be running.

InnoDB shutdown assumes that once the server has entered
SRV_SHUTDOWN_FLUSH_PHASE, no change to persistent data is allowed.
It was possible for the master thread to wake up while shutdown
is executing in SRV_SHUTDOWN_FLUSH_PHASE or
even in SRV_SHUTDOWN_LAST_PHASE.

We do not yet know if further crashes at shutdown are possible.
Also, we do not know if all the observed crashes could be explained
by the race conditions that we are now fixing.

srv_shutdown_print_master_pending(): Remove a redundant ut_time() call.

srv_shutdown(): Renamed from srv_master_do_shutdown_tasks().

srv_master_thread(): Do not resume after shutdown has been initiated.

Don't rebuild the table for ALTER TABLE delay_key_write changes.

After that, delay_key_write value in .frm may differ from the
value in .MYI. We'll do what .frm says.

Test fails, because of a bug: global delay_key_write is
cached in the THD and the cached value becomes out-of-date
when the global value changes.

This is fixed in MDEV-11335

cast pointer(64)->long(32)

Make `mysqladmin --local` use `FLUSH LOCAL` for all flush-* commands,
and only do `SET SQL_LOG_BIN=OFF` for create/drop/old_password/password.

Additionally, --local is ignored for all commands that never write
to binlog, so e.g. `mysqladmin --local version` no longer needs SUPER

MDEV-11650 plugins.cracklib_password_check, plugins.two_password_validations fail in buildbot with valgrind (Conditional jump or move depends on uninitialised value)

FederatedX wasn't discovering prefix keys correctly.
Of course, as it had the HA_NO_PREFIX_CHAR_KEYS table_flag set...

fixed valgrind warning in a debug output

Before MDEV-6812, it did not matter that merge_files[].offset was
uninitialized when no files were created.

This problem was introduced in MDEV-6812. There could be a user-visible
impact that the progress reports spit into the error log are bogus.

row_merge_build_indexes(): Initialize merge_files[i].offset.

As documented in MSDN, DeviceIoControl() needs valid (not NULL) OVERLAPPED
parameter, for files  opened with for OVERLAPPED access.

Additional tests have been done and conclusion was to update maturity to stable

due to different packaging issues.

Also, Percona thinks that tar support has  many limitations
and should be removed as well( see discussion  in
https://bugs.launchpad.net/percona-xtrabackup/+bug/1681721)

there is an alternative streaming format xbstream that is supported and
does not have these limitations.

Snappy compression method require that output buffer
used for compression is bigger than input buffer.
Similarly lzo require additional work memory buffer.
Increase the allocated buffer accordingly.

buf_tmp_buffer_t: removed unnecessary lzo_mem, crypt_buf_free and
comp_buf_free.

buf_pool_reserve_tmp_slot: use alligned_alloc and if snappy
available allocate size based on snappy_max_compressed_length and
if lzo is available increase buffer by LZO1X_1_15_MEM_COMPRESS.

fil_compress_page: Remove unneeded lzo mem (we use same buffer)
and if output buffer is not yet allocated allocate based similarly
as above.

Decompression does not require additional work area.

    Modify test to use same test as other compression method tests.

Changed global parameters will be reset by server restart.
There is no need to save and restore the original values.

In commit 360a4a03
some debug assertions were introduced to the page flushing code
in XtraDB. Add these assertions to InnoDB as well, and adjust
the InnoDB shutdown so that these assertions will not fail.

logs_empty_and_mark_files_at_shutdown(): Advance
srv_shutdown_state from the first phase SRV_SHUTDOWN_CLEANUP
only after no page-dirtying activity is possible
(well, except by srv_master_do_shutdown_tasks(), which will be
fixed separately in MDEV-12052).

rotate_thread_t::should_shutdown(): Already exit the key rotation
threads at the first phase of shutdown (SRV_SHUTDOWN_CLEANUP).

page_cleaner_sleep_if_needed(): Do not sleep during shutdown.
This change is originally from XtraDB.

Significantly reduce the amount of InnoDB, XtraDB and Mariabackup
code changes by defining pfs_os_file_t as something that is
transparently compatible with os_file_t.

These were reported when running tests after compiling the debug server
with -O3 -fsanitize=undefined in GCC 7.1.0.

log_crypt(): Do not cast byte* to uint32_t*, because it may break
strict aliasing rules. Instead, cast in the opposite direction.

buf_read_ahead_linear(): Do not display a message if the tablespace
is being deleted.

dtype_print(): Add a missing break statement.

In my merge of the MySQL fix for Oracle Bug#23333990 / WL#9513
I overlooked some subsequent revisions to the test, and I also
failed to notice that the test is actually always failing.

Oracle introduced the parameter innodb_stats_include_delete_marked
but failed to consistently take it into account in FOREIGN KEY
constraints that involve CASCADE or SET NULL.

When innodb_stats_include_delete_marked=ON, obviously the purge of
delete-marked records should update the statistics as well.

One more omission was that statistics were never updated on ROLLBACK.
We are fixing that as well, properly taking into account the
parameter innodb_stats_include_delete_marked.

dict_stats_analyze_index_level(): Simplify an expression.
(Using the ternary operator with a constant operand is unnecessary
obfuscation.)

page_scan_method_t: Revert the change done by Oracle. Instead,
examine srv_stats_include_delete_marked directly where it is needed.

dict_stats_update_if_needed(): Renamed from
row_update_statistics_if_needed().

row_update_for_mysql_using_upd_graph(): Assert that the table statistics
are initialized, as guaranteed by ha_innobase::open(). Update the
statistics in a consistent way, both for FOREIGN KEY triggers and
for the main table. If FOREIGN KEY constraints exist, do not dereference
a freed pointer, but cache the proper value of node->is_delete so that
it matches prebuilt->table.

row_purge_record_func(): Update statistics if
innodb_stats_include_delete_marked=ON.

row_undo_ins(): Update statistics (on ROLLBACK of a fresh INSERT).
This is independent of the parameter; the record is not delete-marked.

row_undo_mod(): Update statistics on the ROLLBACK of updating key columns,
or (if innodb_stats_include_delete_marked=OFF) updating delete-marks.

innodb.innodb_stats_persistent: Renamed and extended from
innodb.innodb_stats_del_mark. Reduced the unnecessarily large dataset
from 262,144 to 32 rows. Test both values of the configuration
parameter innodb_stats_include_delete_marked.
Test that purge is updating the statistics.

innodb_fts.innodb_fts_multiple_index: Adjust the result. The test
is performing a ROLLBACK of an INSERT, which now affects the statistics.

include/wait_all_purged.inc: Moved from innodb.innodb_truncate_debug
to its own file.