Handling of write sets, which fail in certification happens differently than
with write sets which pass certification. When certification fails, the
write set applying can be skipped and applier needs only to take care of
wsrep XID checkpointing. With current implementation, this can rush ahead
of wsrep XID checkpointing of successful write sets.

The fix in this PR registers wsrep XID checkpointing of certification failure cases in group commit,
which guarantees that XID ceckpointing order is synchronized with real committing transactions.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

If log_slave_updates==OFF, wsrep applier threads used to be configured
with option: thd->variables.option_bits&= ~(OPTION_BIN_LOG);
(i.e. like sql_log_bin=ON). And this was regardless of log-bin configuration.

With this, having configuration of: --log-bin && --log-slave-updates=OFF,
local threads used binlogging, but applier threads did not. And further:
local threads went through binlog group commit, while applier threads did
direct commits. This resulted in situation, where applier threads entered
earlier in wsrep XID checkpointing, and could sync their wsrep XID out of order.
Later local thread commit would see that higher seqno was already checkpointed,
and fire an assert because of this.

As a fix, applier threads are now forced to enable binlogging regardless of
log-slave-updates configuration.

This PR comes with new mtr test: galera.MDEV-24327, which causes a scenario
where applier transaction is applied and committed while earlier local transaction
is parked before commit order monitor enter. A buggy mariadb versoin would fail
for assertion because of wsrep XID checkpoint order violation.

The main goal of this patch is to prevent MariaDB's native_password_plugin
from "parsing" the hex (or non hex) authentication_string. Due to how the
current code is written, we convert any string (within native_password_get_salt)
that has the appropriate length to a "binary" representation, that can
potentially match a real password.

More specifically,
"*THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE" produces the same results as
"*d13c3c78dafa52d9bce09bdd1adcb7befced1ebe".

The length indicator is the main indicator of an invalid password. We use
use same trick with "invalid" to change its internal representation.

The "parsing" mentioned is by get_salt_from_password down to char_val()
and because if where it is, its effectively a static plugin API that cannot
change.

In supporting these, we support the SHOW CREATE USER from MySQL may have the
hashed password string: *THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE.

Obviously this isn't a hash because it contains non-hex characters.

After this patch we do however recognise the pattern;

 [any char, notionally *]{40 chars not all are hex}

as a pattern for an invalid password. This was determined to be the general
pattern that MySQL used.

Reviewers: Sergei G, Vicentiu

following the same masquerading logic

Add the new file

Move the testcase into a separate file: embedded server
doesn't have optimizer trace.

Basic variant of the fix: do not consider conditions in form

  unique_key NOT IN (c1,c2...)

to be sargable. If there are only a few constants, the condition
is not selective. If there are a lot constants, the overhead of
processing such a huge range list is not worth it.

- There is no reason to collect EITS statistics
- The test is sporadically failing on some platforms. I believe the
  issue is in InnoDB. Let's rule out EITS code as a possible source
  of the issue.

MDEV-19716: ASAN use-after-poison in Query_log_event::Query_log_event / THD::log_events_and_free_tmp_shares

Post push fix to address test failure.

Problem:
=======
rpl.rpl_drop_temp_table_invaid_lex added as part bug fix has occasional
failures in build bot.

MTR's internal check of the test case
'rpl.rpl_drop_temp_table_invaid_lex' failed.

 Variable_name    Value
-Slave_open_temp_tables    0
+Slave_open_temp_tables    1

Analysis:
=========
The reason for the failure is that the DROP TEMPORARY TABLE command which
gets generated on connection disconnect might not have reached the slave
and hence the temp table remains on the slave.

Fix:
===
On master, upon disconnect, wait till connection is completely gone.  Then
ensure that DROP TEMPORARY table statement is available in the binary log.
Sync the slave with master and check that temporary table count is zero on
slave. Fixed a typo in test name.

Atomic_relaxed<T>: add fetch_or() and fetch_and()

innodb_init(): rely on a zero-initialization of a global variable

monitor_set_tbl: make Atomic_relaxed<ulint> array and use proper operations
for setting bit, unsetting bit and reading bit

Reviewed by: Marko Mäkelä

os_n_file_reads: make Atomic_counter and correct the semantics of an imprecise
counter.

Reviewed by: Marko Mäkelä

The Galera tests were massively failing with debug assertions.

Post-push Windows compilation errors fix.

Change thd->mdl_context.release_transactional_locks() to
thd->mdl_release_transactional_locks()

row_undo_ins_parse_undo_rec(): Do not try to read non-existing
virtual column information for the metadata record.

The new option --log-innodb-page-corruption is introduced.

When this option is set, backup is not interrupted if innodb corrupted
page is detected. Instead it logs all found corrupted pages in
innodb_corrupted_pages file in backup directory and finishes with error.

For incremental backup corrupted pages are also copied to .delta file,
because we can't do LSN check for such pages during backup,
innodb_corrupted_pages will also be created in incremental backup
directory.

During --prepare, corrupted pages list is read from the file just after
redo log is applied, and each page from the list is checked if it is allocated
in it's tablespace or not. If it is not allocated, then it is zeroed out,
flushed to the tablespace and removed from the list. If all pages are removed
from the list, then --prepare is finished successfully and
innodb_corrupted_pages file is removed from backup directory. Otherwise
--prepare is finished with error message and innodb_corrupted_pages contains
the list of the pages, which are detected as corrupted during backup, and are
allocated in their tablespaces, what means backup directory contains corrupted
innodb pages, and backup can not be considered as consistent.

For incremental --prepare corrupted pages from .delta files are applied
to the base backup, innodb_corrupted_pages is read from both base in
incremental directories, and the same action is proceded for corrupted
pages list as for full --prepare. innodb_corrupted_pages file is
modified or removed only in base directory.

If DDL happens during backup, it is also processed at the end of backup
to have correct tablespace names in innodb_corrupted_pages.

The reason for the failure is that
thd->mdl_context.release_transactional_locks()
was called after commit & rollback even in cases where the current
transaction is still active.

For 10.2, 10.3 and 10.4 the fix is simple:
- Replace all calls to thd->mdl_context.release_transactional_locks() with
  thd->release_transactional_locks(). The thd function will only call
  the mdl_context function if there are no active transactional locks.
  In 10.6 we will better fix where we will change the return value for
  some trans_xxx() functions to indicate if transaction did close the
  transaction or not. This will avoid the need of the indirect call.

Other things:
- trans_xa_commit() and trans_xa_rollback() will automatically
  call release_transactional_locks() if the transaction is closed.
- We can't do that for the other functions as the caller of many of these
  are doing additional work (like close_thread_tables) before calling
  release_transactional_locks().
- Added missing abort_result_set() and missing DBUG_RETURN in
  select_create::send_eof()
- Fixed wrong indentation in injector::transaction::commit()

The real fix for MDEV-15532 will be pushed into 10.2 and 10.6
This is an additional fix for 10.4.

In 10.4 trans_xa_detach was introduced.  However THD::cleanup() assumes
that after trans_xa_detach() is done, there is no registered transactions
anymore. In the 10.2 patch there will be an assert to ensure this, which
will cause 10.4 to fail.

The fix used is to reset the transaction flags in trans_xa_detach().

MDEV-21265: IN predicate conversion to IN subquery should be allowed for a broader set of datatype comparison

Allow materialization strategy when collations on the
inner and outer sides of an IN subquery are the same and the
character set of the inner side is a proper subset of the character
set on the outer side.
This allows conversion from utf8mb3 to utf8mb4
as the former is a subset of the later.
This is only allowed when IN predicate is converted to an IN subquery

Backported part of the patch (d6a00d9b) of MDEV-17905.

Reviewed by:serg@mariadb.com

disable thd->count_cuted_fields when populating internal temporary
tables for I_S, because this is how SELECT works standalone.
And if the SELECT is a part of INSERT or UPDATE or RETURN or SET or
anything else that enables thd->count_cuted_fields, this counting should
only apply when storing the result of the SELECT in a field or a
variable, not when populating internal temporary tables for I_S.

This is a fixup patch for MDEV-23991 afc9d00c

We really should read result.n_leaf_pages, which was set previously.

Analysis and fix was provided by Jukka Santala. Thanks!

Reviewed by: Marko Mäkelä

The nonnull attribute is not applicable to parameters that are
passed by reference, at least not in the Intel compiler.
Let us remove the reference indirection, which was only there
so that the pointer could be assigned to NULL, and let the
callers perform that task.

row_log_allocate(): Fix a bug in out-of-memory error handling
that would leave a pointer to freed memory.

Prior to this patch, it is possible to access freed memory
(THD::event_scheduler) from tp_post_kill_notification().

With this patch, memory is freed only when THD is no more accessible
from other threads, i.e after it is removed from the thread_list.

The idea of this fix is that it's enough to prevent the
next_auto_inc_val from incrementing if an error, to fix this problem
and also the MDEV-17333.
So this patch basically reverts the existing fix to the MDEV-17333.

Problem:
========
When O_TMPFILE is not supported mysqlbinlog outputs the error to standard
stream as a warning which breaks PITR:

ERROR 1064 (42000) at line 382: You have an error in your SQL syntax; check
the manual that corresponds to your MariaDB server version for the right
syntax to use near 'mysqlbinlog: O_TMPFILE is not supported on /tmp (disabling
    future attempts)

Analysis:
=========
'mysqlbinlog' utility is used to perform point-in-time-recovery based on binary
log. It converts the events in the binary log files, from binary format to text
so that they can be viewed or applied. This output can be saved to a file and
it can be sourced back to mysql client. The mysqlbinlog utility stores the
text output into IO_CACHE and when it is full the data is written to a temp
file.  The temporary file creation is attempted using 'O_TMPFILE' flag. If the
underlying filesystem doesn't support this operation, a note is printed on to
standard error and file creation is done without O_TMPFILE' flag. If standard
error is redirected to standard output the note gets written to the sql file
as shown below.

/bld/client/mysqlbinlog: O_TMPFILE is not supported on /tmp (disabling future
    attempts)
table id 32

When the sql file is used for PITR, it leads to a syntax error as it is not a
valid sql command.

Fix:
====
Make 'my_message_stderr' to ignore messages which are flagged as ME_NOTE and
ME_ERROR_LOG_ONLY. ME_ERROR_LOG_ONLY flag is applicable to server. In order to
print an informational note to stderr stream, ME_NOTE flag without
ME_ERROR_LOG_ONLY flag should be specified. 'my_message_stderr' should print
messages flagged with ME_WARNING or ME_FATAL to stderr stream.

In galera_3nodes.galera_safe_to_bootstrap node restart can happen too soon, when earlier SST joiner process is still active in the node.
Similar issue may hurt other mtr tests as well.

This is second variant of fix for this issue. Here we only change rsync SST script to wait a little bit if lingering SST rsync is observed to be in execution.
We assume that the previous mysqld and SST processes have been already signaled to abort during earlier stataup attempt.

If other SST methods (than rsync) suffer from similar overlapping SST execution, they should be sorted out separately within each SST method handler scripts.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

Add test case

A bogus error message was issued when a condition was pushed into a
materialized derived table or view specified as union of selects with
aggregation when the corresponding columns of the selects had different
names. This happened because the expression pushed into having clauses of
the selects was adjusted for the names of the first select of the union.
The easiest solution was to rename the columns of the other selects to be
name compatible with the columns of the first select.

Approved by Oleksandr Byelkin <sanja@mariadb.com>

Removed
* lp1376747-4
* MDEV-16509
* galera_defaults

Make sure that we operate with correct Galera library version
and do not print wsrep_provider_options field.

Test galera_parallel_apply_3nodes started to failed occasionally.
The test assumes that one round of autocommit retry is sufficient in
order to avoid a deadlock error when two conflicting UPDATE statements
run concurrently.
This assumption no longer holds after galera library has changed
last_committed() to return the seqno of the last transaction that left
apply monitor, rather than commit monitor. So it is possible that
after a BF abort, a command is re-executed before it's BF abortee has
left the apply monitor. Thus causing another retry or a deadlock error.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>