XID_STATE::rm_error is never used by internal 2PC, it is intended to be
used by explicit XA only.

Also removed redundant xid reset from THD::init_for_queries(). Must've
been done already either by THD::transaction constructor or by
THD::cleanup().

Part of MDEV-7974 - backport fix for mysql bug#12161 (XA and binlog)

Part of MDEV-7974 - backport fix for mysql bug#12161 (XA and binlog)

    This bug is caused by pushdown from HAVING into WHERE.
    It appears because condition that is pushed wasn't fixed.

    It is also discovered that condition pushdown from HAVING into
    WHERE is done wrong. There is no need to build clones for some
    conditions that can be pushed. They can be simply moved from HAVING
    into WHERE without cloning.
    build_pushable_cond_for_having_pushdown(),
    remove_pushed_top_conjuncts_for_having() methods are changed.

    It is found that there is no transformation made for fields of
    pushed condition.
    field_transformer_for_having_pushdown transformer is added.

    New tests are added. Some comments are changed.

Do not register intermediate tables created by inplace ALTER TABLE in
THD::temporary_tables.

Regular ALTER TABLE doesn't create .frm for temporary and discoverable
tables anymore. For inplace ALTER TABLE moved .frm creation to
create_table_for_inplace_alter().

Removed open_in_engine argument of create_and_open_tmp_table() and
open_temporary_table(): it became unused after this patch.

Part of MDEV-17805 - Remove InnoDB cache for temporary tables.

CREATE TEMPORARY TABLE locks SE plugin 6 times. 5 of these locks are
released by the end of the statement. And only 1 acquired by
init_from_binary_frm_image() / plugin_lock() remains.

The lock removed in this patch was clearly redundant.

Part of MDEV-17805 - Remove InnoDB cache for temporary tables.

Closes #1225

This patch contains a fix for the MDEV-17262/17243 issues and
new mtr test.

These issues (MDEV-17262/17243) have two reasons:

1) After an intermediate commit, a transaction loses its status
of "transaction that registered in the MySQL for 2pc coordinator"
(in the InnoDB) due to the fact that since version 10.2 the
write_row() function (which located in the ha_innodb.cc) does
not call trx_register_for_2pc(m_prebuilt->trx) during the processing
of split transactions. It is necessary to restore this call inside
the write_row() when an intermediate commit was made (for a split
transaction).

Similarly, we need to set the flag of the started transaction
(m_prebuilt->sql_stat_start) after intermediate commit.

The table->file->extra(HA_EXTRA_FAKE_START_STMT) called from the
wsrep_load_data_split() function (which located in sql_load.cc)
will also do this, but it will be too late. As a result, the call
to the wsrep_append_keys() function from the InnoDB engine may be
lost or function may be called with invalid transaction identifier.

2) If a transaction with the LOAD DATA statement is divided into
logical mini-transactions (of the 10K rows) and binlog is rotated,
then in rare cases due to the wsrep handler re-registration at the
boundary of the split, the last portion of data may be lost. Since
splitting of the LOAD DATA into mini-transactions is technical,
I believe that we should not allow these mini-transactions to fall
into separate binlogs. Therefore, it is necessary to prohibit the
rotation of binlog in the middle of processing LOAD DATA statement.

https://jira.mariadb.org/browse/MDEV-17262 and
https://jira.mariadb.org/browse/MDEV-17243

* MDEV-16509 Improve wsrep commit performance with binlog disabled

Release commit order critical section early after trx_commit_low() if
binlog is not transaction coordinator. In order to avoid two phase commit,
binlog_hton is not registered for THD during IO_CACHE population.

Implemented a test which verifies that the transactions release
commit order early.

This optimization will change behavior during recovery as the commit
is not two phase when binlog is off. Fixed and recorded wsrep-recover-v25
and wsrep-recover to match the behavior.

* MDEV-18730 Ordering for wsrep binlog group commit

Previously out of order execution was allowed for wsrep commits.
Established proper ordering by populating wait_for_commit
for every wsrep THD and making group commit leader to wait for
prior commits before proceeding to trx_group_commit_leader().

* MDEV-18730 Added a test case to verify correct commit ordering

* MDEV-16509, MDEV-18730 Review fixes

Use WSREP_EMULATE_BINLOG() macro to decide if the binlog_hton
should be registered. Whitespace/syntax fixes and cleanups.

* MDEV-16509 Require binlog for galera_var_innodb_disallow_writes test

If the commit to InnoDB is done in one phase, the native InnoDB behavior
is that the transaction is committed in memory before it is persisted to
disk. This means that the innodb_disallow_writes=ON may not prevent
transaction to become visible to other readers before commit is completely
over. On the other hand, if the commit is two phase (as it is with binlog),
the transaction will be blocked in prepare phase.

Fixed the test to use binlog, which enforces two phase commit, which
in turn makes commit to block before the changes become visible to
other connections. This guarantees that the test produces expected
result.

The patches features an optional shutdown behavior to hold on until
after all connected slaves have been sent the last binlogged event.
The connected slave is one whose START SLAVE has been acknowledged and
that was not stopped since that though it could be technically
reconnecting in background.

The solution therefore disallows killing the dump thread until is has
found EOF of the latest binlog file.  It is up to the shutdown
requester (DBA) to set up a sufficiently large shutdown timeout value
for shudown to wait patiently until lagging behind slaves have been
synchronized. On the other hand if a specific slave needs exclusion
from synchronization the DBA would have to stop it manually which
would terminate its dump thread.

`mysqladmin shutdown' is extended with a `--wait_for_all_slaves' option
which translates to `SHUTDOW WAIT FOR ALL SLAVES' sql query
to enable the feature on the client side.

The patch also performs a small refactoring of the server shutdown
around close_connections() to introduce kill thread phases which
are two as of current.

Make mysqltest to use --ps-protocol more

use prepared statements for everything that server supports
with the exception of CALL (for now).

Fix discovered test failures and bugs.

tests:
* PROCESSLIST shows Execute state, not Query
* SHOW STATUS increments status variables more than in text protocol
* multi-statements should be avoided (see tests with a wrong delimiter)
* performance_schema events have different names in --ps-protocol
* --enable_prepare_warnings

mysqltest.cc:
* make sure run_query_stmt() doesn't crash if there's
  no active connection (in wait_until_connected_again.inc)
* prepare all statements that server supports

protocol.h
* Protocol_discard::send_result_set_metadata() should not send
  anything to the client.

sql_acl.cc:
* extract the functionality of getting the user for SHOW GRANTS
  from check_show_access(), so that mysql_test_show_grants() could
  generate the correct column names in the prepare step

sql_class.cc:
* result->prepare() can fail, don't ignore its return value
* use correct number of decimals for EXPLAIN columns

sql_parse.cc:
* discard profiling for SHOW PROFILE. In text protocol it's done in
  prepare_schema_table(), but in --ps it is called on prepare only,
  so nothing was discarding profiling during execute.
* move the permission checking code for SHOW CREATE VIEW to
  mysqld_show_create_get_fields(), so that it would be called during
  prepare step too.
* only set sel_result when it was created here and needs to be
  destroyed in the same block. Avoid destroying lex->result.
* use the correct number of tables in check_show_access(). Saying
  "as many as possible" doesn't work when first_not_own_table isn't
  set yet.

sql_prepare.cc:
* use correct user name for SHOW GRANTS columns
* don't ignore verbose flag for SHOW SLAVE STATUS
* support preparing REVOKE ALL and ROLLBACK TO SAVEPOINT
* don't ignore errors from thd->prepare_explain_fields()
* use select_send result for sending ANALYZE and EXPLAIN, but don't
  overwrite lex->result, because it might be needed to issue execute-time
  errors (select_dumpvar - too many rows)

sql_show.cc:
* check grants for SHOW CREATE VIEW here, not in mysql_execute_command

sql_view.cc:
* use the correct function to check privileges. Old code was doing
  check_access() for thd->security_ctx, which is invoker's sctx,
  not definer's sctx. Hide various view related errors from the invoker.

sql_yacc.yy:
* initialize lex->select_lex for LOAD, otherwise it'll contain garbage
  data that happen to fail tests with views in --ps (but not otherwise).

This solves the following issues:

* unlike lex->m_sql_cmd and lex->sql_command, thd->query_plan_flags
  is not reset in Prepared_statement::execute, it survives
  till the log_slow_statement(), so slow logging behaves correctly in --ps

* using thd->query_plan_flags for both slow_log_filter and
  log_slow_admin_statements means the definition of "admin" statements
  for the slow log is the same no matter how it is filtered out.

Added metadate info after prepare EXPLAIN/ANALYZE.

After-merge fix for MDEV-18333 Slow_queries count doesn't increase when slow_query_log is turned off

slave_list was used to provide data for SHOW SLAVE HOSTS and
Slaves_connected status variable.

Introduced binlog_dump_thread_count which is exposed via Slaves_connected
(replaces slave_list.records).

Store Slave_info on THD and access it by iterating server_threads
(replaces slave_list).

Added:
THD::slave_info
binlog_dump_thread_count
show_slave_hosts_callback()

Removed:
slave_list
SLAVE_LIST_CHUNK
SLAVE_ERRMSG_SIZE
slave_list_key()
slave_info_free()
init_slave_list()
end_slave_list()
all_slave_list_mutexes
init_all_slave_list_mutexes()
key_LOCK_slave_list
LOCK_slave_list

Moved:
SLAVE_INFO -> Slave_info
register_slave() -> THD::register_slave()
unregister_slave() -> THD::unregister_slave()

Also removed redundant end_slave() from close_connections(): it is called
again soon afterwards by clean_up().

Pre-requisite for clean MDEV-18450 solution.

If we have a 2+ node cluster which is replicating from an async master
and the binlog_format is set to STATEMENT and multi-row inserts are executed
on a table with an auto_increment column such that values are automatically
generated by MySQL, then the server node generates wrong auto_increment
values, which are different from what was generated on the async master.

In the title of the MDEV-9519 it was proposed to ban start slave on a Galera
if master binlog_format = statement and wsrep_auto_increment_control = 1,
but the problem can be solved without such a restriction.

The causes and fixes:

1. We need to improve processing of changing the auto-increment values
after changing the cluster size.

2. If wsrep auto_increment_control switched on during operation of
the node, then we should immediately update the auto_increment_increment
and auto_increment_offset global variables, without waiting of the next
invocation of the wsrep_view_handler_cb() callback. In the current version
these variables retain its initial values if wsrep_auto_increment_control
is switched on during operation of the node, which leads to inconsistent
results on the different nodes in some scenarios.

3. If wsrep auto_increment_control switched off during operation of the node,
then we must return the original values of the auto_increment_increment and
auto_increment_offset global variables, as the user has set. To make this
possible, we need to add a "shadow copies" of these variables (which stores
the latest values set by the user).

https://jira.mariadb.org/browse/MDEV-9519

If we have a 2+ node cluster which is replicating from an async master
and the binlog_format is set to STATEMENT and multi-row inserts are executed
on a table with an auto_increment column such that values are automatically
generated by MySQL, then the server node generates wrong auto_increment
values, which are different from what was generated on the async master.

In the title of the MDEV-9519 it was proposed to ban start slave on a Galera
if master binlog_format = statement and wsrep_auto_increment_control = 1,
but the problem can be solved without such a restriction.

The causes and fixes:

1. We need to improve processing of changing the auto-increment values
after changing the cluster size.

2. If wsrep auto_increment_control switched on during operation of
the node, then we should immediately update the auto_increment_increment
and auto_increment_offset global variables, without waiting of the next
invocation of the wsrep_view_handler_cb() callback. In the current version
these variables retain its initial values if wsrep_auto_increment_control
is switched on during operation of the node, which leads to inconsistent
results on the different nodes in some scenarios.

3. If wsrep auto_increment_control switched off during operation of the node,
then we must return the original values of the auto_increment_increment and
auto_increment_offset global variables, as the user has set. To make this
possible, we need to add a "shadow copies" of these variables (which stores
the latest values set by the user).

https://jira.mariadb.org/browse/MDEV-9519

If we have a 2+ node cluster which is replicating from an async master
and the binlog_format is set to STATEMENT and multi-row inserts are executed
on a table with an auto_increment column such that values are automatically
generated by MySQL, then the server node generates wrong auto_increment
values, which are different from what was generated on the async master.

In the title of the MDEV-9519 it was proposed to ban start slave on a Galera
if master binlog_format = statement and wsrep_auto_increment_control = 1,
but the problem can be solved without such a restriction.

The causes and fixes:

1. We need to improve processing of changing the auto-increment values
after changing the cluster size.

2. If wsrep auto_increment_control switched on during operation of
the node, then we should immediately update the auto_increment_increment
and auto_increment_offset global variables, without waiting of the next
invocation of the wsrep_view_handler_cb() callback. In the current version
these variables retain its initial values if wsrep_auto_increment_control
is switched on during operation of the node, which leads to inconsistent
results on the different nodes in some scenarios.

3. If wsrep auto_increment_control switched off during operation of the node,
then we must return the original values of the auto_increment_increment and
auto_increment_offset global variables, as the user has set. To make this
possible, we need to add a "shadow copies" of these variables (which stores
the latest values set by the user).

https://jira.mariadb.org/browse/MDEV-9519

This patch adds support for expiring user passwords.
The following statements are extended:
  CREATE USER user@localhost PASSWORD EXPIRE [option]
  ALTER USER user@localhost PASSWORD EXPIRE [option]
If no option is specified, the password is expired with immediate
effect. If option is DEFAULT, global policy applies according to
the default_password_lifetime system var (if 0, password never
expires, if N, password expires every N days). If option is NEVER,
the password never expires and if option is INTERVAL N DAY, the
password expires every N days.
The feature also supports the disconnect_on_expired_password system
var and the --connect-expired-password client option.

Closes #1166

* implicit period constraint is hidden and cannot be dropped independently
* create...like and create...select support

Apparently DBUG_ASSERT() can co-exist with DBUG_OFF when
-DCMAKE_CXX_FLAGS="-DDBUG_ASSERT_AS_PRINTF".

Removed assertion as it is useless now, since the type is unsigned.

The variable controls the amount of sampling analyze table performs.

If ANALYZE table with histogram collection is too slow, one can reduce the
time taken by setting analyze_sample_percentage to a lower value of the
total number of rows.
Setting it to 0 will use a formula to compute how many rows to sample:

The number of rows collected is capped to a minimum of 50000 and
increases logarithmically with a coffecient of 4096. The coffecient is
chosen so that we expect an error of less than 3% in our estimations
according to the paper:
"Random Sampling for Histogram Construction: How much is enough?”
– Surajit Chaudhuri, Rajeev Motwani, Vivek Narasayya, ACM SIGMOD, 1998.

The drawback of sampling is that avg_frequency number is computed
imprecisely and will yeild a smaller number than the real one.

Condition can be pushed from the HAVING clause into the WHERE clause
if it depends only on the fields that are used in the GROUP BY list
or depends on the fields that are equal to grouping fields.
Aggregate functions can't be pushed down.

How the pushdown is performed on the example:

SELECT t1.a,MAX(t1.b)
FROM t1
GROUP BY t1.a
HAVING (t1.a>2) AND (MAX(c)>12);

=>

SELECT t1.a,MAX(t1.b)
FROM t1
WHERE (t1.a>2)
GROUP BY t1.a
HAVING (MAX(c)>12);

The implementation scheme:

1. Extract the most restrictive condition cond from the HAVING clause of
   the select that depends only on the fields that are used in the GROUP BY
   list of the select (directly or indirectly through equalities)
2. Save cond as a condition that can be pushed into the WHERE clause
   of the select
3. Remove cond from the HAVING clause if it is possible

The optimization is implemented in the function
st_select_lex::pushdown_from_having_into_where().

New test file having_cond_pushdown.test is created.

Too easy to abuse. One tends to use it even when
microseconds are not needed and will be ignored.

This task involves the implementation for the optimizer trace.

This feature produces a trace for any SELECT/UPDATE/DELETE/,
which contains information about decisions taken by the optimizer during
the optimization phase (choice of table access method, various costs,
transformations, etc). This feature would help to tell why some decisions were
taken by the optimizer and why some were rejected.

Trace is session-local, controlled by the @@optimizer_trace variable.
To enable optimizer trace we need to write:
   set @@optimizer_trace variable= 'enabled=on';

To display the trace one can run:
   SELECT trace FROM INFORMATION_SCHEMA.OPTIMIZER_TRACE;

This task also involves:
    MDEV-18489: Limit the memory used by the optimizer trace
    introduces a switch optimizer_trace_max_mem_size which limits
    the memory used by the optimizer trace. This was implemented by
    Sergei Petrunia.

Store original charset during client authentication, and restore it for
COM_RESET_CONNECTION

Store original charset during client authentication, and restore it for
COM_RESET_CONNECTION

Replaced wait on COND_thread_count with busy waiting with 1 millisecond
sleep.

Aim is to reduce usage of LOCK_thread_count and COND_thread_count.

Signal handler is now respoinsible for setting abort_loop and breaking
poll() in main thread. The rest is handled by main thread itself.

Removed redundant LOCK_error_log init/destroy wrappers.
Removed redundant unireg_end(): it is trivial and it has only one caller.
Removed unused ready_to_exit from PFS.
Removed kill_in_progress: duplicates abort_loop.
Removed shutdown_in_progress: duplicates abort_loop.
Removed ready_to_exit: was used to make sure main thread waits for
cleanups, which are now done by main thread itself.
Removed SIGNALS_DONT_BREAK_READ, MAYBE_BROKEN_SYSCALL,
kill_broken_server: never defined/used.
Make clean_up() static.

modifications (insert/erase) are protected by write lock
iteration over list is protected by read lock.

This way, threads that iterate over the list (as in SHOW PROCESSLIST,
SHOW GLOBAL STATUS) do not block each other.

In contrast to thread_count, which is decremented by THD destructor,
this one was most probably intended to be decremented after all THD
destructors are done.

THD_count class was added to achieve similar effect with thread_count.

Aim is to reduce usage of LOCK_thread_count and COND_thread_count.
Part of MDEV-15135.

Implemented and integrated THD_list as a replacement for the global
thread list. It uses own mutex instead of LOCK_thread_count for THD
list protection.

Removed unused first_global_thread() and next_global_thread().

delayed_insert_threads is now protected by LOCK_delayed_insert. Although
this patch doesn't fix very wrong synchronization of this variable.

After this patch there are only 2 legitimate uses of LOCK_thread_count
left, both in mysqld.cc: thread_count and ready_to_exit.

Aim is to reduce usage of LOCK_thread_count and COND_thread_count.
Part of MDEV-15135.

LOG_INFO::lock was useless. It could've only protect against concurrent
iterators execution, which was already protected by LOCK_thread_count.

Use LOCK_thd_data instead of LOCK_thread_count as a protection against
THD::current_linfo reset.

Aim is to reduce usage of LOCK_thread_count and COND_thread_count.
Part of MDEV-15135.

Part of MDEV-5336 Implement LOCK FOR BACKUP

- Changed check of Global_only_lock to also include BACKUP lock.
- We store latest MDL_BACKUP_DDL lock in thd->mdl_backup_ticket to be able
  to downgrade lock during copy_data_between_tables()

Part of MDEV-5336 Implement LOCK FOR BACKUP

Other things:
- Added printing of MDL locks to DBUG.