An after-fix for MDEV-14008 Assertion failing: `!is_set() || (m_status == DA_OK_BULK && is_bulk_op())

Fixing an additional failure discovered after a merge to 10.2

The comment became stale in commit 9f57e595
which removed the parameter "flags".

Whenever we call merge_role_privileges on a role, we make use of
the role->counter variable to check if all it's children have had their
privileges merged. Only if all children have had their privileges merged,
do we update the privileges on parent. This is done to prevent extra work.
The same idea is employed during flush privileges. You only begin merging
from "leaf" roles. The recursive calls will merge their parents at some point.
A problem arises when we try to "re-merge" a parent. Take the following graph:

{noformat}
     A (0)  ----  C (2) ---- D (2)  ---- USER
                 /          /
     B (0)  ----/          /
                          /
     E (0) --------------/
{noformat}

In parentheses we have the "counter" value right before we start to iterate
through the roles hash and propagate values. It represents the number of roles
granted to the current role. The order in which we iterate through the roles
hash is alphabetical.

* First merge A, which leads to decreasing the counter for C to 1. Since C is
not 0, we don't proceed with merging into C.

* Second we merge B, which leads to decreasing the counter for C to 0. Now
we proceed with merging into C. This leads to reducing the counter for D to 1
as part of C merge process.

* Third as we iterate through the hash, we see that C has counter 0, thus we
start the merge process *again*. This leads to reducing the counter for
D to 0! We then attempt to merge D.

* Fourth we start merging E. When E sees D as it's parent (according to the code)
it attempts to reduce D's counter, which leads to overflow. Now D's counter is
a very large number, thus E's privileges are not forwarded to D yet.

To correct this behavior we must make sure to only start merging from initial
leaf nodes.

When granting a role to another role, DB privileges get propagated. If
the grantee had no previous DB privileges, an extra ACL_DB entry is created to
house those "indirectly received" privileges. If, afterwards, DB
privileges are granted to the grantee directly, we must make sure to not
create a duplicate ACL_DB entry.

The InnoDB background DROP TABLE queue is something that we should
really remove, but are unable to until we remove dict_operation_lock
so that DDL and DML operations can be combined in a single transaction.

Because the queue is not persistent, it is not crash-safe. In stable
versions of MariaDB, we can only try harder to drop all enqueued
tables before server shutdown.

row_mysql_drop_t::table_id: Replaces table_name.

row_drop_tables_for_mysql_in_background():
Do not remove the entry from the list as long as the table exists.
In this way, the table should eventually be dropped.

MDEV-14641 Incompatible key or row definition between the MariaDB .frm file and the information in the storage engine

make sure that mysql_create_frm_image() and fast_alter_partition_table()
use the same code to derive HA_OPTION_PACK_RECORD from
create_info->row_type.

trx_rollback_active(): When aborting the rollback, free the query graph.

MDEV-12323 Rollback progress log messages during crash recovery are intermixed with unrelated log messages

trx_roll_must_shutdown(): During the rollback of recovered transactions,
report progress and check if the rollback should be interrupted because
of a pending shutdown.

trx_roll_max_undo_no, trx_roll_progress_printed_pct: Remove, along with
the messages that were interleaved with other messages.

row_undo_step(), trx_rollback_active(): Abort the rollback of a
recovered ordinary transaction if fast shutdown has been initiated.

trx_rollback_resurrected(): Convert an aborted-rollback transaction
into a fake XA PREPARE transaction, so that fast shutdown can proceed.

MDEV-13797 InnoDB may hang if shutdown is initiated soon after startup while rolling back recovered incomplete transactions

trx_rollback_resurrected(): If shutdown was initiated, fake all
remaining active transactions to XA PREPARE state, so that shutdown
can proceed. Also, make the parameter "all" an output that will be
assigned to FALSE in this case.

trx_rollback_or_clean_recovered(): Remove the shutdown check
(it was moved to trx_rollback_resurrected()).

trx_undo_free_prepared(): Relax assertions.

Add missing instrumentation to row0ins.cc.

MariaDB adjustments to test case innodb-replace-debug.

MariaDB 10.0 does not seem to be affected.

Imported missing test case from MySQL 5.7 for

commit 25781c154396dbbc21023786aa3be070057d6999
Author: Annamalai Gurusami <annamalai.gurusami@oracle.com>
Date:   Mon Feb 24 14:00:03 2014 +0530

    Bug #17604730 ASSERTION: *CURSOR->INDEX->NAME == TEMP_INDEX_PREFIX

Use utf-mb4 if it is possible.

MDEV-12372 mysqlbinlog --version output is the same on 10.x as on 5.5.x, and contains not only version

don't print usage() for --version

Some tests are skipped by checks in suite.pm. It is redundant to
have an sql-level run-time check in the .inc file itself.

In some cases it's not only redundant, but dangerous.
After one bug in 10.2 innodb.create_isl_with_direct failed
to start InnoDB, but the server started fine (just without InnoDB)
and instead of failing, the test was skipped by run-time check in
have_innodb.inc.

# Conflicts:
#	mysql-test/include/not_embedded.inc
#	mysql-test/r/change_user_notembedded.result
#	mysql-test/suite.pm
#	mysql-test/t/change_user_notembedded.test

MDEV-14164: Unknown column error when adding aggregate to function in oracle style procedure FOR loop

Make differentiation between pullout for merge and pulout of outer field during exists2in transformation.
In last case the field was outer and so we can safely start from name resolution context of the SELECT where it was pulled.
Old behavior lead to inconsistence between list of tables and outer name resolution context (which skips one SELECT for merge purposes) which creates problem vor name resolution.

ibuf_check_bitmap_on_import(): Only access the pages that
are below FSP_FREE_LIMIT. It is possible that especially with
ROW_FORMAT=COMPRESSED, the FSP_SIZE will be much bigger than
the FSP_FREE_LIMIT, and the bitmap pages (page_size*N, 1+page_size*N)
are filled with zero bytes.

buf_page_is_corrupted(), buf_page_io_complete(): Make the
fault injection compatible with MariaDB 10.2.

Backport the IMPORT tests from 10.2.

- innodb_buffer_pool_dump_now_basic is modified to make sure it
  really performs a dump and waits till it completion, to avoid
  the apparent or hidden failure similar to MDEV-9713 / MDEV-10651
- innodb_buffer_pool_dump_pct_basic is modified to re-use the new
  code from innodb_buffer_pool_dump_now_basic and thus avoid
  the failure MDEV-10651
- innodb_buffer_pool_load_now_basic is re-written to simplify
  the logic by re-using the code innodb_buffer_pool_dump_now_basic
  and is given an opt file to avoid race conditions with
  buffer pool load performed upon server startup, which causes
  MDEV-14196 failure

Add a check for #sql* files in test and mysql subdirs to the testcase check

The test would pass even with skipped partitioning, because
CHECK PARTITION for a view works identically with enabled/disabled
partitioning; but if the server is compiled without partitioning
at all, it cannot execute the statement, and the test would fail.

Check for the presence of partitioning allows to skip the test
in this case, rather than let it fail

present.

        thread_pool_server_audit.test fixed.
        plugin version updated.

present.

        thread_pool_server_audit.test fixed.
        plugin version updated.

present.

        Escape special characters (like \r \n \t) instead of
        replacing them with spaces.

present.

        Escape special characters (like \r \n \t) instead of
        replacing them with spaces.

If InnoDB or XtraDB recovered committed transactions at server
startup, but the processing of recovered transactions was
prevented by innodb_read_only or by innodb_force_recovery,
an assertion would fail at shutdown.

This bug was originally reproduced when Mariabackup executed
InnoDB shutdown after preparing (applying redo log into) a backup.

trx_free_prepared(): Allow TRX_STATE_COMMITTED_IN_MEMORY.

trx_undo_free_prepared(): Allow any undo log state. For transactions
that were resurrected in TRX_STATE_COMMITTED_IN_MEMORY
the undo log state would have been reset by trx_undo_set_state_at_finish().

Re-apply the patch cb1b466c,
the change disappeared after the merge

Replace all references in InnoDB and XtraDB error log messages
to bugs.mysql.com with references to https://jira.mariadb.org/.

The original merge
commit 4274d0bf
was accidentally reverted by the subsequent merge
commit 3b35d745