Prepared statements which were run over binary protocol crashed
a server if the statement did not have CF_PS_ARRAY_BINDING_OPTIMIZED
flag and the statement was executed in bulk mode and a BF abort occrurred.
This was because the bulk execution resulted in several statements without
calling wsrep_after_statement() between, which confused wsrep transaction
state tracking.

As a fix, call wsrep_after_statement() in bulk loop after each execution
if CF_PS_ARRAY_BINDING_OPTIMIZED is not set.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

instant_alter_column_possible(): Relax a too strict debug assertion.
The existence of an index stub or a corrupted index on virtual columns
does not imply that virtual columns exist.

Per b9f3f068, mysql_system_tables_data.sql creates
a mysql_native_password with a salted hash of "invalid" so that `set password`
will detect a native password can be applied:.

SHOW CREATE USER; diligently uses this value in its output
generating the SQL:

   MariaDB [(none)]> show create user;

   +---------------------------------------------------------------------------------------------------+
   | CREATE USER for dan@localhost                                                                     |
   +---------------------------------------------------------------------------------------------------+
   | CREATE USER `dan`@`localhost` IDENTIFIED VIA mysql_native_password USING 'invalid' OR unix_socket |
   +---------------------------------------------------------------------------------------------------+

Attempting to execute this before this patch results in:

  MariaDB [(none)]>  CREATE USER `dan2`@`localhost` IDENTIFIED VIA mysql_native_password USING 'invalid' OR unix_socket;
  ERROR 1372 (HY000): Password hash should be a 41-digit hexadecimal number

As such, deep the implementation of mysql_native_password we make "invalid" valid (pun intended)
such that the above create user will succeed. We do this by storing
"*THISISNOTAVALIDPASSWORDTHATCANBEUSEDHERE" (credit: Oracle MySQL), that is of an INCORRECT
length for a scramble.

In native_password_authenticate we check the length of this cached value
and immediately fail if it is anything other than the scramble length.

native_password_get_salt is only called in the context of set_user_salt, so all setting of native
passwords to hashed content of 'invalid', quite literally create an invalid password.

So other forms of "invalid" are valid SQL in creating invalid passwords:

   MariaDB [(none)]> set password = 'invalid';
   Query OK, 0 rows affected (0.001 sec)

   MariaDB [(none)]> alter user dan@localhost IDENTIFIED BY PASSWORD 'invalid';
   Query OK, 0 rows affected (0.000 sec)

closes #1628

Reviewer: serg@mariadb.com

Thanks to Varun Gupta for suggesting this. This seems to
make main.innodb_ext_key,off more stable.

Invoking memcpy() on a NULL pointer is undefined behaviour
(even if the length is 0) and gives the compiler permission to
assume that the pointer is nonnull. Recent versions of GCC
(starting with version 8) are more aggressively optimizing away
checks for NULL pointers. This undefined behaviour would cause
a SIGSEGV in the test main.func_encrypt on an optimized debug build
on GCC 10.2.0.

Also, revert the work-around for the test that was attempted in
commit 85613a32.

This issue was caught by MemorySanitizer as well as on the
Microsoft Windows debug builds, thanks to /MD being used
starting with 10.4.

The code fix will also be applied to 10.2 because the regression
was introduced in commit afc9d00c.

Disable galera_var_replicate_myisam until fixed on 10.4

This bug was already fixed in a previous commit.
Added test case from the MDEV to prove it's fixed.

For some reason, in the test main,innodb_ext_key,off
we frequently get unexpected EXPLAIN output, in particular
on Microsoft Windows debug builders. Let us comment out that
EXPLAIN statement for now.

Additional case for PS protocol: UPDATE is converted to multi-update
in mysql_multi_update_prepare().

An assertion inadvertently contained an assignment and an implicit
comparison to zero. The intention was to test equality.

Tighten access control - deny FILE_CREATE_PIPE_INSTANCE permission to
everyone except current user (the one that runs mysqld)

A wsrep transaction was started for EXECUTE IMMEDIATE, which
caused assertion failure when the executed statement was
CREATE TABLE which should be executed in TOI mode.

As a fix, don't start wsrep transaction for EXECUTE IMMEDIATE
to let the wsrep state logic to be handled from inside stored
procedure codepath.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

All relevant InnoDB changes from MySQL 5.7.32 have been applied
in preceding commits.

Fix assertion `thd->in_active_multi_stmt_transaction() ||
thd->m_transaction_psi == __null' failed on MTR test
galera_sr.GCF-1051.

Add a new MTR test MDEV-23623 that reproduces the issue
deterministically and update wsrep-lib submodule, containing
the actual fix.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

Patch removes dict_index_t::stats_latch. Table/index statistics now
protected with dict_sys->mutex. That way statistics computation can
happen in parallel in several threads and dict_sys->mutex will be locked
only for a short period of time.

This patch is a joint work with Marko Mäkelä

dict_index_t::lock: make mutable which allows to pass const pointer
when only lock is touched in an object

btr_height_get()
btr_get_size(): make index argument const for better type safety

btr_estimate_number_of_different_key_vals(): now returns computed values
instead of setting fields in dict_index_t directly

remove everything related to dict_index_t::stats_latch

dict_stats_index_set_n_diff(): now returns computed values instead
of setting fields in dict_index_t directly

dict_stats_analyze_index():  now returns computed values instead
of setting fields in dict_index_t directly

Reviewed by: Marko Mäkelä

Reviewed-by: wlad@mariadb.com

Let us introduce a dummy variable innodb_max_purge_lag_wait
for waiting that the InnoDB history list length is below
the user-specified limit. Specifically,

SET GLOBAL innodb_max_purge_lag_wait=0;

should wait for all history to be purged. This could be useful
when upgrading from an older version to MariaDB 10.3 or later,
to avoid hitting MDEV-15912.

Note: the history cannot be purged if there exist transactions
that may see old versions.

Reviewed by: Vladislav Vaintroub

session_track_system_variables and max_relay_log_size.

lock LOCK_global_system_variables around the get_one_variable() call
in the Session_sysvars_tracker::store_variable().

MDEV-23693 Failing assertion: my_atomic_load32_explicit(&lock->lock_word, MY_MEMORY_ORDER_RELAXED) == X_LOCK_DECR

InnoDB frees the block lock during buffer pool shrinking when other
thread is yet to release the block lock.  While shrinking the
buffer pool, InnoDB allows the page to be freed unless it is buffer
fixed. In some cases, InnoDB releases the latch after unfixing the
block.

Fix:
====
- InnoDB should unfix the block after releases the latch.

- Add more assertion to check buffer fix while accessing the page.

- Introduced block_hint structure to store buf_block_t pointer
and allow accessing the buf_block_t pointer only by passing a
functor. It returns original buf_block_t* pointer if it is valid
or nullptr if the pointer become stale.

- Replace buf_block_is_uncompressed() with
buf_pool_t::is_block_pointer()

This change is motivated by a change in mysql-5.7.32:
mysql/mysql-server@46e60de444a8fbd876cc6778a7e64a1d3426a48d
Bug #31036301 ASSERTION FAILURE: SYNC0RW.IC:429:LOCK->LOCK_WORD

For debug build of MariaDB server running of the following test case
will hit the assert `thd->lex->sql_command == SQLCOM_UPDATE' in the function
check_fields() on attempt to execute the UPDATE statement.

  CREATE TABLE t1 (a INT);
  UPDATE t1 FOR PORTION OF APPTIME FROM (SELECT 1 FROM t1) TO 2 SET a = 1;

Stack trace to the fired assert statement
  DBUG_ASSERT(thd->lex->sql_command == SQLCOM_UPDATE)
listed below:
  mysql_execute_command() ->
    mysql_multi_update_prepare() -->
      Multiupdate_prelocking_strategy::handle_end() -->
        check_fiels()

It's worth to note that this stack trace looks like a multi update
statement is being executed. The fired assert is checked inside the
function check_fields() in case table->has_period() returns the value
true that in turns happens when temporal period specified in the UPDATE
statement. Condition specified in the DEBUG_ASSERT statement returns
the false value since the data member thd->lex->sql_command have the
value SQLCOM_UPDATE_MULTI. So, the main question is why a program control
flow go to the path prescribed for handling MULTI update statement
despite of the fact that the ordinary UPDATE statement being executed.

The answer is a way that SQL grammar rules written.

When the statement
  UPDATE t1 FOR PORTION OF APPTIME FROM (SELECT 1 FROM t1) TO 2 SET a = 1;
being parsed an action for the rule 'table_primary_ident' (part of this action
is listed below to simplify description) is  invoked to handle the table
name 't1' specified in the clause 'SELECT 1 FROM t1'.

table_primary_ident:
  table_ident opt_use_partition opt_for_system_time_clause
  opt_table_alias_clause opt_key_definition
  {
    SELECT_LEX *sel= Select;
    sel->table_join_options= 0;
    if (!($$= Select->add_table_to_list(thd, $1, $4,

This action calls the method st_select_lex::add_table_to_list()
to add the table name 't1' to the list of tables being used by the statement.

Later, an action for the following grammar rule
update_table_list:
  table_ident opt_use_partition for_portion_of_time_clause
  opt_table_alias_clause opt_key_definition
  {
    SELECT_LEX *sel= Select;
    sel->table_join_options= 0;
    if (!($$= Select->add_table_to_list(thd, $1, $4,

is invoked to handle the clause 't1 FOR PORTION OF APPTIME FROM ... TO 2'.
This action also calls the method st_select_lex::add_table_to_list()
to add the table name 't1' to the list of tables being used by the statement.

In result the table name 't1' contained twice in this list.

Presence of duplicate names for the table 't1' in a list of table used by
a statement leads to the fact that the function unique_table() called
from the function mysql_update() returns the value true that forces
implementation of the function mysql_update() to return the value 2 as
a signal to fall through the case boundary of the switch statement placed
in the function mysql_execute_statement() and start handling of the case
for sql_command SQLCOM_UPDATE_MULTI. The compound statement block for the
case SQLCOM_UPDATE_MULTI invokes the function mysql_multi_update_prepare()
that executes the statement
  set thd->lex->sql_command= SQLCOM_UPDATE_MULTI;
and after that calls the method
  Multiupdate_prelocking_strategy::handle_end(). Finally, this method
invokes the check_field() function and assert is fired.

The above analysis shows that update for a table that simultaneously specified
both as a destination table of UPDATE statement and as a table taking part in
subquery is actually treated by MariaDB server as multi-update statement.
Taking into account that multi-update statement for temporal period
table is not supported yet by MariaDB, correct way to fix the bug is to return
the error ER_NOT_SUPPORTED_YET for this case.

Deadlock is possible between applier thread and local committing thread with active FLUSH TABLE.
Applier thread should skip table share checks and locks when opening table.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

MDEV-23356 InnoDB: Failing assertion: field->col->mtype == type, crash or ASAN failures in row_sel_convert_mysql_key_to_innobase, InnoDB indexes are inconsistent after INDEX changes

innobase_rename_indexes_cache(): fix corruption of index cache. Index ids
help distinguish indexes when their names clash.

innobase_rename_indexes_cache(): fix corruption of index statistics table.
Use unique temporary names to avoid names clashing.

Reviewed by: Marko Mäkelä

Problem:
1. The server terminates abnormally when phrase search doesn't
filter out doc_ids correctly. This problem has been fixed in bug
2. Wrong query result: It's a regression from the bug #22709692 fix.
This fix optimize full-text search query with limit clause.
when FTS expression involves only union operation, we fetch only
number of doc_ids specified with the limit clause.
Fulltext phrase search is not an union operation and we consider
phrase search with plugin parser a union operation.
In phrase search with limit clause, we fetch limited doc_ids for
each token and if any of the selected doc_id does not contain all
tokens in correct order then we do not include that row_id in the
result set.
Therefore phrase search gets fewer number of rows than the qualified
rows exist in the table.

Fix:
Added a condition that phrase search with plugin parser is not a
union operation.

RB: 24925
Reviewed by : Annamalai Gurusami <annamalai.gurusami@oracle.com>

This is a cherry-pick of
mysql/mysql-server@5549920b7a33ef33034461d973a9ecb17ce49799
without a test case, because the test case depends on an n-gram
tokenizer that will be missing from MariaDB until MDEV-10267 is added.

Problem:
In Full-text phrase search, we filter out row that do not contain
all the tokens in the phrase.
If we do not filter out doc_id that doesn't appear in all the
token's doc_id lists then we hit an assert.

Fix:
if any of the token has last doc_id equal to ith doc_id of the first
token doc_id list then filter out rest of the higher doc_ids.

RB: 24909
Reviewed by : Annamalai Gurusami <annamalai.gurusami@oracle.com>

This is a cherry-pick of
mysql/mysql-server@5aa075277dfe84a17a0331c57a6fe9b91dafb4cf
but without a test case, because the test case depends on an n-gram
tokenizer that will be missing from MariaDB until MDEV-10267 is added.

Recent gcc/clang versions failed to compile the existing code.

Updating a later upstream SDK version was simple and required
only implementing a flush method. This was left blank as
there was no strong requirement to keep the error log
atomic or durable.

Reviewed-by: wlad@mariadb.com

The upstream SDK version added a flush method which was simple
to complete.

was left over from testing

MDEV-23370 innodb_fts.innodb_fts_misc failed in buildbot, server crashed in dict_table_autoinc_destroy

This issue is caused by MDEV-22456 ad6171b9. Fix involves the backported version of 10.4 patch
MDEV-22778 5f2628d1 and few parts of
MDEV-17441 (e9a5f288).

dict_table_t::stats_latch_created: Removed

dict_table_t::stats_latch: make value member and always lock it for
simplicity even for stats cloned table.

zip_pad_info_t::mutex_created: Removed

zip_pad_info_t::mutex: make member value instead of pointer

os0once.h: Removed

dict_table_remove_from_cache_low(): Ensure that fts_free() is always
called, even if dict_mem_table_free() is deferred until
btr_search_lazy_free().

InnoDB would always zip_pad_info_t::mutex and
dict_table_t::autoinc_mutex, even for tables are not in
ROW_FORMAT=COMPRESSED nor include any AUTO_INCREMENT column.

MariaDB 10.2.2 inherited from MySQL 5.7 a perceived optimization
of ALTER TABLE, which skips the writing of redo log records.
In MDEV-16809 we introduced a parameter that allows the redo log to
be written, so that Mariabackup would not be impacted, but we kept
the MySQL 5.7 behaviour enabled by default (innodb_log_optimize_ddl=ON).

As noted in MDEV-19747 (Deprecate and ignore innodb_log_optimize_ddl,
implemented in MariaDB 10.5.1), omitting the redo log writes can
actually reduce performance, because we will have to wait for the data
pages to be written out. When the redo log file is configured to be
large enough, it actually can be much faster to write the redo log and
avoid the extra page flushing.

When the redo log is omitted (innodb_log_optimize_ddl=ON), also
Mariabackup may have to perform a lot of extra work, to re-copy the
entire data file if it is possible that any log was omitted during
the backup.

Starting with MariaDB 10.5.1, the parameter innodb_log_optimize_ddl
is deprecated and ignored. We hereby deprecate (but will not ignore)
the parameter in earlier versions as well.

Reviewed-by: serg@mariadb.com

There are 2 issues here:

Issue #1: memory allocation.
An IO_CACHE that uses encryption uses a larger buffer (it needs space for the encrypted data,
decrypted data, IO_CACHE_CRYPT struct to describe encryption parameters etc).

Issue #2: IO_CACHE::seek_not_done
When IO_CACHE objects are cloned, they still share the file descriptor.
This means, operation on one IO_CACHE may change the file read position
which will confuse other IO_CACHEs using it.

The fix of these issues would be:
Allocate the buffer to also include the extra size needed for encryption.
Perform seek again after one IO_CACHE reads the file.