innobase_get_stmt_safe(): Copy also the last byte.

dict_init_free(): Make global, and move the call from
dict_close() to srv_free(), because this is initialized
earlier than dict_sys.

innobase_space_shutdown(): Do not leak srv_allow_writes_event.

Write only one encryption key to the checkpoint page.
Use 4 bytes of nonce. Encrypt more of each redo log block,
only skipping the 4-byte field LOG_BLOCK_HDR_NO which the
initialization vector is derived from.

Issue notes, not warning messages for rewriting the redo log files.

recv_recovery_from_checkpoint_finish(): Do not generate any redo log,
because we must avoid that before rewriting the redo log files, or
otherwise a crash during a redo log rewrite (removing or adding
encryption) may end up making the database unrecoverable.
Instead, do these tasks in innobase_start_or_create_for_mysql().

Issue a firm "Missing MLOG_CHECKPOINT" error message. Remove some
unreachable code and duplicated error messages for log corruption.

LOG_HEADER_FORMAT_ENCRYPTED: A flag for identifying an encrypted redo
log format.

log_group_t::is_encrypted(), log_t::is_encrypted(): Determine
if the redo log is in encrypted format.

recv_find_max_checkpoint(): Interpret LOG_HEADER_FORMAT_ENCRYPTED.

srv_prepare_to_delete_redo_log_files(): Display NOTE messages about
adding or removing encryption. Do not issue warnings for redo log
resizing any more.

innobase_start_or_create_for_mysql(): Rebuild the redo logs also when
the encryption changes.

innodb_log_checksums_func_update(): Always use the CRC-32C checksum
if innodb_encrypt_log. If needed, issue a warning
that innodb_encrypt_log implies innodb_log_checksums.

log_group_write_buf(): Compute the checksum on the encrypted
block contents, so that transmission errors or incomplete blocks can be
detected without decrypting.

Rewrite most of the redo log encryption code. Only remember one
encryption key at a time (but remember up to 5 when upgrading from the
MariaDB 10.1 format.)

The InnoDB redo log consists of a list of files that logically form
a bigger file, as if the individual files were concatenated together.

The first file will always be written on redo log checkpoint, because
the two checkpoint pages are at the start of the single logical
redo log file.

There is no technical reason why InnoDB requires at least 2 files
to exist. Let us reduce the minimum number to 1. In that way,
restoring from backups will become easier, since InnoDB can directly
deal with a single backed-up redo log file.

the test restarts the server, giving it 60 seconds to shutdown
and then killing it mercilessly.

make sure the server closes all MyISAM tables before shutdown,
as we cannot reliably expect it to make the deadline.

wait for the first connection to reach the debug sync point
before letting the second connection do its stuff

options --log-slow-admin-statements, --log-queries-not-using-indexes and --log-slow-slave-statements have no effect if --log_slow_queries is not set

1. s/--log_slow_queries/--log-slow-queries/
2. disable log-slow-admin-statement/etc in mytr when doing mysqld --help

Use JSON_COMPACT(X) instead of CAST(X AS JSON).
Bonus - X is validated to be a valid JSON.
Fix a typo in the error message.

fixes crashes on win32-debug

Change output of mysql_config --libs to have -lmariadb instead of
-lmysqlclient

Static linking is not yet supported, because static library has different
base name ( libmariadbclient.a static vs libmariadb.so shared)

work.

        json_detailed() fixed

Fix a test result for the narrower type of innodb_fast_shutdown.

Ever since MDEV-5800 enabled indexed virtual columns for InnoDB,
the InnoDB shutdown relied on close_connections() that would set
thd->killed for the InnoDB purge threads. Alas, the embedded server
shutdown is not invoking close_connections(), and thus InnoDB purge
threads fail to initiate shutdown, causing a hang.

innodb_inited: Remove. Use srv_was_started instead.

innobase_fast_shutdown: Remove. Use srv_fast_shutdown instead.

srv_running: Renamed from thd_destructor_myvar, and made global.
The value NULL means that shutdown was requested or the purge threads
should not be running because of innodb_read_only_mode=1.

innobase_init(): Set srv_was_started after ensuring that srv_running
was initialized. (In innodb_read_only mode, the purge threads are not
started and we do not care if srv_running==NULL.)

innobase_start_or_create_for_mysql(): Do not set srv_was_started.
Let it be set by the only caller innobase_init().

srv_purge_should_exit(): Check also srv_was_started and srv_running
when evaluating thd->killed.

PART 2 of the fix adds the logic of not using password column, unless it
exists. If password column is missing we attempt to use plugin &&
authentication_string columns.

PART 1 of the fix requires a bit of refactoring to not use hard-coded
field indices any more. Create classes that express the grant tables structure,
without exposing the underlying field indices.

Most of the code is converted to use these classes, except parts which
are not directly affected by the MDEV-11170. These however are TODO
items for subsequent refactoring.

On construction the Sql_mode_save class stores the current THD's
sql_mode. On destruction, the THD's mode is restored.

The same approach is needed for LAST_VALUE, otherwise the LAST_VALUE sum
functions are not cleared correctly. Now LAST_VALUE behaves as NTH_VALUE
with 0 offset, only that the frame that it is examining is the bottom bound,
not the top bound.

Reimplement FIRST_VALUE to act as NTH_VALUE with 0 offset. The previous
implementation was flawed when the window frame would remove values.

The problematic queries involve unions. For unions we have an
optimization where we skip the ORDER BY clause in a query from one side
of the union if it will be performed later due to UNION.
EX:
(SELECT a from t1 ORDER BY a) ORDER BY b;
The first ordering by a is not necessary and it gets removed.

The problem is that we still need to resolve the Items before removing the
ORDER BY list from the
SELECT_LEX structure. During this final resolve step however, we forgot to
allow SET functions within the ORDER BY clause. This caused us to return
an "Invalid use of group function" error during the checking performed
by fix_fields in Item_sum::init_sum_func_check.

Fatal error: mysql.user table is damaged or in unsupported 3.20 format

The problem stems from MySQL 5.7.6. According to MySQL documentation:
In MySQL 5.7.6, the Password column was removed and all credentials are
stored in the authentication_string column.

If opening a MySQL 5.7.6 (and up) datadir with MariaDB 10.2, the user table
appears corrupted. In order to fix this, the server must be started with
--skip-grant-tables and then a subsequent mysql_upgrade command must be
issued.

This patch updates the mysql_upgrade command to also add the removed
Password column. The password column is necessary, otherwise
the mysql_upgrade script fails due to the Event_scheduler not being able
to start, as it can't find Event_priv in the table where it ought to be.
MySQL's version has column position 28 (0 index) vs our datadir version
expects position 29.

This reverts commit 1f372cf1.
Wasn't supposed to be pushed just yet.

remove CAST(... AS JSON) from the grammar for 10.2.4

relax innodb assertion, because Field_blob::store() clearly says
that a data pointer can be zero if the length is zero.

change the parser not to allow SERIAL as a normal data type.
make a special rule for it, where it could be used for define
fields, but not generated fields, not return type of a stored function, etc.

JOIN_CACHE's were initialized in  check_join_cache_usage()
from make_join_readinfo(). After that make_join_readinfo() was looking
whether it's possible to use keyread. Later, after make_join_readinfo(),
optimizer decided whether to use filesort. And even later, at the
execution time, from join_read_first(), keyread was actually enabled.

The problem is, that if a query uses a vcol, base columns that it
depends on are automatically added to the read_set - because they're
needed to calculate the vcol. But if we're doing keyread, vcol is taken
from the index, not calculated, and base columns do not need to  be
in the read set (even should not be - as they aren't getting values).

The bug was that JOIN_CACHE used read_set with base columns,
they were not read because of keyread, so it was caching garbage.

So read_set is only known after the keyread was decided. And after the
filesort was decided, as filesort doesn't use keyread. But
check_join_cache_usage() needs to be done in make_join_readinfo(),
as the code below depends on these checks,

Fix: keep JOIN_CACHE checks where they were, but move initialization
down to the very end of JOIN::optimize_inner. If keyread was enabled,
update the read_set to include only columns that are part of the index.
Copy the keyread logic from join_read_first() to happen at optimize time.

make init_read_record() to detect enabled keyread
and use index_* access methods, not rnd_*

this makes MariaDB to use keyread a lot more often than before

because it doesn't work, vcols are never calculated for ICP

with --sorted_result

* rename to "keyread" (to avoid conflicts with tokudb),
* change from bool to uint and store the keyread index number there
* provide a bool accessor to check if keyread is enabled

Filesort temporarily changes read_set to be tmp_set and marks only
fields needed for filesort. Add an assert to ensure that it doesn't
overwrite the old value of tmp_set, that is that read_set was *not*
already tmp_set when filesort was invoked.

Fix sql_update.cc that was was doing exactly that - changing read_set to
tmp_set, configuring tmp_set for keyread, and then invoking filesort.

mark_columns_used_by_index used to do
reset + mark_columns_used_by_index_no_reset + start keyread + set bitmaps

Now prepare_for_keyread does that, while mark_columns_used_by_index
does only reset + mark_columns_used_by_index_no_reset,
just as its name suggests.

old code didn't calculate vcols that were part of keyread,
but calculated other vcols. It was wrong - there was no guarantee
that vcol's base columns were part of keyread.

Technically it's possible for the vcol not be a part of keyread,
but all its base columns being part of keyread. But currently the
optimizer doesn't do that, keyread is only used if it covers
all columns used in the query.

This fixes crashes of vcol.vcol_trigger_sp_innodb

TABLE::add_read_columns_used_by_index() is conceptually wrong,
it *adds* columns used by index to the bitmap, without clearing
it first. But it also enables keyread, meaning that *only* columns
from the index will be read. It is supposed to be used to
add columns used by an index to a bitmap that already has columns
of a primary key - for engines where a primary key is part of every
index.

The correct fix is to change mark_columns_used_by_index() to
take into account extended keys.

this reverts 1d0acc77 and cf97cbd1

it should not mark base columns that a vcol depends on, because
keyread (on a vcol) will not read them

use table->mark_columns_used_by_index, don't copy it

use table->mark_columns_used_by_index, don't copy it