This patch adds PACKAGE support with SQL/PSM dialect for sql_mode=DEFAULT:

- CREATE PACKAGE
- DROP PACKAGE
- CREATE PACKAGE BODY
- DROP PACKAGE BODY
- Package function and procedure invocation from outside of the package:
    -- using two step identifiers
    SELECT pkg.f1();
    CALL pkg.p1()

    -- using three step identifiers
    SELECT db.pkg.f1();
    CALL db.pkg.p1();

This is a non-standard MariaDB extension.

However, later this code can be used to implement
the SQL Standard and DB2 dialects of CREATE MODULE.

This file isn't used by client or server. Its
functionality is purely in the generation of *.sys
files.

As such omit from packaging and save ~1MB.

We use a bool to indicate that the key part used is a descending
index, which will flip the functions and flags used in
get_index_max_value() and get_index_min_value(), that allows correct
optimization for max/min for descending index.

This is a follow-up for the initial MDEV-24486 commit. It renames
the view sys.table_privileges to sys.privileges_by_table_by_level
and adds some more tests displaying privilege levels GLOBAL and SCHEMA

1. WITHOUT/WITH VALIDATION may be added to EXCHANGE PARTITION or CONVERT TABLE:

  alter table tp exchange partition p1 with table t with validation;
  alter table tp exchange partition p1 with table t; -- same as with validation
  alter table tp exchange partition p1 with table t without validation;

2. Optional THAN keyword for RANGE partitioning. Normally you type:

  create table tp (a int primary key) partition by range (a) (
    partition p0 values less than (100),
    partition p1 values less than maxvalue);

  Now you may type (PARTITION keyword is also optional):

  create table tp (a int primary key) partition by range (a) (
    p0 values less (100),
    p1 values less maxvalue);

DIGEST independed testing for performance_schema events.

The existing INFORMATION_SCHEMA.TABLE_PRIVILEGES displays only those
privileges that were specifically granted on the table level,
whereas it may be useful to see privileges granted at the database
and global level.

This commit adds a new view `table_privileges` to the `sys` schema
for that purpose. The view shows privileges on existing tables
and views, combining all possible levels:
 - user_privileges
 - schema_privileges
 - table_privileges

With the default datadir path of /var/lib/mysql the same
as the path of the unix socket, this tmpfiles.d file
was generated to "create" a datadir. This wasn't intended.

In this case we comment out the entry and explain why.

Add extra ideas on other temporary directories that
may be specified in this file.

This patch provides a new mode FULL_NODUP to binlog_row_image system
variable. With FULL_NODUP mode, all columns are included in before
image, but only updated columns are included in after image for UPDATE.
While all columns are included in the after image for INSERT.

FULL_NODUP is for replacing FULL mode. It includes all data of
the before and after image as FULL mode, but it uses less storage
especially in the case that only a few columns are updated.

Note: It will binlog full before and after image for all modes if the
      table has no primary key. FULL_NODUP follows the behavior.

Systemd socket activation cannot handle a shutdown on the file
descriptor[1].

Enumerate past the socket activation descriptors.

If there was no shutdown to trigger the breaking of the event loop,
then write to the termination_event_fd that was setup during
the socket activation code for this purpose.

As abort_loop= true is already set at the top of break_connect_loop,
and this is checked in loop before sockets are processed, no
additional checking to read from the termination_event_fd is needed.

Without socket activation defined, or used, termination_event_fd keeps
its -1 default value.

Close the eventfd outside the while loop so retries can happen if
the write fails for some reason.

ref[1]: https://www.freedesktop.org/software/systemd/man/latest/sd_listen_fds.html

Reviewed by: Vladislav Vaintroub

Remove alarm() remnants

- Replace thread-unsafe use of alarm() inside my_lock.c with a
  timed loop.
- Remove configure time checks
- Remove mysys my_alarm.c/my_alarm.h

This allows to simplify net_real_read() and net_real_write() a bit.

Removed some superfluous #ifdef/ifndef MYSQL_SERVER from net_serv.cc
The code always runs in server, either normal or embedded.
Dead code for switching socket between blocking and non-blocking modes,
is also removed.

Removed pthread_kill() with alarm signal that woke up main thread on
server shutdown. Used shutdown(2) on polling sockets instead, to the same
effect.

Removed yet another superstitious pthread_kill(), that ran on non-Windows
in terminate_slave_thread().

Use ICU to work with timezones, to retrieve current timezone name,
abbreviation, and offset from GMT. However in case TZ environment variable
is used to set timezone, and ICU does not have corresponding one,
C runtime functions will be used.

Moved some of timezone handling to mysys.
Added unit tests.

After b8f9f796 Format_description_log_event constructor
requires binlog checksum algorithm or BINLOG_CHECKSUM_ALG_OFF
for Galera also.

Thanks to Kristian Nielsen <knielsen@knielsen-hq.org>
for pointing a fix.

INSERT IGNORE had a pecular undocumented case that when one row was
inserted, there was an error rather than a warning.

As LOAD DATA IGNORE, UPDATE IGNORE, INSERT IGNORE SELECT, and INSERT
IGNORE VALUES (single row, for foreign key violation) all behave the same
way with a warning lets keep the behaviour normalized.

In compatibility, previously a error was generated, now a warning is
generated.

This behaviour is now consistent with MySQL-8.0 too.

and to related methods and their parameters:
- The return value of Spvar_definition::m_column_type_ref()
- The parameter of Spvar_definition::set_column_type_ref()
- The method Qualified_column_ident::resolve_type_ref()
- The parameter of LEX::sp_variable_declarations_column_type_finalize()

The reason was that Event e11 was re-executed before
"ALTER EVENT e11 DISABLE" had been executed.

Fixed by increasing re-schedule time

Other things:
- Removed double accounting of 'execution_count'. It was incremented in
  top->mark_last_executed(thd) that was executed a few lines earlier.

MDEV-32441 SENT_ROWS shows random wrong values when stored function
           is selected.
MDEV-32281 EXAMINED_ROWS is not populated in
           information_schema.processlist upon SELECT.

Added ROWS_SENT to information_schema.processlist
This is to have the same information as Percona server (SENT_ROWS)

To ensure that information_schema.processlist has correct values for
sent_rows and examined_rows I introduced two new variables to hold the
total counts so far. This was needed as stored functions and stored
procedures will reset the normal counters to be able to count rows for
each statement individually for slow query log.

Other things:
- Selects with functions shows in processlist the total examined_rows
  and sent_rows by the main statement and all functions.
- Stored procedures shows in processlist examined_rows and sent_rows
  per stored procedure statement.
- Fixed some double accounting for sent_rows and examined_rows.
- HANDLER operations now also supports send_rows and examined_rows.
- Display sizes for MEMORY_USED, MAX_MEMORY_USED, EXAMINED_ROWS and
  QUERY_ID in information_schema.processlist changed to 10 characters.
- EXAMINED_ROWS and SENT_ROWS changed to bigint.
- INSERT RETURNING and DELETE RETURNING now updates SENT_ROWS.
- As thd is always up to date with examined_rows, we do not need
  to handle examined row counting for unions or filesort.
- I renamed SORT_INFO::examined_rows to m_examined_rows to ensure that
  we don't get bugs in merges that tries to use examined_rows.
- Removed calls of type "thd->set_examined_row_count(0)" as they are
  not needed anymore.
- Removed JOIN::join_examined_rows
- Removed not used functions:
  THD::set_examined_row_count()
- Made inline some functions that where called for each row.

Compute binlog checksums (when enabled) already when writing events
into the statement or transaction caches, where before it was done
when the caches are copied to the real binlog file. This moves the
checksum computation outside of holding LOCK_log, improving
scalabitily.

At stmt/trx cache write time, the final end_log_pos values are not
known, so with this patch these will be set to 0. Events that are
written directly to the binlog file (not through stmt/trx cache) keep
the correct end_log_pos value. The GTID and COMMIT/XID events at the
start and end of event groups are written directly, so the zero
end_log_pos is only for events in the middle of event groups, which
do not negatively affect replication.

An option --binlog-legacy-event-pos, off by default, is provided to
disable this behavior to provide backwards compatibility with any
external applications that might rely on end_log_pos in events in the
middle of event groups.

Checksums cannot be pre-computed when binlog encryption is enabled, as
encryption relies on correct end_log_pos to provide part of the
nonce/IV.

Checksum pre-computation is also disabled for WSREP/Galera, as it uses
events differently in its write-sets and so on. Extending pre-computation of
checksums to Galera where it makes sense could be added in a future patch.

The current --binlog-checksum configuration is saved in
binlog_cache_data at transaction start and used to pre-compute
checksums in cache, if applicable. When the cache is later copied to
the binlog, a check is made if the saved value still matches the
configured global value; if so, the events are block-copied directly
into the binlog file. If --binlog-checksum was changed during the
transaction, events are re-written to the binlog file one-by-one and
the checksums recomputed/discarded as appropriate.
Reviewed-by: Monty <monty@mariadb.org>
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

MDEV-27106 added REMOTE_TABLE, REMOTE_DATABASE, REMOTE_SERVER spider
table options. In this commit, we add all remaining options for table
params that are not marked to be deprecated.

All these options are parsed as strings from sql statements and have
string values at the sql level, so that we can determine whether it is
specified by checking its nullness.

The string values are further parsed by Spider into their actual types
in the SPIDER_SHARE, including string list, bounded nonnegative int,
bounded nonnegative int list, nonnegative longlong, boolean, and key
hints. Except for string lists, all other types are validated during
this parsing process.

Most of the options are backward compatible, i.e. they accept any
values that is accepted by there corresponding param parser. The only
exception is the index hint IDX which corresponds to the idxNNN param
name. For example,

'idx000 "f PRIMARY", idx001 "u k1"'

translates to

IDX="f PRIMARY u k1".

We include a test with all options specified, and tests involving
spider table options of all actual types.

Any table options, if present, will cause comments to be ignored with
a warning. The warning can be disabled by setting a new spider
global/session system variable spider_suppress_comment_ignored_warning
to 1.

Another global/session variable introduced is spider_ignore_comments,
which if set to 1, will cause COMMENT and CONNECTION strings to be
ignored unconditionally, whether or not table options are specified.

- Introduced the variable "innodb_truncate_temporary_tablespace_now"
to shrink the temporary tablespace.

Steps for shrinking the temporary tablespace:

1) Find the last used extent in temporary tablespace
by iterating through the BITMAP in extent descriptor pages

2) If the last used extent is lesser than user specified size
then set desired target size to user specified size

3) Store the page contents of "to be modified" extent
descriptor pages, latches the "to be modified" extent
descriptor pages and check for buffer pool memory availability

4) Update the FSP_SIZE and FSP_FREE_LIMIT in header page

5) Remove the "to be truncated" pages from FSP_FREE and
FSP_FREE_FRAG list

6) Reset the bitmap in the last descriptor pages for the
"to be truncated" pages.

7) Clear the freed range in temporary tablespace which
are to be truncated.

8) Evict the "to be truncated" temporary tablespace pages
from LRU list.

9) In case of multiple files, calculate the truncated last
file size and do truncation in last file

10) Commit the mini-transaction for shrinking the tablespace

Preparatory patch for pre-computing binlog checksums outside of holding
LOCK_log.

The existing code for MYSQL_BIN_LOG::write_cache() was needlessly complex
and very hard to understand and modify for handling the new case where
pre-computed checksums are already present in the IO_CACHE.

Greatly simplify the logic by replacing the (implicit) state machine with
direct code that pulls the events one by one from the IO_CACHE. This removes
a lot of state flags and avoids duplicate code for handling full vs. split
headers.

This also removes the need for the CacheWriter class. As a bonus, this fixes
the bug that CacheWriter::write() was completely ignoring write errors. No
other functional changes are done with this patch, only code cleanup.
Reviewed-by: Monty <monty@mariadb.org>
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

This is a preparatory commit for pre-computing checksums outside of
holding LOCK_log, no functional changes.

Which checksum algorithm is used (if any) when writing an event does not
belong in the event, it is a property of the log being written to.

Instead decide the checksum algorithm when constructing the
Log_event_writer object, and store it there.

Introduce a client-only Log_event::read_checksum_alg to be able to
print the checksum read, and a
Format_description_log_event::source_checksum_alg which is the
checksum algorithm (if any) to use when reading events from a log.

Also eliminate some redundant `enum` keywords on the enum_binlog_checksum_alg
type.
Reviewed-by: Monty <monty@mariadb.org>
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

This is a preparatory patch for precomputing binlog checksums outside
of holding LOCK_log, no functional changes.

Replace Log_event::writer with just passing the writer object as a
function parameter to Log_event::write().

This is mainly for code clarity. Having to set ev->writer before every
call to ev->write() is error-prone (what if it's forgotten in some
code place?), while passing it as parameter as usual makes it explicit
how the dataflow is.

As a minor point, it also improves the code, as the compiler now can
save the function parameter in a register across nested calls (when it
is a class member, compiler needs to reload across nested calls in
case the object would be modified during the call).
Reviewed-by: Monty <monty@mariadb.org>
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

Like sql_mode, we factor out of ON_CHECK function for export, to be
used in get_options() during server startup, for validation of
--redirect_url value.

Adding a global/session var `redirect_url' of string type. The initial
value is empty. Can be supplied in mysqld with --redirect-url or set
in --init-connect. A valid redirect_url should be of the format

{mysql,mariadb}://host[:port]

where <host> is an arbitrary string not containing colons, and <port>
is a number between 0 and 65535 inclusive.

The variable will be used by the server to notify clients that they
should connect to another server, specified by the value of the
variable, if not empty.

The notification is done by the inclusion of the variable in
session_track_system_variable.

Merge sys_var_charptr with sys_var_charptr_base, as well as merge
Sys_var_session_lexstring into Sys_var_lexstring. Also refactored
update methods of sys_var_charptr accordingly.

Because the class is more generic, session_update() calls
sys_var_charptr::session_update() which does not assume a buffer field
associated with THD, but instead call strdup/free, we get rid of
THD::default_master_connection_buff accordingly. This also makes THD
smaller by ~192 bytes, and there can be many thousands of concurrent
THDs.

New Feature:
============
This patch extends the START SLAVE UNTIL command with options
SQL_BEFORE_GTIDS and SQL_AFTER_GTIDS to allow user control of
whether the replica stops before or after a provided GTID state. Its
syntax is:

START SLAVE UNTIL (SQL_BEFORE_GTIDS|SQL_AFTER_GTIDS)=”<gtid_list>”

When providing SQL_BEFORE_GTIDS=”<gtid_list>”, for each domain
specified in the gtid_list, the replica will execute transactions up
to the GTID found, and immediately stop processing events in that
domain (without executing the transaction of the specified GTID).
Once all domains have stopped, the replica will stop. Events
originating from domains that are not specified in the list are not
replicated.

START SLAVE UNTIL SQL_AFTER_GTIDS=”<gtid_list>” is an alias to the
default behavior of START SLAVE UNTIL master_gtid_pos=”<gtid_list>”.
That is, the replica will only execute transactions originating from
domain ids provided in the list, and will stop once all transactions
provided in the UNTIL list have all been executed.

Example:
=========
If a primary server has a binary log consisting of the following GTIDs:

0-1-1
1-1-1
0-1-2
1-1-2
0-1-3
1-1-3

If a fresh replica (i.e. one with an empty GTID position,
@@gtid_slave_pos='') is started with SQL_BEFORE_GTIDS, i.e.

START SLAVE UNTIL SQL_BEFORE_GTIDS=”1-1-2”

The resulting gtid_slave_pos of the replica will be “1-1-1”.
This is because the replica will execute only events from domain 1
until it sees the transaction with sequence number 2, and
immediately stop without executing it.

If the replica is started with SQL_AFTER_GTIDS, i.e.

START SLAVE UNTIL SQL_AFTER_GTIDS=”1-1-2”

then the resulting gtid_slave_pos of the replica will be “1-1-2”.
This is because it will only execute events from domain 1 until it
has executed the provided GTID.

Reviewed By:
============
Kristian Nielson <knielsen@knielsen-hq.org>

utf8mb4_general_ci has been outdated for a while and contained loosely
standardized collations.

UCA-14.0.0 has a more defined collation with multiple benefit that new
users may not immediately consider, or may assume to be default.

By defining default collation for utf8mb4 to be uc1400_ai_ci newly
created tables will have a modern standard collation.

In MemorySanitizer builds of 10.10 and 10.11, we would rather often
have the assertion fail in innodb_init() during mariadb-backup --prepare.
The assertion could also fail during InnoDB startup, but less often.

Before commit 685d958e in 10.8 the
log file cleanup after a successfully applied backup is different,
and the os_aio_pending_writes() assertion is in srv0start.cc.

IORequest::write_complete(): Invoke node->complete_write() before
releasing the page latch, so that a log checkpoint that is about to
execute concurrently will not miss a fdatasync() or fsync() on the
file, in case this was the first write since the last such call.

create_log_file(), srv_start(): Replace the debug assertion with
a debug check. For all intents and purposes, all writes could have
been completed but some write_io_callback() may not have invoked
io_slots::release() yet.