sprintf() format of double changed from '%lg' to '%-.11lg'

The change was to make it easier to read optimizer trace output
with tables that has millions of records.

The reason for this is to make all temporary file names similar and
also to be able to figure out from where a #sql-xxx name orginates.

New format is for most cases:
'#sql-name-current_pid-thread_id[-increment]'
Where name is one of subselect, alter, exchange, temptable or backup

The exceptions are:
ALTER PARTITION shadow files:
'#sql-shadow-thread_id-'original_table_name'

Names used with temp pool:
'#sql-name-current_pid-pool_number'

MDEV-22088 S3 partitioning support

All ALTER PARTITION commands should now work on S3 tables except

REBUILD PARTITION
TRUNCATE PARTITION
REORGANIZE PARTITION

In addition, PARTIONED S3 TABLES can also be replicated.
This is achived by storing the partition tables .frm and .par file on S3
for partitioned shared (S3) tables.

The discovery methods are enchanced by allowing engines that supports
discovery to also support of the partitioned tables .frm and .par file

Things in more detail

- The .frm and .par files of partitioned tables are stored in S3 and kept
  in sync.
- Added hton callback create_partitioning_metadata to inform handler
  that metadata for a partitoned file has changed
- Added back handler::discover_check_version() to be able to check if
  a table's or a part table's definition has changed.
- Added handler::check_if_updates_are_ignored(). Needed for partitioning.
- Renamed rebind() -> rebind_psi(), as it was before.
- Changed CHF_xxx hadnler flags to an enum
- Changed some checks from using table->file->ht to use
  table->file->partition_ht() to get discovery to work with partitioning.
- If TABLE_SHARE::init_from_binary_frm_image() fails, ensure that we
  don't leave any .frm or .par files around.
- Fixed that writefrm() doesn't leave unusable .frm files around
- Appended extension to path for writefrm() to be able to reuse to function
  for creating .par files.
- Added DBUG_PUSH("") to a a few functions that caused a lot of not
  critical tracing.

MDEV-22199 Add VISIBLE attribute for indexes in CREATE TABLE

This was done to make it easier to read in dumps from MySQL 8.0 generated
with MySQL workbench

This was done to increase performance when not using temporary tables
as checking if a table is a temporary table involves a lot of code.

Other things:
- Handler errors from ha_maria::implict_commit
- Disable DBUG in safe_mutex_lock to get trace file easier to read

- MDEV-22062 Assertion `!table->file->keyread_enabled()' failed in
  close_thread_table()
- MDEV-22077 table->no_keyread .. failed in join_read_first()
- MDEV-22237 Assertion `!table->file->keyread_enabled()' failed in
  handler::ha_reset on DELETE

              >= M_TOT_PARTS' FAILED.

This patch is taken from MySQL, originally written by Mattias Jonsson
Here follows the original commit message:

Problem in handle_alter_part_error(),
result in altered partition_info object was still used
if table was under LOCK TABLES.

Solution was to always close and destroy all table
and table_share instances if exclusive mdl lock was
possible.
If not succeeding in get an exlusive lock (only possible
during rollback of DDL), at least close and destroy this
table instance.

rb#7361.
Approved by Mikael and Aditya.

With these in place Lintian will pass without errors (only warnings) and
the CI systems will at least be able to stop anybody from introducing
new Lintian errors.

Fix Lintian complaints on:
 P: maintainer-script-without-set-e
 W: mariadb-server-10.5: script-not-executable usr/share/mysql/mysql.init
 W: mariadb-client-10.5: incorrect-path-for-interpreter usr/bin/innotop
    (#!/usr/bin/env perl != /usr/bin/perl)

Manages the security risk in way that also fixes Lintian warning:
  W: mariadb-server-10.5: setuid-binary
  usr/lib/mysql/plugin/auth_pam_tool_dir/auth_pam_tool 4755 root/root

Applied downstream in
https://salsa.debian.org/mariadb-team/mariadb-10.4/-/commit/9605a48a99a61f439698676da192799de45d732a

There are still some differences in due to systemd and service triggers,
but these differences make sense and are likely to be synced the other
way later so that downstream Debian official packaging adopts them.

The synced changes include among others:
- https://salsa.debian.org/mariadb-team/mariadb-10.4/-/commit/9f49e4b494f3dad8c403972996f7a1ebceb4b34f
- https://salsa.debian.org/mariadb-team/mariadb-10.4/-/commit/6440c0d6e75aeda134f4f942eee4fbfd62b1ce20
- https://salsa.debian.org/mariadb-team/mariadb-10.4/-/commit/6e5ee72d647744bf84c42e02c3dfe888912e739e
- https://salsa.debian.org/mariadb-team/mariadb-10.4/-/commit/e62e67ae4b7679c43e80a20ab3ba3c51f01f0714
- https://salsa.debian.org/mariadb-team/mariadb-10.4/-/commit/df2415a53d5604755768f00aad0c640df1ad815c
- https://salsa.debian.org/mariadb-team/mariadb-10.4/-/commit/643558da74990f8648b3c86039e253dbae337c84

These changes are in line with what has been changes in downstream Debian
and they fix the following errors:

 $ check_for_missing_breaks_replaces.py -o ${WORKING_DIR}/missing_breaks.xml --changes-file ${WORKING_DIR}/*.changes
 [ERROR] mariadb-client-10.5 conflicts with mytop files: {'/usr/bin/mytop'}
 [ERROR] mariadb-client-10.5 conflicts with mariadb-client-core-10.3 files: {'/usr/bin/mysqlcheck'}
 [ERROR] mariadb-server-10.5 conflicts with mariadb-server-core-10.3 files: {'/usr/share/man/man1/resolveip.1.gz', '/usr/bin/resolveip', '/usr/share/mysql/mysql_test_data_timezone.sql'}
 [ERROR] mariadb-client-10.5 conflicts with mysql-client-core-5.7 files: {'/usr/bin/mysqlcheck'} [ERROR] mariadb-server-10.5 conflicts with mariadb-server-core-10.3 files: {'/usr/share/man/man1/resolveip.1.gz', '/usr/bin/resolveip', '/usr/share/mysql/mysql_test_data_timezone.sql'}
 [ERROR] mariadb-server-10.5 conflicts with mysql-client-5.7 files: {'/usr/bin/myisam_ftdump', '/usr/share/man/man1/myisam_ftdump.1.gz'}
 [ERROR] mariadb-server-core-10.5 conflicts with mysql-server-5.7 files: {'/usr/share/mysql/mysql_test_data_timezone.sql', '/usr/bin/resolveip', '/usr/share/man/man1/resolveip.1.gz'}
 [ERROR] mariadb-test conflicts with mysql-server-5.7 files: {'/usr/lib/mysql/plugin/adt_null.so', '/usr/lib/mysql/plugin/mypluglib.so'}

- Ensure cmake builds also apply CPPFLAGS flags for hardening to fully work
  https://salsa.debian.org/mariadb-team/mariadb-10.4/-/commit/fc4f33cf40d0a10ef5d1992accd2af734ba96356
- Install Ubuntu Apport files in the same way with same filename.
- MDEV-21705: Build flags to keep WolfSSL from crashing
- Clean away sql-bench from packaging immediately after build step.
- Delete private files from libraries so they don't get shipped in the -dev
  packages.

- Make full contents, also header files, explicit in -dev packages
  https://salsa.debian.org/mariadb-team/mariadb-10.4/-/commit/fb77df0c75422449c32ab1f5f8650a7e9e91091f
- Don't ship private header files at all.
- Ship both libmariadb.pc and maridb.pc.
- Ship mysql_config only as a symlink.
- Keep 'mariadbcheck' as symlink as somebody could have used it already
  https://salsa.debian.org/mariadb-team/mariadb-10.3/commit/d0466f74d543ac574dcd832e081fdc18538f3edd
- The mariadb.pc is the server pkg-config, ship in libmariadbd-dev.
- The client pkg-config is called libmariadb.pc. Ship only it in the client
  library packages.

Use versioned breaks/replaces for mysql* packages we also provide and fix:

  [ERROR] libmariadb-dev conflicts with libmysqlclient-dev files:
    {'/usr/share/man/man1/mysql_config.1.gz', '/usr/share/aclocal/mysql.m4'}
  [ERROR] libmariadb-dev conflicts with libmysqld-dev files:
    {'/usr/lib/x86_64-linux-gnu/libmysqlservices.a'}

- Remove versioned depends that are far outdated and not relevant anymore
  in 2020.
- Link with libedit instead of unmaintained readline5
  https://salsa.debian.org/mariadb-team/mariadb-10.4/-/commit/d8796a0e3b5fc4450f4dc80fd159599a7f03997a
- Add [linux-any] for Linux specific depends and other smaller control
  file tweaks.
- Update .travis-ci.yml to match.
- Also fixes the Debian autopkgtest detected error:
    ERROR 1231 (42000) at line 1: Variable 'innodb_compression_algorithm'
    can't be set to the value of 'lz4'

Drop Aliases from the service file directive. Aliases in the service
file only take effect when the service is enabled. This is the case
because Aliases in service files do not have to be unique across various
services.

Shipping symlinks guarantees that one can always enable mariadb service
using `systemctl enable mysql` or `systemctl enable mysqld` and makes
the commands indempotent.

Post change of 8447edb7, --plugin-dir
is set in the default my.cnf file under [client]. The value is automatically
detected during mtr startup.

MDEV-20502 Queries against spider tables return wrong values for columns following constant declarations.

When executing a query like "select id, 0 as const, val from ...", there are 3 columns(items) in Query->select at handlerton->create_group_by(). After that, MariaDB makes a temporary table with 2 columns. The skipped items are const item, so fixing Spider to skip const items for items at Query->select.

- On Windows, do not handle lack of SeLockMemory privilege as fatal error.
Just like on any other platform, there is a fallback to ordinary pages.
It is better than server that does not start, silently.

- On Windows, remove incorrect  irritating "fallback to conventional pages failed"
from the warning, when allocating large pages fails.

The test innodb.innodb_wl6326 that had been disabled in 10.4 due to
MDEV-21535 is failing on 10.5 due to a different reason: the removal
of the MLOG_COMP_END_COPY_CREATED operations in MDEV-12353
commit 276f996a caused PAGE_LAST_INSERT
to be set to something nonzero by the function page_copy_rec_list_end().

This in turn would cause btr_page_get_split_rec_to_right() to behave
differently: we would not attempt to split the page at all, but simply
insert the new record into the new, empty, right leaf page.

Even though the change reduced the sizes of some tables, it is better
to aim for balanced trees.

page_copy_rec_list_end(), PageBulk::finishPage():
Preserve PAGE_LAST_INSERT, PAGE_N_DIRECTION, PAGE_DIRECTION.

PageBulk::finish(): Move some common code from PageBulk::finishPage().

Without this, the mtr will fail to load client plugins and the following
tests would fail:

- main.plugin_auth_qa_2
- main.plugin_auth_qa_3
- main.plugin_auth
- plugins.multiauth
- plugins.auth_ed25519

When testing installed packages, the client (mariadb or mysql) can have
plugins stored in a separate location compared to server. This change
ensures that the correct paths are used for the plugin-dir directive in
the my.cnf generated by mysql-test-run.

The directives in my.cnf will be replaced like so:

[client]
plugin-dir=$client_plugindir

[mysqld]
plugin-dir=$plugindir

where $<variable-name> corresponds to the variable name in mtr.

plugin-dir is provided via --defaults-file=<config-file> already part of
the exec call.

As this was missing, the libmariadb/install.cmake section for Debian
builds was not triggered at all, and existing fixes to make the
Debian paths correct did not apply.

Update libmariadb3 paths in Debian packaging after MariaDB Connector C
3.1.8 upgrade

Also now include caching_sha2_password.so which was previously omitted.

Keep old libmariadbclient.a as symbolic link for backwards compatibility.

Closes: CONC-304
Related: CONC-456, MDEV-22150

MDEV-22218 InnoDB: Failing assertion: node->pcur->rel_pos == BTR_PCUR_ON upon LOAD DATA with NO_BACKSLASH_ESCAPES in SQL_MODE and unique blob in table

`inited == NONE` at the initialization time does not always mean
that it'll be `NONE` later, at the execution time. Use a more complex
caller-specific logic to decide whether to create a cloned lookup handler.

Besides LOAD (as in the original bug report) make sure that all
prepare_for_insert() invocations are covered by tests. Add tests for
CREATE ... SELECT, multi-UPDATE, and multi-DELETE.

Don't enable write cache with long uniques.

CMake parameter

This includes compiler-specific dependencies
- tiny part of CRT, mostly memcpy/memset in msvcr140.dll
- exception handling in msvc140_1.dll
- C++ standard library in msvcp140
not much else

CMake is smart enough to copy the dependencies into the bin directory
itself.

Stop linking plugins to the server executable on Windows.
Instead, extract whole server functionality into a large DLL, called
server.dll. Link both plugins, and small server "stub" exe to it.

This eliminates plugin dependency on the name of the server executable.
It also reduces the size of the packages (since tiny mysqld.exe
and mariadbd.exe are now both linked to one big DLL)

Also, simplify the functionality of exporing all symbols from selected
static libraries. Rely on WINDOWS_EXPORT_ALL_SYMBOLS, rather than old
self-backed solution.

fix compile error

replace GetProcAddress(GetModuleHandle(NULL), "variable_name")
for server exported data with actual variable names.

Runtime loading was never required,was error prone
, since symbols could be missing at runtime, and now it actually failed,
because we do not export symbols from executable anymore, but from a shared
library

This did require a MYSQL_PLUGIN_IMPORT decoration for the plugin,
but made the code more straightforward, and avoids missing symbols at
runtime (as mentioned before).

The audit plugin is still doing some dynamic loading, as it aims to work
cross-version. Now it won't work cross-version on Windows, as it already
uses some symbols that are *not* dynamically loaded, e.g fn_format
and those symbols now exported from server.dll , when earlier they were
exported by mysqld.exe

Windows, fixes for storage engine plugin loading
after various rebranding stuff

Create server.dll containing functionality of the whole server
make mariadbd.exe/mysqld.exe a stub that is only  calling mysqld_main()

fix build

This broke clang-cl, apparently when compiling version resource file with
RC tool, because RC would not aceept -Wxxx at all.

- InnoDB is doing un-necessary redo log page initialisation during
recovery and unnecessary traversal of redo log during last phase.
This patch does the optimization of removing unnecessary redo log page
initialisation and detects the memory exhaust earlier.