* mariadb-service-convert to use mariadbd-safe
* galera_recovery to use mariadbd
* mtr - wsrep use mariadb executables
* debian/mariadb-server.mariadb.init use mariadbd-safe
* debian/tests/smoke uses mariadb instead of mysql as client.

Co-Author: Daniel Black <daniel@mariadb.org>

move mariadb_capi_rename.h out of private server headers,
because it's included by mysql.h which is not private.

custom code in `case OPT_SKIP_RESOLVE`
was overriding the correct value from handle_options().

covers `git commit -a` and `git citool` (`git add`, `git status`, etc).
they can still be added explicitly

the parser couldn't parse `1=2 not between 3 and 5`
after `2` it expected only NOT2_SYM, but not NOT_SYM
(visible from the sql_yacc.output file), which resulted in
Syntax error ... near 'not between 3 and 4'

The parser was confused by a rather low NOT_SYM precedence and
%prec BETWEEN_SYM didn't resolve this confusion.

As a fix, let's remove any %precedence from NOT_SYM and
specify %prec explicitly in the only place where it matters for NOT_SYM.

In other places, such as for NOT BETWEEN, NOT_SYM won't have a
precedence, so bison won't be confused about it.

Without Galera, mariabackup should ignore the --galera-info option
and not fail with rc != 0 like it does now. This commit fixes this flaw.

  * Avoid some pessimization
  * Slightly smaller upgrade dataset
  * Simplify vers_row_same_trx() and its caller

1. In case of system-versioned table add row_end into FTS_DOC_ID index
   in fts_create_common_tables() and innobase_create_key_defs().
   fts_n_uniq() returns 1 or 2 depending on whether the table is
   system-versioned.

   After this patch recreate of FTS_DOC_ID index is required for
   existing system-versioned tables. If you see this message in error
   log or server warnings: "InnoDB: Table db/t1 contains 2 indexes
   inside InnoDB, which is different from the number of indexes 1
   defined in the MariaDB" use this command to fix the table:

      ALTER TABLE db.t1 FORCE;

2. Fix duplicate history for secondary unique index like it was done
   in MDEV-23644 for clustered index (932ec586). In case of
   existing history row which conflicts with currently inseted row we
   check in row_ins_scan_sec_index_for_duplicate() whether that row
   was inserted as part of current transaction. In that case we
   indicate with DB_FOREIGN_DUPLICATE_KEY that new history row is not
   needed and should be silently skipped.

3. Some parts of MDEV-21138 (7410ff43) reverted. Skipping of
   FTS_DOC_ID index for history rows made problems with purge
   system. Now this is fixed differently by p.2.

4. wait_all_purged.inc checks that we didn't affect non-history rows
   so they are deleted and purged correctly.

Additional FTS fixes

  fts_init_get_doc_id(): exclude history rows from max_doc_id
  calculation. fts_init_get_doc_id() callback is used only for crash
  recovery.

  fts_add_doc_by_id(): set max value for row_end field.

  fts_read_stopword(): stopwords table can be system-versioned too. We
  now read stopwords only for current data.

  row_insert_for_mysql(): exclude history rows from doc_id validation.

  row_merge_read_clustered_index(): exclude history_rows from doc_id
  processing.

  fts_load_user_stopword(): for versioned table retrieve row_end field
  and skip history rows. For non-versioned table we retrieve 'value'
  field twice (just for uniformity).

FTS tests for System Versioning now include maybe_versioning.inc which
adds 3 combinations:

'vers'     for debug build sets sysvers_force and
	   sysvers_hide. sysvers_force makes every created table
	   system-versioned, sysvers_hide hides WITH SYSTEM VERSIONING
	   for SHOW CREATE.

	   Note: basic.test, stopword.test and versioning.test do not
	   require debug for 'vers' combination. This is controlled by
	   $modify_create_table in maybe_versioning.inc and these
	   tests run WITH SYSTEM VERSIONING explicitly which allows to
	   test 'vers' combination on non-debug builds.

'vers_trx' like 'vers' sets sysvers_force_trx and sysvers_hide. That
	   tests FTS with trx_id-based System Versioning.

'orig' 	   works like before: no System Versioning is added, no debug is
	   required.

Upgrade/downgrade test for System Versioning is done by
innodb_fts.versioning. It has 2 combinations:

'prepare' makes binaries in std_data (requires old server and OLD_BINDIR).
	  It tests upgrade/downgrade against old server as well.

'upgrade' tests upgrade against binaries in std_data.

Cleanups:

Removed innodb-fts-stopword.test as it duplicates stopword.test

Adds new parameter $restart_bindir for restart_mysqld.inc.

Example:

let $restart_bindir= /home/midenok/src/mariadb/10.3b/build;
--source include/restart_mysqld.inc

It is good to return back original server before check_mysqld will be
run at the test end:

let $restart_bindir=;
--source include/restart_mysqld.inc

Works like vers_force but forces trx_id-based system-versioned tables
if the storage supports it (currently InnoDB-only). Otherwise creates
timestamp-based system-versioned table.

wait until all three concurrent statements are truly completely
finished before quering P_S.

In particular "Logging slow query" stage happens after sending the
OK packet but before the statement appears in events_statements_history

MDEV-29852 SIGSEGV in mysql_create_routine or is_acl_user on 2nd execution, ASAN use-after-poison in get_current_user (sql_acl.cc)

if lex->definer is replaced, take care to restore it at the
end of PS EXECUTE

ha_spider::create(): Pass the correct length of the argument of the
CHARSET attribute. The macro STRING_WITH_LEN() is intended to be used
with NUL terminated string constants only. Here, it would incorrectly
pass sizeof(char*)-1 as the length.

The incorrect type handler caused an incorrect result_type() for
Item_cache_row (STRING_RESULT rather than ROW_RESULT). By updating the
constructor of Item_cache_row with the correct type handler, it fixes
this problem.
Signed-off-by: Yuchen Pei <yuchen.pei@mariadb.com>
Reviewed-by: Sergei Golubchik <serg@mariadb.com>

io_watching_thread(): Declare as a detachable thread, similar to
log_copying_thread().

stop_backup_threads(): Wait for both log_copying_thread and
io_watching_thread to clear their flags. Expect log_sys.mutex
to be held by the caller.

xtrabackup_backup_func(): Initialize log_copying_stop before
creating io_watching_thread. This prevents a race condition
where io_watching_thread() could wait on the condition variable
before it had been fully initialized. This race condition would
cause a hang in the GNU libc implementation of pthread_cond_destroy()
at the end of stop_backup_threads().

This race condition was introduced in
commit 38fd7b7d (MDEV-21452).

When trying to create a spider table with banned charsets including
utf32, utf16, ucs2 and utf16le[1], spider should emit an error
immediately, rather than wait until a separate statement that
establishes a connection (e.g. SELECT). This also applies to ALTER
TABLE statement that changes charsets.

[1] https://mariadb.com/kb/en/server-system-variables/#character_set_clientSigned-off-by: Yuchen Pei <yuchen.pei@mariadb.com>
Reviewed-by: Nayuta Yanagisawa <nayuta.yanagisawa@mariadb.com>

Sanitizer tests were introduced in 617f45b for GitLab CI, but started
failing on latest Fedora version with error:

    $ yum install -y /usr/lib64/libasan.so.6.0.0 /usr/lib64/libtsan.so.0.0.0 /usr/lib64/libubsan.so.1.0.0
    Last metadata expiration check: 0:00:51 ago on Fri Dec  9 20:05:01 2022.
    No match for argument: /usr/lib64/libasan.so.6.0.0
    No match for argument: /usr/lib64/libtsan.so.0.0.0
    Error: Unable to find a match: /usr/lib64/libasan.so.6.0.0 /usr/lib64/libtsan.so.0.0.0

The reason for using specific library versions is unknown. Switch to
simply using latest package versions, as is works and is likely to work
best in the long run.

Also, enclose "../rpmlist-$CI_JOB_NAME-$CI_COMMIT_REF_SLUG.log" in
quotes to avoid `ambiguous redirect` error when $CI_JOB_NAME has spaces.

Additionally use "needs" statements to allow tests to run immediately
after dependent jobs passed instead of waiting for the full stage to
complete.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the BSD-new
license. I am contributing on behalf of my employer Amazon Web Services

Before the fix next-key lock was requested only if a record was
delete-marked for locking unique search in RR isolation level.
There can be several delete-marked records for the same unique key,
that's why InnoDB scans the records until eighter non-delete-marked record
is reached or all delete-marked records with the same unique key are
scanned.

For range scan next-key locks are used for RR to protect scanned range from
inserting new records by other transactions. And this is the reason of why
next-key locks are used for delete-marked records for unique searches.

If a record is not delete-marked, the requested lock type was "not-gap".
When a record is not delete-marked during lock request by trx 1, and
some other transaction holds conflicting lock, trx 1 creates waiting
not-gap lock on the record and suspends. During trx 1 suspending the
record can be delete-marked. And when the lock is granted on conflicting
transaction commit or rollback, its type is still "not-gap". So we have
"not-gap" lock on delete-marked record for RR. And this let some other
transaction to insert some record with the same unique key when trx 1 is
not committed, what can cause isolation level violation.

The fix is to set next-key locks for both delete-marked and
non-delete-marked records for unique search in RR.

When trying to create a spider table with banned charsets including
utf32, utf16, ucs2 and utf16le[1], spider should emit an error
immediately, rather than wait until a separate statement that
establishes a connection (e.g. SELECT). This also applies to ALTER
TABLE statement that changes charsets.

[1] https://mariadb.com/kb/en/server-system-variables/#character_set_clientSigned-off-by: Yuchen Pei <yuchen.pei@mariadb.com>
Reviewed-by: Nayuta Yanagisawa <nayuta.yanagisawa@mariadb.com>

The variable was not really being used for anything. The parameters
innodb_read_io_threads, innodb_write_io_threads have replaced
innodb_file_io_threads.

Excluded one case from view-protocol in gis.test

An unfortunate change to the default behavior of the handling of
core dumps was implemented in
commit e9be5428
by making MTR_PRINT_CORE=small the default value, that is, to
only display the stack trace of one thread in crash reports.
Many if not most failures that occur in regression tests are
sporadic and involve race conditions or deadlocks. To be able to
analyze such failures, having the stack traces of all active threads
is a must, because CI environments typically do not save any core dumps.

While the environment variable MTR_PRINT_CORE could be set in CI
environments to compensate for the unfortunate change, it is better
to revert to the old default (dumping all threads) so that
no explicit action will be required from maintainers of independent
CI systems. In that case, if something fails once in a blue moon,
we can have some hope of diagnosing it based on the output.

We fix this regression by defaulting the unset environment variable
MTR_PRINT_CORE to "medium".

I have not been able to repeat the problem, but the stack trace indicates
that ha_maria::extra() is called with a null file pointer.

This indicates the table has either never been opened or opened and closed,
with file pointer set to NULL, but ha_maria::extra() is still called.

In JOIN::partial_cleanup() we are only checking of table->is_created(),
which will fail if table was created and later closed.

Fixed by clearing table->created if table is dropped.

I added an assert to is_created() to catch the case that the create
flag does not match 'file'.

commit_try_rebuild(): Only invoke trx_t::drop_table_statistics()
if both InnoDB statistics tables are accessible (and exclusively
locked by the current transaction). This avoids a crash due to
ut_a(sym_node->table != NULL) in pars_retrieve_table_def().

The crash was repeated on a partial copy of a MariaDB 10.3 data
directory that lacked the *.ibd files for the statistics tables.

fil_space_t::flush<false>(): If the CLOSING flag is set,
the file may already have been closed, resulting in EBADF
being returned by fdatasync(). In any case, the
thread that had set the flag should take care of invoking
os_file_flush_func().

The crash occurred during the execution of FLUSH TABLES...FOR EXPORT.

Tested by: Matthias Leich

Some tests drop the default mtr database "test". This may fail due
to the directory not being empty. InnoDB may not delete all tables
immediately, due to the "background drop table queue" or its
replacement in commit 1bd681c8
(the purge of history would clean up after a DDL operation during
which the server was killed).

Let us try to avoid "drop database test" whenever it is easily possible.
Where it is not, SET GLOBAL innodb_max_purge_lag_wait=0 will ensure
that the replacement of the "background drop table queue" will have
completed its job.