MDEV-35055 ASAN errors in TABLE_SHARE::lock_share upon committing transaction after FLUSH on table with vector key

MHNSW_Trx cannot store a pointer to the TABLE_SHARE for the duration
of a transaction, because the share can be evicted from the cache any
time.

Use a pointer to the MDL_ticket instead, it is stored in the THD
and has a duration of MDL_TRANSACTION, so won't go away.

When we need a TABLE_SHARE - on commit - get a new one from tdc.
Normally, it'll be already in the cache, so it'd be fast.
We don't optimize for the edge case when TABLE_SHARE was evicted.

disallow USING RTREE for not SPATIAL index

disallow hlindexes in MERGE - because we cannot create the secondary
table *in the same engine*

MDEV-35044 ALTER on a table with vector index attempts to bypass unsupported locking limitation, server crashes in THD::free_tmp_table_share

open secondary tables early enough for the cleanup on error to see
them and remove their underlying files

MEMORY engine doesn't support blobs

support SQL semantics for SELECT ... WHERE ... ORDER BY ... LIMIT

* switch from returning k nearest neighbors to returning
  as many as needed, in k-neighbor chunks, with increasing distance
* make search_layer() skips nodes that are closer than a threshold
* read_next keeps a search context - list of k found nodes,
  threshold, ctx, etc.
* when the list of found nodes is exhausted, it repeats the search
  starting from last found nodes and a threshold
* search context kepts ctx->refcount incremented, so ctx won't go away
* but commit_lock is unlocked between calls, so InnoDB can modify the table
* use ctx version to detect that, switch to MHNSW_Trx when it happens

bugfix:
* use the correct lock in ha_external_lock() for the graph table
* InnoDB didn't reset locks on ha_external_lock(F_UNLCK) and previous
  LOCK_X leaked into the next statement

renames

MDEV-35029 ASAN errors in Lex_ident<Compare_ident_ci>::is_valid_ident upon DDL on table with vector index

in ALTER TABLE or CREATE TABLE LIKE, a create a copy of key->option_list,
because it can be extended later on the thd->mem_root, so it has
to be a copy

MDEV-35033 LeakSanitizer errors in my_malloc / safe_mutex_lazy_init_deadlock_detection / MHNSW_Context::alloc_node and alike

ctx wasn't released on errors

MDEV-35020 After a failed attempt to create vector index temporary file remains and prevents further operation

MDEV-35035 Assertion failure in ha_blackhole::position upon INSERT into blackhole table with vector index

let's allow ::position() and ::rnd_pos() in blackhole.
::position() can be called directly after insert, it doesn't need
a search to happen, so it's possible.

::rnd_pos() can be called with a value that ::position() produced,
so, possible too.

MDEV-35037 Invalid (old?) table or database name 't#i#00' upon creating RocksDB table with vector index

disallow it, for now

also fixes

MDEV-35036 Assertion failure in myrocks::ha_rocksdb::position upon INSERT into RocksDB table with vector index

MDEV-35038 Server crash in Index_statistics::get_avg_frequency upon EITS collection for vector index

don't collect eits for vector indexes

MDEV-35039 Number of indexes inside InnoDB differs from that defined in MariaDB after altering table with vector key

don't show table->s->total_keys to engine in inplace alter

* hlindexes cannot be extended with pk
* hlindexes cannot be covering

add missing casts to float4store/float8store for bigendian.
fix a typo in float4store() usage
remove unnecessary one-byte-at-time appends

make generosity depend on

1. M. Keep small M's fast, increase generosity for larger M's to get
   better recall.

2. distance. Keep generosity small when vectors are far from the
   target, increase generosity when the search gets closer. This
   allows to examine more relevant vectors but doesn't waste time
   examining irrelevant vectors. Particularly important with cosine
   metric when the distance is bounded

... until a few bugs that cause server crash are fixed.

also: renames, s/const/constexpr/ for consistency

Replaced obscure FRM_ONLY, NO_FRM_RENAME, NO_HA_TABLE, NO_PAR_TABLE with
straightforward explicit flags:

QRMT_FRM - [re]moves .frm
QRMT_PAR - [re]moves .par
QRMT_HANDLER - calls ha_delete_table()/ha_rename_table() and [re]moves
               high-level indexes
QRMT_DEFAULT - same as QRMT_FRM | QRMT_HANDLER, which is regular table
               drop/rename.

quick_rm_table() expects .frm to exist when it removes high-level indexes.
For cases like ALTER TABLE t1 RENAME TO t2, ENGINE=other_engine .frm was
removed earlier.

Another option would be removing high-level indexes explicitly before the
first quick_rm_table() and skipping high-level indexes for subsequent
quick_rm_table(NO_FRM_RENAME).

But this suggested order may also help with ddl log recovery. That is
if we crash before high-level indexes are removed, .frm is going to
exist.

Disable non-copy ALTER algorithms when VECTOR index is affected. Engines
are not supposed to handle high-level indexes anyway.

Also fixed misbehaving IF [NOT] EXISTS variants.

Fixes for ALTER TABLE ... ADD/DROP COLUMN, ALGORITHM=COPY.

Let quick_rm_table() remove high-level indexes along with original table.

Avoid locking uninitialized LOCK_share for INTERNAL_TMP_TABLEs.

Don't enable bulk insert when altering a table containing vector index.
InnoDB can't handle situation when bulk insert is enabled for one table
but disabled for another. We can't do bulk insert on vector index as it
does table updates currently.

and create a parent Item_func_vec_distance_common class

into a separate transaction_participant structure

handlerton inherits it, so handlerton itself doesn't change.
but entities that only need to participate in a transaction,
like binlog or online alter log, use a transaction_participant
and no longer need to pretend to be a full-blown but invisible
storage engine which doesn't support create table.

remove unused methods, reorder methods, add comments

Rename high-level indexes along with a table.

* fix the truncate-by-handler variant, used by InnoDB
* test that insert works after truncate, meaning graph table was emptied
* test that the vector index size is zero after truncate in MyISAM