Bug#35220: ALTER TABLE too picky on reserved word "foreign"

In ALTER TABLE, change the internal parser to search for
``FOREIGN[[:space:]]'' instead of only ``FOREIGN'' when parsing
ALTER TABLE ... DROP FOREIGN KEY ...; otherwise it could be mistaken
with ALTER TABLE ... DROP foreign_col;

(This fix is already present in MySQL 5.1 and higher.)

innodb-5.0-ss2475.

Bug #34286  Assertion failure in thread 2816 in file .\row\row0sel.c line 3500
Since autoinc init performs a MySQL SELECT query to determine the auto-inc
value, set prebuilt->sql_stat_start = TRUE so that it is performed like any
normal SELECT, regardless of the context in which it was invoked.


Bug #35352  If InnoDB crashes with UNDO slots full error the error persists on restart
We've added a heuristic that checks the size of the UNDO slots cache lists
(insert and upate). If either of cached lists has more than 500 entries then we
add any UNDO slots that are freed, to the common free list instead of the cache
list, this is to avoid the case where all the free slots end up in only one of
the lists on startup after a crash.

Tested with test case for 26590 and passes all mysql-test(s).

Bug #36600  SHOW STATUS takes a lot of CPU in buf_get_latched_pages_number
Fixed by removing the Innodb_buffer_pool_pages_latched variable from SHOW
STATUS output in non-UNIV_DEBUG compilation.

      
Reset session sql_mode before creating system tables as it
is done in the mysql_fix_privilege_tables.sql script.

scripts/mysql_system_tables.sql:
  reset sql mode

      
min() and max() functions are implemented in MySQL as macros.
This means that max(a,b) is expanded to: ((a) > (b) ? (a) : (b))
Note how 'a' is quoted two times.
Now imagine 'a' is a recursive function call that's several 10s of levels deep.
And the recursive function does max() with a function arg as well to dive into
recursion.
This means that simple function call can take most of the clock time.
Identified and fixed several such calls to max()/min() : including the IF() 
sql function implementation.

mysql-test/r/func_if.result:
  Bug#37662 test case
mysql-test/t/func_if.test:
  Bug#37662 test case
sql/item.cc:
  Bug#37662 don't call expensive functions as arguments to min/max
sql/item_cmpfunc.cc:
  Bug#37662 don't call expensive functions as arguments to min/max
sql/item_func.cc:
  Bug#37662 don't call expensive functions as arguments to min/max

Calling List<Cached_item>::delete_elements for the same list twice
caused a crash of the server in the function JOIN::cleaunup.
Ensured that delete_elements() in JOIN::cleanup would be called only once.


mysql-test/r/subselect.result:
  Added a test case for bug #38191.
mysql-test/t/subselect.test:
  Added a test case for bug #38191.
sql/sql_select.cc:
  Fixed bug #38191.
  Ensured that delete_elements() in JOIN::cleanup would be called only once.

                  
Range scan in descending order for c <= <col> <= c type of
ranges was ignoring the DESC flag.
However some engines like InnoDB have the primary key parts 
as a suffix for every secondary key.
When such primary key suffix is used for ordering ignoring 
the DESC is not valid.
But we generally would like to do this because it's faster.
            
Fixed by performing only reverse scan if the primary key is used.
Removed some dead code in the process.

mysql-test/r/innodb_mysql.result:
  Bug#37830 : test case
mysql-test/t/innodb_mysql.test:
  Bug#37830 : test case
sql/opt_range.cc:
  Bug#37830 : 
  - preserve and use used_key_parts to
    distinguish when a primary key suffix is used
  - removed some dead code
sql/opt_range.h:
  Bug#37830 : 
  - preserve used_key_parts
  - dead code removed
sql/sql_select.cc:
  Bug#37830 : Do only reverse order traversal
  if the primary key suffix is used.

If the server failed to expired log files during start up it could crash.

sql/log.cc:
  Added predicate to protect againt cases when current_thd might be NULL.

- Revert the fix for bug 33812
- fixed a win32 warning

client/mysql.cc:
  revert the fix for bug 33812
mysql-test/r/mysql.result:
  revert the fix for bug 33812
mysql-test/t/mysql_delimiter.sql:
  revert the fix for bug 33812
mysys/default.c:
  fixed a win32 warning

Problem: missing initialization, if the previous test
fails leaving table t1, client_xml fails as well.
Fix: adding initialization.

Includes fix for Bug #38180, "options are read from ~/my.cnf instead of ~/.my.cnf"

Don't initalize federated if it's disabled by a command line option.

sql/ha_federated.cc:
  Folow-up on Bug#37069: Don't initalize federated if it's disabled 
  by a command line option.

Pull out some of unpack_dirname() into normalize_dirname(); this
new function does not expand "~" to the home directory.  Use this
function in unpack_dirname(), and use it during init_default_directories()
to remove duplicate entries without losing track of which directory
is a user's home dir.

Reverting the previous patch

- In QUICK_INDEX_MERGE_SELECT::read_keys_and_merge: when we got table->sort from Unique,
  tell init_read_record() not to use rr_from_cache() because a) rowids are already sorted
  and b) it might be that the the data is used by filesort(), which will need record rowids
  (which rr_from_cache() cannot provide).
- Fully de-initialize the table->sort read in QUICK_INDEX_MERGE_SELECT::get_next(). This fixes BUG#35477.
(bk trigger: file as fix for BUG#35478).

sql/filesort.cc:
  BUG#35478: sort_union() returns bad data when sort_buffer_size is hit
  - make find_all_keys() use quick->get_next() instead of init_read_record(r)/r.read_record() calls
  - added dbug printout
sql/mysql_priv.h:
  BUG#35478: sort_union() returns bad data when sort_buffer_size is hit
  - Added parameter to init_read_record
sql/opt_range.cc:
  BUG#35478: sort_union() returns bad data when sort_buffer_size is hit
  - In QUICK_INDEX_MERGE_SELECT::read_keys_and_merge: when we got table->sort from Unique,
    tell init_read_record() not to use rr_from_cache() because a) rowids are already sorted
    and b) it might be that the the data is used by filesort(), which will need record rowids
    (which rr_from_cache() cannot provide).
  - Fully de-initialize the table->sort read in QUICK_INDEX_MERGE_SELECT::get_next().
sql/records.cc:
  BUG#35478: sort_union() returns bad data when sort_buffer_size is hit
  - Added disable_rr_cache parameter to init_read_record
  - Added comment
sql/sql_acl.cc:
  BUG#35478: sort_union() returns bad data when sort_buffer_size is hit
  - Added parameter to init_read_record
sql/sql_delete.cc:
  BUG#35478: sort_union() returns bad data when sort_buffer_size is hit
  - Added parameter to init_read_record
sql/sql_help.cc:
  BUG#35478: sort_union() returns bad data when sort_buffer_size is hit
  - Added parameter to init_read_record
sql/sql_select.cc:
  BUG#35478: sort_union() returns bad data when sort_buffer_size is hit
  - Added parameter to init_read_record
sql/sql_table.cc:
  BUG#35478: sort_union() returns bad data when sort_buffer_size is hit
  - Added parameter to init_read_record
sql/sql_udf.cc:
  BUG#35478: sort_union() returns bad data when sort_buffer_size is hit
  - Added parameter to init_read_record
sql/sql_update.cc:
  BUG#35478: sort_union() returns bad data when sort_buffer_size is hit
  - Added parameter to init_read_record

build)

The crash was caused by freeing the internal parser stack during the parser
execution.
This occured only for complex stored procedures, after reallocating the parser
stack using my_yyoverflow(), with the following C call stack:
- MYSQLparse()
- any rule calling sp_head::restore_lex()
- lex_end()
- x_free(lex->yacc_yyss), xfree(lex->yacc_yyvs)

The root cause is the implementation of stored procedures, which breaks the
assumption from 4.1 that there is only one LEX structure per parser call.

The solution is to separate the LEX structure into:
- attributes that represent a statement (the current LEX structure),
- attributes that relate to the syntax parser itself (Yacc_state),
so that parsing multiple statements in stored programs can create multiple
LEX structures while not changing the unique Yacc_state.

Now, Yacc_state and the existing Lex_input_stream are aggregated into
Parser_state, a structure that represent the complete state of the (Lexical +
Syntax) parser.


mysql-test/r/parser_stack.result:
  Bug#35577 (CREATE PROCEDURE causes either crash or syntax error depending on
  build)
mysql-test/t/parser_stack.test:
  Bug#35577 (CREATE PROCEDURE causes either crash or syntax error depending on
  build)
sql/sp.cc:
  Bug#35577 (CREATE PROCEDURE causes either crash or syntax error depending on
  build)
sql/sp_head.cc:
  Bug#35577 (CREATE PROCEDURE causes either crash or syntax error depending on
  build)
sql/sql_class.cc:
  Bug#35577 (CREATE PROCEDURE causes either crash or syntax error depending on
  build)
sql/sql_class.h:
  Bug#35577 (CREATE PROCEDURE causes either crash or syntax error depending on
  build)
sql/sql_lex.cc:
  Bug#35577 (CREATE PROCEDURE causes either crash or syntax error depending on
  build)
sql/sql_lex.h:
  Bug#35577 (CREATE PROCEDURE causes either crash or syntax error depending on
  build)
sql/sql_parse.cc:
  Bug#35577 (CREATE PROCEDURE causes either crash or syntax error depending on
  build)
sql/sql_prepare.cc:
  Bug#35577 (CREATE PROCEDURE causes either crash or syntax error depending on
  build)
sql/sql_trigger.cc:
  Bug#35577 (CREATE PROCEDURE causes either crash or syntax error depending on
  build)
sql/sql_view.cc:
  Bug#35577 (CREATE PROCEDURE causes either crash or syntax error depending on
  build)
sql/sql_yacc.yy:
  Bug#35577 (CREATE PROCEDURE causes either crash or syntax error depending on
  build)

offset for time part in UUIDs was 1/1000 of what it
should be. In other words, offset was off.

Also handle the case where we count into the future
when several UUIDs are generated in one "tick", and
then the next call is late enough for us to unwind
some but not all of those borrowed ticks.

Lastly, handle the case where we keep borrowing and
borrowing until the tick-counter overflows by also
changing into a new "numberspace" by creating a new
random suffix.


mysql-test/r/func_misc.result:
  Show that time-part of UUIDs is correct now.
mysql-test/t/func_misc.test:
  Show that time-part of UUIDs is correct now
  by replicating the C-code's resultin SQL.
  Results also decode to expect date-data on
  command-line (external validation).
  
  No test for unwinding of borrowed ticks as
  this a) is a race and b) depends on what timer
  we get.
sql/item_strfunc.cc:
  correct offset for date/time-part of UUID.
  also make sure that when we counted into
  the future earlier (several UUIDs generated
  in same tick), we only give back as many
  "borrowed" ticks as we can without duplicating
  past timestamps. If our tick-counter overflows
  before we can give back, or if the system-clock
  is set back (by user or Daylight Saving Time),
  we create a new random suffix to avoid
  collisions and clear the tick-counter.

   "funcs_2: The tests do not check if optional character
               sets exist."
2. Minor cleanup